Skip to content

Commit 72741cb

Browse files
authored
Merge pull request quantcast#205 from mckurt/mckurt/wiki
wiki: fix paragraph indentations and bullet numbering in S3 guide
2 parents a529f87 + 0eea64b commit 72741cb

File tree

1 file changed

+22
-23
lines changed

1 file changed

+22
-23
lines changed

wiki/QFS-on-S3.md

Lines changed: 22 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -54,14 +54,13 @@ The changes for each is presented below.
5454
instead of pointing to an existing directory in chunkserver, it points to an hypothetical
5555
directory in which chunkservers can place S3 files. For example, the corresponding line
5656
for this parameter can read:
57-
```
58-
chunkServer.objectDir = s3://myfirstbucket.
59-
```
60-
In example, the part after "s3://" in the specified object directory, i.e. "myfirstbucket."
57+
```
58+
chunkServer.objectDir = s3://myfirstbucket.
59+
```
60+
In the example, the part after "s3://" in the specified object directory, i.e. "myfirstbucket."
6161
(including the . sign at the end), is treated as the configuration suffix. Metaserver
6262
associates the configuration suffix with an S3 bucket name and AWS credentials to access
6363
the given bucket. Corresponding metaserver configuration changes are described in the next section.
64-
6564
This is the only required parameter that needs to be set up in chunkserver configuration file for S3 support. For the optional S3 parameters in chunkserver configuration file, please visit [here](https://github.com/quantcast/qfs/blob/master/conf/ChunkServer.prp#L290).
6665

6766
\* **One can specify multiple object directories with chunkServer.objectDir parameter.
@@ -71,21 +70,21 @@ Please refer to [Specifying Multiple Object Store Directories](#MultipleObjectSt
7170
1. *Add and set [metaServer.objectStoreEnabled](https://github.com/quantcast/qfs/blob/master/conf/MetaServer.prp#L1200) parameter.*
7271

7372
This parameter enables object store in metaserver. For example:
74-
```
75-
metaServer.objectStoreEnabled = 1
76-
```
73+
```
74+
metaServer.objectStoreEnabled = 1
75+
```
7776
2. *For each configuration suffix specified in the chunkserver configuration file, specify
7877
the [chunkServer.diskQueue.<object-store-directory-prefix>bucketName](https://github.com/quantcast/qfs/blob/master/conf/MetaServer.prp#L1245),
7978
[chunkServer.diskQueue.<object-store-directory-prefix>accessKeyId](https://github.com/quantcast/qfs/blob/master/conf/MetaServer.prp#L1248),
8079
and [chunkServer.diskQueue.<object-store-directory-prefix>secretAccessKey](https://github.com/quantcast/qfs/blob/master/conf/MetaServer.prp#L1251)
8180
parameters.*
8281

8382
For example, using the same object directory specified in chunkserver configuration example, one can have:
84-
```
85-
metaServer.diskQueue.myfirstbucket.bucketName = <my-first-bucket-name>
86-
metaServer.diskQueue.myfirstbucket.accessKeyId = <access-key-ID>
87-
metaServer.diskQueue.myfirstbucket.secretAccessKey = <secret-access-key>
88-
```
83+
```
84+
metaServer.diskQueue.myfirstbucket.bucketName = <my-first-bucket-name>
85+
metaServer.diskQueue.myfirstbucket.accessKeyId = <access-key-ID>
86+
metaServer.diskQueue.myfirstbucket.secretAccessKey = <secret-access-key>
87+
```
8988

9089
The actual values for &lt;my-first-bucket-name&gt;, &lt;access-key-ID&gt; and
9190
&lt;secret-access-key&gt; depend on the existing S3 bucket and the IAM
@@ -96,10 +95,10 @@ and [metaServer.maxReplicasPerRSFile](https://github.com/quantcast/qfs/blob/mast
9695

9796
These two parameters tell metaserver to store all files in S3 object store,
9897
regardless of the file create parameters sent by QFS clients. For example:
99-
```
100-
metaServer.maxReplicasPerFile = 0
101-
metaServer.maxReplicasPerRSFile = 0
102-
```
98+
```
99+
metaServer.maxReplicasPerFile = 0
100+
metaServer.maxReplicasPerRSFile = 0
101+
```
103102

104103
These parameters are optional, and typically used in the case where
105104
QFS instance is configured with S3 storage only (in mixed mode, where
@@ -108,9 +107,9 @@ In their absence, QFS clients need to specifically indicate each time
108107
that a file is to be stored in S3 object store. For instance, to copy
109108
a file in S3 object store, one would need to append “-r 0” argument to
110109
cptoqfs call:
111-
```
112-
$QFS_BUILD_DIR/bin/tools/cptoqfs -s 127.0.0.1 -p 20000 -d TestFile.txt -k /TestFile.txt -r 0
113-
```
110+
```
111+
$QFS_BUILD_DIR/bin/tools/cptoqfs -s 127.0.0.1 -p 20000 -d TestFile.txt -k /TestFile.txt -r 0
112+
```
114113

115114
“-r 0” tells the metaserver to use 0 replicas for TestFile.txt and that
116115
internally is mapped to S3 object store by the metaserver. If “-r 0” were omitted,
@@ -123,9 +122,9 @@ may not succeed depending on chunkserver configurations.
123122
web UI reveals all metaserver configuration parameters including AWS credentials
124123
used to access the specified bucket. One can prevent this by setting
125124
this parameter like the following:
126-
```
127-
metaServer.pingDoNotShow = chunkServer.diskQueue.myfirstbucket.accessKeyId chunkServer.diskQueue.myfirstbucket.secretAccessKey
128-
```
125+
```
126+
metaServer.pingDoNotShow = chunkServer.diskQueue.myfirstbucket.accessKeyId chunkServer.diskQueue.myfirstbucket.secretAccessKey
127+
```
129128

130129
For the other optional S3 parameters in the metaserver configuration file, please visit [here](https://github.com/quantcast/qfs/blob/master/conf/MetaServer.prp#L1196).
131130

0 commit comments

Comments
 (0)