p-daos.2.4.1.source-code.daos-config.txt Maven / Gradle / Ivy
The newest version!
-> DAOS URIs
1. daos:////.
2. daos:////.
3. daos://[authority starts with "uns-id"]/.
If the authority doesn't start with "uns-id", it will be considered as either pool UUID or label. You
will get unexpected parse error.
-> DAOS FS Configs
The following config items starting from 6th can put into application, Hadoop config files and DAOS container
via "daos cont set-attr" command (see examples in next section for the command). The config priority is,
application > Hadoop config files > DAOS container. The items from 1st to 5th are item-specific.
name default value description
1. fs.defaultFS no default one of above DAOS URIs. Read from application or Hadoop
config files
2. fs.daos.server.group daos_server DAOS server group. Read from application or Hadoop
config files
3. fs.daos.pool.id UUID or label user should not set it directly
parsed from DAOS URI
4. fs.daos.container.id UUID or label user should not set it directly
parsed from DAOS URI
5. fs.daos.pool.flags 2 daos pool access flags, 1 for readonly,
2 for read/write, 4 for execute. Read from application
or Hadoop config files
6. fs.daos.choice no default multiple applications can use different choices to
apply their own configurations or override default
values. E.g., if Spark set "fs.daos.choice" to "spark".
DAOS FS will try to read "spark.fs.daos.*" (from item 6)
first. Then fall back to "fs.daos.*".
7. fs.daos.read.buffer.size 1048576 size of direct buffer for reading data from DAOS.
Default is 1m. Value range is 64k - 2g
8. fs.daos.read.min.size 65536 minimum size of direct buffer for reading data from
DAOS. Default is 64k. Value range is 64k - 2g. It should
be no more than fs.daos.read.buffer.size
9. fs.daos.write.buffer.size 1048576 size of direct buffer for writing data to DAOS. Default
is 1m. Value range is 64k - 2g
10. fs.daos.block.size 134217728 size for splitting large file into blocks when read by
Hadoop. Default is 128m. Value range is 16m - 2m.
11. fs.daos.chunk.size 1048576 size of DAOS file chunk. Default is 1m. Value range is
4k - 2g.
12. fs.daos.io.async true perform DAOS IO asynchronously. Default is true.
Set to false to use synchronous IO.
-> DAOS FS Config to DAOS Container Examples
1. Set/get the choice to "spark" and set "read buffer size" for Spark
"daos cont set-attr bce96e2f-5a52-4aeb-b0b4-8994eee79a5f e7814443-c8a1-4014-b94b-571ac7f5cff3
--attr=fs.daos.choice --value=spark"
"daos cont get-attr bce96e2f-5a52-4aeb-b0b4-8994eee79a5f e7814443-c8a1-4014-b94b-571ac7f5cff3
fs.daos.choice"
"daos cont set-attr bce96e2f-5a52-4aeb-b0b4-8994eee79a5f e7814443-c8a1-4014-b94b-571ac7f5cff3
--attr=spark.fs.daos.read.buffer.size --value=2048576"
2. Set container specific default value
"daos cont set-attr bce96e2f-5a52-4aeb-b0b4-8994eee79a5f e7814443-c8a1-4014-b94b-571ac7f5cff3
--attr=fs.daos.read.buffer.size --value=2048576"