测试hdfs写
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar  TestDFSIO  
-write -nrFiles 10 -fileSize 1000
测试hdfs读
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar  TestDFSIO  
-read -nrFiles 10 -fileSize 1000
清理数据
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar  TestDFSIO  -clean

Total MBytes processed : 总共需要写入的数据量 100MB
Throughput mb/sec :总共需要写入的数据量/(每个map任务实际写入数据的执行时间之和(这个时间会远小于
Test exec time sec))==》100/(map1写时间+map2写时间+...)
Average IO rate mb/sec :(每个map需要写入的数据量/每个map任务实际写入数据的执行时间)之和/任务数
==》(20/map1写时间+20/map2写时间+...)/1000,所以这个值跟上面一个值总是存在差异。
IO rate std deviation :上一个值的标准差
Test exec time sec :整个job的执行时间


mapreduces测试
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar mrbench 
-baseDir /tmp/mrbench -maps 100 -reduces 100 -numRuns 1

results matching ""

    No results matching ""