本文整理汇总了Java中org.apache.hadoop.fs.loadGenerator.DataGenerator类的典型用法代码示例。如果您正苦于以下问题:Java DataGenerator类的具体用法?Java DataGenerator怎么用?Java DataGenerator使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
DataGenerator类属于org.apache.hadoop.fs.loadGenerator包,在下文中一共展示了DataGenerator类的2个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: MapredTestDriver
import org.apache.hadoop.fs.loadGenerator.DataGenerator; //导入依赖的package包/类
public MapredTestDriver(ProgramDriver pgd) {
this.pgd = pgd;
try {
pgd.addClass("testsequencefile", TestSequenceFile.class,
"A test for flat files of binary key value pairs.");
pgd.addClass("threadedmapbench", ThreadedMapBenchmark.class,
"A map/reduce benchmark that compares the performance " +
"of maps with multiple spills over maps with 1 spill");
pgd.addClass("mrbench", MRBench.class,
"A map/reduce benchmark that can create many small jobs");
pgd.addClass("mapredtest", TestMapRed.class, "A map/reduce test check.");
pgd.addClass("testsequencefileinputformat",
TestSequenceFileInputFormat.class,
"A test for sequence file input format.");
pgd.addClass("testtextinputformat", TestTextInputFormat.class,
"A test for text input format.");
pgd.addClass("testmapredsort", SortValidator.class,
"A map/reduce program that validates the " +
"map-reduce framework's sort.");
pgd.addClass("testbigmapoutput", BigMapOutput.class,
"A map/reduce program that works on a very big " +
"non-splittable file and does identity map/reduce");
pgd.addClass("loadgen", GenericMRLoadGenerator.class,
"Generic map/reduce load generator");
pgd.addClass("MRReliabilityTest", ReliabilityTest.class,
"A program that tests the reliability of the MR framework by " +
"injecting faults/failures");
pgd.addClass("fail", FailJob.class, "a job that always fails");
pgd.addClass("sleep", SleepJob.class,
"A job that sleeps at each map and reduce task.");
pgd.addClass("nnbench", NNBench.class,
"A benchmark that stresses the namenode.");
pgd.addClass("testfilesystem", TestFileSystem.class,
"A test for FileSystem read/write.");
pgd.addClass(TestDFSIO.class.getSimpleName(), TestDFSIO.class,
"Distributed i/o benchmark.");
pgd.addClass("DFSCIOTest", DFSCIOTest.class, "" +
"Distributed i/o benchmark of libhdfs.");
pgd.addClass("DistributedFSCheck", DistributedFSCheck.class,
"Distributed checkup of the file system consistency.");
pgd.addClass("filebench", FileBench.class,
"Benchmark SequenceFile(Input|Output)Format " +
"(block,record compressed and uncompressed), " +
"Text(Input|Output)Format (compressed and uncompressed)");
pgd.addClass(JHLogAnalyzer.class.getSimpleName(), JHLogAnalyzer.class,
"Job History Log analyzer.");
pgd.addClass(SliveTest.class.getSimpleName(), SliveTest.class,
"HDFS Stress Test and Live Data Verification.");
pgd.addClass("minicluster", MiniHadoopClusterManager.class,
"Single process HDFS and MR cluster.");
pgd.addClass("largesorter", LargeSorter.class,
"Large-Sort tester");
pgd.addClass("NNloadGenerator", LoadGenerator.class,
"Generate load on Namenode using NN loadgenerator run WITHOUT MR");
pgd.addClass("NNloadGeneratorMR", LoadGeneratorMR.class,
"Generate load on Namenode using NN loadgenerator run as MR job");
pgd.addClass("NNstructureGenerator", StructureGenerator.class,
"Generate the structure to be used by NNdataGenerator");
pgd.addClass("NNdataGenerator", DataGenerator.class,
"Generate the data to be used by NNloadGenerator");
} catch(Throwable e) {
e.printStackTrace();
}
}
开发者ID:naver,项目名称:hadoop,代码行数:65,代码来源:MapredTestDriver.java
示例2: MapredTestDriver
import org.apache.hadoop.fs.loadGenerator.DataGenerator; //导入依赖的package包/类
public MapredTestDriver(ProgramDriver pgd) {
this.pgd = pgd;
try {
pgd.addClass("testsequencefile", TestSequenceFile.class,
"A test for flat files of binary key value pairs.");
pgd.addClass("threadedmapbench", ThreadedMapBenchmark.class,
"A map/reduce benchmark that compares the performance " +
"of maps with multiple spills over maps with 1 spill");
pgd.addClass("mrbench", MRBench.class,
"A map/reduce benchmark that can create many small jobs");
pgd.addClass("mapredtest", TestMapRed.class, "A map/reduce test check.");
pgd.addClass("testsequencefileinputformat",
TestSequenceFileInputFormat.class,
"A test for sequence file input format.");
pgd.addClass("testtextinputformat", TestTextInputFormat.class,
"A test for text input format.");
pgd.addClass("testmapredsort", SortValidator.class,
"A map/reduce program that validates the " +
"map-reduce framework's sort.");
pgd.addClass("testbigmapoutput", BigMapOutput.class,
"A map/reduce program that works on a very big " +
"non-splittable file and does identity map/reduce");
pgd.addClass("loadgen", GenericMRLoadGenerator.class,
"Generic map/reduce load generator");
pgd.addClass("MRReliabilityTest", ReliabilityTest.class,
"A program that tests the reliability of the MR framework by " +
"injecting faults/failures");
pgd.addClass("fail", FailJob.class, "a job that always fails");
pgd.addClass("sleep", SleepJob.class,
"A job that sleeps at each map and reduce task.");
pgd.addClass("timelineperformance", TimelineServicePerformance.class,
"A job that launches mappers to test timlineserver performance.");
pgd.addClass("nnbench", NNBench.class,
"A benchmark that stresses the namenode w/ MR.");
pgd.addClass("nnbenchWithoutMR", NNBenchWithoutMR.class,
"A benchmark that stresses the namenode w/o MR.");
pgd.addClass("testfilesystem", TestFileSystem.class,
"A test for FileSystem read/write.");
pgd.addClass(TestDFSIO.class.getSimpleName(), TestDFSIO.class,
"Distributed i/o benchmark.");
pgd.addClass("DFSCIOTest", DFSCIOTest.class, "" +
"Distributed i/o benchmark of libhdfs.");
pgd.addClass("DistributedFSCheck", DistributedFSCheck.class,
"Distributed checkup of the file system consistency.");
pgd.addClass("filebench", FileBench.class,
"Benchmark SequenceFile(Input|Output)Format " +
"(block,record compressed and uncompressed), " +
"Text(Input|Output)Format (compressed and uncompressed)");
pgd.addClass(JHLogAnalyzer.class.getSimpleName(), JHLogAnalyzer.class,
"Job History Log analyzer.");
pgd.addClass(SliveTest.class.getSimpleName(), SliveTest.class,
"HDFS Stress Test and Live Data Verification.");
pgd.addClass("minicluster", MiniHadoopClusterManager.class,
"Single process HDFS and MR cluster.");
pgd.addClass("largesorter", LargeSorter.class,
"Large-Sort tester");
pgd.addClass("NNloadGenerator", LoadGenerator.class,
"Generate load on Namenode using NN loadgenerator run WITHOUT MR");
pgd.addClass("NNloadGeneratorMR", LoadGeneratorMR.class,
"Generate load on Namenode using NN loadgenerator run as MR job");
pgd.addClass("NNstructureGenerator", StructureGenerator.class,
"Generate the structure to be used by NNdataGenerator");
pgd.addClass("NNdataGenerator", DataGenerator.class,
"Generate the data to be used by NNloadGenerator");
} catch(Throwable e) {
e.printStackTrace();
}
}
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:69,代码来源:MapredTestDriver.java
注:本文中的org.apache.hadoop.fs.loadGenerator.DataGenerator类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论