本文整理汇总了Java中org.apache.giraph.io.VertexReader类的典型用法代码示例。如果您正苦于以下问题:Java VertexReader类的具体用法?Java VertexReader怎么用?Java VertexReader使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
VertexReader类属于org.apache.giraph.io包,在下文中一共展示了VertexReader类的19个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
HiveVertexReader<I, V, E> reader = new HiveVertexReader<I, V, E>();
reader.setTableSchema(getTableSchema());
RecordReader<WritableComparable, HiveReadableRecord> baseReader;
try {
baseReader = hiveInputFormat.createRecordReader(split, context);
} catch (InterruptedException e) {
throw new IOException("Could not create vertex reader", e);
}
reader.setHiveRecordReader(baseReader);
return new VertexReaderWrapper<I, V, E>(reader);
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:17,代码来源:HiveVertexInputFormat.java
示例2: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public final VertexReader<I, V, E>
createVertexReader(final InputSplit split,
final TaskAttemptContext context)
throws IOException {
try {
HCatalogVertexReader reader = createVertexReader();
reader.initialize(hCatInputFormat.
createVertexRecordReader(split, context));
return reader;
} catch (InterruptedException e) {
throw new IllegalStateException(
"createVertexReader: " +
"Interrupted creating reader.", e);
}
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:17,代码来源:HCatalogVertexInputFormat.java
示例3: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
HiveVertexReader<I, V, E> reader = new HiveVertexReader<I, V, E>();
reader.setTableSchema(hiveInputFormat.getTableSchema(getConf()));
RecordReader<WritableComparable, HiveReadableRecord> baseReader;
try {
baseReader = hiveInputFormat.createRecordReader(split, context);
} catch (InterruptedException e) {
throw new IOException("Could not create vertex reader", e);
}
reader.setHiveRecordReader(baseReader);
return new VertexReaderWrapper<I, V, E>(reader);
}
开发者ID:zfighter,项目名称:giraph-research,代码行数:17,代码来源:HiveVertexInputFormat.java
示例4: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader createVertexReader(final InputSplit split, final TaskAttemptContext context) throws IOException {
try {
final GiraphVertexReader reader = new GiraphVertexReader();
reader.initialize(split, context);
return reader;
} catch (final InterruptedException e) {
throw new IOException(e.getMessage(), e);
}
}
开发者ID:PKUSilvester,项目名称:LiteGraph,代码行数:11,代码来源:GiraphVertexInputFormat.java
示例5: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
* Creates a new VertexReader
* @param split the split to be read
* @param context the information about the task
* @return VertexReader for LinkRank
* @throws IOException
*/
public VertexReader<Text, DoubleWritable, NullWritable>
createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
return new NutchTableEdgeVertexReader(split, context);
}
开发者ID:AGMLab,项目名称:giranking,代码行数:15,代码来源:Nutch2WebpageInputFormat.java
示例6: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
* Creates a new VertexReader
* @param split the split to be read
* @param context the information about the task
* @return VertexReader for LinkRank
* @throws java.io.IOException
*/
public VertexReader<Text, DoubleWritable, NullWritable>
createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
return new NutchTableEdgeVertexReader(split, context);
}
开发者ID:AGMLab,项目名称:giranking,代码行数:15,代码来源:Nutch2HostInputFormat.java
示例7: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
public VertexReader<Text, Text, Text>
createVertexReader(InputSplit split, TaskAttemptContext context)
throws IOException {
try {
return new AccumuloEdgeVertexReader(
accumuloInputFormat.createRecordReader(split, context)) {
};
} catch (InterruptedException e) {
throw new IOException(e);
}
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:14,代码来源:AccumuloEdgeInputFormat.java
示例8: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable,
FloatWritable> createVertexReader(InputSplit split,
TaskAttemptContext context)
throws IOException {
return new SimpleVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:8,代码来源:AggregatorsTestComputation.java
示例9: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable,
FloatWritable> createVertexReader(InputSplit split,
TaskAttemptContext context)
throws IOException {
return new SimplePageRankVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:8,代码来源:SimplePageRankComputation.java
示例10: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
final VertexReader<I, V, E> vertexReader =
originalInputFormat.createVertexReader(split,
HadoopUtils.makeTaskAttemptContext(getConf(), context));
return new WrappedVertexReader<I, V, E>(vertexReader, getConf());
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:9,代码来源:WrappedVertexInputFormat.java
示例11: WrappedVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
* Constructor
*
* @param baseVertexReader VertexReader to delegate all the methods to
* @param conf Configuration
*/
public WrappedVertexReader(VertexReader<I, V, E> baseVertexReader,
ImmutableClassesGiraphConfiguration<I, V, E> conf) {
this.baseVertexReader = baseVertexReader;
super.setConf(conf);
baseVertexReader.setConf(conf);
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:13,代码来源:WrappedVertexReader.java
示例12: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<Text, RyaTypeWritable, RyaStatementWritable> createVertexReader(InputSplit split, TaskAttemptContext context) throws IOException {
return new RyaVertexReader((RyaStatementRecordReader) ryaInputFormat.createRecordReader(split, context),
rdfTableLayout, tripleContext,
context.getConfiguration());
}
开发者ID:apache,项目名称:incubator-rya,代码行数:7,代码来源:RyaVertexInputFormat.java
示例13: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, IntWritable, FloatWritable>
createVertexReader(InputSplit split, TaskAttemptContext context)
throws IOException {
return new SimpleSuperstepVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:SimpleSuperstepComputation.java
示例14: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit inputSplit,
TaskAttemptContext context) throws IOException {
return new InMemoryVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:6,代码来源:InMemoryVertexInputFormat.java
示例15: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<IntWritable, FloatWritable, NullWritable>
createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
return new PseudoRandomVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:PseudoRandomIntNullVertexInputFormat.java
示例16: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable, DoubleWritable>
createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
return new WattsStrogatzVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:WattsStrogatzVertexInputFormat.java
示例17: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
TaskAttemptContext context) throws IOException {
return new SequenceFileVertexReader<I, V, E, X>(
sequenceFileInputFormat.createRecordReader(split, context));
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:SequenceFileVertexInputFormat.java
示例18: createVertexReader
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable, DoubleWritable>
createVertexReader(InputSplit split, TaskAttemptContext context)
throws IOException {
return new PseudoRandomVertexReader();
}
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:PseudoRandomVertexInputFormat.java
示例19: readInputSplit
import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
* Read vertices from input split. If testing, the user may request a
* maximum number of vertices to be read from an input split.
*
* @param inputSplit Input split to process with vertex reader
* @param graphState Current graph state
* @return Vertices and edges loaded from this input split
* @throws IOException
* @throws InterruptedException
*/
@Override
protected VertexEdgeCount readInputSplit(
InputSplit inputSplit,
GraphState<I, V, E, M> graphState)
throws IOException, InterruptedException {
VertexInputFormat<I, V, E> vertexInputFormat =
configuration.createVertexInputFormat();
VertexReader<I, V, E> vertexReader =
vertexInputFormat.createVertexReader(inputSplit, context);
vertexReader.setConf(
(ImmutableClassesGiraphConfiguration<I, V, E, Writable>) configuration);
vertexReader.initialize(inputSplit, context);
long inputSplitVerticesLoaded = 0;
long edgesSinceLastUpdate = 0;
long inputSplitEdgesLoaded = 0;
while (vertexReader.nextVertex()) {
Vertex<I, V, E, M> readerVertex =
(Vertex<I, V, E, M>) vertexReader.getCurrentVertex();
if (readerVertex.getId() == null) {
throw new IllegalArgumentException(
"readInputSplit: Vertex reader returned a vertex " +
"without an id! - " + readerVertex);
}
if (readerVertex.getValue() == null) {
readerVertex.setValue(configuration.createVertexValue());
}
readerVertex.setConf(configuration);
readerVertex.setGraphState(graphState);
PartitionOwner partitionOwner =
bspServiceWorker.getVertexPartitionOwner(readerVertex.getId());
graphState.getWorkerClientRequestProcessor().sendVertexRequest(
partitionOwner, readerVertex);
context.progress(); // do this before potential data transfer
++inputSplitVerticesLoaded;
edgesSinceLastUpdate += readerVertex.getNumEdges();
// Update status every VERTICES_UPDATE_PERIOD vertices
if (inputSplitVerticesLoaded % VERTICES_UPDATE_PERIOD == 0) {
totalVerticesMeter.mark(VERTICES_UPDATE_PERIOD);
totalEdgesMeter.mark(edgesSinceLastUpdate);
inputSplitEdgesLoaded += edgesSinceLastUpdate;
edgesSinceLastUpdate = 0;
LoggerUtils.setStatusAndLog(
context, LOG, Level.INFO,
"readVertexInputSplit: Loaded " +
totalVerticesMeter.count() + " vertices at " +
totalVerticesMeter.meanRate() + " vertices/sec " +
totalEdgesMeter.count() + " edges at " +
totalEdgesMeter.meanRate() + " edges/sec " +
MemoryUtils.getRuntimeMemoryStats());
}
// For sampling, or to limit outlier input splits, the number of
// records per input split can be limited
if (inputSplitMaxVertices > 0 &&
inputSplitVerticesLoaded >= inputSplitMaxVertices) {
if (LOG.isInfoEnabled()) {
LOG.info("readInputSplit: Leaving the input " +
"split early, reached maximum vertices " +
inputSplitVerticesLoaded);
}
break;
}
}
vertexReader.close();
return new VertexEdgeCount(inputSplitVerticesLoaded,
inputSplitEdgesLoaded + edgesSinceLastUpdate);
}
开发者ID:zfighter,项目名称:giraph-research,代码行数:81,代码来源:VertexInputSplitsCallable.java
注:本文中的org.apache.giraph.io.VertexReader类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论