• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java VertexReader类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.giraph.io.VertexReader的典型用法代码示例。如果您正苦于以下问题:Java VertexReader类的具体用法?Java VertexReader怎么用?Java VertexReader使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



VertexReader类属于org.apache.giraph.io包,在下文中一共展示了VertexReader类的19个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
    TaskAttemptContext context) throws IOException {
  HiveVertexReader<I, V, E> reader = new HiveVertexReader<I, V, E>();
  reader.setTableSchema(getTableSchema());

  RecordReader<WritableComparable, HiveReadableRecord> baseReader;
  try {
    baseReader = hiveInputFormat.createRecordReader(split, context);
  } catch (InterruptedException e) {
    throw new IOException("Could not create vertex reader", e);
  }

  reader.setHiveRecordReader(baseReader);
  return new VertexReaderWrapper<I, V, E>(reader);
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:17,代码来源:HiveVertexInputFormat.java


示例2: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public final VertexReader<I, V, E>
createVertexReader(final InputSplit split,
                   final TaskAttemptContext context)
  throws IOException {
  try {
    HCatalogVertexReader reader = createVertexReader();
    reader.initialize(hCatInputFormat.
        createVertexRecordReader(split, context));
    return reader;
  } catch (InterruptedException e) {
    throw new IllegalStateException(
        "createVertexReader: " +
            "Interrupted creating reader.", e);
  }
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:17,代码来源:HCatalogVertexInputFormat.java


示例3: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
    TaskAttemptContext context) throws IOException {
  HiveVertexReader<I, V, E> reader = new HiveVertexReader<I, V, E>();
  reader.setTableSchema(hiveInputFormat.getTableSchema(getConf()));

  RecordReader<WritableComparable, HiveReadableRecord> baseReader;
  try {
    baseReader = hiveInputFormat.createRecordReader(split, context);
  } catch (InterruptedException e) {
    throw new IOException("Could not create vertex reader", e);
  }

  reader.setHiveRecordReader(baseReader);
  return new VertexReaderWrapper<I, V, E>(reader);
}
 
开发者ID:zfighter,项目名称:giraph-research,代码行数:17,代码来源:HiveVertexInputFormat.java


示例4: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader createVertexReader(final InputSplit split, final TaskAttemptContext context) throws IOException {
    try {
        final GiraphVertexReader reader = new GiraphVertexReader();
        reader.initialize(split, context);
        return reader;
    } catch (final InterruptedException e) {
        throw new IOException(e.getMessage(), e);
    }
}
 
开发者ID:PKUSilvester,项目名称:LiteGraph,代码行数:11,代码来源:GiraphVertexInputFormat.java


示例5: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
 * Creates a new VertexReader
 * @param split the split to be read
 * @param context the information about the task
 * @return VertexReader for LinkRank
 * @throws IOException
 */
public VertexReader<Text, DoubleWritable, NullWritable>
createVertexReader(InputSplit split,
                   TaskAttemptContext context) throws IOException {

  return new NutchTableEdgeVertexReader(split, context);

}
 
开发者ID:AGMLab,项目名称:giranking,代码行数:15,代码来源:Nutch2WebpageInputFormat.java


示例6: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
 * Creates a new VertexReader
 * @param split the split to be read
 * @param context the information about the task
 * @return VertexReader for LinkRank
 * @throws java.io.IOException
 */
public VertexReader<Text, DoubleWritable, NullWritable>
createVertexReader(InputSplit split,
                   TaskAttemptContext context) throws IOException {

  return new NutchTableEdgeVertexReader(split, context);

}
 
开发者ID:AGMLab,项目名称:giranking,代码行数:15,代码来源:Nutch2HostInputFormat.java


示例7: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
public VertexReader<Text, Text, Text>
createVertexReader(InputSplit split, TaskAttemptContext context)
    throws IOException {
  try {

    return new AccumuloEdgeVertexReader(
        accumuloInputFormat.createRecordReader(split, context)) {
    };
  } catch (InterruptedException e) {
    throw new IOException(e);
  }

}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:14,代码来源:AccumuloEdgeInputFormat.java


示例8: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable,
FloatWritable> createVertexReader(InputSplit split,
  TaskAttemptContext context)
  throws IOException {
  return new SimpleVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:8,代码来源:AggregatorsTestComputation.java


示例9: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable,
FloatWritable> createVertexReader(InputSplit split,
  TaskAttemptContext context)
  throws IOException {
  return new SimplePageRankVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:8,代码来源:SimplePageRankComputation.java


示例10: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
    TaskAttemptContext context) throws IOException {
  final VertexReader<I, V, E> vertexReader =
      originalInputFormat.createVertexReader(split,
          HadoopUtils.makeTaskAttemptContext(getConf(), context));
  return new WrappedVertexReader<I, V, E>(vertexReader, getConf());
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:9,代码来源:WrappedVertexInputFormat.java


示例11: WrappedVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
 * Constructor
 *
 * @param baseVertexReader VertexReader to delegate all the methods to
 * @param conf Configuration
 */
public WrappedVertexReader(VertexReader<I, V, E> baseVertexReader,
    ImmutableClassesGiraphConfiguration<I, V, E> conf) {
  this.baseVertexReader = baseVertexReader;
  super.setConf(conf);
  baseVertexReader.setConf(conf);
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:13,代码来源:WrappedVertexReader.java


示例12: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<Text, RyaTypeWritable, RyaStatementWritable> createVertexReader(InputSplit split, TaskAttemptContext context) throws IOException {
     return new RyaVertexReader((RyaStatementRecordReader) ryaInputFormat.createRecordReader(split, context),
            rdfTableLayout, tripleContext,
            context.getConfiguration());
}
 
开发者ID:apache,项目名称:incubator-rya,代码行数:7,代码来源:RyaVertexInputFormat.java


示例13: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, IntWritable, FloatWritable>
createVertexReader(InputSplit split, TaskAttemptContext context)
  throws IOException {
  return new SimpleSuperstepVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:SimpleSuperstepComputation.java


示例14: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit inputSplit,
    TaskAttemptContext context) throws IOException {
  return new InMemoryVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:6,代码来源:InMemoryVertexInputFormat.java


示例15: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<IntWritable, FloatWritable, NullWritable>
createVertexReader(InputSplit split,
    TaskAttemptContext context) throws IOException {
  return new PseudoRandomVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:PseudoRandomIntNullVertexInputFormat.java


示例16: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable, DoubleWritable>
createVertexReader(InputSplit split,
    TaskAttemptContext context) throws IOException {
  return new WattsStrogatzVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:WattsStrogatzVertexInputFormat.java


示例17: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<I, V, E> createVertexReader(InputSplit split,
    TaskAttemptContext context) throws IOException {
  return new SequenceFileVertexReader<I, V, E, X>(
      sequenceFileInputFormat.createRecordReader(split, context));
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:SequenceFileVertexInputFormat.java


示例18: createVertexReader

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
@Override
public VertexReader<LongWritable, DoubleWritable, DoubleWritable>
createVertexReader(InputSplit split, TaskAttemptContext context)
  throws IOException {
  return new PseudoRandomVertexReader();
}
 
开发者ID:renato2099,项目名称:giraph-gora,代码行数:7,代码来源:PseudoRandomVertexInputFormat.java


示例19: readInputSplit

import org.apache.giraph.io.VertexReader; //导入依赖的package包/类
/**
 * Read vertices from input split.  If testing, the user may request a
 * maximum number of vertices to be read from an input split.
 *
 * @param inputSplit Input split to process with vertex reader
 * @param graphState Current graph state
 * @return Vertices and edges loaded from this input split
 * @throws IOException
 * @throws InterruptedException
 */
@Override
protected VertexEdgeCount readInputSplit(
    InputSplit inputSplit,
    GraphState<I, V, E, M> graphState)
  throws IOException, InterruptedException {
  VertexInputFormat<I, V, E> vertexInputFormat =
      configuration.createVertexInputFormat();
  VertexReader<I, V, E> vertexReader =
      vertexInputFormat.createVertexReader(inputSplit, context);
  vertexReader.setConf(
      (ImmutableClassesGiraphConfiguration<I, V, E, Writable>) configuration);
  vertexReader.initialize(inputSplit, context);
  long inputSplitVerticesLoaded = 0;
  long edgesSinceLastUpdate = 0;
  long inputSplitEdgesLoaded = 0;
  while (vertexReader.nextVertex()) {
    Vertex<I, V, E, M> readerVertex =
        (Vertex<I, V, E, M>) vertexReader.getCurrentVertex();
    if (readerVertex.getId() == null) {
      throw new IllegalArgumentException(
          "readInputSplit: Vertex reader returned a vertex " +
              "without an id!  - " + readerVertex);
    }
    if (readerVertex.getValue() == null) {
      readerVertex.setValue(configuration.createVertexValue());
    }
    readerVertex.setConf(configuration);
    readerVertex.setGraphState(graphState);

    PartitionOwner partitionOwner =
        bspServiceWorker.getVertexPartitionOwner(readerVertex.getId());
    graphState.getWorkerClientRequestProcessor().sendVertexRequest(
        partitionOwner, readerVertex);
    context.progress(); // do this before potential data transfer
    ++inputSplitVerticesLoaded;
    edgesSinceLastUpdate += readerVertex.getNumEdges();

    // Update status every VERTICES_UPDATE_PERIOD vertices
    if (inputSplitVerticesLoaded % VERTICES_UPDATE_PERIOD == 0) {
      totalVerticesMeter.mark(VERTICES_UPDATE_PERIOD);
      totalEdgesMeter.mark(edgesSinceLastUpdate);
      inputSplitEdgesLoaded += edgesSinceLastUpdate;
      edgesSinceLastUpdate = 0;

      LoggerUtils.setStatusAndLog(
          context, LOG, Level.INFO,
          "readVertexInputSplit: Loaded " +
              totalVerticesMeter.count() + " vertices at " +
              totalVerticesMeter.meanRate() + " vertices/sec " +
              totalEdgesMeter.count() + " edges at " +
              totalEdgesMeter.meanRate() + " edges/sec " +
              MemoryUtils.getRuntimeMemoryStats());
    }

    // For sampling, or to limit outlier input splits, the number of
    // records per input split can be limited
    if (inputSplitMaxVertices > 0 &&
        inputSplitVerticesLoaded >= inputSplitMaxVertices) {
      if (LOG.isInfoEnabled()) {
        LOG.info("readInputSplit: Leaving the input " +
            "split early, reached maximum vertices " +
            inputSplitVerticesLoaded);
      }
      break;
    }
  }
  vertexReader.close();
  return new VertexEdgeCount(inputSplitVerticesLoaded,
      inputSplitEdgesLoaded + edgesSinceLastUpdate);
}
 
开发者ID:zfighter,项目名称:giraph-research,代码行数:81,代码来源:VertexInputSplitsCallable.java



注:本文中的org.apache.giraph.io.VertexReader类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java ChoosableIterable类代码示例发布时间:2022-05-22
下一篇:
Java OperationHistoryEvent类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap