本文整理汇总了Java中org.apache.flink.core.io.InputSplitAssigner类的典型用法代码示例。如果您正苦于以下问题:Java InputSplitAssigner类的具体用法?Java InputSplitAssigner怎么用?Java InputSplitAssigner使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
InputSplitAssigner类属于org.apache.flink.core.io包,在下文中一共展示了InputSplitAssigner类的15个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: requestNextInputSplit
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplit requestNextInputSplit(JobID jobID, JobVertexID vertexId, ExecutionAttemptID executionAttempt) throws IOException {
final ExecutionGraph graph = this.currentJobs.get(jobID);
if (graph == null) {
LOG.error("Cannot find execution graph to job ID " + jobID);
return null;
}
final ExecutionJobVertex vertex = graph.getJobVertex(vertexId);
if (vertex == null) {
LOG.error("Cannot find execution vertex for vertex ID " + vertexId);
return null;
}
InputSplitAssigner splitAssigner = vertex.getSplitAssigner();
if (splitAssigner == null) {
LOG.error("No InputSplitAssigner for vertex ID " + vertexId);
return null;
}
// get hostname for input split assignment
String host = null;
Execution execution = graph.getRegisteredExecutions().get(executionAttempt);
if(execution == null) {
LOG.error("Can not find Execution for attempt " + executionAttempt);
} else {
AllocatedSlot slot = execution.getAssignedResource();
if(slot != null) {
host = slot.getInstance().getInstanceConnectionInfo().getHostname();
}
}
return splitAssigner.getNextInputSplit(host);
}
开发者ID:citlab,项目名称:vs.msc.ws14,代码行数:36,代码来源:JobManager.java
示例2: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(TweetFileInputSplit[] inputSplits) {
return new DefaultInputSplitAssigner(inputSplits);
}
开发者ID:mushketyk,项目名称:flink-examples,代码行数:5,代码来源:StanfordTweetsDataSetInputFormat.java
示例3: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(PravegaInputSplit[] inputSplits) {
return new DefaultInputSplitAssigner(inputSplits);
}
开发者ID:pravega,项目名称:flink-connectors,代码行数:5,代码来源:FlinkPravegaInputFormat.java
示例4: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(final SourceInputSplit[] sourceInputSplits) {
return new DefaultInputSplitAssigner(sourceInputSplits);
}
开发者ID:apache,项目名称:beam,代码行数:5,代码来源:SourceInputFormat.java
示例5: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(InputSplit[] inputSplits) {
return new DefaultInputSplitAssigner(inputSplits);
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:CassandraInputFormat.java
示例6: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(HadoopInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:HadoopInputFormatBase.java
示例7: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(TableInputSplit[] inputSplits) {
return new LocatableInputSplitAssigner(inputSplits);
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:AbstractTableInputFormat.java
示例8: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(CustomInputSplit[] inputSplits) {
return new CustomSplitAssigner(inputSplits);
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:CustomInputSplitProgram.java
示例9: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(S[] inputSplits) {
return new ReplicatingInputSplitAssigner(inputSplits);
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:ReplicatingInputFormat.java
示例10: requestNextInputSplit
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public CompletableFuture<SerializedInputSplit> requestNextInputSplit(
final JobVertexID vertexID,
final ExecutionAttemptID executionAttempt) {
final Execution execution = executionGraph.getRegisteredExecutions().get(executionAttempt);
if (execution == null) {
// can happen when JobManager had already unregistered this execution upon on task failure,
// but TaskManager get some delay to aware of that situation
if (log.isDebugEnabled()) {
log.debug("Can not find Execution for attempt {}.", executionAttempt);
}
// but we should TaskManager be aware of this
return FutureUtils.completedExceptionally(new Exception("Can not find Execution for attempt " + executionAttempt));
}
final ExecutionJobVertex vertex = executionGraph.getJobVertex(vertexID);
if (vertex == null) {
log.error("Cannot find execution vertex for vertex ID {}.", vertexID);
return FutureUtils.completedExceptionally(new Exception("Cannot find execution vertex for vertex ID " + vertexID));
}
final InputSplitAssigner splitAssigner = vertex.getSplitAssigner();
if (splitAssigner == null) {
log.error("No InputSplitAssigner for vertex ID {}.", vertexID);
return FutureUtils.completedExceptionally(new Exception("No InputSplitAssigner for vertex ID " + vertexID));
}
final LogicalSlot slot = execution.getAssignedResource();
final int taskId = execution.getVertex().getParallelSubtaskIndex();
final String host = slot != null ? slot.getTaskManagerLocation().getHostname() : null;
final InputSplit nextInputSplit = splitAssigner.getNextInputSplit(host, taskId);
if (log.isDebugEnabled()) {
log.debug("Send next input split {}.", nextInputSplit);
}
try {
final byte[] serializedInputSplit = InstantiationUtil.serializeObject(nextInputSplit);
return CompletableFuture.completedFuture(new SerializedInputSplit(serializedInputSplit));
} catch (Exception ex) {
log.error("Could not serialize the next input split of class {}.", nextInputSplit.getClass(), ex);
IOException reason = new IOException("Could not serialize the next input split of class " +
nextInputSplit.getClass() + ".", ex);
vertex.fail(reason);
return FutureUtils.completedExceptionally(reason);
}
}
开发者ID:axbaretto,项目名称:flink,代码行数:49,代码来源:JobMaster.java
示例11: getSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
public InputSplitAssigner getSplitAssigner() {
return splitAssigner;
}
开发者ID:axbaretto,项目名称:flink,代码行数:4,代码来源:ExecutionJobVertex.java
示例12: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(InputSplit[] inputSplits) {
return null;
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:InputFormatSourceFunctionTest.java
示例13: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(FileCopyTaskInputSplit[] inputSplits) {
return new FileCopyTaskAssigner(inputSplits);
}
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:FileCopyTaskInputFormat.java
示例14: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(HadoopInputSplit[] inputSplits) {
return new DefaultInputSplitAssigner(inputSplits);
}
开发者ID:citlab,项目名称:vs.msc.ws14,代码行数:5,代码来源:HadoopRecordInputFormat.java
示例15: getInputSplitAssigner
import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
/**
* Gets the type of the input splits that are processed by this input format.
*
* @return The type of the input splits.
*/
@Override
InputSplitAssigner getInputSplitAssigner(T[] inputSplits);
开发者ID:axbaretto,项目名称:flink,代码行数:8,代码来源:InputFormat.java
注:本文中的org.apache.flink.core.io.InputSplitAssigner类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论