• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java InputSplitAssigner类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.flink.core.io.InputSplitAssigner的典型用法代码示例。如果您正苦于以下问题:Java InputSplitAssigner类的具体用法?Java InputSplitAssigner怎么用?Java InputSplitAssigner使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



InputSplitAssigner类属于org.apache.flink.core.io包,在下文中一共展示了InputSplitAssigner类的15个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: requestNextInputSplit

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplit requestNextInputSplit(JobID jobID, JobVertexID vertexId, ExecutionAttemptID executionAttempt) throws IOException {

	final ExecutionGraph graph = this.currentJobs.get(jobID);
	if (graph == null) {
		LOG.error("Cannot find execution graph to job ID " + jobID);
		return null;
	}

	final ExecutionJobVertex vertex = graph.getJobVertex(vertexId);
	if (vertex == null) {
		LOG.error("Cannot find execution vertex for vertex ID " + vertexId);
		return null;
	}

	InputSplitAssigner splitAssigner = vertex.getSplitAssigner();
	if (splitAssigner == null) {
		LOG.error("No InputSplitAssigner for vertex ID " + vertexId);
		return null;
	}
	
	// get hostname for input split assignment
	String host = null;
	Execution execution = graph.getRegisteredExecutions().get(executionAttempt);
	if(execution == null) {
		LOG.error("Can not find Execution for attempt " + executionAttempt);
	} else {
		AllocatedSlot slot = execution.getAssignedResource();
		if(slot != null) {
			host = slot.getInstance().getInstanceConnectionInfo().getHostname();
		}
	}
	
	return splitAssigner.getNextInputSplit(host);
}
 
开发者ID:citlab,项目名称:vs.msc.ws14,代码行数:36,代码来源:JobManager.java


示例2: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(TweetFileInputSplit[] inputSplits) {
    return new DefaultInputSplitAssigner(inputSplits);
}
 
开发者ID:mushketyk,项目名称:flink-examples,代码行数:5,代码来源:StanfordTweetsDataSetInputFormat.java


示例3: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(PravegaInputSplit[] inputSplits) {
    return new DefaultInputSplitAssigner(inputSplits);
}
 
开发者ID:pravega,项目名称:flink-connectors,代码行数:5,代码来源:FlinkPravegaInputFormat.java


示例4: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(final SourceInputSplit[] sourceInputSplits) {
  return new DefaultInputSplitAssigner(sourceInputSplits);
}
 
开发者ID:apache,项目名称:beam,代码行数:5,代码来源:SourceInputFormat.java


示例5: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(InputSplit[] inputSplits) {
	return new DefaultInputSplitAssigner(inputSplits);
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:CassandraInputFormat.java


示例6: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(HadoopInputSplit[] inputSplits) {
	return new LocatableInputSplitAssigner(inputSplits);
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:HadoopInputFormatBase.java


示例7: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(TableInputSplit[] inputSplits) {
	return new LocatableInputSplitAssigner(inputSplits);
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:AbstractTableInputFormat.java


示例8: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(CustomInputSplit[] inputSplits) {
	return new CustomSplitAssigner(inputSplits);
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:CustomInputSplitProgram.java


示例9: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(S[] inputSplits) {
	return new ReplicatingInputSplitAssigner(inputSplits);
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:ReplicatingInputFormat.java


示例10: requestNextInputSplit

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public CompletableFuture<SerializedInputSplit> requestNextInputSplit(
		final JobVertexID vertexID,
		final ExecutionAttemptID executionAttempt) {

	final Execution execution = executionGraph.getRegisteredExecutions().get(executionAttempt);
	if (execution == null) {
		// can happen when JobManager had already unregistered this execution upon on task failure,
		// but TaskManager get some delay to aware of that situation
		if (log.isDebugEnabled()) {
			log.debug("Can not find Execution for attempt {}.", executionAttempt);
		}
		// but we should TaskManager be aware of this
		return FutureUtils.completedExceptionally(new Exception("Can not find Execution for attempt " + executionAttempt));
	}

	final ExecutionJobVertex vertex = executionGraph.getJobVertex(vertexID);
	if (vertex == null) {
		log.error("Cannot find execution vertex for vertex ID {}.", vertexID);
		return FutureUtils.completedExceptionally(new Exception("Cannot find execution vertex for vertex ID " + vertexID));
	}

	final InputSplitAssigner splitAssigner = vertex.getSplitAssigner();
	if (splitAssigner == null) {
		log.error("No InputSplitAssigner for vertex ID {}.", vertexID);
		return FutureUtils.completedExceptionally(new Exception("No InputSplitAssigner for vertex ID " + vertexID));
	}

	final LogicalSlot slot = execution.getAssignedResource();
	final int taskId = execution.getVertex().getParallelSubtaskIndex();
	final String host = slot != null ? slot.getTaskManagerLocation().getHostname() : null;
	final InputSplit nextInputSplit = splitAssigner.getNextInputSplit(host, taskId);

	if (log.isDebugEnabled()) {
		log.debug("Send next input split {}.", nextInputSplit);
	}

	try {
		final byte[] serializedInputSplit = InstantiationUtil.serializeObject(nextInputSplit);
		return CompletableFuture.completedFuture(new SerializedInputSplit(serializedInputSplit));
	} catch (Exception ex) {
		log.error("Could not serialize the next input split of class {}.", nextInputSplit.getClass(), ex);
		IOException reason = new IOException("Could not serialize the next input split of class " +
				nextInputSplit.getClass() + ".", ex);
		vertex.fail(reason);
		return FutureUtils.completedExceptionally(reason);
	}
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:49,代码来源:JobMaster.java


示例11: getSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
public InputSplitAssigner getSplitAssigner() {
	return splitAssigner;
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:4,代码来源:ExecutionJobVertex.java


示例12: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(InputSplit[] inputSplits) {
	return null;
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:InputFormatSourceFunctionTest.java


示例13: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(FileCopyTaskInputSplit[] inputSplits) {
	return new FileCopyTaskAssigner(inputSplits);
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:5,代码来源:FileCopyTaskInputFormat.java


示例14: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
@Override
public InputSplitAssigner getInputSplitAssigner(HadoopInputSplit[] inputSplits) {
	return new DefaultInputSplitAssigner(inputSplits);
}
 
开发者ID:citlab,项目名称:vs.msc.ws14,代码行数:5,代码来源:HadoopRecordInputFormat.java


示例15: getInputSplitAssigner

import org.apache.flink.core.io.InputSplitAssigner; //导入依赖的package包/类
/**
 * Gets the type of the input splits that are processed by this input format.
 * 
 * @return The type of the input splits.
 */
@Override
InputSplitAssigner getInputSplitAssigner(T[] inputSplits);
 
开发者ID:axbaretto,项目名称:flink,代码行数:8,代码来源:InputFormat.java



注:本文中的org.apache.flink.core.io.InputSplitAssigner类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java FontPolicy类代码示例发布时间:2022-05-23
下一篇:
Java ApiException类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap