本文整理汇总了Java中org.apache.pig.StoreMetadata类的典型用法代码示例。如果您正苦于以下问题:Java StoreMetadata类的具体用法?Java StoreMetadata怎么用?Java StoreMetadata使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
StoreMetadata类属于org.apache.pig包,在下文中一共展示了StoreMetadata类的3个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: storeCleanup
import org.apache.pig.StoreMetadata; //导入依赖的package包/类
static public void storeCleanup(POStore store, Configuration conf)
throws IOException {
StoreFuncInterface storeFunc = store.getStoreFunc();
if (storeFunc instanceof StoreMetadata) {
Schema schema = store.getSchema();
if (schema != null) {
((StoreMetadata) storeFunc).storeSchema(
new ResourceSchema(schema, store.getSortInfo()), store.getSFile()
.getFileName(), new Job(conf));
}
}
}
开发者ID:sigmoidanalytics,项目名称:spork-streaming,代码行数:13,代码来源:PigOutputCommitter.java
示例2: write
import org.apache.pig.StoreMetadata; //导入依赖的package包/类
public static void write(String out) throws IOException, ParserException,
InterruptedException, ExecException {
{
StringBuilder schemaString = new StringBuilder("a0: chararray");
for (int i = 1; i < COLUMN_COUNT; i++) {
schemaString.append(", a" + i + ": chararray");
}
String location = out;
String schema = schemaString.toString();
StoreFuncInterface storer = new ParquetStorer();
Job job = new Job(conf);
storer.setStoreFuncUDFContextSignature("sig");
String absPath = storer.relToAbsPathForStoreLocation(location, new Path(new File(".").getAbsoluteFile().toURI()));
storer.setStoreLocation(absPath, job);
storer.checkSchema(new ResourceSchema(Utils.getSchemaFromString(schema)));
@SuppressWarnings("unchecked") // that's how the base class is defined
OutputFormat<Void, Tuple> outputFormat = storer.getOutputFormat();
// it's ContextUtil.getConfiguration(job) and not just conf !
JobContext jobContext = ContextUtil.newJobContext(ContextUtil.getConfiguration(job), new JobID("jt", jobid ++));
outputFormat.checkOutputSpecs(jobContext);
if (schema != null) {
ResourceSchema resourceSchema = new ResourceSchema(Utils.getSchemaFromString(schema));
storer.checkSchema(resourceSchema);
if (storer instanceof StoreMetadata) {
((StoreMetadata)storer).storeSchema(resourceSchema, absPath, job);
}
}
TaskAttemptContext taskAttemptContext = ContextUtil.newTaskAttemptContext(ContextUtil.getConfiguration(job), new TaskAttemptID("jt", jobid, true, 1, 0));
RecordWriter<Void, Tuple> recordWriter = outputFormat.getRecordWriter(taskAttemptContext);
storer.prepareToWrite(recordWriter);
for (int i = 0; i < ROW_COUNT; i++) {
Tuple tuple = TupleFactory.getInstance().newTuple(COLUMN_COUNT);
for (int j = 0; j < COLUMN_COUNT; j++) {
tuple.set(j, "a" + i + "_" + j);
}
storer.putNext(tuple);
}
recordWriter.close(taskAttemptContext);
OutputCommitter outputCommitter = outputFormat.getOutputCommitter(taskAttemptContext);
outputCommitter.commitTask(taskAttemptContext);
outputCommitter.commitJob(jobContext);
}
}
开发者ID:apache,项目名称:parquet-mr,代码行数:49,代码来源:PerfTest2.java
示例3: init
import org.apache.pig.StoreMetadata; //导入依赖的package包/类
private void init(StoreMetadata storeMetadata) {
setStoreFunc(storeMetadata);
}
开发者ID:sigmoidanalytics,项目名称:spork,代码行数:4,代码来源:ParquetStorer.java
注:本文中的org.apache.pig.StoreMetadata类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论