Java DistCpConstants类代码示例

OStack程序员社区-中国程序员成长平台 › 门户 › 编程› Java›Java编程经验

原作者: [db:作者] 来自: [db:来源] 收藏邀请

本文整理汇总了Java中org.apache.hadoop.tools.DistCpConstants类的典型用法代码示例。如果您正苦于以下问题：Java DistCpConstants类的具体用法？Java DistCpConstants怎么用？Java DistCpConstants使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

DistCpConstants类属于org.apache.hadoop.tools包，在下文中一共展示了DistCpConstants类的9个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: getTmpFile

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
private Path getTmpFile(Path target, Mapper.Context context) {
  Path targetWorkPath = new Path(context.getConfiguration().
      get(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH));

  Path root = target.equals(targetWorkPath)? targetWorkPath.getParent() : targetWorkPath;
  LOG.info("Creating temp file: " +
      new Path(root, ".distcp.tmp." + context.getTaskAttemptID().toString()));
  return new Path(root, ".distcp.tmp." + context.getTaskAttemptID().toString());
}

开发者ID:naver，项目名称:hadoop，代码行数:10，代码来源:RetriableFileCopyCommand.java

示例2: assertDefaultValues

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
private void assertDefaultValues(DistCpOptions distCpOptions) {
  assertThat(distCpOptions, is(not(nullValue())));
  assertThat(distCpOptions.preserveAttributes().hasNext(), is(false));
  assertThat(distCpOptions.shouldPreserveRawXattrs(), is(false));
  assertThat(distCpOptions.shouldAppend(), is(false));
  assertThat(distCpOptions.shouldAtomicCommit(), is(false));
  assertThat(distCpOptions.getAtomicWorkPath(), is(nullValue()));
  assertThat(distCpOptions.shouldBlock(), is(true));
  assertThat(distCpOptions.getCopyStrategy(), is(DistCpConstants.UNIFORMSIZE));
  assertThat(distCpOptions.shouldDeleteMissing(), is(false));
  assertThat(distCpOptions.shouldIgnoreFailures(), is(false));
  assertThat(distCpOptions.getLogPath(), is(nullValue()));
  assertThat(distCpOptions.getMapBandwidth(), is(DistCpConstants.DEFAULT_BANDWIDTH_MB));
  assertThat(distCpOptions.getMaxMaps(), is(DistCpConstants.DEFAULT_MAPS));
  assertThat(distCpOptions.shouldOverwrite(), is(false));
  assertThat(distCpOptions.shouldSkipCRC(), is(false));
  assertThat(distCpOptions.getSslConfigurationFile(), is(nullValue()));
  assertThat(distCpOptions.shouldSyncFolder(), is(false));
  assertThat(distCpOptions.getTargetPathExists(), is(true));
}

开发者ID:HotelsDotCom，项目名称:circus-train，代码行数:21，代码来源:DistCpOptionsParserTest.java

示例3: setup

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
/**
 * Implementation of the Mapper::setup() method. This extracts the DistCp-
 * options specified in the Job's configuration, to set up the Job.
 * @param context Mapper's context.
 * @throws IOException On IO failure.
 * @throws InterruptedException If the job is interrupted.
 */
@Override
public void setup(Context context) throws IOException, InterruptedException {
  conf = context.getConfiguration();

  syncFolders = conf.getBoolean(DistCpOptionSwitch.SYNC_FOLDERS.getConfigLabel(), false);
  ignoreFailures = conf.getBoolean(DistCpOptionSwitch.IGNORE_FAILURES.getConfigLabel(), false);
  skipCrc = conf.getBoolean(DistCpOptionSwitch.SKIP_CRC.getConfigLabel(), false);
  overWrite = conf.getBoolean(DistCpOptionSwitch.OVERWRITE.getConfigLabel(), false);
  append = conf.getBoolean(DistCpOptionSwitch.APPEND.getConfigLabel(), false);
  preserve = DistCpUtils.unpackAttributes(conf.get(DistCpOptionSwitch.
      PRESERVE_STATUS.getConfigLabel()));

  targetWorkPath = new Path(conf.get(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH));
  Path targetFinalPath = new Path(conf.get(
          DistCpConstants.CONF_LABEL_TARGET_FINAL_PATH));
  targetFS = targetFinalPath.getFileSystem(conf);

  if (targetFS.exists(targetFinalPath) && targetFS.isFile(targetFinalPath)) {
    overWrite = true; // When target is an existing file, overwrite it.
  }

  if (conf.get(DistCpConstants.CONF_LABEL_SSL_CONF) != null) {
    initializeSSLConf(context);
  }
}

开发者ID:naver，项目名称:hadoop，代码行数:33，代码来源:CopyMapper.java

示例4: getConfiguration

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
private static Configuration getConfiguration() throws IOException {
  Configuration configuration = getConfigurationForCluster();
  final FileSystem fs = cluster.getFileSystem();
  Path workPath = new Path(TARGET_PATH)
          .makeQualified(fs.getUri(), fs.getWorkingDirectory());
  configuration.set(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH,
          workPath.toString());
  configuration.set(DistCpConstants.CONF_LABEL_TARGET_FINAL_PATH,
          workPath.toString());
  configuration.setBoolean(DistCpOptionSwitch.OVERWRITE.getConfigLabel(),
          false);
  configuration.setBoolean(DistCpOptionSwitch.SKIP_CRC.getConfigLabel(),
          false);
  configuration.setBoolean(DistCpOptionSwitch.SYNC_FOLDERS.getConfigLabel(),
          true);
  configuration.set(DistCpOptionSwitch.PRESERVE_STATUS.getConfigLabel(),
          "br");
  return configuration;
}

开发者ID:naver，项目名称:hadoop，代码行数:20，代码来源:TestCopyMapper.java

示例5: testSetCommitDirectory

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
@Test
public void testSetCommitDirectory() {
  try {
    Job job = Job.getInstance(new Configuration());
    Assert.assertEquals(null, CopyOutputFormat.getCommitDirectory(job));

    job.getConfiguration().set(DistCpConstants.CONF_LABEL_TARGET_FINAL_PATH, "");
    Assert.assertEquals(null, CopyOutputFormat.getCommitDirectory(job));

    Path directory = new Path("/tmp/test");
    CopyOutputFormat.setCommitDirectory(job, directory);
    Assert.assertEquals(directory, CopyOutputFormat.getCommitDirectory(job));
    Assert.assertEquals(directory.toString(), job.getConfiguration().
        get(DistCpConstants.CONF_LABEL_TARGET_FINAL_PATH));
  } catch (IOException e) {
    LOG.error("Exception encountered while running test", e);
    Assert.fail("Failed while testing for set Commit Directory");
  }
}

开发者ID:naver，项目名称:hadoop，代码行数:20，代码来源:TestCopyOutputFormat.java

示例6: testSetWorkingDirectory

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
@Test
public void testSetWorkingDirectory() {
  try {
    Job job = Job.getInstance(new Configuration());
    Assert.assertEquals(null, CopyOutputFormat.getWorkingDirectory(job));

    job.getConfiguration().set(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH, "");
    Assert.assertEquals(null, CopyOutputFormat.getWorkingDirectory(job));

    Path directory = new Path("/tmp/test");
    CopyOutputFormat.setWorkingDirectory(job, directory);
    Assert.assertEquals(directory, CopyOutputFormat.getWorkingDirectory(job));
    Assert.assertEquals(directory.toString(), job.getConfiguration().
        get(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH));
  } catch (IOException e) {
    LOG.error("Exception encountered while running test", e);
    Assert.fail("Failed while testing for set Working Directory");
  }
}

开发者ID:naver，项目名称:hadoop，代码行数:20，代码来源:TestCopyOutputFormat.java

示例7: testDynamicInputChunkContext

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
@Test
public void testDynamicInputChunkContext() throws IOException {
  Configuration configuration = new Configuration();
  configuration.set(DistCpConstants.CONF_LABEL_LISTING_FILE_PATH,
      "/tmp/test/file1.seq");
  DynamicInputFormat firstInputFormat = new DynamicInputFormat();
  DynamicInputFormat secondInputFormat = new DynamicInputFormat();
  DynamicInputChunkContext firstContext =
      firstInputFormat.getChunkContext(configuration);
  DynamicInputChunkContext secondContext =
      firstInputFormat.getChunkContext(configuration);
  DynamicInputChunkContext thirdContext =
      secondInputFormat.getChunkContext(configuration);
  DynamicInputChunkContext fourthContext =
      secondInputFormat.getChunkContext(configuration);
  Assert.assertTrue("Chunk contexts from the same DynamicInputFormat " +
      "object should be the same.",firstContext.equals(secondContext));
  Assert.assertTrue("Chunk contexts from the same DynamicInputFormat " +
      "object should be the same.",thirdContext.equals(fourthContext));
  Assert.assertTrue("Contexts from different DynamicInputFormat " +
      "objects should be different.",!firstContext.equals(thirdContext));
}

开发者ID:aliyun-beta，项目名称:aliyun-oss-hadoop-fs，代码行数:23，代码来源:TestDynamicInputFormat.java

示例8: setup

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
/**
 * Implementation of the Mapper::setup() method. This extracts the DistCp-
 * options specified in the Job's configuration, to set up the Job.
 * @param context Mapper's context.
 * @throws IOException On IO failure.
 * @throws InterruptedException If the job is interrupted.
 */
@Override
public void setup(Context context) throws IOException, InterruptedException {
  conf = context.getConfiguration();

  syncFolders = conf.getBoolean(DistCpOptionSwitch.SYNC_FOLDERS.getConfigLabel(), false);
  ignoreFailures = conf.getBoolean(DistCpOptionSwitch.IGNORE_FAILURES.getConfigLabel(), false);
  skipCrc = conf.getBoolean(DistCpOptionSwitch.SKIP_CRC.getConfigLabel(), false);
  overWrite = conf.getBoolean(DistCpOptionSwitch.OVERWRITE.getConfigLabel(), false);
  preserve = DistCpUtils.unpackAttributes(conf.get(DistCpOptionSwitch.
      PRESERVE_STATUS.getConfigLabel()));

  targetWorkPath = new Path(conf.get(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH));
  Path targetFinalPath = new Path(conf.get(
          DistCpConstants.CONF_LABEL_TARGET_FINAL_PATH));
  targetFS = targetFinalPath.getFileSystem(conf);

  if (targetFS.exists(targetFinalPath) && targetFS.isFile(targetFinalPath)) {
    overWrite = true; // When target is an existing file, overwrite it.
  }

  if (conf.get(DistCpConstants.CONF_LABEL_SSL_CONF) != null) {
    initializeSSLConf(context);
  }
}

开发者ID:ict-carch，项目名称:hadoop-plus，代码行数:32，代码来源:CopyMapper.java

示例9: getInputStream

import org.apache.hadoop.tools.DistCpConstants; //导入依赖的package包/类
private static ThrottledInputStream getInputStream(Path path,
    Configuration conf) throws IOException {
  try {
    FileSystem fs = path.getFileSystem(conf);
    long bandwidthMB = conf.getInt(DistCpConstants.CONF_LABEL_BANDWIDTH_MB,
            DistCpConstants.DEFAULT_BANDWIDTH_MB);
    FSDataInputStream in = fs.open(path);
    return new ThrottledInputStream(in, bandwidthMB * 1024 * 1024);
  }
  catch (IOException e) {
    throw new CopyReadException(e);
  }
}

开发者ID:naver，项目名称:hadoop，代码行数:14，代码来源:RetriableFileCopyCommand.java

注：本文中的org.apache.hadoop.tools.DistCpConstants类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。

鲜花

握手

雷人

路过

鸡蛋

该文章已有0人参与评论

请发表评论

全部评论

专题导读

More+

10-27 六六分期app的软件客服如何联系？(六六分期

11-06 可心卡盟:win10系统火狐flash插件崩溃怎么

11-06 亲亲特价:怎么删除回收站图标

11-06 济南大学虚拟社区:鲁大师节能降温的具体办

11-06 xlueops.exe:无线网络安装向导

11-06 女斗合众国:win7系统cf与主机连接不稳定怎

11-06 0xc000022-[cf烟雾头]cf怎么调烟雾头

11-06 qizideyouhuo:应用程序无法正常启动0xc0000

11-06 ipz-185:win7系统vcf文件怎么打开

11-06 傻哥蹦迪:win10系统s4怎么打开usb调试

11-06 八神浩树gtaste:回收站清空了怎么恢复

11-06 妖尾之黑色守护:win10系统电脑没有1440x900

11-06 校园至尊魔王小说:win7系统浏览网页时字体

11-06 女斗合众国:win10系统访问共享文件夹提示请

11-06 tokyo hot n0654:恢复win7系统默认字体一招

11-06 雨酷仙境:设置win7系统转移临时文件夹腾出

11-06 阿穆纳伊之杖:win7系统开始菜单在右边还原

11-06 tunespotting:win10系统火狐flash插件总是

11-06 甘尔葛分析师：计谋网站seo关键词暴涨有什

11-06 蔡贵霖: 计谋网站seo关键词暴涨有什么秘密

11-06 博益网首页:ao3网页版进入不了解决方法

11-06 漏斗子专栏: 网站数据分析小白易懂精华篇

11-06 见证双虹怎么做:win7系统开启telnet命令的

11-06 颾狐蝶蜋:系统资源不足无法完成请求的服务

11-06 国光中学校歌:提交网站到alexa查询详细步骤

11-06 西安有情天:静态网页和动态网页的区别

11-06 红木雅尚斋:外部链接构造对网站的好处

11-06 前官礼遇：防止域名劫持–增强域安全性的10

11-06 密传二转答案: 中文分词算法有哪些

11-06 金泉家园邮编:百度快照劫持的表现及应对方

Java TransformC14N11类代码示例发布时间：2022-05-22

Java DefaultRoutePlanner类代码示例发布时间：2022-05-22

剪的笔顺,诠释剪的笔画,认识剪的部首

1 六六分期app的软件客服如何联系？(六六分期

六六分期app的软件客服如何联系？不知道吗？加qq群【895510560】即可！标题：六六分期

阅读：19240|2023-10-27

2 可心卡盟:win10系统火狐flash插件崩溃怎么

今天小编告诉大家如何处理win10系统火狐flash插件总是崩溃的问题，可能很多用户都不知

阅读：10001|2022-11-06

3 亲亲特价:怎么删除回收站图标

今天小编告诉大家如何对win10系统删除桌面回收站图标进行设置，可能很多用户都不知道

阅读：8331|2022-11-06

4 济南大学虚拟社区:鲁大师节能降温的具体办

今天小编告诉大家如何对win10系统电脑设置节能降温的设置方法，想必大家都遇到过需要

阅读：8701|2022-11-06

5 xlueops.exe:无线网络安装向导

我们在使用xp系统的过程中,经常需要对xp系统无线网络安装向导设置进行设置，可能很多

阅读：8647|2022-11-06

6 女斗合众国:win7系统cf与主机连接不稳定怎

今天小编告诉大家如何处理win7系统玩cf老是与主机连接不稳定的问题，可能很多用户都不

阅读：9672|2022-11-06

7 0xc000022-[cf烟雾头]cf怎么调烟雾头

电脑对日常生活的重要性小编就不多说了，可是一旦碰到win7系统设置cf烟雾头的问题，很

阅读：8632|2022-11-06

8 qizideyouhuo:应用程序无法正常启动0xc0000

我们在日常使用电脑的时候，有的小伙伴们可能在打开应用的时候会遇见提示应用程序无法

阅读：8005|2022-11-06

9 ipz-185:win7系统vcf文件怎么打开

今天小编告诉大家如何对win7系统打开vcf文件进行设置，可能很多用户都不知道怎么对win

阅读：8668|2022-11-06

10 傻哥蹦迪:win10系统s4怎么打开usb调试

今天小编告诉大家如何对win10系统s4开启USB调试模式进行设置，可能很多用户都不知道怎

阅读：7540|2022-11-06

客服电话

电子邮件

Java DistCpConstants类代码示例

示例1: getTmpFile

示例2: assertDefaultValues

示例3: setup

示例4: getConfiguration

示例5: testSetCommitDirectory

示例6: testSetWorkingDirectory

示例7: testDynamicInputChunkContext

示例8: setup

示例9: getInputStream

请发表评论

全部评论

上一篇：

下一篇：

chasinginfinity/ml-from-scratch: Machine

mkyong/spring3-mvc-maven-annotation-hell

床的笔顺,关于床的笔画,体会床的部首

zendesk/android-floating-action-button:

csimmonds/procrank_linux: procrank from

剪的笔顺,诠释剪的笔画,认识剪的部首

六六分期app的软件客服如何联系？(六六分期

florent37/ViewAnimator: A fluent Android

florent37/Shrine-MaterialDesign2: implem

CVE-2020-36276

SimpleSoftwareIO/simple-sms: Send and re

关于我们

产品与服务

解决方案

139-2527-9053