Java ResourceCalculatorPlugin类代码示例

OStack程序员社区-中国程序员成长平台 › 门户 › 编程› Java›Java编程经验

原作者: [db:作者] 来自: [db:来源] 收藏邀请

本文整理汇总了Java中org.apache.hadoop.util.ResourceCalculatorPlugin类的典型用法代码示例。如果您正苦于以下问题：Java ResourceCalculatorPlugin类的具体用法？Java ResourceCalculatorPlugin怎么用？Java ResourceCalculatorPlugin使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

ResourceCalculatorPlugin类属于org.apache.hadoop.util包，在下文中一共展示了ResourceCalculatorPlugin类的13个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testDefaultResourceValues

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
/**
 * Test that verifies default values are configured and reported correctly.
 * 
 * @throws Exception
 */
public void testDefaultResourceValues()
    throws Exception {
  JobConf conf = new JobConf();
  try {
    // Memory values are disabled by default.
    conf.setClass(
        org.apache.hadoop.mapred.TaskTracker.TT_RESOURCE_CALCULATOR_PLUGIN,       
        DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    setUpCluster(conf);
    JobConf jobConf = miniMRCluster.createJobConf();
    jobConf.setClass(
        org.apache.hadoop.mapred.TaskTracker.TT_RESOURCE_CALCULATOR_PLUGIN,
        DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    runSleepJob(jobConf);
    verifyTestResults();
  } finally {
    tearDownCluster();
  }
}

开发者ID:Nextzero，项目名称:hadoop-2.6.0-cdh5.4.3，代码行数:25，代码来源:TestTTResourceReporting.java

示例2: testDefaultResourceValues

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
/**
 * Test that verifies default values are configured and reported correctly.
 * 
 * @throws Exception
 */
@Test(timeout=60000)
public void testDefaultResourceValues()
    throws Exception {
  JobConf conf = new JobConf();
  try {
    // Memory values are disabled by default.
    conf.setClass(
        org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
        DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    setUpCluster(conf);
    JobConf jobConf = miniMRCluster.createJobConf();
    jobConf.setClass(
        org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
        DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    runSleepJob(jobConf);
    verifyTestResults();
  } finally {
    tearDownCluster();
  }
}

开发者ID:rhli，项目名称:hadoop-EAR，代码行数:26，代码来源:TestTTResourceReporting.java

示例3: getMaxActualSlots

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
@Override
int getMaxActualSlots(JobConf conf, int numCpuOnTT, TaskType type) {
  Map<Integer, Map<ResourceType, Integer>> cpuToResourcePartitioning =
    CoronaConf.getUncachedCpuToResourcePartitioning(conf);
  if (numCpuOnTT == ResourceCalculatorPlugin.UNAVAILABLE) {
    numCpuOnTT = 1;
  }
  Map<ResourceType, Integer> resourceTypeToCountMap =
    ClusterNode.getResourceTypeToCountMap(numCpuOnTT,
                                          cpuToResourcePartitioning);
  switch (type) {
  case MAP:
    return resourceTypeToCountMap.get(ResourceType.MAP);
  case REDUCE:
    return resourceTypeToCountMap.get(ResourceType.REDUCE);
  default:
    throw new RuntimeException("getMaxActualSlots: Illegal type " + type);
  }
}

开发者ID:rhli，项目名称:hadoop-EAR，代码行数:20，代码来源:CoronaTaskTracker.java

示例4: testDefaultResourceValues

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
/**
 * Test that verifies default values are configured and reported correctly.
 * 
 * @throws Exception
 */
public void testDefaultResourceValues()
    throws Exception {
  JobConf conf = new JobConf();
  try {
    // Memory values are disabled by default.
    conf.setClass(
        org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
        DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    setUpCluster(conf);
    JobConf jobConf = miniMRCluster.createJobConf();
    jobConf.setClass(
        org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
        DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    runSleepJob(jobConf);
    verifyTestResults();
  } finally {
    tearDownCluster();
  }
}

开发者ID:iVCE，项目名称:RDFS，代码行数:25，代码来源:TestTTResourceReporting.java

示例5: initialize

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
public void initialize(JobConf job, JobID id, 
                       Reporter reporter,
                       boolean useNewApi) throws IOException, 
                                                 ClassNotFoundException,
                                                 InterruptedException {
  jobContext = new JobContext(job, id, reporter);
  taskContext = new TaskAttemptContext(job, taskId, reporter);
  if (getState() == TaskStatus.State.UNASSIGNED) {
    setState(TaskStatus.State.RUNNING);
  }
  if (useNewApi) {
    LOG.debug("using new api for output committer");
    outputFormat =
      ReflectionUtils.newInstance(taskContext.getOutputFormatClass(), job);
    committer = outputFormat.getOutputCommitter(taskContext);
  } else {
    committer = conf.getOutputCommitter();
  }
  Path outputPath = FileOutputFormat.getOutputPath(conf);
  if (outputPath != null) {
    if ((committer instanceof FileOutputCommitter)) {
      FileOutputFormat.setWorkOutputPath(conf, 
        ((FileOutputCommitter)committer).getTempTaskOutputPath(taskContext));
    } else {
      FileOutputFormat.setWorkOutputPath(conf, outputPath);
    }
  }
  committer.setupTask(taskContext);
  Class<? extends ResourceCalculatorPlugin> clazz = conf.getClass(
      TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
          null, ResourceCalculatorPlugin.class);
  resourceCalculator = ResourceCalculatorPlugin
          .getResourceCalculatorPlugin(clazz, conf);
  LOG.info(" Using ResourceCalculatorPlugin : " + resourceCalculator);
  if (resourceCalculator != null) {
    initCpuCumulativeTime =
      resourceCalculator.getProcResourceValues().getCumulativeCpuTime();
  }
}

开发者ID:iVCE，项目名称:RDFS，代码行数:40，代码来源:Task.java

示例6: initialize

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
public void initialize(JobConf job, JobID id, 
                       Reporter reporter,
                       boolean useNewApi) throws IOException, 
                                                 ClassNotFoundException,
                                                 InterruptedException {
  jobContext = new JobContextImpl(job, id, reporter);
  taskContext = new TaskAttemptContextImpl(job, taskId, reporter);
  if (getState() == TaskStatus.State.UNASSIGNED) {
    setState(TaskStatus.State.RUNNING);
  }
  if (useNewApi) {
    if (LOG.isDebugEnabled()) {
      LOG.debug("using new api for output committer");
    }
    outputFormat =
      ReflectionUtils.newInstance(taskContext.getOutputFormatClass(), job);
    committer = outputFormat.getOutputCommitter(taskContext);
  } else {
    committer = conf.getOutputCommitter();
  }
  Path outputPath = FileOutputFormat.getOutputPath(conf);
  if (outputPath != null) {
    if ((committer instanceof FileOutputCommitter)) {
      FileOutputFormat.setWorkOutputPath(conf, 
        ((FileOutputCommitter)committer).getTempTaskOutputPath(taskContext));
    } else {
      FileOutputFormat.setWorkOutputPath(conf, outputPath);
    }
  }
  committer.setupTask(taskContext);
  Class<? extends ResourceCalculatorPlugin> clazz =
      conf.getClass(TaskTracker.TT_RESOURCE_CALCULATOR_PLUGIN,
          null, ResourceCalculatorPlugin.class);
  resourceCalculator = ResourceCalculatorPlugin
          .getResourceCalculatorPlugin(clazz, conf);
  LOG.info(" Using ResourceCalculatorPlugin : " + resourceCalculator);
  if (resourceCalculator != null) {
    initCpuCumulativeTime =
      resourceCalculator.getProcResourceValues().getCumulativeCpuTime();
  }
}

开发者ID:Nextzero，项目名称:hadoop-2.6.0-cdh5.4.3，代码行数:42，代码来源:Task.java

示例7: testConfiguredResourceValues

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
/**
 * Test that verifies that configured values are reported correctly.
 * 
 * @throws Exception
 */
public void testConfiguredResourceValues()
    throws Exception {
  JobConf conf = new JobConf();
  conf.setLong("totalVmemOnTT", 4 * 1024 * 1024 * 1024L);
  conf.setLong("totalPmemOnTT", 2 * 1024 * 1024 * 1024L);
  conf.setLong("mapSlotMemorySize", 1 * 512L);
  conf.setLong("reduceSlotMemorySize", 1 * 1024L);
  conf.setLong("availableVmemOnTT", 4 * 1024 * 1024 * 1024L);
  conf.setLong("availablePmemOnTT", 2 * 1024 * 1024 * 1024L);
  conf.setLong("cumulativeCpuTime", 10000L);
  conf.setLong("cpuFrequency", 2000000L);
  conf.setInt("numProcessors", 8);
  conf.setFloat("cpuUsage", 15.5F);
  conf.setLong("procCumulativeCpuTime", 1000L);
  conf.setLong("procVirtualMemorySize", 2 * 1024 * 1024 * 1024L);
  conf.setLong("procPhysicalMemorySize", 1024 * 1024 * 1024L);

  conf.setClass(
      org.apache.hadoop.mapred.TaskTracker.TT_RESOURCE_CALCULATOR_PLUGIN,       
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
  conf.setLong(DummyResourceCalculatorPlugin.MAXVMEM_TESTING_PROPERTY,
      4 * 1024 * 1024 * 1024L);
  conf.setLong(DummyResourceCalculatorPlugin.MAXPMEM_TESTING_PROPERTY,
      2 * 1024 * 1024 * 1024L);
  conf.setLong(JobTracker.MAPRED_CLUSTER_MAP_MEMORY_MB_PROPERTY, 512L);
  conf.setLong(JobTracker.MAPRED_CLUSTER_REDUCE_MEMORY_MB_PROPERTY, 1024L);
  conf.setLong(DummyResourceCalculatorPlugin.CUMULATIVE_CPU_TIME, 10000L);
  conf.setLong(DummyResourceCalculatorPlugin.CPU_FREQUENCY, 2000000L);
  conf.setInt(DummyResourceCalculatorPlugin.NUM_PROCESSORS, 8);
  conf.setFloat(DummyResourceCalculatorPlugin.CPU_USAGE, 15.5F);
  try {
    setUpCluster(conf);
    JobConf jobConf = miniMRCluster.createJobConf();
    jobConf.setMemoryForMapTask(1 * 1024L);
    jobConf.setMemoryForReduceTask(2 * 1024L);
    jobConf.setClass(
      org.apache.hadoop.mapred.TaskTracker.TT_RESOURCE_CALCULATOR_PLUGIN,
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_CUMULATIVE_CPU_TIME, 1000L);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_VMEM_TESTING_PROPERTY,
                    2 * 1024 * 1024 * 1024L);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_PMEM_TESTING_PROPERTY,
                    1024 * 1024 * 1024L);
    runSleepJob(jobConf);
    verifyTestResults();
  } finally {
    tearDownCluster();
  }
}

开发者ID:Nextzero，项目名称:hadoop-2.6.0-cdh5.4.3，代码行数:55，代码来源:TestTTResourceReporting.java

示例8: initialize

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
public void initialize(JobConf job, JobID id, 
                       Reporter reporter,
                       boolean useNewApi) throws IOException, 
                                                 ClassNotFoundException,
                                                 InterruptedException {
  jobContext = new JobContext(job, id, reporter);
  taskContext = new TaskAttemptContext(job, taskId, reporter);
  if (getState() == TaskStatus.State.UNASSIGNED) {
    setState(TaskStatus.State.RUNNING);
  }
  if (useNewApi) {
    LOG.debug("using new api for output committer");
    outputFormat =
      ReflectionUtils.newInstance(taskContext.getOutputFormatClass(), job);
    committer = outputFormat.getOutputCommitter(taskContext);
  } else {
    committer = conf.getOutputCommitter();
  }
  Path outputPath = FileOutputFormat.getOutputPath(conf);
  if (outputPath != null) {
    if ((committer instanceof FileOutputCommitter)) {
      FileOutputFormat.setWorkOutputPath(conf, 
        ((FileOutputCommitter)committer).getTempTaskOutputPath(taskContext));
    } else {
      FileOutputFormat.setWorkOutputPath(conf, outputPath);
    }
  }
  committer.setupTask(taskContext);
  Class<? extends ResourceCalculatorPlugin> clazz = conf.getClass(
      TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
          null, ResourceCalculatorPlugin.class);
  resourceCalculator = ResourceCalculatorPlugin
          .getResourceCalculatorPlugin(clazz, conf);
  LOG.info(" Using ResourceCalculatorPlugin : " + resourceCalculator);
  if (resourceCalculator != null) {
    initCpuCumulativeTime =
      resourceCalculator.getProcResourceValues().getCumulativeCpuTime();
  }
  
  jmxThreadInfoTracker = new JMXThreadBasedMetrics();
  jmxThreadInfoTracker.registerThreadToTask(
      "MAIN_TASK", Thread.currentThread().getId());
  this.initJvmCpuCumulativeTime = 
      jmxThreadInfoTracker.getCumulativeCPUTime();
  
  cgResourceTracker = new CGroupResourceTracker (
                        job, CGroupResourceTracker.RESOURCE_TRAKCER_TYPE.TASK, 
                        taskId.toString(), resourceCalculator);
}

开发者ID:rhli，项目名称:hadoop-EAR，代码行数:50，代码来源:Task.java

示例9: CGroupResourceTracker

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
public CGroupResourceTracker(
         JobConf conf, RESOURCE_TRAKCER_TYPE trackerType, 
         String target,
         ResourceCalculatorPlugin plugin) {
  this.resourceCalculaotr = plugin;
  
  boolean taskMemoryControlGroupEnabled = conf.getBoolean(
      TaskTracker.MAPRED_TASKTRACKER_CGROUP_MEM_ENABLE_PROPERTY,
      TaskTracker.DEFAULT_MAPRED_TASKTRACKER_CGROUP_MEM_ENABLE_PROPERTY);
  
  if (taskMemoryControlGroupEnabled) {
    if (MemoryControlGroup.isAvailable()) {
      switch (trackerType) {
        case JOB_TRACKER:
          String jtRootpath = conf.get(
              TaskTrackerMemoryControlGroup.CGROUP_MEM_JT_ROOT, 
              TaskTrackerMemoryControlGroup.DEFAULT_JT_ROOT);
          memControlGroup= new MemoryControlGroup(jtRootpath);
          
          if (isMemTrackerAvailable()) {
            LOG.info("A CGroupResourceTracker for JOB_TRACKER created.");
          }
          break;
        case TASK_TRACKER:
          String ttRootpath = conf.get(
              TaskTrackerMemoryControlGroup.CGROUP_MEM_TT_ROOT, 
              TaskTrackerMemoryControlGroup.DEFAULT_TT_ROOT);
            memControlGroup = new MemoryControlGroup(ttRootpath);
            
          if (isMemTrackerAvailable()) {
            LOG.info("A CGroupResourceTracker for TASK_TRACKER created.");
          }
          break;
        case TASK:
          String rootpath = conf.get(
            TaskTrackerMemoryControlGroup.CGROUP_MEM_ROOT_PROPERTY, 
            TaskTrackerMemoryControlGroup.DEFAULT_CGROUP_MEM_ROOT);
          MemoryControlGroup container = new MemoryControlGroup(rootpath);
          memControlGroup = 
            container.getSubGroup(target);
          
          if (isMemTrackerAvailable()) {
            LOG.info("A CGroupResourceTracker for TASK:" + 
              target + " created.");
          }
          break;
      }
    }
  }
}

开发者ID:rhli，项目名称:hadoop-EAR，代码行数:51，代码来源:CGroupResourceTracker.java

示例10: TestCpuToMapTasksConfig

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
@Test
public void TestCpuToMapTasksConfig() throws Exception {
  JobConf conf = new JobConf();
  conf.set("mapred.tasktracker.map.tasks.maximum", "3");
  conf.set("mapred.tasktracker.reduce.tasks.maximum", "1");
  // Test with the original settings
  try {
    miniMRCluster =
        new MiniMRCluster(1, "file:///", 3, null, null, conf);
    taskTracker = miniMRCluster.getTaskTrackerRunner(0).getTaskTracker();
    Assert.assertEquals(3, taskTracker.getMaxCurrentMapTasks());
    Assert.assertEquals(1, taskTracker.getMaxCurrentReduceTasks());
  } finally {
    if (miniMRCluster != null) {
      miniMRCluster.shutdown();
    }
  }

  // Test with the # CPU -> mappers settings
  conf.setClass(org.apache.hadoop.mapred.TaskTracker.
      MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
  conf.set("mapred.tasktracker.cpus.to.maptasks", "4:6, 8:9, 16:15");
  conf.set("mapred.tasktracker.cpus.to.reducetasks", "4:3, 8:7, 16:12");

  // 4 CPU -> 6 mappers, 3 reducers
  try {
    conf.set(DummyResourceCalculatorPlugin.NUM_PROCESSORS, "4");
    miniMRCluster =
        new MiniMRCluster(1, "file:///", 3, null, null, conf);
    taskTracker = miniMRCluster.getTaskTrackerRunner(0).getTaskTracker();
    Assert.assertEquals(6, taskTracker.getMaxCurrentMapTasks());
    Assert.assertEquals(3, taskTracker.getMaxCurrentReduceTasks());
  } finally {
    if (miniMRCluster != null) {
      miniMRCluster.shutdown();
    }
  }

  // 8 CPU -> 9 mappers, 7 reduces
  try {
    conf.set(DummyResourceCalculatorPlugin.NUM_PROCESSORS, "8");
    miniMRCluster =
        new MiniMRCluster(1, "file:///", 3, null, null, conf);
    taskTracker = miniMRCluster.getTaskTrackerRunner(0).getTaskTracker();
    Assert.assertEquals(9, taskTracker.getMaxCurrentMapTasks());
    Assert.assertEquals(7, taskTracker.getMaxCurrentReduceTasks());
  } finally {
    if (miniMRCluster != null) {
      miniMRCluster.shutdown();
    }
  }

  // 16 CPU -> 15 mappers, 12 reduces
  try {
    conf.set(DummyResourceCalculatorPlugin.NUM_PROCESSORS, "16");
    miniMRCluster =
        new MiniMRCluster(1, "file:///", 3, null, null, conf);
    taskTracker = miniMRCluster.getTaskTrackerRunner(0).getTaskTracker();
    Assert.assertEquals(15, taskTracker.getMaxCurrentMapTasks());
    Assert.assertEquals(12, taskTracker.getMaxCurrentReduceTasks());
  } finally {
    if (miniMRCluster != null) {
      miniMRCluster.shutdown();
    }
  }

  // 11 CPU -> 3 mappers, 1 reduce (back to default)
  try {
    conf.set(DummyResourceCalculatorPlugin.NUM_PROCESSORS, "11");
    miniMRCluster =
        new MiniMRCluster(1, "file:///", 3, null, null, conf);
    taskTracker = miniMRCluster.getTaskTrackerRunner(0).getTaskTracker();
    Assert.assertEquals(3, taskTracker.getMaxCurrentMapTasks());
    Assert.assertEquals(1, taskTracker.getMaxCurrentReduceTasks());
  } finally {
    if (miniMRCluster != null) {
      miniMRCluster.shutdown();
    }
  }
}

开发者ID:rhli，项目名称:hadoop-EAR，代码行数:82，代码来源:TestTTCpuToTaskSlots.java

示例11: testConfiguredResourceValues

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
/**
 * Test that verifies that configured values are reported correctly.
 * 
 * @throws Exception
 */
@Test(timeout=60000)
public void testConfiguredResourceValues()
    throws Exception {
  JobConf conf = new JobConf();
  conf.setLong("totalVmemOnTT", 4 * 1024 * 1024 * 1024L);
  conf.setLong("totalPmemOnTT", 2 * 1024 * 1024 * 1024L);
  conf.setLong("mapSlotMemorySize", 1 * 512L);
  conf.setLong("reduceSlotMemorySize", 1 * 1024L);
  conf.setLong("availableVmemOnTT", 4 * 1024 * 1024 * 1024L);
  conf.setLong("availablePmemOnTT", 2 * 1024 * 1024 * 1024L);
  conf.setLong("cumulativeCpuTime", 10000L);
  conf.setLong("cpuFrequency", 2000000L);
  conf.setInt("numProcessors", 8);
  conf.setFloat("cpuUsage", 15.5F);
  conf.setLong("procCumulativeCpuTime", 1000L);
  conf.setLong("procVirtualMemorySize", 1 * 1024 * 1024 * 1024L);
  conf.setLong("procPhysicalMemorySize", 1024 * 1024 * 1024L);

  conf.setClass(
      org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
  conf.setLong(DummyResourceCalculatorPlugin.MAXVMEM_TESTING_PROPERTY,
      4 * 1024 * 1024 * 1024L);
  conf.setLong(DummyResourceCalculatorPlugin.MAXPMEM_TESTING_PROPERTY,
      2 * 1024 * 1024 * 1024L);
  conf.setLong(JobTracker.MAPRED_CLUSTER_MAP_MEMORY_MB_PROPERTY, 512L);
  conf.setLong(
      JobTracker.MAPRED_CLUSTER_REDUCE_MEMORY_MB_PROPERTY, 1024L);
  conf.setLong(DummyResourceCalculatorPlugin.CUMULATIVE_CPU_TIME, 10000L);
  conf.setLong(DummyResourceCalculatorPlugin.CPU_FREQUENCY, 2000000L);
  conf.setInt(DummyResourceCalculatorPlugin.NUM_PROCESSORS, 8);
  conf.setFloat(DummyResourceCalculatorPlugin.CPU_USAGE, 15.5F);
  try {
    setUpCluster(conf);
    JobConf jobConf = miniMRCluster.createJobConf();
    jobConf.setMemoryForMapTask(4 * 1024L);
    jobConf.setMemoryForReduceTask(4 * 1024L);
    jobConf.setClass(
      org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_CUMULATIVE_CPU_TIME, 1000L);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_VMEM_TESTING_PROPERTY,
                    1 * 1024 * 1024 * 1024L);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_PMEM_TESTING_PROPERTY,
                    1024 * 1024 * 1024L);
    runSleepJob(jobConf);
    verifyTestResults();
  } finally {
    tearDownCluster();
  }
}

开发者ID:rhli，项目名称:hadoop-EAR，代码行数:57，代码来源:TestTTResourceReporting.java

示例12: initialize

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
public void initialize(JobConf job, JobID id, 
                       Reporter reporter,
                       boolean useNewApi) throws IOException, 
                                                 ClassNotFoundException,
                                                 InterruptedException {
  jobContext = new JobContext(job, id, reporter);
  taskContext = new TaskAttemptContext(job, taskId, reporter);
  if (getState() == TaskStatus.State.UNASSIGNED) {
    setState(TaskStatus.State.RUNNING);
  }
  if (useNewApi) {
    if (LOG.isDebugEnabled()) {
      LOG.debug("using new api for output committer");
    }
    outputFormat =
      ReflectionUtils.newInstance(taskContext.getOutputFormatClass(), job);
    committer = outputFormat.getOutputCommitter(taskContext);
  } else {
    committer = conf.getOutputCommitter();
  }
  Path outputPath = FileOutputFormat.getOutputPath(conf);
  if (outputPath != null) {
    if ((committer instanceof FileOutputCommitter)) {
      FileOutputFormat.setWorkOutputPath(conf, 
        ((FileOutputCommitter)committer).getTempTaskOutputPath(taskContext));
    } else {
      FileOutputFormat.setWorkOutputPath(conf, outputPath);
    }
  }
  committer.setupTask(taskContext);
  Class<? extends ResourceCalculatorPlugin> clazz =
      conf.getClass(TaskTracker.TT_RESOURCE_CALCULATOR_PLUGIN,
          null, ResourceCalculatorPlugin.class);
  resourceCalculator = ResourceCalculatorPlugin
          .getResourceCalculatorPlugin(clazz, conf);
  LOG.info(" Using ResourceCalculatorPlugin : " + resourceCalculator);
  if (resourceCalculator != null) {
    initCpuCumulativeTime =
      resourceCalculator.getProcResourceValues().getCumulativeCpuTime();
  }
}

开发者ID:Seagate，项目名称:hadoop-on-lustre，代码行数:42，代码来源:Task.java

示例13: testConfiguredResourceValues

import org.apache.hadoop.util.ResourceCalculatorPlugin; //导入依赖的package包/类
/**
 * Test that verifies that configured values are reported correctly.
 * 
 * @throws Exception
 */
public void testConfiguredResourceValues()
    throws Exception {
  JobConf conf = new JobConf();
  conf.setLong("totalVmemOnTT", 4 * 1024 * 1024 * 1024L);
  conf.setLong("totalPmemOnTT", 2 * 1024 * 1024 * 1024L);
  conf.setLong("mapSlotMemorySize", 1 * 512L);
  conf.setLong("reduceSlotMemorySize", 1 * 1024L);
  conf.setLong("availableVmemOnTT", 4 * 1024 * 1024 * 1024L);
  conf.setLong("availablePmemOnTT", 2 * 1024 * 1024 * 1024L);
  conf.setLong("cumulativeCpuTime", 10000L);
  conf.setLong("cpuFrequency", 2000000L);
  conf.setInt("numProcessors", 8);
  conf.setFloat("cpuUsage", 15.5F);
  conf.setLong("procCumulativeCpuTime", 1000L);
  conf.setLong("procVirtualMemorySize", 2 * 1024 * 1024 * 1024L);
  conf.setLong("procPhysicalMemorySize", 1024 * 1024 * 1024L);

  conf.setClass(
      org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
  conf.setLong(DummyResourceCalculatorPlugin.MAXVMEM_TESTING_PROPERTY,
      4 * 1024 * 1024 * 1024L);
  conf.setLong(DummyResourceCalculatorPlugin.MAXPMEM_TESTING_PROPERTY,
      2 * 1024 * 1024 * 1024L);
  conf.setLong(JobTracker.MAPRED_CLUSTER_MAP_MEMORY_MB_PROPERTY, 512L);
  conf.setLong(
      JobTracker.MAPRED_CLUSTER_REDUCE_MEMORY_MB_PROPERTY, 1024L);
  conf.setLong(DummyResourceCalculatorPlugin.CUMULATIVE_CPU_TIME, 10000L);
  conf.setLong(DummyResourceCalculatorPlugin.CPU_FREQUENCY, 2000000L);
  conf.setInt(DummyResourceCalculatorPlugin.NUM_PROCESSORS, 8);
  conf.setFloat(DummyResourceCalculatorPlugin.CPU_USAGE, 15.5F);
  try {
    setUpCluster(conf);
    JobConf jobConf = miniMRCluster.createJobConf();
    jobConf.setMemoryForMapTask(1 * 1024L);
    jobConf.setMemoryForReduceTask(2 * 1024L);
    jobConf.setClass(
      org.apache.hadoop.mapred.TaskTracker.MAPRED_TASKTRACKER_MEMORY_CALCULATOR_PLUGIN_PROPERTY,
      DummyResourceCalculatorPlugin.class, ResourceCalculatorPlugin.class);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_CUMULATIVE_CPU_TIME, 1000L);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_VMEM_TESTING_PROPERTY,
                    2 * 1024 * 1024 * 1024L);
    jobConf.setLong(DummyResourceCalculatorPlugin.PROC_PMEM_TESTING_PROPERTY,
                    1024 * 1024 * 1024L);
    runSleepJob(jobConf);
    verifyTestResults();
  } finally {
    tearDownCluster();
  }
}

开发者ID:iVCE，项目名称:RDFS，代码行数:56，代码来源:TestTTResourceReporting.java

注：本文中的org.apache.hadoop.util.ResourceCalculatorPlugin类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。

鲜花

握手

雷人

路过

鸡蛋

该文章已有0人参与评论

请发表评论

全部评论

专题导读

More+

10-27 六六分期app的软件客服如何联系？(六六分期

11-06 可心卡盟:win10系统火狐flash插件崩溃怎么

11-06 亲亲特价:怎么删除回收站图标

11-06 济南大学虚拟社区:鲁大师节能降温的具体办

11-06 xlueops.exe:无线网络安装向导

11-06 女斗合众国:win7系统cf与主机连接不稳定怎

11-06 0xc000022-[cf烟雾头]cf怎么调烟雾头

11-06 qizideyouhuo:应用程序无法正常启动0xc0000

11-06 ipz-185:win7系统vcf文件怎么打开

11-06 傻哥蹦迪:win10系统s4怎么打开usb调试

11-06 八神浩树gtaste:回收站清空了怎么恢复

11-06 妖尾之黑色守护:win10系统电脑没有1440x900

11-06 校园至尊魔王小说:win7系统浏览网页时字体

11-06 女斗合众国:win10系统访问共享文件夹提示请

11-06 tokyo hot n0654:恢复win7系统默认字体一招

11-06 雨酷仙境:设置win7系统转移临时文件夹腾出

11-06 阿穆纳伊之杖:win7系统开始菜单在右边还原

11-06 tunespotting:win10系统火狐flash插件总是

11-06 甘尔葛分析师：计谋网站seo关键词暴涨有什

11-06 蔡贵霖: 计谋网站seo关键词暴涨有什么秘密

11-06 博益网首页:ao3网页版进入不了解决方法

11-06 漏斗子专栏: 网站数据分析小白易懂精华篇

11-06 见证双虹怎么做:win7系统开启telnet命令的

11-06 颾狐蝶蜋:系统资源不足无法完成请求的服务

11-06 国光中学校歌:提交网站到alexa查询详细步骤

11-06 西安有情天:静态网页和动态网页的区别

11-06 红木雅尚斋:外部链接构造对网站的好处

11-06 前官礼遇：防止域名劫持–增强域安全性的10

11-06 密传二转答案: 中文分词算法有哪些

11-06 金泉家园邮编:百度快照劫持的表现及应对方

Java Fade类代码示例发布时间：2022-05-22

Java Converters类代码示例发布时间：2022-05-22

剪的笔顺,诠释剪的笔画,认识剪的部首

1 六六分期app的软件客服如何联系？(六六分期

六六分期app的软件客服如何联系？不知道吗？加qq群【895510560】即可！标题：六六分期

阅读：19168|2023-10-27

2 可心卡盟:win10系统火狐flash插件崩溃怎么

今天小编告诉大家如何处理win10系统火狐flash插件总是崩溃的问题，可能很多用户都不知

阅读：9982|2022-11-06

3 亲亲特价:怎么删除回收站图标

今天小编告诉大家如何对win10系统删除桌面回收站图标进行设置，可能很多用户都不知道

阅读：8322|2022-11-06

4 济南大学虚拟社区:鲁大师节能降温的具体办

今天小编告诉大家如何对win10系统电脑设置节能降温的设置方法，想必大家都遇到过需要

阅读：8691|2022-11-06

5 xlueops.exe:无线网络安装向导

我们在使用xp系统的过程中,经常需要对xp系统无线网络安装向导设置进行设置，可能很多

阅读：8635|2022-11-06

6 女斗合众国:win7系统cf与主机连接不稳定怎

今天小编告诉大家如何处理win7系统玩cf老是与主机连接不稳定的问题，可能很多用户都不

阅读：9651|2022-11-06

7 0xc000022-[cf烟雾头]cf怎么调烟雾头

电脑对日常生活的重要性小编就不多说了，可是一旦碰到win7系统设置cf烟雾头的问题，很

阅读：8618|2022-11-06

8 qizideyouhuo:应用程序无法正常启动0xc0000

我们在日常使用电脑的时候，有的小伙伴们可能在打开应用的时候会遇见提示应用程序无法

阅读：7995|2022-11-06

9 ipz-185:win7系统vcf文件怎么打开

今天小编告诉大家如何对win7系统打开vcf文件进行设置，可能很多用户都不知道怎么对win

阅读：8649|2022-11-06

10 傻哥蹦迪:win10系统s4怎么打开usb调试

今天小编告诉大家如何对win10系统s4开启USB调试模式进行设置，可能很多用户都不知道怎

阅读：7532|2022-11-06

客服电话

电子邮件

Java ResourceCalculatorPlugin类代码示例

示例1: testDefaultResourceValues

示例2: testDefaultResourceValues

示例3: getMaxActualSlots

示例4: testDefaultResourceValues

示例5: initialize

示例6: initialize

示例7: testConfiguredResourceValues

示例8: initialize

示例9: CGroupResourceTracker

示例10: TestCpuToMapTasksConfig

示例11: testConfiguredResourceValues

示例12: initialize

示例13: testConfiguredResourceValues

请发表评论

全部评论

上一篇：

下一篇：

CVE-2022-33173

bradtraversy/iweather: Ionic 3 mobile we

Ramotion/android-ui-animation-components

一个小时快速搭建微信小程序教程

joaomh/curso-de-matlab

剪的笔顺,诠释剪的笔画,认识剪的部首

六六分期app的软件客服如何联系？(六六分期

florent37/ViewAnimator: A fluent Android

florent37/Shrine-MaterialDesign2: implem

CVE-2020-36276

SimpleSoftwareIO/simple-sms: Send and re

关于我们

产品与服务

解决方案

139-2527-9053