• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java SchedulerApplicationAttempt类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt的典型用法代码示例。如果您正苦于以下问题:Java SchedulerApplicationAttempt类的具体用法?Java SchedulerApplicationAttempt怎么用?Java SchedulerApplicationAttempt使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



SchedulerApplicationAttempt类属于org.apache.hadoop.yarn.server.resourcemanager.scheduler包,在下文中一共展示了SchedulerApplicationAttempt类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: unreserveResource

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public synchronized void unreserveResource(
    SchedulerApplicationAttempt application) {
  // Cannot unreserve for wrong application...
  ApplicationAttemptId reservedApplication = 
      getReservedContainer().getContainer().getId().getApplicationAttemptId(); 
  if (!reservedApplication.equals(
      application.getApplicationAttemptId())) {
    throw new IllegalStateException("Trying to unreserve " +  
        " for application " + application.getApplicationId() + 
        " when currently reserved " + 
        " for application " + reservedApplication.getApplicationId() + 
        " on node " + this);
  }
  
  setReservedContainer(null);
  this.reservedAppSchedulable = null;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:19,代码来源:FSSchedulerNode.java


示例2: unreserveResource

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public synchronized void unreserveResource(
    SchedulerApplicationAttempt application) {

  // adding NP checks as this can now be called for preemption
  if (getReservedContainer() != null
      && getReservedContainer().getContainer() != null
      && getReservedContainer().getContainer().getId() != null
      && getReservedContainer().getContainer().getId()
        .getApplicationAttemptId() != null) {

    // Cannot unreserve for wrong application...
    ApplicationAttemptId reservedApplication =
        getReservedContainer().getContainer().getId()
          .getApplicationAttemptId();
    if (!reservedApplication.equals(
        application.getApplicationAttemptId())) {
      throw new IllegalStateException("Trying to unreserve " +
          " for application " + application.getApplicationAttemptId() +
          " when currently reserved " +
          " for application " + reservedApplication.getApplicationId() +
          " on node " + this);
    }
  }
  setReservedContainer(null);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:27,代码来源:FiCaSchedulerNode.java


示例3: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), node.getLabels());
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:ParentQueue.java


示例4: allocateResource

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
synchronized void allocateResource(Resource clusterResource,
    SchedulerApplicationAttempt application, Resource resource,
    Set<String> nodeLabels) {
  super.allocateResource(clusterResource, resource, nodeLabels);
  
  // Update user metrics
  String userName = application.getUser();
  User user = getUser(userName);
  user.assignContainer(resource, nodeLabels);
  // Note this is a bit unconventional since it gets the object and modifies
  // it here, rather then using set routine
  Resources.subtractFrom(application.getHeadroom(), resource); // headroom
  metrics.setAvailableResourcesToUser(userName, application.getHeadroom());
  
  if (LOG.isDebugEnabled()) {
    LOG.info(getQueueName() + 
        " user=" + userName + 
        " used=" + queueUsage.getUsed() + " numContainers=" + numContainers +
        " headroom = " + application.getHeadroom() +
        " user-resources=" + user.getUsed()
        );
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:24,代码来源:LeafQueue.java


示例5: testAddAndRemoveAppFromCapacityScheduler

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Test
public void testAddAndRemoveAppFromCapacityScheduler() throws Exception {
  CapacitySchedulerConfiguration conf = new CapacitySchedulerConfiguration();
  setupQueueConfiguration(conf);
  conf.setClass(YarnConfiguration.RM_SCHEDULER, CapacityScheduler.class,
    ResourceScheduler.class);
  MockRM rm = new MockRM(conf);
  @SuppressWarnings("unchecked")
  AbstractYarnScheduler<SchedulerApplicationAttempt, SchedulerNode> cs =
      (AbstractYarnScheduler<SchedulerApplicationAttempt, SchedulerNode>) rm
        .getResourceScheduler();

  SchedulerApplication<SchedulerApplicationAttempt> app =
      TestSchedulerUtils.verifyAppAddedAndRemovedFromScheduler(
        cs.getSchedulerApplications(), cs, "a1");
  Assert.assertEquals("a1", app.getQueue().getQueueName());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:TestCapacityScheduler.java


示例6: checkCSLeafQueue

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
private void checkCSLeafQueue(MockRM rm,
    SchedulerApplication<SchedulerApplicationAttempt> app,
    Resource clusterResource, Resource queueResource, Resource usedResource,
    int numContainers) {
  LeafQueue leafQueue = (LeafQueue) app.getQueue();
  // assert queue used resources.
  assertEquals(usedResource, leafQueue.getUsedResources());
  assertEquals(numContainers, leafQueue.getNumContainers());

  ResourceCalculator calc =
      ((CapacityScheduler) rm.getResourceScheduler()).getResourceCalculator();
  float usedCapacity =
      Resources.divide(calc, clusterResource, usedResource, queueResource);
  // assert queue used capacity
  assertEquals(usedCapacity, leafQueue.getUsedCapacity(), 1e-8);
  float absoluteUsedCapacity =
      Resources.divide(calc, clusterResource, usedResource, clusterResource);
  // assert queue absolute capacity
  assertEquals(absoluteUsedCapacity, leafQueue.getAbsoluteUsedCapacity(),
    1e-8);
  // assert user consumed resources.
  assertEquals(usedResource, leafQueue.getUser(app.getUser())
    .getUsed());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:25,代码来源:TestWorkPreservingRMRestart.java


示例7: waitForNumContainersToRecover

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
public static void waitForNumContainersToRecover(int num, MockRM rm,
    ApplicationAttemptId attemptId) throws Exception {
  AbstractYarnScheduler scheduler =
      (AbstractYarnScheduler) rm.getResourceScheduler();
  SchedulerApplicationAttempt attempt =
      scheduler.getApplicationAttempt(attemptId);
  while (attempt == null) {
    System.out.println("Wait for scheduler attempt " + attemptId
        + " to be created");
    Thread.sleep(200);
    attempt = scheduler.getApplicationAttempt(attemptId);
  }
  while (attempt.getLiveContainers().size() < num) {
    System.out.println("Wait for " + num
        + " containers to recover. currently: "
        + attempt.getLiveContainers().size());
    Thread.sleep(200);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:20,代码来源:TestWorkPreservingRMRestart.java


示例8: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), node.getPartition(), false);
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:18,代码来源:ParentQueue.java


示例9: waitForNumContainersToRecover

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@SuppressWarnings("rawtypes")
public static void waitForNumContainersToRecover(int num, MockRM rm,
    ApplicationAttemptId attemptId) throws Exception {
  AbstractYarnScheduler scheduler =
      (AbstractYarnScheduler) rm.getResourceScheduler();
  SchedulerApplicationAttempt attempt =
      scheduler.getApplicationAttempt(attemptId);
  while (attempt == null) {
    System.out.println("Wait for scheduler attempt " + attemptId
        + " to be created");
    Thread.sleep(200);
    attempt = scheduler.getApplicationAttempt(attemptId);
  }
  while (attempt.getLiveContainers().size() < num) {
    System.out.println("Wait for " + num
        + " containers to recover. currently: "
        + attempt.getLiveContainers().size());
    Thread.sleep(200);
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:21,代码来源:TestWorkPreservingRMRestartForNodeLabel.java


示例10: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), node.getLabels(),false);
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:18,代码来源:ParentQueue.java


示例11: allocateResource

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
synchronized void allocateResource(Resource clusterResource,
    SchedulerApplicationAttempt application, Resource resource,
    Set<String> nodeLabels,boolean isResume) {
  super.allocateResource(clusterResource, resource, nodeLabels,isResume);
  
  // Update user metrics
  String userName = application.getUser();
  User user = getUser(userName);
  user.assignContainer(resource, nodeLabels);
  // Note this is a bit unconventional since it gets the object and modifies
  // it here, rather then using set routine
  if(!isResume){
  Resources.subtractFrom(application.getHeadroom(), resource); // headroom
  metrics.setAvailableResourcesToUser(userName, application.getHeadroom());
  }
  //if (LOG.isDebugEnabled()) {
    LOG.info(getQueueName() + 
        " used=" + queueUsage.getUsed() + " numContainers=" + numContainers +
        " headroom = " + application.getHeadroom() +
        " user-resources=" + user.getUsed()+"allocate resource:"+resource+
        " absUsed= "+getAbsoluteUsedCapacity()
        );
//  }
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:25,代码来源:LeafQueue.java


示例12: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), labelManager.getLabelsOnNode(rmContainer
        .getContainer().getNodeId()));
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:17,代码来源:ParentQueue.java


示例13: allocateResource

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
synchronized void allocateResource(Resource clusterResource,
    SchedulerApplicationAttempt application, Resource resource,
    Set<String> nodeLabels) {
  super.allocateResource(clusterResource, resource, nodeLabels);
  
  // Update user metrics
  String userName = application.getUser();
  User user = getUser(userName);
  user.assignContainer(resource, nodeLabels);
  // Note this is a bit unconventional since it gets the object and modifies
  // it here, rather then using set routine
  Resources.subtractFrom(application.getHeadroom(), resource); // headroom
  metrics.setAvailableResourcesToUser(userName, application.getHeadroom());
  
  if (LOG.isDebugEnabled()) {
    LOG.info(getQueueName() + 
        " user=" + userName + 
        " used=" + usedResources + " numContainers=" + numContainers +
        " headroom = " + application.getHeadroom() +
        " user-resources=" + user.getTotalConsumedResources()
        );
  }
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:24,代码来源:LeafQueue.java


示例14: checkCSQueue

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
private void checkCSQueue(MockRM rm,
    SchedulerApplication<SchedulerApplicationAttempt> app,
    Resource clusterResource, Resource queueResource, Resource usedResource,
    int numContainers)
    throws Exception {
  checkCSLeafQueue(rm2, app, clusterResource, queueResource, usedResource,
    numContainers);

  LeafQueue queue = (LeafQueue) app.getQueue();
  Resource availableResources = Resources.subtract(queueResource, usedResource);
  // ************* check Queue metrics ************
  QueueMetrics queueMetrics = queue.getMetrics();
  asserteMetrics(queueMetrics, 1, 0, 1, 0, 2, availableResources.getMemory(),
    availableResources.getVirtualCores(), usedResource.getMemory(),
    usedResource.getVirtualCores());

  // ************ check user metrics ***********
  QueueMetrics userMetrics =
      queueMetrics.getUserMetrics(app.getUser());
  asserteMetrics(userMetrics, 1, 0, 1, 0, 2, availableResources.getMemory(),
    availableResources.getVirtualCores(), usedResource.getMemory(),
    usedResource.getVirtualCores());
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:24,代码来源:TestWorkPreservingRMRestart.java


示例15: checkCSLeafQueue

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
private void checkCSLeafQueue(MockRM rm,
    SchedulerApplication<SchedulerApplicationAttempt> app,
    Resource clusterResource, Resource queueResource, Resource usedResource,
    int numContainers) {
  LeafQueue leafQueue = (LeafQueue) app.getQueue();
  // assert queue used resources.
  assertEquals(usedResource, leafQueue.getUsedResources());
  assertEquals(numContainers, leafQueue.getNumContainers());

  ResourceCalculator calc =
      ((CapacityScheduler) rm.getResourceScheduler()).getResourceCalculator();
  float usedCapacity =
      Resources.divide(calc, clusterResource, usedResource, queueResource);
  // assert queue used capacity
  assertEquals(usedCapacity, leafQueue.getUsedCapacity(), 1e-8);
  float absoluteUsedCapacity =
      Resources.divide(calc, clusterResource, usedResource, clusterResource);
  // assert queue absolute capacity
  assertEquals(absoluteUsedCapacity, leafQueue.getAbsoluteUsedCapacity(),
    1e-8);
  // assert user consumed resources.
  assertEquals(usedResource, leafQueue.getUser(app.getUser())
    .getTotalConsumedResources());
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:25,代码来源:TestWorkPreservingRMRestart.java


示例16: checkFifoQueue

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
private void checkFifoQueue(SchedulerApplication schedulerApp,
    Resource usedResources, Resource availableResources) throws Exception {
  FifoScheduler scheduler = (FifoScheduler) rm2.getResourceScheduler();
  // ************ check cluster used Resources ********
  assertEquals(usedResources, scheduler.getUsedResource());

  // ************ check app headroom ****************
  SchedulerApplicationAttempt schedulerAttempt =
      schedulerApp.getCurrentAppAttempt();
  assertEquals(availableResources, schedulerAttempt.getHeadroom());

  // ************ check queue metrics ****************
  QueueMetrics queueMetrics = scheduler.getRootQueueMetrics();
  asserteMetrics(queueMetrics, 1, 0, 1, 0, 2, availableResources.getMemory(),
    availableResources.getVirtualCores(), usedResources.getMemory(),
    usedResources.getVirtualCores());
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:18,代码来源:TestWorkPreservingRMRestart.java


示例17: unreserveResource

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public synchronized void unreserveResource(
    SchedulerApplicationAttempt application) {
  // adding NP checks as this can now be called for preemption
  if (getReservedContainer() != null
      && getReservedContainer().getContainer() != null
      && getReservedContainer().getContainer().getId() != null
      && getReservedContainer().getContainer().getId()
        .getApplicationAttemptId() != null) {

    // Cannot unreserve for wrong application...
    ApplicationAttemptId reservedApplication =
        getReservedContainer().getContainer().getId()
          .getApplicationAttemptId();
    if (!reservedApplication.equals(
        application.getApplicationAttemptId())) {
      throw new IllegalStateException("Trying to unreserve " +
          " for application " + application.getApplicationAttemptId() +
          " when currently reserved " +
          " for application " + reservedApplication.getApplicationId() +
          " on node " + this);
    }
  }
  setReservedContainer(null);
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:26,代码来源:FiCaSchedulerNode.java


示例18: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    allocateResource(clusterResource,
        rmContainer.getContainer().getResource(), node.getPartition(), false);
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:18,代码来源:ParentQueue.java


示例19: decreaseContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
@Override
protected void decreaseContainer(SchedContainerChangeRequest decreaseRequest,
    SchedulerApplicationAttempt attempt) {
  RMContainer rmContainer = decreaseRequest.getRMContainer();
  // Check container status before doing decrease
  if (rmContainer.getState() != RMContainerState.RUNNING) {
    LOG.info("Trying to decrease a container not in RUNNING state, container="
        + rmContainer + " state=" + rmContainer.getState().name());
    return;
  }
  FiCaSchedulerApp app = (FiCaSchedulerApp)attempt;
  LeafQueue queue = (LeafQueue) attempt.getQueue();
  try {
    queue.decreaseContainer(clusterResource, decreaseRequest, app);
    // Notify RMNode that the container can be pulled by NodeManager in the
    // next heartbeat
    this.rmContext.getDispatcher().getEventHandler()
        .handle(new RMNodeDecreaseContainerEvent(
            decreaseRequest.getNodeId(),
            Collections.singletonList(rmContainer.getContainer())));
  } catch (InvalidResourceRequestException e) {
    LOG.warn("Error happens when checking decrease request, Ignoring.."
        + " exception=", e);
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:26,代码来源:CapacityScheduler.java


示例20: waitForNumContainersToRecover

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt; //导入依赖的package包/类
public static void waitForNumContainersToRecover(int num, MockRM rm,
    ApplicationAttemptId attemptId) throws Exception {
  AbstractYarnScheduler scheduler =
      (AbstractYarnScheduler) rm.getResourceScheduler();
  SchedulerApplicationAttempt attempt =
      scheduler.getApplicationAttempt(attemptId);
  while (attempt == null) {
    LOG.info("Wait for scheduler attempt " + attemptId
        + " to be created");
    Thread.sleep(200);
    attempt = scheduler.getApplicationAttempt(attemptId);
  }
  while (attempt.getLiveContainers().size() < num) {
    LOG.info("Wait for " + num
        + " containers to recover. currently: "
        + attempt.getLiveContainers().size());
    Thread.sleep(200);
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:20,代码来源:TestWorkPreservingRMRestart.java



注:本文中的org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java ScriptableOutputStream类代码示例发布时间:2022-05-23
下一篇:
Java ContextThemeWrapper类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap