• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java SchedulerEvent类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent的典型用法代码示例。如果您正苦于以下问题:Java SchedulerEvent类的具体用法?Java SchedulerEvent怎么用?Java SchedulerEvent使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



SchedulerEvent类属于org.apache.hadoop.yarn.server.resourcemanager.scheduler.event包,在下文中一共展示了SchedulerEvent类的18个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: handle

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void handle(SchedulerEvent event) {
  try {
    int qSize = eventQueue.size();
    if (qSize !=0 && qSize %1000 == 0) {
      LOG.info("Size of scheduler event-queue is " + qSize);
    }
    int remCapacity = eventQueue.remainingCapacity();
    if (remCapacity < 1000) {
      LOG.info("Very low remaining capacity on scheduler event queue: "
          + remCapacity);
    }
    this.eventQueue.put(event);
  } catch (InterruptedException e) {
    LOG.info("Interrupted. Trying to exit gracefully.");
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:ResourceManager.java


示例2: verifyAppAddedAndRemovedFromScheduler

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
public static SchedulerApplication<SchedulerApplicationAttempt>
    verifyAppAddedAndRemovedFromScheduler(
        Map<ApplicationId, SchedulerApplication<SchedulerApplicationAttempt>> applications,
        EventHandler<SchedulerEvent> handler, String queueName)
        throws Exception {
  ApplicationId appId =
      ApplicationId.newInstance(System.currentTimeMillis(), 1);
  AppAddedSchedulerEvent appAddedEvent =
      new AppAddedSchedulerEvent(appId, queueName, "user");
  handler.handle(appAddedEvent);
  SchedulerApplication<SchedulerApplicationAttempt> app =
      applications.get(appId);
  // verify application is added.
  Assert.assertNotNull(app);
  Assert.assertEquals("user", app.getUser());

  AppRemovedSchedulerEvent appRemoveEvent =
      new AppRemovedSchedulerEvent(appId, RMAppState.FINISHED);
  handler.handle(appRemoveEvent);
  Assert.assertNull(applications.get(appId));
  return app;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:23,代码来源:TestSchedulerUtils.java


示例3: setUp

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Before
public void setUp() {
  dispatcher = new DrainDispatcher();
  this.rm = new MockRM() {
    @Override
    protected EventHandler<SchedulerEvent> createSchedulerEventDispatcher() {
      return new SchedulerEventDispatcher(this.scheduler) {
        @Override
        public void handle(SchedulerEvent event) {
          scheduler.handle(event);
        }
      };
    }

    @Override
    protected Dispatcher createDispatcher() {
      return dispatcher;
    }
  };
  rm.start();
  amService = rm.getApplicationMasterService();
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:23,代码来源:TestAMRMRPCNodeUpdates.java


示例4: handle

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void handle(SchedulerEvent event) {
  try {
    int qSize = eventQueue.size();
    if (qSize !=0 && qSize %1000 == 0) {
      LOG.info("Size of scheduler event-queue is " + qSize);
    }
    int remCapacity = eventQueue.remainingCapacity();
    if (remCapacity < 1000) {
      LOG.info("Very low remaining capacity on scheduler event queue: "
          + remCapacity);
    }
    this.eventQueue.put(event);
  } catch (InterruptedException e) {
    throw new YarnRuntimeException(e);
  }
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:18,代码来源:ResourceManager.java


示例5: handle

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void handle(SchedulerEvent event) {
  try {
    int qSize = eventQueue.size();
    if (qSize != 0 && qSize % 1000 == 0
        && lastEventQueueSizeLogged != qSize) {
      lastEventQueueSizeLogged = qSize;
      LOG.info("Size of scheduler event-queue is " + qSize);
    }
    int remCapacity = eventQueue.remainingCapacity();
    if (remCapacity < 1000) {
      LOG.info("Very low remaining capacity on scheduler event queue: "
          + remCapacity);
    }
    this.eventQueue.put(event);
  } catch (InterruptedException e) {
    LOG.info("Interrupted. Trying to exit gracefully.");
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:20,代码来源:ResourceManager.java


示例6: startRM

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
private MockRM startRM(YarnConfiguration conf) {

    MemoryRMStateStore memStore = new MemoryRMStateStore();
    memStore.init(conf);

    MockRM rm1 = new MockRM(conf, memStore) {
      @Override
      protected EventHandler<SchedulerEvent> createSchedulerEventDispatcher() {
        return new SchedulerEventDispatcher(this.scheduler) {
          @Override
          public void handle(SchedulerEvent event) {
            super.handle(event);
          }
        };
      }

      @Override
      protected Dispatcher createDispatcher() {
        return new DrainDispatcher();
      }
    };

    rm1.start();
    return rm1;
  }
 
开发者ID:hopshadoop,项目名称:hops,代码行数:26,代码来源:TestNodeBlacklistingOnAMFailures.java


示例7: afterSchedulerEventHandled

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void afterSchedulerEventHandled(SchedulerEvent event) {

  try {
    switch (event.getType()) {
      case NODE_UPDATE:
        onNodeUpdated((NodeUpdateSchedulerEvent) event);
        break;

      case NODE_REMOVED:
        onNodeRemoved((NodeRemovedSchedulerEvent) event);
        break;

      default:
        break;
    }
  } catch (ClassCastException e) {
    LOGGER.error("incorrect event object", e);
  }
}
 
开发者ID:apache,项目名称:incubator-myriad,代码行数:21,代码来源:LeastAMNodesFirstPolicy.java


示例8: verifyAppAddedAndRemovedFromScheduler

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
public static <T> SchedulerApplication verifyAppAddedAndRemovedFromScheduler(
    final Map<ApplicationId, SchedulerApplication> applications,
    EventHandler<SchedulerEvent> handler, String queueName) throws Exception {
  ApplicationId appId =
      ApplicationId.newInstance(System.currentTimeMillis(), 1);
  AppAddedSchedulerEvent appAddedEvent =
      new AppAddedSchedulerEvent(appId, queueName, "user");
  handler.handle(appAddedEvent);
  SchedulerApplication app = applications.get(appId);
  // verify application is added.
  Assert.assertNotNull(app);
  Assert.assertEquals("user", app.getUser());

  AppRemovedSchedulerEvent appRemoveEvent =
      new AppRemovedSchedulerEvent(appId, RMAppState.FINISHED);
  handler.handle(appRemoveEvent);
  Assert.assertNull(applications.get(appId));
  return app;
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:20,代码来源:TestSchedulerUtils.java


示例9: run

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void run() {

  SchedulerEvent event;

  while (!stopped && !Thread.currentThread().isInterrupted()) {
    try {
      event = eventQueue.take();
    } catch (InterruptedException e) {
      LOG.error("Returning, interrupted : " + e);
      return; // TODO: Kill RM.
    }

    try {
      scheduler.handle(event);
    } catch (Throwable t) {
      // An error occurred, but we are shutting down anyway.
      // If it was an InterruptedException, the very act of 
      // shutdown could have caused it and is probably harmless.
      if (stopped) {
        LOG.warn("Exception during shutdown: ", t);
        break;
      }
      LOG.fatal("Error in handling event type " + event.getType()
          + " to the scheduler", t);
      if (shouldExitOnError
          && !ShutdownHookManager.get().isShutdownInProgress()) {
        LOG.info("Exiting, bbye..");
        System.exit(-1);
      }
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:34,代码来源:ResourceManager.java


示例10: setUp

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Before
public void setUp() {
  dispatcher = new DrainDispatcher();
  this.rm = new MockRM() {
    @Override
    public void init(Configuration conf) {
      conf.set(
        CapacitySchedulerConfiguration.MAXIMUM_APPLICATION_MASTERS_RESOURCE_PERCENT,
        "1.0");
      super.init(conf);
    }
    @Override
    protected EventHandler<SchedulerEvent> createSchedulerEventDispatcher() {
      return new SchedulerEventDispatcher(this.scheduler) {
        @Override
        public void handle(SchedulerEvent event) {
          scheduler.handle(event);
        }
      };
    }

    @Override
    protected Dispatcher createDispatcher() {
      return dispatcher;
    }
  };
  rm.start();
  amService = rm.getApplicationMasterService();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:30,代码来源:TestAMRMRPCNodeUpdates.java


示例11: testAppAttemptMetrics

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Test(timeout=5000)
public void testAppAttemptMetrics() throws Exception {
  AsyncDispatcher dispatcher = new InlineDispatcher();
  
  FifoScheduler scheduler = new FifoScheduler();
  RMApplicationHistoryWriter writer = mock(RMApplicationHistoryWriter.class);
  RMContext rmContext = new RMContextImpl(dispatcher, null,
      null, null, null, null, null, null, null, writer, scheduler);
  ((RMContextImpl) rmContext).setSystemMetricsPublisher(
      mock(SystemMetricsPublisher.class));

  Configuration conf = new Configuration();
  scheduler.setRMContext(rmContext);
  scheduler.init(conf);
  scheduler.start();
  scheduler.reinitialize(conf, rmContext);
  QueueMetrics metrics = scheduler.getRootQueueMetrics();
  int beforeAppsSubmitted = metrics.getAppsSubmitted();

  ApplicationId appId = BuilderUtils.newApplicationId(200, 1);
  ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
      appId, 1);

  SchedulerEvent appEvent = new AppAddedSchedulerEvent(appId, "queue", "user");
  scheduler.handle(appEvent);
  SchedulerEvent attemptEvent =
      new AppAttemptAddedSchedulerEvent(appAttemptId, false);
  scheduler.handle(attemptEvent);

  appAttemptId = BuilderUtils.newApplicationAttemptId(appId, 2);
  SchedulerEvent attemptEvent2 =
      new AppAttemptAddedSchedulerEvent(appAttemptId, false);
  scheduler.handle(attemptEvent2);

  int afterAppsSubmitted = metrics.getAppsSubmitted();
  Assert.assertEquals(1, afterAppsSubmitted - beforeAppsSubmitted);
  scheduler.stop();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:39,代码来源:TestFifoScheduler.java


示例12: testBlackListNodes

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Test
public void testBlackListNodes() throws Exception {
  Configuration conf = new Configuration();
  conf.setClass(YarnConfiguration.RM_SCHEDULER, FifoScheduler.class,
      ResourceScheduler.class);
  MockRM rm = new MockRM(conf);
  rm.start();
  FifoScheduler fs = (FifoScheduler) rm.getResourceScheduler();

  String host = "127.0.0.1";
  RMNode node =
      MockNodes.newNodeInfo(0, MockNodes.newResource(4 * GB), 1, host);
  fs.handle(new NodeAddedSchedulerEvent(node));

  ApplicationId appId = BuilderUtils.newApplicationId(100, 1);
  ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
      appId, 1);

  createMockRMApp(appAttemptId, rm.getRMContext());

  SchedulerEvent appEvent =
      new AppAddedSchedulerEvent(appId, "default",
        "user");
  fs.handle(appEvent);
  SchedulerEvent attemptEvent =
      new AppAttemptAddedSchedulerEvent(appAttemptId, false);
  fs.handle(attemptEvent);

  // Verify the blacklist can be updated independent of requesting containers
  fs.allocate(appAttemptId, Collections.<ResourceRequest>emptyList(),
      Collections.<ContainerId>emptyList(),
      Collections.singletonList(host), null);
  Assert.assertTrue(fs.getApplicationAttempt(appAttemptId).isBlacklisted(host));
  fs.allocate(appAttemptId, Collections.<ResourceRequest>emptyList(),
      Collections.<ContainerId>emptyList(), null,
      Collections.singletonList(host));
  Assert.assertFalse(fs.getApplicationAttempt(appAttemptId).isBlacklisted(host));
  rm.stop();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:40,代码来源:TestFifoScheduler.java


示例13: createSchedulerEventDispatcher

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
protected EventHandler<SchedulerEvent> createSchedulerEventDispatcher() {
  // Dispatch inline for test sanity
  return new EventHandler<SchedulerEvent>() {
    @Override
    public void handle(SchedulerEvent event) {
      scheduler.handle(event);
    }
  };
}
 
开发者ID:naver,项目名称:hadoop,代码行数:11,代码来源:TestAMRMClientOnRMRestart.java


示例14: handle

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void handle(SchedulerEvent event) {
  switch (event.getType()) {
  case NODE_LABELS_UPDATE:
    receivedEvent = true;
    updatedNodeToLabels =
        ((NodeLabelsUpdateSchedulerEvent) event).getUpdatedNodeToLabels();
    break;
  default:
    break;
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:13,代码来源:TestRMNodeLabelsManager.java


示例15: setUp

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Before
public void setUp() throws Exception {
  InlineDispatcher rmDispatcher = new InlineDispatcher();

  rmContext =
      new RMContextImpl(rmDispatcher, null, null, null,
        null, null, null, null, null);
  rmContext.setSystemMetricsPublisher(new SystemMetricsPublisher());
  rmContext.setRMApplicationHistoryWriter(mock(RMApplicationHistoryWriter.class));

  scheduler = mock(YarnScheduler.class);
  doAnswer(
      new Answer<Void>() {

        @Override
        public Void answer(InvocationOnMock invocation) throws Throwable {
          final SchedulerEvent event = (SchedulerEvent)(invocation.getArguments()[0]);
          eventType = event.getType();
          if (eventType == SchedulerEventType.NODE_UPDATE) {
            //DO NOTHING
          }
          return null;
        }
      }
      ).when(scheduler).handle(any(SchedulerEvent.class));

  rmDispatcher.register(SchedulerEventType.class,
      new TestSchedulerEventDispatcher());

  appId = ApplicationId.newInstance(System.currentTimeMillis(), 1);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:32,代码来源:TestRMAppLogAggregationStatus.java


示例16: testAppAttemptMetrics

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Test(timeout=5000)
public void testAppAttemptMetrics() throws Exception {
  AsyncDispatcher dispatcher = new InlineDispatcher();
  
  FifoScheduler scheduler = new FifoScheduler();
  RMApplicationHistoryWriter writer = mock(RMApplicationHistoryWriter.class);
  RMContext rmContext = new RMContextImpl(dispatcher, null,
      null, null, null, null, null, null, null, scheduler);
  ((RMContextImpl) rmContext).setSystemMetricsPublisher(
      mock(SystemMetricsPublisher.class));

  Configuration conf = new Configuration();
  ((RMContextImpl) rmContext).setScheduler(scheduler);
  scheduler.setRMContext(rmContext);
  scheduler.init(conf);
  scheduler.start();
  scheduler.reinitialize(conf, rmContext);
  QueueMetrics metrics = scheduler.getRootQueueMetrics();
  int beforeAppsSubmitted = metrics.getAppsSubmitted();

  ApplicationId appId = BuilderUtils.newApplicationId(200, 1);
  ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
      appId, 1);

  SchedulerEvent appEvent = new AppAddedSchedulerEvent(appId, "queue", "user");
  scheduler.handle(appEvent);
  SchedulerEvent attemptEvent =
      new AppAttemptAddedSchedulerEvent(appAttemptId, false);
  scheduler.handle(attemptEvent);

  appAttemptId = BuilderUtils.newApplicationAttemptId(appId, 2);
  SchedulerEvent attemptEvent2 =
      new AppAttemptAddedSchedulerEvent(appAttemptId, false);
  scheduler.handle(attemptEvent2);

  int afterAppsSubmitted = metrics.getAppsSubmitted();
  Assert.assertEquals(1, afterAppsSubmitted - beforeAppsSubmitted);
  scheduler.stop();
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:40,代码来源:TestFifoScheduler.java


示例17: handle

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@Override
public void handle(SchedulerEvent event) {
  try {
    if(sleepFlag) {
      Thread.sleep(sleepTime);
    }
  }
  catch(InterruptedException ie) {
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:11,代码来源:TestCapacityScheduler.java


示例18: testSchedulerEventDispatcherForPreemptionEvents

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent; //导入依赖的package包/类
@SuppressWarnings("unchecked")
@Test(timeout=10000)
public void testSchedulerEventDispatcherForPreemptionEvents() {
  AsyncDispatcher rmDispatcher = new AsyncDispatcher();
  CapacityScheduler sched = spy(new CapacityScheduler());
  YarnConfiguration conf = new YarnConfiguration();
  SchedulerEventDispatcher schedulerDispatcher =
      new SchedulerEventDispatcher(sched);
  rmDispatcher.register(SchedulerEventType.class, schedulerDispatcher);
  rmDispatcher.init(conf);
  rmDispatcher.start();
  schedulerDispatcher.init(conf);
  schedulerDispatcher.start();
  try {
    ApplicationAttemptId appAttemptId = mock(ApplicationAttemptId.class);
    RMContainer container = mock(RMContainer.class);
    ContainerPreemptEvent event1 = new ContainerPreemptEvent(
        appAttemptId, container, SchedulerEventType.DROP_RESERVATION);
    rmDispatcher.getEventHandler().handle(event1);
    ContainerPreemptEvent event2 = new ContainerPreemptEvent(
         appAttemptId, container, SchedulerEventType.KILL_CONTAINER);
    rmDispatcher.getEventHandler().handle(event2);
    ContainerPreemptEvent event3 = new ContainerPreemptEvent(
        appAttemptId, container, SchedulerEventType.PREEMPT_CONTAINER);
    rmDispatcher.getEventHandler().handle(event3);
    // Wait for events to be processed by scheduler dispatcher.
    Thread.sleep(1000);
    verify(sched, times(3)).handle(any(SchedulerEvent.class));
    verify(sched).dropContainerReservation(container);
    verify(sched).preemptContainer(appAttemptId, container);
    verify(sched).killContainer(container);
  } catch (InterruptedException e) {
    Assert.fail();
  } finally {
    schedulerDispatcher.stop();
    rmDispatcher.stop();
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:39,代码来源:TestRMDispatcher.java



注:本文中的org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java CsrfException类代码示例发布时间:2022-05-22
下一篇:
Java RetrievalMethodResolver类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap