• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java YarnTwillRunnerService类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.twill.yarn.YarnTwillRunnerService的典型用法代码示例。如果您正苦于以下问题:Java YarnTwillRunnerService类的具体用法?Java YarnTwillRunnerService怎么用?Java YarnTwillRunnerService使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



YarnTwillRunnerService类属于org.apache.twill.yarn包,在下文中一共展示了YarnTwillRunnerService类的5个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: PeriodicNotificationTwillRunner

import org.apache.twill.yarn.YarnTwillRunnerService; //导入依赖的package包/类
/**
 *
 * @param yarnZookeepers - The zookeeper connect string used by the Hadoop YARN cluster.
 * @param configFile - The config file used by {@link PeriodicNotificationTwillApp}.  Typically notification.properties.
 */
public PeriodicNotificationTwillRunner(final String yarnZookeepers, final File configFile) {
    Preconditions.checkArgument(configFile.exists(), "Config File must exist");
    Objects.requireNonNull(yarnZookeepers, "YARN Zookeepers must not be null.");
    this.configFile = configFile;
    yarnConfiguration = new YarnConfiguration();
    twillRunner = new YarnTwillRunnerService(yarnConfiguration, yarnZookeepers);
    twillRunner.start();

    // sleep to give the YarnTwillRunnerService time to retrieve state from zookeeper
    try {
        Thread.sleep(1000);
    } catch (final InterruptedException e) {
        throw new IllegalStateException(e);
    }
}
 
开发者ID:apache,项目名称:incubator-rya,代码行数:21,代码来源:PeriodicNotificationTwillRunner.java


示例2: getTwillRunner

import org.apache.twill.yarn.YarnTwillRunnerService; //导入依赖的package包/类
private synchronized TwillRunnerService getTwillRunner(FluoConfiguration config) {
  if (!twillRunners.containsKey(config.getApplicationName())) {
    YarnConfiguration yarnConfig = new YarnConfiguration();
    yarnConfig.addResource(new Path(hadoopPrefix + "/etc/hadoop/core-site.xml"));
    yarnConfig.addResource(new Path(hadoopPrefix + "/etc/hadoop/yarn-site.xml"));

    TwillRunnerService twillRunner =
        new YarnTwillRunnerService(yarnConfig, config.getAppZookeepers() + ZookeeperPath.TWILL);
    twillRunner.start();

    twillRunners.put(config.getApplicationName(), twillRunner);

    // sleep to give twill time to retrieve state from zookeeper
    try {
      Thread.sleep(1000);
    } catch (InterruptedException e) {
      throw new IllegalStateException(e);
    }
  }
  return twillRunners.get(config.getApplicationName());
}
 
开发者ID:apache,项目名称:fluo,代码行数:22,代码来源:YarnAppRunner.java


示例3: startTwillRunner

import org.apache.twill.yarn.YarnTwillRunnerService; //导入依赖的package包/类
public TwillRunnerService startTwillRunner(YarnConfiguration yarnConfiguration) {
  String zkStr = dremioConfig.getString(DremioConfig.ZOOKEEPER_QUORUM);
  String clusterId = yarnConfiguration.get(YARN_CLUSTER_ID);
  Preconditions.checkNotNull(clusterId, "Cluster ID can not be null");
  TwillRunnerService twillRunner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
  TwillRunnerService previousOne = twillRunners.putIfAbsent(new ClusterId(clusterId), twillRunner);
  if (previousOne == null) {
    // start one we are planning to add - if it is already in collection it should be started
    twillRunner.start();
    return twillRunner;
  }
  return previousOne;
}
 
开发者ID:dremio,项目名称:dremio-oss,代码行数:14,代码来源:YarnController.java


示例4: main

import org.apache.twill.yarn.YarnTwillRunnerService; //导入依赖的package包/类
public static void main(String[] args) {
  if (args.length < 1) {
    System.err.println("Arguments format: <host:port of zookeeper server>");
    System.exit(1);
  }

  String zkStr = args[0];
  YarnConfiguration yarnConfiguration = new YarnConfiguration();
  final TwillRunnerService twillRunner = new YarnTwillRunnerService(yarnConfiguration, zkStr);
  twillRunner.start();

  String yarnClasspath =
    yarnConfiguration.get(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
                          Joiner.on(",").join(YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH));
  List<String> applicationClassPaths = Lists.newArrayList();
  Iterables.addAll(applicationClassPaths, Splitter.on(",").split(yarnClasspath));
  final TwillController controller = twillRunner.prepare(new HelloWorldRunnable())
      .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out, true)))
      .withApplicationClassPaths(applicationClassPaths)
      .withBundlerClassAcceptor(new HadoopClassExcluder())
      .start();

  Runtime.getRuntime().addShutdownHook(new Thread() {
    @Override
    public void run() {
      try {
        Futures.getUnchecked(controller.terminate());
      } finally {
        twillRunner.stop();
      }
    }
  });

  try {
    controller.awaitTerminated();
  } catch (ExecutionException e) {
    e.printStackTrace();
  }
}
 
开发者ID:apache,项目名称:twill,代码行数:40,代码来源:HelloWorld.java


示例5: main

import org.apache.twill.yarn.YarnTwillRunnerService; //导入依赖的package包/类
public static void main(String[] args) {
  if (args.length < 3) {
    System.err.println("Arguments format: <host:port of zookeeper server>"
                         + " <bundle jar path> <main class name> <extra args>");
    System.exit(1);
  }

  String zkStr = args[0];
  BundledJarRunner.Arguments arguments = new BundledJarRunner.Arguments(
          args[1], "/lib", args[2], Arrays.copyOfRange(args, 3, args.length));

  File jarFile = new File(arguments.getJarFileName());
  Preconditions.checkState(jarFile.exists());
  Preconditions.checkState(jarFile.canRead());

  final TwillRunnerService twillRunner = new YarnTwillRunnerService(new YarnConfiguration(), zkStr);
  twillRunner.start();


  final TwillController controller = twillRunner.prepare(
    new ExampleBundledJarApp(jarFile.getName(), jarFile.toURI()))
    .withArguments("BundledJarRunnable", arguments.toArray())
    .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out, true)))
    .start();

  Runtime.getRuntime().addShutdownHook(new Thread() {
    @Override
    public void run() {
      try {
        Futures.getUnchecked(controller.terminate());
      } finally {
        twillRunner.stop();
      }
    }
  });

  try {
    controller.awaitTerminated();
  } catch (ExecutionException e) {
    LOG.error("Error", e);
  }
}
 
开发者ID:apache,项目名称:twill,代码行数:43,代码来源:BundledJarExample.java



注:本文中的org.apache.twill.yarn.YarnTwillRunnerService类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java DurabilityData类代码示例发布时间:2022-05-23
下一篇:
Java DOMSerializerImpl类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap