• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java MiniQJMHACluster类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster的典型用法代码示例。如果您正苦于以下问题:Java MiniQJMHACluster类的具体用法?Java MiniQJMHACluster怎么用?Java MiniQJMHACluster使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



MiniQJMHACluster类属于org.apache.hadoop.hdfs.qjournal包,在下文中一共展示了MiniQJMHACluster类的15个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: assertEpochFilesCopied

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private static void assertEpochFilesCopied(MiniQJMHACluster jnCluster)
    throws IOException {
  for (int i = 0; i < 3; i++) {
    File journalDir = jnCluster.getJournalCluster().getJournalDir(i, "ns1");
    File currDir = new File(journalDir, "current");
    File prevDir = new File(journalDir, "previous");
    for (String fileName : new String[]{ Journal.LAST_PROMISED_FILENAME,
        Journal.LAST_WRITER_EPOCH }) {
      File prevFile = new File(prevDir, fileName);
      // Possible the prev file doesn't exist, e.g. if there has never been a
      // writer before the upgrade.
      if (prevFile.exists()) {
        PersistentLongFile prevLongFile = new PersistentLongFile(prevFile, -10);
        PersistentLongFile currLongFile = new PersistentLongFile(new File(currDir,
            fileName), -11);
        assertTrue("Value in " + fileName + " has decreased on upgrade in "
            + journalDir, prevLongFile.get() <= currLongFile.get());
      }
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:22,代码来源:TestDFSUpgradeWithHA.java


示例2: setup

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Before
public void setup() throws Exception {
  Configuration conf = new Configuration();
  // Turn off IPC client caching, so that the suite can handle
  // the restart of the daemons between test cases.
  conf.setInt(
      CommonConfigurationKeysPublic.IPC_CLIENT_CONNECTION_MAXIDLETIME_KEY,
      0);

  MiniQJMHACluster miniQjmHaCluster = new MiniQJMHACluster.Builder(conf).build();
  cluster = miniQjmHaCluster.getDfsCluster();
  jCluster = miniQjmHaCluster.getJournalCluster();
  
  // make nn0 active
  cluster.transitionToActive(0);
  // do sth to generate in-progress edit log data
  DistributedFileSystem dfs = (DistributedFileSystem) 
      HATestUtil.configureFailoverFs(cluster, conf);
  dfs.mkdirs(new Path("/test2"));
  dfs.close();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:22,代码来源:TestBootstrapStandbyWithQJM.java


示例3: setUpHaCluster

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private void setUpHaCluster(boolean security) throws Exception {
  conf = new Configuration();
  conf.setBoolean(CommonConfigurationKeys.HADOOP_SECURITY_AUTHORIZATION,
      security);
  cluster = new MiniQJMHACluster.Builder(conf).build();
  setHAConf(conf, cluster.getDfsCluster().getNameNode(0).getHostAndPort(),
      cluster.getDfsCluster().getNameNode(1).getHostAndPort());
  cluster.getDfsCluster().getNameNode(0).getHostAndPort();
  admin = new DFSAdmin();
  admin.setConf(conf);
  assertTrue(HAUtil.isHAEnabled(conf, "ns1"));

  originOut = System.out;
  originErr = System.err;
  System.setOut(new PrintStream(out));
  System.setErr(new PrintStream(err));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:TestDFSAdminWithHA.java


示例4: setup

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Before
public void setup() throws Exception {
  Configuration conf = new Configuration();
  // Turn off IPC client caching, so that the suite can handle
  // the restart of the daemons between test cases.
  conf.setInt(
      CommonConfigurationKeysPublic.IPC_CLIENT_CONNECTION_MAXIDLETIME_KEY,
      0);

  MiniQJMHACluster miniQjmHaCluster =
      new MiniQJMHACluster.Builder(conf).setNumNameNodes(nnCount).build();
  cluster = miniQjmHaCluster.getDfsCluster();
  jCluster = miniQjmHaCluster.getJournalCluster();
  
  // make nn0 active
  cluster.transitionToActive(0);
  // do sth to generate in-progress edit log data
  DistributedFileSystem dfs = (DistributedFileSystem) 
      HATestUtil.configureFailoverFs(cluster, conf);
  dfs.mkdirs(new Path("/test2"));
  dfs.close();
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:23,代码来源:TestBootstrapStandbyWithQJM.java


示例5: setup

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Before
public void setup() throws Exception {
  Configuration conf = new Configuration();
  // Turn off IPC client caching, so that the suite can handle
  // the restart of the daemons between test cases.
  conf.setInt(
      CommonConfigurationKeysPublic.IPC_CLIENT_CONNECTION_MAXIDLETIME_KEY,
      0);

  miniQjmHaCluster = new MiniQJMHACluster.Builder(conf).build();
  cluster = miniQjmHaCluster.getDfsCluster();
  jCluster = miniQjmHaCluster.getJournalCluster();
  
  // make nn0 active
  cluster.transitionToActive(0);
  // do sth to generate in-progress edit log data
  DistributedFileSystem dfs = (DistributedFileSystem) 
      HATestUtil.configureFailoverFs(cluster, conf);
  dfs.mkdirs(new Path("/test2"));
  dfs.close();
}
 
开发者ID:yncxcw,项目名称:FlexMap,代码行数:22,代码来源:TestBootstrapStandbyWithQJM.java


示例6: testNNFailover

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Test(timeout = 120000)
public void testNNFailover() throws IOException, URISyntaxException,
    MissingEventsException {
  Configuration conf = new HdfsConfiguration();
  MiniQJMHACluster cluster = new MiniQJMHACluster.Builder(conf).build();

  try {
    cluster.getDfsCluster().waitActive();
    cluster.getDfsCluster().transitionToActive(0);
    DFSClient client = ((DistributedFileSystem) HATestUtil.configureFailoverFs
        (cluster.getDfsCluster(), conf)).dfs;
    DFSInotifyEventInputStream eis = client.getInotifyEventStream();
    for (int i = 0; i < 10; i++) {
      client.mkdirs("/dir" + i, null, false);
    }
    cluster.getDfsCluster().shutdownNameNode(0);
    cluster.getDfsCluster().transitionToActive(1);
    EventBatch batch = null;
    // we can read all of the edits logged by the old active from the new
    // active
    for (int i = 0; i < 10; i++) {
      batch = waitForNextEvents(eis);
      Assert.assertEquals(1, batch.getEvents().length);
      Assert.assertTrue(batch.getEvents()[0].getEventType() == Event.EventType.CREATE);
      Assert.assertTrue(((Event.CreateEvent) batch.getEvents()[0]).getPath().equals("/dir" +
          i));
    }
    Assert.assertTrue(eis.poll() == null);
  } finally {
    cluster.shutdown();
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:33,代码来源:TestDFSInotifyEventInputStream.java


示例7: testReadEventsWithTimeout

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Test(timeout = 120000)
public void testReadEventsWithTimeout() throws IOException,
    InterruptedException, MissingEventsException {
  Configuration conf = new HdfsConfiguration();
  MiniQJMHACluster cluster = new MiniQJMHACluster.Builder(conf).build();

  try {
    cluster.getDfsCluster().waitActive();
    cluster.getDfsCluster().transitionToActive(0);
    final DFSClient client = new DFSClient(cluster.getDfsCluster()
        .getNameNode(0).getNameNodeAddress(), conf);
    DFSInotifyEventInputStream eis = client.getInotifyEventStream();
    ScheduledExecutorService ex = Executors
        .newSingleThreadScheduledExecutor();
    ex.schedule(new Runnable() {
      @Override
      public void run() {
        try {
          client.mkdirs("/dir", null, false);
        } catch (IOException e) {
          // test will fail
          LOG.error("Unable to create /dir", e);
        }
      }
    }, 1, TimeUnit.SECONDS);
    // a very generous wait period -- the edit will definitely have been
    // processed by the time this is up
    EventBatch batch = eis.poll(5, TimeUnit.SECONDS);
    Assert.assertNotNull(batch);
    Assert.assertEquals(1, batch.getEvents().length);
    Assert.assertTrue(batch.getEvents()[0].getEventType() == Event.EventType.CREATE);
    Assert.assertEquals("/dir", ((Event.CreateEvent) batch.getEvents()[0]).getPath());
  } finally {
    cluster.shutdown();
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:37,代码来源:TestDFSInotifyEventInputStream.java


示例8: testQuery

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Test (timeout = 300000)
public void testQuery() throws Exception {
  final Configuration conf = new Configuration();
  MiniQJMHACluster cluster = null;
  try {
    cluster = new MiniQJMHACluster.Builder(conf).build();
    MiniDFSCluster dfsCluster = cluster.getDfsCluster();
    dfsCluster.waitActive();

    dfsCluster.transitionToActive(0);
    DistributedFileSystem dfs = dfsCluster.getFileSystem(0);

    dfsCluster.shutdownNameNode(1);

    // start rolling upgrade
    RollingUpgradeInfo info = dfs
        .rollingUpgrade(RollingUpgradeAction.PREPARE);
    Assert.assertTrue(info.isStarted());

    info = dfs.rollingUpgrade(RollingUpgradeAction.QUERY);
    Assert.assertFalse(info.createdRollbackImages());

    dfsCluster.restartNameNode(1);

    queryForPreparation(dfs);

    // The NN should have a copy of the fsimage in case of rollbacks.
    Assert.assertTrue(dfsCluster.getNamesystem(0).getFSImage()
        .hasRollbackFSImage());
  } finally {
    if (cluster != null) {
      cluster.shutdown();
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:36,代码来源:TestRollingUpgrade.java


示例9: setUpCluster

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
@Before
public void setUpCluster() throws Exception {
  conf = new Configuration();
  conf.setInt(DFSConfigKeys.DFS_NAMENODE_CHECKPOINT_CHECK_PERIOD_KEY, 1);
  conf.setInt(DFSConfigKeys.DFS_NAMENODE_CHECKPOINT_TXNS_KEY, 1);
  conf.setInt(DFSConfigKeys.DFS_NAMENODE_NUM_CHECKPOINTS_RETAINED_KEY, 10);
  conf.setInt(DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_KEY, 1);
  HAUtil.setAllowStandbyReads(conf, true);
  
  if (clusterType == TestType.SHARED_DIR_HA) {
    MiniDFSNNTopology topology = MiniQJMHACluster.createDefaultTopology(10000);
    cluster = new MiniDFSCluster.Builder(conf)
      .nnTopology(topology)
      .numDataNodes(0)
      .checkExitOnShutdown(false)
      .build();
  } else {
    Builder builder = new MiniQJMHACluster.Builder(conf);
    builder.getDfsBuilder().numDataNodes(0).checkExitOnShutdown(false);
    miniQjmHaCluster = builder.build();
    cluster = miniQjmHaCluster.getDfsCluster();
  }
  cluster.waitActive();
  
  nn0 = cluster.getNameNode(0);
  nn1 = cluster.getNameNode(1);
  
  cluster.transitionToActive(0);
  fs = HATestUtil.configureFailoverFs(cluster, conf);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:31,代码来源:TestFailureToReadEdits.java


示例10: checkJnPreviousDirExistence

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private static void checkJnPreviousDirExistence(MiniQJMHACluster jnCluster,
    boolean shouldExist) throws IOException {
  for (int i = 0; i < 3; i++) {
    checkPreviousDirExistence(
        jnCluster.getJournalCluster().getJournalDir(i, "ns1"), shouldExist);
  }
  if (shouldExist) {
    assertEpochFilesCopied(jnCluster);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:11,代码来源:TestDFSUpgradeWithHA.java


示例11: getCommittedTxnIdValue

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private long getCommittedTxnIdValue(MiniQJMHACluster qjCluster)
    throws IOException {
  Journal journal1 = qjCluster.getJournalCluster().getJournalNode(0)
      .getOrCreateJournal(MiniQJMHACluster.NAMESERVICE);
  BestEffortLongFile committedTxnId = (BestEffortLongFile) Whitebox
      .getInternalState(journal1, "committedTxnId");
  return committedTxnId != null ? committedTxnId.get() :
      HdfsConstants.INVALID_TXID;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:10,代码来源:TestDFSUpgradeWithHA.java


示例12: testQuery

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private void testQuery(int nnCount) throws Exception{
  final Configuration conf = new Configuration();
  MiniQJMHACluster cluster = null;
  try {
    cluster = new MiniQJMHACluster.Builder(conf).setNumNameNodes(nnCount).build();
    MiniDFSCluster dfsCluster = cluster.getDfsCluster();
    dfsCluster.waitActive();

    dfsCluster.transitionToActive(0);
    DistributedFileSystem dfs = dfsCluster.getFileSystem(0);

    // shutdown other NNs
    for (int i = 1; i < nnCount; i++) {
      dfsCluster.shutdownNameNode(i);
    }

    // start rolling upgrade
    RollingUpgradeInfo info = dfs
        .rollingUpgrade(RollingUpgradeAction.PREPARE);
    Assert.assertTrue(info.isStarted());

    info = dfs.rollingUpgrade(RollingUpgradeAction.QUERY);
    Assert.assertFalse(info.createdRollbackImages());

    // restart other NNs
    for (int i = 1; i < nnCount; i++) {
      dfsCluster.restartNameNode(i);
    }
    // check that one of the other NNs has created the rollback image and uploaded it
    queryForPreparation(dfs);

    // The NN should have a copy of the fsimage in case of rollbacks.
    Assert.assertTrue(dfsCluster.getNamesystem(0).getFSImage()
            .hasRollbackFSImage());
  } finally {
    if (cluster != null) {
      cluster.shutdown();
    }
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:41,代码来源:TestRollingUpgrade.java


示例13: testCheckpoint

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
public void testCheckpoint(int nnCount) throws IOException, InterruptedException {
  final Configuration conf = new Configuration();
  conf.setInt(DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_KEY, 1);
  conf.setInt(DFSConfigKeys.DFS_NAMENODE_CHECKPOINT_PERIOD_KEY, 1);

  MiniQJMHACluster cluster = null;
  final Path foo = new Path("/foo");

  try {
    cluster = new MiniQJMHACluster.Builder(conf).setNumNameNodes(nnCount).build();
    MiniDFSCluster dfsCluster = cluster.getDfsCluster();
    dfsCluster.waitActive();

    dfsCluster.transitionToActive(0);
    DistributedFileSystem dfs = dfsCluster.getFileSystem(0);

    // start rolling upgrade
    RollingUpgradeInfo info = dfs
        .rollingUpgrade(RollingUpgradeAction.PREPARE);
    Assert.assertTrue(info.isStarted());

    queryForPreparation(dfs);

    dfs.mkdirs(foo);
    long txid = dfs.rollEdits();
    Assert.assertTrue(txid > 0);

    for(int i=1; i< nnCount; i++) {
      verifyNNCheckpoint(dfsCluster, txid, i);
    }

  } finally {
    if (cluster != null) {
      cluster.shutdown();
    }
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:38,代码来源:TestRollingUpgrade.java


示例14: getCommittedTxnIdValue

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private long getCommittedTxnIdValue(MiniQJMHACluster qjCluster)
    throws IOException {
  Journal journal1 = qjCluster.getJournalCluster().getJournalNode(0)
      .getOrCreateJournal(MiniQJMHACluster.NAMESERVICE);
  BestEffortLongFile committedTxnId = (BestEffortLongFile) Whitebox
      .getInternalState(journal1, "committedTxnId");
  return committedTxnId != null ? committedTxnId.get() :
      HdfsServerConstants.INVALID_TXID;
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:10,代码来源:TestDFSUpgradeWithHA.java


示例15: setUpHaCluster

import org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster; //导入依赖的package包/类
private void setUpHaCluster(boolean security) throws Exception {
  conf = new Configuration();
  conf.setBoolean(CommonConfigurationKeys.HADOOP_SECURITY_AUTHORIZATION,
      security);
  cluster = new MiniQJMHACluster.Builder(conf).build();
  setHAConf(conf, cluster.getDfsCluster().getNameNode(0).getHostAndPort(),
      cluster.getDfsCluster().getNameNode(1).getHostAndPort());
  cluster.getDfsCluster().getNameNode(0).getHostAndPort();
  admin = new DFSAdmin();
  admin.setConf(conf);
  assertTrue(HAUtil.isHAEnabled(conf, "ns1"));

  System.setOut(new PrintStream(out));
  System.setErr(new PrintStream(err));
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:16,代码来源:TestDFSAdminWithHA.java



注:本文中的org.apache.hadoop.hdfs.qjournal.MiniQJMHACluster类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java PropertyMessageResources类代码示例发布时间:2022-05-22
下一篇:
Java Layer类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap