客服电话
APP下载
迪恩网络APP
随时随地掌握行业动态
官方微信
扫描二维码
关注迪恩网络微信公众号
|
对siwft有些了解的人都知道，Ring是swift中非常核心的组件，它决定着数据如何在集群中分布。Swift根据设置的partition_power决定集群中的分区数量（2的partition_power次方），并根据一致性哈希算法将分区分配到不同的node上，并将数据分布到对应的分区上。 因此，构建Ring就成为swift初始化必须经历的过程。简单说来：
那么swift-ring-builder命令又是如何执行的呢？本文简单旨在介绍swift-ring-builder命令，通过源码可以发现，swift-ring-builder命令的功能基本上都是通过RingBuilder实例的相关方法实现的，因此更加原理和细节的东西，将会在后续阅读RingBuilder的源码后再进行总结。So，莫喷我挂羊头卖狗肉啦 ^_~
1. swift-ring-builder 做了什么？ Rings是通过swift-ring-builder这个工具手动创建的，swift-ring-builder将分区与设备关联，并将该数据写入一个优化过的Python数据结构，压缩、序列化后写入磁盘，以供rings创建的数据可以被导入到服务器中。更新rings的机制非常简单，服务器通过检查创建rings的文件的最后更新日期来判断它和自己内存中的版本哪一个更新，从而决定是否需要重新载入rings创建数据。本段中所说的“Python数据结构”是一个如下所示的字典输出结构： def to_dict(self): """ Returns a dict that can be used later with copy_from to restore a RingBuilder. swift-ring-builder uses this to pickle.dump the dict to a file and later load that dict into copy_from. """ return {'part_power': self.part_power, 'replicas': self.replicas, 'min_part_hours': self.min_part_hours, 'parts': self.parts, 'devs': self.devs, 'devs_changed': self.devs_changed, 'version': self.version, '_replica2part2dev': self._replica2part2dev, '_last_part_moves_epoch': self._last_part_moves_epoch, '_last_part_moves': self._last_part_moves, '_last_part_gather_start': self._last_part_gather_start, '_remove_devs': self._remove_devs} swift-ring-builder命令的基本结构为： swift-ring-builder <builder_file> <action> [params] swift-ring-builder根据<action>执行相应的动作，生成builder file存储在<builder_file>指定的文件中，生成指导创建ring的文件xxx.ring.gz。在此之前，它会将原来的<builder_file>和xxx.ring.gz备份到backups文件夹中。
图1 swift-ring-builder创建的builder file和ring.gz 图2 swift-ring-builder备份的builder file和ring.gz 对<builder_file>的保存时非常重要的，因此你需要存储ring创建文件的多个副本。因为一旦ring创建文件完全丢失，就意味着我们需要重头完全重新创建一个ring，这样几乎所有的分区都会被分配到新的不同的设备上，因此数据副本也都会被移动到新的位置，造成大量数据迁移，导致系统在一段时间内不可用。
2. swift-ring-builder 命令
swift-ring-builder中包含多种命令：
add
create
list_parts
rebalance
remove
search
set_info
set_min_part_hours
set_weight
set_replicas
validate
write_ring
接下来我们对这些命令进行罗列，并作出相关解释。英文的文档内容可以通过直接运行“swift-ring-builder”命令获得。
swift-ring-builder <builder_file> Shows information about the ring and the devices within. swift-ring-builder <builder_file> create <part_power> <replicas> <min_part_hours> Creates <builder_file> with 2^<part_power> partitions and <replicas>. <min_part_hours> is number of hours to restrict moving a partition more than once. swift-ring-builder <builder_file> rebalance Attempts to rebalance the ring by reassigning partitions that haven't been recently reassigned. swift-ring-builder <builder_file> remove <search-value> [search-value ...] Removes the device(s) from the ring. This should normally just be used for a device that has failed. For a device you wish to decommission, it's best to set its weight to 0, wait for it to drain all its data, then use this remove command. This will not take effect until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder <builder_file> search <search-value> Shows information about matching devices. swift-ring-builder <builder_file> set_min_part_hours <hours> Changes the <min_part_hours> to the given <hours>. This should be set to however long a full replication/update cycle takes. We're working on a way to determine this more easily than scanning logs. swift-ring-builder <builder_file> set_weight <search-value> <weight> [<search-value> <weight] ... Resets the devices' weights. No partitions will be reassigned to or from the device until after running 'rebalance'. This is so you can make multiple device changes and rebalance them all just once. swift-ring-builder <builder_file> set_replicas <replicas> set_replicas命令用于使用参数中的<replicas>来设置副本数。 需要执行一个rebalance命令来使副本设置生效。该命令是swift-1.8.0新增的。 swift-ring-builder <builder_file> validate Just runs the validation routines on the ring. 仅运行builder的validate方法，使ring生效
3. 参数格式 在进行search设备的时候，<search_value>的格式如下： d<device_id>z<zone>-<ip>:<port>/<device_name>_<meta> 这个格式中的任意一个部分都是可选的，例如： z1 Matches devices in zone 1 z1-1.2.3.4 Matches devices in zone 1 with the ip 1.2.3.4 1.2.3.4 Matches devices in any zone with the ip 1.2.3.4 z1:5678 Matches devices in zone 1 using port 5678 :5678 Matches devices that use port 5678 /sdb1 Matches devices with the device name sdb1 _shiny Matches devices with shiny in the meta data _"snet: 5.6.7.8" Matches devices with snet: 5.6.7.8 in the meta data [::1] Matches devices in any zone with the ip ::1 z1-[::1]:5678 Matches devices in zone 1 with ip ::1 and port 5678 下面是一个指定最精确的例子： d74z1-1.2.3.4:5678/sdb1_"snet: 5.6.7.8"
4. 返回码含义 0 = operation successful 1 = operation completed with warnings 2 = error
|
请发表评论