在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称:redis-migrate-tool开源软件地址:https://gitee.com/vipshop/redis-migrate-tool开源软件介绍:redis-migrate-toolredis-migrate-tool is a convenient and useful tool for migrating data between redis. 中文介绍Features
DependencePlease install automake, libtool, autoconf and bzip2 at first. BuildTo build redis-migrate-tool: $ cd redis-migrate-tool$ autoreconf -fvi$ ./configure$ make$ src/redis-migrate-tool -h RUNsrc/redis-migrate-tool -c rmt.conf -o log -d WARNINGBefore run this tool, make sure your source redis machines have enough memory allowed at least one redis generate rdb file. If your source machines have large enough memory allowed all the redis generate rdb files at one time, you can set 'source_safe: false' in the rmt.conf. Not supported redis commandThe following commands are not supported to be propagated to the target redis group, because the keys in those commands maybe cross different target redis nodes: RENAME,RENAMENX,RPOPLPUSH,BRPOPLPUSH,FLUSHALL,FLUSHDB,BITOP,MOVE,GEORADIUS,GEORADIUSBYMEMBER,EVAL,EVALSHA,SCRIPT,PFMERGE ConfigurationConfig file has three parts: source, target and common. source OR target:
common:
filter supported glob-style patterns:
Use \ to escape special characters if you want to match them verbatim. For example, the configuration file shown below is to migrate data from single to twemproxy. [source]type: singleservers: - 127.0.0.1:6379 - 127.0.0.1:6380 - 127.0.0.1:6381 - 127.0.0.1:6382[target]type: twemproxyhash: fnv1a_64hash_tag: "{}"distribution: ketamaservers: - 127.0.0.1:6380:1 server1 - 127.0.0.1:6381:1 server2 - 127.0.0.1:6382:1 server3 - 127.0.0.1:6383:1 server4[common]listen: 0.0.0.0:8888threads: 2step: 1mbuf_size: 1024source_safe: true Migrate data from twemproxy to redis cluster. [source]type: twemproxyhash: fnv1a_64hash_tag: "{}"distribution: ketamaservers: - 127.0.0.1:6379 - 127.0.0.1:6380 - 127.0.0.1:6381 - 127.0.0.1:6382[target]type: redis clusterservers: - 127.0.0.1:7379[common]listen: 0.0.0.0:8888step: 1mbuf_size: 512 Migrate data from a redis cluster to another redis cluster with key filter(key's prefix is "abc"). [source]type: redis clusterservers: - 127.0.0.1:8379[target]type: redis clusterservers: - 127.0.0.1:7379[common]listen: 0.0.0.0:8888filter: abc* Load data from rdb file to redis cluster. [source]type: rdb fileservers: - /data/redis/dump1.rdb - /data/redis/dump2.rdb[target]type: redis clusterservers: - 127.0.0.1:7379[common]listen: 0.0.0.0:8888step: 2mbuf_size: 512source_safe: false Just save rdb file from redis cluster. [source]type: redis clusterservers: - 127.0.0.1:7379[target]type: rdb file[common]listen: 0.0.0.0:8888source_safe: true Load data from aof file to redis cluster. [source]type: aof fileservers: - /data/redis/appendonly1.aof - /data/redis/appendonly2.aof[target]type: redis clusterservers: - 127.0.0.1:7379[common]listen: 0.0.0.0:8888step: 2 STATUSYou can use redis-cli to connect with redis-migrate-tool. The listening address and port can be setted at common config. info commandFor example, you try the info command: $redis-cli -h 127.0.0.1 -p 8888127.0.0.1:8888> info# Serverversion:0.1.0os:Linux 2.6.32-573.12.1.el6.x86_64 x86_64multiplexing_api:epollgcc_version:4.4.7process_id:9199tcp_port:8888uptime_in_seconds:1662uptime_in_days:0config_file:/ect/rmt.conf# Clientsconnected_clients:1max_clients_limit:100total_connections_received:3# Memorymem_allocator:jemalloc-4.0.4# Groupsource_nodes_count:32target_nodes_count:48# Statsall_rdb_received:1all_rdb_parsed:1all_aof_loaded:0rdb_received_count:32rdb_parsed_count:32aof_loaded_count:0total_msgs_recv:7753587total_msgs_sent:7753587total_net_input_bytes:234636318total_net_output_bytes:255384129total_net_input_bytes_human:223.77Mtotal_net_output_bytes_human:243.55Mtotal_mbufs_inqueue:0total_msgs_outqueue:0127.0.0.1:8888> info command response instruction: Server:
Clients:
Group:
Stats:
OTHER COMMANDSshutdown [seconds|asap]The command behavior is the following:
Parameter:
For example, you try the shutdown command: $redis-cli -h 127.0.0.1 -p 8888127.0.0.1:8888> shutdownOK CHECK THE DATAAfter migrate the data, you can use redis_check command to check data in the source group and target group. Try the redis_check command: $src/redis-migrate-tool -c rmt.conf -o log -C redis_checkCheck job is running...Checked keys: 1000Inconsistent value keys: 0Inconsistent expire keys : 0Other check error keys: 0Checked OK keys: 1000All keys checked OK!Check job finished, used 1.041s If you want check more keys, try the follow: $src/redis-migrate-tool -c rmt.conf -o log -C "redis_check 200000"Check job is running...Checked keys: 200000Inconsistent value keys: 0Inconsistent expire keys : 0Other check error keys: 0Checked OK keys: 200000All keys checked OK!Check job finished, used 11.962s INSERT SOME KEYS JUST FOR TESTTry the redis_testinsert command: $src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert"Test insert job is running...Insert string keys: 200Insert list keys : 200Insert set keys : 200Insert zset keys : 200Insert hash keys : 200Insert total keys : 1000Correct inserted keys: 1000Test insert job finished, used 0.525s If you want insert more keys, try the follow: $src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert 30000"Test insert job is running...Insert string keys: 6000Insert list keys : 6000Insert set keys : 6000Insert zset keys : 6000Insert hash keys : 6000Insert total keys : 30000Correct inserted keys: 30000Test insert job finished, used 15.486s If you want insert only string type keys, try the follow: $src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert string"Test insert job is running...Insert string keys: 1000Insert list keys : 0Insert set keys : 0Insert zset keys : 0Insert hash keys : 0Insert total keys : 1000Correct inserted keys: 1000Test insert job finished, used 0.024s If you want insert some special type keys, try the follow: $src/redis-migrate-tool -c rmt.conf -o log -C "redis_testinsert string|set|list 10000"Test insert job is running...Insert string keys: 3336Insert list keys : 3336Insert set keys : 3328Insert zset keys : 0Insert hash keys : 0Insert total keys : 10000Correct inserted keys: 10000Test insert job finished, used 5.539s LicenseCopyright © 2016 VIPSHOP Inc. Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0 |
请发表评论