场景
项目中很多地方使用Redis,有的用于缓存,有的直接做为存储,有的key设置有过期,有的key没有过期时间。
随着时间增长,Redis存储数据越来越多,消耗内存不断增长;
无论测试或生产环境,总内存是有限的;
有的key可能临时或测试使用的;
于是有了清理Redis key的需求。
Redis命令
查看key个数:
dbsize
info keyspace
查看内存情况:
info memory
通配符扫描key:
SCAN cursor [MATCH pattern] [COUNT count]
Python小程序
Linux服务器一般都自带Python,这里用Python编写2个小程序,分别用于扫描key和扫描并删除key。
redis-scan.py(扫描key):
#!/usr/bin/env python
# Scan keys in Redis.
# Author: cdfive
from redis import Redis
import time
def RedisScan(host, port, password, db, cursor, pattern, count):
start_time = time.time()
client = Redis(host=host, port=port, password=password, db=db)
counts, other_cursor_counts = 0, 0
while True:
cursor, keys = client.scan(cursor, pattern, count)
length = len(keys)
if length > 0:
for key in keys:
counts += 1
print("[%s][%s]%s,cursor=%s" % (
int(time.time() - start_time), counts, key.decode("utf-8"), cursor))
else:
other_cursor_counts += 1
print("[%s][other_curosr]other_cursor_counts=%s,cursor=%s" % (
int(time.time() - start_time), other_cursor_counts, cursor))
if cursor == 0:
break
client.close()
print("[%s]counts=%s,cursor=%s,other_cursor_counts=%s"
% (int(time.time() - start_time), counts, cursor, other_cursor_counts))
RedisScan("localhost", 6379, "123456", 0, 0, "*xxx*", 1000)
redis-scan-and-delete.py(扫描并删除key):
#!/usr/bin/env python
# Scan and delete keys in Redis.
# Author: cdfive
from redis import Redis
import time
def RedisScanAndDelete(host, port, password, db, cursor, pattern, count, batch_delete_size):
start_time = time.time()
client = Redis(host=host, port=port, password=password, db=db)
counts, other_cursor_counts = 0, 0
delete_keys = []
while True:
cursor, keys = client.scan(cursor, pattern, count)
length = len(keys)
if length > 0:
index = 0
for key in keys:
index += 1
counts += 1
print("[%s][%s]%s,cursor=%s" % (
int(time.time() - start_time), counts, key.decode("utf-8"), cursor))
delete_keys.append(key)
if (len(delete_keys) >= batch_delete_size or index >= length) and len(delete_keys) > 0:
client.delete(*delete_keys)
delete_keys = []
else:
other_cursor_counts += 1
print("[%s][other_curosr]other_cursor_counts=%s,cursor=%s" % (
int(time.time() - start_time), other_cursor_counts, cursor))
if cursor == 0:
break
client.close()
print("[%s]counts=%s,cursor=%s,other_cursor_counts=%s"
% (int(time.time() - start_time), counts, cursor, other_cursor_counts))
RedisScanAndDelete("localhost", 6379, "123456", 0, 0, "*xxx*", 1000, 100)
注:RedisScanAndDelete 最后1个参数为batch_delete_size ,用于通过pipeline批量删除key,提高删除效率。
|
请发表评论