客服电话
APP下载
迪恩网络APP
随时随地掌握行业动态
官方微信
扫描二维码
关注迪恩网络微信公众号
|
一，uniq干什么用的 文本中的重复行，基本上不是我们所要的，所以就要去除掉。linux下有其他命令可以去除重复行，但是我觉得uniq还是比较方便的一个。使用uniq的时候要注意以下二点 二，uniq参数说明 复制代码 代码如下:[zhangy@BlackGhost ~]$ uniq --help 用法：uniq [选项]... [文件] 从输入文件或者标准输入中筛选相邻的匹配行并写入到输出文件或标准输出。</p> <p>不附加任何选项时匹配行将在首次出现处被合并。</p> <p>长选项必须使用的参数对于短选项时也是必需使用的。 -c, --count //在每行前加上表示相应行目出现次数的前缀编号 -d, --repeated //只输出重复的行 -D, --all-repeated //只输出重复的行，不过有几行输出几行 -f, --skip-fields=N //-f 忽略的段数，-f 1 忽略第一段 -i, --ignore-case //不区分大小写 -s, --skip-chars=N //根-f有点像，不过-s是忽略，后面多少个字符 -s 5就忽略后面5个字符 -u, --unique //去除重复的后，全部显示出来，根mysql的distinct功能上有点像 -z, --zero-terminated end lines with 0 byte, not newline -w, --check-chars=N //对每行第N 个字符以后的内容不作对照 --help //显示此帮助信息并退出 --version //显示版本信息并退出 其中-z不知道有什么用 三，测试文本文件uniqtest 复制代码 代码如下:this is a test this is a test this is a test i am tank i love tank i love tank this is a test whom have a try WhoM have a try you have a try i want to abroad those are good men we are good men 四，实例详解 复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -c uniqtest 3 this is a test 1 i am tank 2 i love tank 1 this is a test //和第一行是重复的 1 whom have a try 1 WhoM have a try 1 you have a try 1 i want to abroad 1 those are good men 1 we are good men 从上例子中我们可以看出，uniq的一个特性，检查重复行的时候，只会检查相邻的行。重复数据，肯定有很多不是相邻在一起的。 复制代码 代码如下:[zhangy@BlackGhost mytest]$ sort uniqtest |uniq -c 1 WhoM have a try 1 i am tank 2 i love tank 1 i want to abroad 4 this is a test 1 those are good men 1 we are good men 1 whom have a try 1 you have a try
复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -d -c uniqtest 3 this is a test 2 i love tank uniq -d 只显示重复的行 复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -D uniqtest this is a test this is a test this is a test i love tank i love tank uniq -D 只显示重复的行，并且把重复几行都显示出来。他不能和-c一起使用 复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -f 1 -c uniqtest 3 this is a test 1 i am tank 2 i love tank 1 this is a test 2 whom have a try 1 you have a try 1 i want to abroad 2 those are good men //只有一行，显示二行 在这里those只有一行，显示的却是重复了，这是因为，-f 1 忽略了第一列，检查重复从第二字段开始的。 复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -i -c uniqtest 3 this is a test 1 i am tank 2 i love tank 1 this is a test 2 whom have a try //一个大写，一个小写 1 you have a try 1 i want to abroad 1 those are good men 1 we are good men 检查的时候，不区分大小写 复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -s 4 -c uniqtest 3 this is a test 1 i am tank 2 i love tank 1 this is a test 3 whom have a try //根上一个例子有什么不同 1 i want to abroad 1 those are good men 1 we are good men
复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -u uniqtest i am tank this is a test whom have a try WhoM have a try you have a try i want to abroad those are good men we are good men 去重复的项，然后全部显示出来 复制代码 代码如下:[zhangy@BlackGhost mytest]$ uniq -w 2 -c uniqtest 3 this is a test 3 i am tank 1 this is a test 1 whom have a try 1 WhoM have a try 1 you have a try 1 i want to abroad 1 those are good men 1 we are good men 对每行第2个字符以后的内容不作检查，所以i am tank 根 i love tank就一样了。 |
请发表评论