python - 删除列表中的重复项(Removing duplicates in lists)

Question

Pretty much I need to write a program to check if a list has any duplicates and if it does it removes them and returns a new list with the items that weren't duplicated/removed.

(我几乎需要编写一个程序来检查列表中是否有重复项，如果删除了重复项，则将其删除，并返回一个新列表，其中包含未重复/删除的项。)

This is what I have but to be honest I do not know what to do.

(这就是我所拥有的，但老实说我不知道??该怎么办。)

def remove_duplicates():
    t = ['a', 'b', 'c', 'd']
    t2 = ['a', 'c', 'd']
    for t in t2:
        t.append(t.remove())
    return t

  ask by Neemaximo translate from so

Answer 1

The common approach to get a unique collection of items is to use a set .

(获取项目唯一集合的常用方法是使用set 。)

Sets are unordered collections of distinct objects.

(集是不同对象的无序集合。)

To create a set from any iterable, you can simply pass it to the built-in set() function.

(要从任何迭代创建集合，您只需将其传递给内置的set()函数。)

If you later need a real list again, you can similarly pass the set to the list() function.

(如果以后再次需要真实列表，则可以类似地将集合传递给list()函数。)

The following example should cover whatever you are trying to do:

(以下示例应涵盖您尝试做的所有事情：)

>>> t = [1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> t
[1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> list(set(t))
[1, 2, 3, 5, 6, 7, 8]
>>> s = [1, 2, 3]
>>> list(set(t) - set(s))
[8, 5, 6, 7]

As you can see from the example result, the original order is not maintained .

(从示例结果中可以看到， 原始订单未维护 。)

As mentioned above, sets themselves are unordered collections, so the order is lost.

(如上所述，集合本身是无序集合，因此顺序丢失。)

When converting a set back to a list, an arbitrary order is created.

(将集合转换回列表时，将创建任意顺序。)

Maintaining order (维持秩序)

If order is important to you, then you will have to use a different mechanism.

(如果订单对您很重要，那么您将不得不使用其他机制。)

A very common solution for this is to rely on OrderedDict to keep the order of keys during insertion:

(一个非常常见的解决方案是在插入过程中依靠OrderedDict来保持键的顺序：)

>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(t))
[1, 2, 3, 5, 6, 7, 8]

Starting with Python 3.7 , the built-in dictionary is guaranteed to maintain the insertion order as well, so you can also use that directly if you are on Python 3.7 or later (or CPython 3.6):

(从Python 3.7开始 ，内置字典也保证可以保持插入顺序，因此，如果您使用的是Python 3.7或更高版本（或CPython 3.6），也可以直接使用它：)

>>> list(dict.fromkeys(t))
[1, 2, 3, 5, 6, 7, 8]

Note that this has the overhead of creating a dictionary first, and then creating a list from it.

(请注意，这样做的开销是先创建字典，然后再从中创建列表。)

If you don't actually need to preserve the order, you're better off using a set.

(如果您实际上不需要保留订单，则最好使用一组。)

Check out this question for more details and alternative ways to preserve the order when removing duplicates.

(请查看此问题，以获取更多详细信息以及删除重复项时保留订单的其他方法。)

---

Finally note that both the set as well as the OrderedDict / dict solutions require your items to be hashable .

(最后请注意， set以及OrderedDict / dict解决方案都要求您的项目是可哈希的 。)

This usually means that they have to be immutable.

(这通常意味着它们必须是不变的。)

If you have to deal with items that are not hashable (eg list objects), then you will have to use a slow approach in which you will basically have to compare every item with every other item in a nested loop.

(如果必须处理不可散列的项目（例如列表对象），则必须使用慢速方法，在这种方法中，您基本上必须将每个项目与嵌套循环中的所有其他项目进行比较。)