You need to create a temporary pair that holds both the privilege (after flattening the list) and the related user, which you can then use in a grouping collector:
Map<String, List<User>> result = users.stream()
.flatMap(user -> user.getPrivileges()
.stream()
.map(priv -> new SimpleEntry<>(priv, user)))
.collect(
Collectors.groupingBy(Entry::getKey,
Collectors.mapping(Entry::getValue, Collectors.toList())));
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…