This an effect of the maximal munch principle, which has the lexical analyzer take as many characters as possible to form a valid token. This is covered in draft C++ standard section 2.5
[lex.pptoken] which says:
Otherwise, the next preprocessing token is the longest sequence of
characters that could constitute a preprocessing token, even if that
would cause further lexical analysis to fail.
Any cases such as the one you cite above need a specific exception carved out such as this case for <::
, we can see an example in the following code:
template<typename T> class SomeClass;
class Class;
SomeClass<::Class>* cls;
which is covered in this question, the exception is listed in the bullet just above the maximal munch rule:
Otherwise, if the next three characters are <:: and the subsequent character is neither : nor >, the <
is treated as a preprocessor token by itself and not as the first character of the alternative token <:.
and of course the non-nested >
which you cite in your question.
Note we can see that >=
is a preprocessor token from section 2.13
[lex.operators] which says:
The lexical representation of C++ programs includes a number of preprocessing tokens which are used in
the syntax of the preprocessor or are converted into tokens for operators and punctuators:
and includes >=
in the list.
The >> fix
We can see from the proposal that fixed the >>
case: N1757: Right Angle Brackets which says (emphasis mine):
Ever since the introduction of angle brackets, C++ programmers have
been surprised by the fact that two consecutive right angle brackets
must be separated by whitespace:
#include <vector>
typedef std::vector<std::vector<int> > Table; // OK
typedef std::vector<std::vector<bool>> Flags; // Error
The problem is an immediate consequence of the the “maximum munch”
principle and the fact that >> is a valid token (right shift) in C++.
This issue is a minor, but persisting, annoying, and somewhat
embarrassing problem. If the cost is reasonable, it seems therefore
worthwhile to eliminate the surprise.
The purpose of this document is to explain ways to allow >> to be
treated as two closing angle brackets, as well as to discuss the
resulting issues. A specific option is proposed along with wording
that would implement the proposal in the current working paper.
Also points out the >=
case:
It is also worth noting that the problem can also occur with the >>=
and >= tokens. For example
void func(List<B>= default_val1);
void func(List<List<B>>= default_val2);
Both of these forms are currently ill-formed. It may be desirable to
also address this issue, but this paper does not propose to do so.
Note, this change broke backward compatibility with C++03.