Your description is correct.
If site A tells your browser to go to B and get the token, that's fine, but as it is a cross-domain request, A will not have access to the token in Javascript (this is a browser feature). So when A tells your browser to go back to B and actually do something, it still cannot include the token in the request.
That is, unless B set the token as a cookie. Evidently, that would be flawed, because the token cookie would also be sent, thus negating any protection. So the token in this case must be sent as either a form value or a request header (or something else that is not sent automatically like a cookie).
This also means that if B is vulnerable to cross-site scripting, it is also vulnerable to CSRF, because the token can then be stolen, but CSRF is the smaller problem then. :)
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…