I'm not entirely sure what you're trying to do so this may or may not generalise well.
That said, one possibility is to use annotate
with coord_cartesian(clip = "off")
to allow text outside the plot area.
ggplot(mtcars, aes(mpg, cyl, color = vs)) +
geom_line() +
annotate("text", x = 12.5, y = 3.5, label = "Arbitrary text") +
coord_cartesian(ylim = c(4, 8), clip = "off")
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…