Unless you really, really have to do IP adress validation, as well, I suggest you simplify the regular expression, because this beast is far too complex for only matching "IP part" and "port part". My suggestion would be
(d{1,3}.d{1,3}.d{1,3}.d{1,3}):(d{1,5})
Groups 1 and 2 will hold IP and port, respectively. And the above is already more complex that it needs to be, IMHO even something as simple as this would be enough:
(d+.d+.d+.d+):(d+)
Note that double backslashes are are requirement of Java strings, not of regex, so I left them out.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…