networking - Direct TCP/IP connections in P2P apps

Question

From a Joel's post on Copilot:

Direct Connect! We’ve always done everything we can to make sure that Fog Creek Copilot can connect in any networking situation, no matter what firewalls or NATs are in place. To make this happen, both parties make outbound connections to our server, which relays traffic on their behalf. Well, in many cases, this isn’t necessary. So version 2.0 does something rather clever: it sets up the initial connection through our servers, so you get connected right away with 100% reliability. But then once you’re all connected, it quietly, in the background, looks for a way to make a direct connection. If it can’t, no big deal: you just keep relaying through our server. If you can make a direct peer-to-peer connection, it silently shifts your data onto the direct connection. You won’t notice anything except, probably, much faster communication.

How do they change the server connection to a P2P connection?

Answer 1

It's pretty tricky and interesting. I'm sure I have some details wrong, but the overview is this:

The programs can already talk to each other through Joel's server, so they can exchange information with each other and Joel's server. Further, Joel has their external IP addresses, and they give joel information about their internal IP addresses.

They decide to try this hole punch technique. Computer A initiates a TCP connection with Computer B using B's external IP address. It won't go through, but what it does is tell's A's router that it needs to allow incoming packets from B on a given port.

Computer B does the same thing, but its message gets through to A since A's router opened a port/ip combination that matches what B sent (there's some port magic that happens here - this is non trivial, but doable).

B's router remembers that B initiated a connection with A on a given port and IP, and so A's packets now flow into B past their router correctly as well.

So it's actually pretty straight forward, but the implementation has details, especially regarding how ports are given to new TCP connections, and how NAT routers typically deal with TCP requests and how they map to external ports. These details are the interesting, and difficult, bit.

-Adam