node.js - Socket.io with Nodejs not working with nginx reverse proxy

Question

I have a Nodejs server app with Express and Socket.io (Ubuntu 18.04). It always worked fine until nGinx (1.14) reverse proxy entered the scene. The nginx server is running on a different machine of Node.js apps, each app on it's own vm, inside the same network.

Server and Client on version 2.1.1.

The nginx server is responsible for multiple app redirects.

I tried several configuration combinations but nothing works.

Here what I've tried (examples for "company1"):

default.conf in /etc/nginx/conf.d

location /company1-srv/ {
     proxy_http_version 1.1;
     proxy_set_header Upgrade $http_upgrade;
     proxy_set_header Connection "Upgrade";
     proxy_set_header Host $host;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-Proto $scheme;
     proxy_set_header X-NginX-Proxy true;
     proxy_redirect off;
     proxy_pass http://172.16.0.25:51001/;
}

Then in the client code I connect using "path" options because socket.io misplace it's library path.

// companySrv and URL is actually returned by another service (following code is for illustrative purposes):
let companyUrl = 'https://api.myserver.com/company1-srv';
let companySrv = '/company1-srv'; 

socket(companyUrl, {
      path: companySrv + '/socket.io/'
});

I also tried to remove the path option and configured a specific /location for the socket.io stuff (for testing purposes):

location /socket.io/ {
   proxy_http_version 1.1;
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection 'upgrade';
   proxy_set_header Host $host;
   proxy_cache_bypass $http_upgrade;
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   proxy_set_header X-Real-IP $remote_addr;
   proxy_set_header X-Forwarded-Proto $scheme;
   proxy_set_header X-NginX-Proxy true;
   proxy_redirect off;
   proxy_pass http://172.16.0.25:51001/socket.io/;
}

Nothing worked.

It connects, but does'n emits anything. And after a short while (a minute or so), it becomes unavailable, raising the "disconnect" (reason: transport close) client event.

Server:

const io = require('socket.io')(https || http, {
   transports: ['polling', 'websocket'],
   allowUpgrades: true, 
   pingInterval: 60000*60*24, 
   pingTimeout: 60000*60*24
});

I also tried to edit the nginx.conf and write the "upstream socket_nodes { ..." and use the proxy_pass http://socket_nodes. It didn't make sense as I need a exact redirect depending on the company, but for the sake of tests I did, but it doesn't work as well.

What I need to do?

Thanks

Answer 1

We as well use socket.io with reverse-proxy from ngnix. I can share a little bit of our setup, maybe it helps to rule things out.

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
}

stream {
        log_format      basic   '$time_iso8601 $remote_addr '
                                '$protocol $status $bytes_sent $bytes_received '
                                '$session_time $upstream_addr '
                                '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';
        
        access_log /var/log/nginx/stream.log basic;
}

http {

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        gzip on;

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;

        ##
        # Server Blocks 
        ##


        # DOMAINEXAMPLE A
        server {
                server_name exampleA.domain.com;
                location / {
                        proxy_set_header        X-Forwarded-For $remote_addr;
                        proxy_set_header        Host $http_host;
                        proxy_pass              http://192.168.21.105:5050;
                 }
        }

        # DOMAINEXAMPLE B
        server {
                server_name exampleB.domain.com;
                location /api {
                        proxy_set_header        X-Forwarded-For $remote_addr;
                        proxy_set_header        Host $http_host;
                        proxy_pass              http://192.168.21.106:5050;
                }
        }

     
}

The most interesting part here are probably the server blocks

        # DOMAINEXAMPLE A
        server {
                server_name exampleA.domain.com;
                location / {
                        proxy_set_header        X-Forwarded-For $remote_addr;
                        proxy_set_header        Host $http_host;
                        proxy_pass              http://192.168.21.105:5050;
                 }
        }

        # DOMAINEXAMPLE B
        server {
                server_name exampleB.domain.com;
                location /api {
                        proxy_set_header        X-Forwarded-For $remote_addr;
                        proxy_set_header        Host $http_host;
                        proxy_pass              http://192.168.21.106:5050;
                }
        }

Domain Example A

For location / at http://192.168.21.105:5050 we have a NodeJS process running, including the setup for socket.io

const express = require('express');
const http    = require('http');
const app     = express();
const server  = http.createServer(app);
const io      = require('socket.io')(server);

Domain Example B

For location /api at http://192.168.21.106:5050 we have another NodeJS process running, including a slightly different setup for socket.io

const express  = require('express');
const http     = require('http');
const app      = express();
const server   = http.createServer(app);
const io       = require('socket.io')(server, {path: '/api/socket.io'});

In both cases socket.io works perfectly fine for us

Connecting from Client (Example B)

What we actually do on the server side here is creating a namespace for socket.io, like

const io= require('socket.io')(server, {path: '/api/socket.io'});
const nsp = io.of('/api/frontend');

and then on the client side , connect to it like

import io from 'socket.io-client'
const socket = io('https://exampleB.domain.com/api/frontend', {path: "/api/socket.io"});