express - How do I flush to event stream, when res.flush() does not work? (node.js)

Question

I use express and node.js

I have been working a long time on getting a loading bar to work, and I finally figured out to use res.flush() since res.write() does not work due to compression middle-ware (to my understanding).

Now when I uploaded it to the server I run into the same problem. The stream does not flush, and the loading bar only gets updated on res.end(). All the other progress that was supposed to be sent continuously is also sent on res.end() (all in one go). Since the res.flush() is not working like it is on my pc, I am out of ideas on what to do, since I can't debug on the server.

I have an EventSource on the client side listening for updates to make to the loading bar, but on the live server the only update is when res.end() is called.

router.get('/statistics/:query/', function(req, res) {
    let query = req.params.query;

    res.writeHead(200, {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive'
    })
    
    //I have absolutely no idea if this is how to transfer progress between two classes
    //If this can be improved please let me know
    //The goal is to send updates back to the router, while the static class is querying the database    

    let progressObject : any = new Object();
    progressObject.progress = 0;
    
    progressObject.update = function(newProgress) { 
        //The update function is called within the static class "Statistics" as database query is handled
    
        let isDone = Math.round(newProgress) == 100 ? true : null; //sets isDone to true if progress=100
        this.progress = newProgress;
        res.write("data: "+JSON.stringify(this)+"

"); //This is the real problem! It works on my PC
        res.flush(); //But on the server it is only sent on res.end() in the bottom
    }

    let barChartLabels = Statistics.getBarChartLabels(progressObject, query);

    barChartLabels.then(labels =>{
        let responseObject = {
            message : labels,
            progress : null,
            chartTitle : "Stats", 
            success : true,
        };
        res.write("data: "+JSON.stringify(responseObject)+"

");
        res.end() //All progress data is only flushed here.
    }).catch(err=>{
        console.log("error!: "+err)
        let responseObject = {
            message : "Something went wrong when querying the database",
            progress : null,
            success : false
        };
        res.write("data: "+JSON.stringify(responseObject)+"

");
        res.end()
    })
});

The most important thing is getting this to work on the server, but I am also very interested in knowing what a good approach for sending the loading progress to the router is.

question from:https://stackoverflow.com/questions/65843913/how-do-i-flush-to-event-stream-when-res-flush-does-not-work-node-js

Answer 1

Waitting for answers