apache - PHP output buffer not flushing

Question

I have a number of scripts that echo progress as they execute because they are long running. Each basically does the following at the end of each looped row of data processed:

echo '.';
@ob_flush();
flush();

This was working just fine for years and then I upgraded to PHP 5.3.x and Apache 2.2.x across several servers. Now, even if I pad the buffer with white space or set "ob_implicit_flush(1)", I can't get it to show the output on command.

One server still shows the output, but it is in chunks. It can take nearly 5 minutes and then suddenly a string of dots appears on the screen. With the other servers, I get nothing until the script finishes executing entirely.

I've tried looking through the php.ini and httpd.conf files to see if I could figure out what had changed between the different servers, but I'm obviously missing something.

I've also tried disabling mod_deflate in .htaccess for the affected scripts, but that doesn't help either (disabling mod_gzip used to fix the problem right away).

Can someone point me in the right direction with this please? Not being able to monitor script execution in real time is causing all sorts of problems but we can't stay on these older PHP versions any longer.

On an even more peculiar side note, I did try downgrading a server to PHP 5.2.17 but the output buffer problem remained after the downgrade. This makes me suspect it is something relating to the way Apache is handling PHP output since Apache 2 was left in place.

Answer 1

ob_flush() (an flush()) only flush the PHP buffer - the webserver maintains a buffer itself. And weird as it may sound, flushing the buffer early actually decreases throughput on the server, hence recent versions of apache buffer more agressively. There's also horrendous problems relating to compression and partial rendering when working with HTTP chunked encoding.

If you want to incrementally add content to a page then use ajax or websockets to add it a bit at a time.