jqgrid - Real-time data in a grid - better method

Question

What is for you the better approach for displaying real-time data (Stock Exchange, Weather, ...) in a grid ?
I use this method:

setInterval(function(){
      jQuery("#list1").trigger('reloadGrid');
}, 5000);

Answer 1

I find your question very interesting. I think the question could be interesting for many other users. So +1 from me.

The usage of setInterval seams me the best way in common browser independent case. One should of cause save the result of setInterval in a variable to be able to use clearInterval to stop it.

One more small improvement would be the usage of [{current:true}] (see the answer for details) as the second parameter of trigger:

var $grid = jQuery("#list1"), timer;

timer = setInterval(function () {
    $grid.trigger('reloadGrid', [{current: true}]);
}, 5000);

It will save selection during the reloading of grid.

Many new web browsers support now WebSocket. So it would be better to use the way in case if the web browser supports it. In the way one can skip unneeded grid reloading in case if the data are not changed on server and prevent permanent pooling of the server.

One more common way seems to me also very interesting. If one would use some kind of timestamp of the last changes of the grid data one could verify whether the data are changed or not. One can either uses ETag or some general additional method on the server for the purpose.

The current success callback of jQuery.ajax which fill jqGrid allows you use to implement beforeProcessing callback to modify the server response, but it's not allows you to stop jqGrid refreshing based on jqXHR.satus value (it would be xhr.status in the current jqGrid code). Small modification of the line of the code of jqGrid would allows you to stop jqGrid refreshing in case of unchanged data on the server. One can either test textStatus (st in the current code of jqGrid) for "notmodified" or test jqXHR.satus (xhr.status in the current jqGrid code) for 304. It's important that to use the scenario you should use prmNames: { nd:null } option and set ETag and Cache-Control: private, max-age=0 (see here, here and here for additional information).

UPDATED: I tried to create the demo project based on my last suggestions and found out that it's not so easy as I described above. Nevertheless I made it working. The difficulty was because one can't see the 304 code from the server response inside of jQuery.ajax. The reason was the following place in the XMLHttpRequest specification

For 304 Not Modified responses that are a result of a user agent generated conditional request the user agent must act as if the server gave a 200 OK response with the appropriate content. The user agent must allow author request headers to override automatic cache validation (e.g. If-None-Match or If-Modified-Since), in which case 304 Not Modified responses must be passed through.

So one sees 200 OK status inside of success handler of $.ajax instead 304 Not Modified from the server response. It seems that XMLHttpRequest get back just full response from the cache inclusive all HTTP headers. So I decide to change the analyse of cached data just saving of ETag from the last HTTP response as new jqGrid parameter and test the ETag of the new response with the saved data.

I seen that you use PHP (which I don't use :-(). Nevertheless I can read and understand PHP code. I hope you can in the same way read C# code and understand the main idea. So you will be able to implement the same in PHP.

Now I describe what I did. First of all I modified the lines of jqGrid source code which use beforeProcessing callback from

if ($.isFunction(ts.p.beforeProcessing)) {
    ts.p.beforeProcessing.call(ts, data, st, xhr);
}

to

if ($.isFunction(ts.p.beforeProcessing)) {
    if (ts.p.beforeProcessing.call(ts, data, st, xhr) === false) {
        endReq();
        return;
    }
}

It will allows to return false from beforeProcessing to skip refreshing of data - to skip processing of data. The implementation of beforeProcessing which I used in the demo are based on the usage ETag:

beforeProcessing: function (data, status, jqXHR) {
    var currentETag = jqXHR.getResponseHeader("ETag"), $this = $(this),
        eTagOfGridData = $this.jqGrid('getGridParam', "eTagOfGridData");
    if (currentETag === eTagOfGridData) {
        $("#isProcessed").text("Processing skipped!!!");
        return false;
    }
    $this.jqGrid('setGridParam', { eTagOfGridData: currentETag });
    $("#isProcessed").text("Processed");
}

The line $("#isProcessed").text("Processed"); or the line $("#isProcessed").text("Processing skipped!!!"); set "Processed" or "Processing skipped!!!" text in the div which I used to indicate visually that the data from the server was used to fill the grid.

In the demo I display two grids with the same data. The fist grid I use for editing of data. The second grid pulls the data from the server every second. If the data are not changed on the server the HTTP traffic looks as following

HTTP Request:

GET http://localhost:34336/Home/DynamicGridData?search=false&rows=10&page=1&sidx=Id&sord=desc&filters= HTTP/1.1
X-Requested-With: XMLHttpRequest
Accept: application/json, text/javascript, */*; q=0.01
Referer: http://localhost:34336/
Accept-Language: de-DE
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; LEN2)
Host: localhost:34336
If-None-Match: D5k+rkf3T7SDQl8b4/Y1aQ==
Connection: Keep-Alive

HTTP Response:

HTTP/1.1 304 Not Modified
Server: ASP.NET Development Server/10.0.0.0
Date: Sun, 06 May 2012 19:44:36 GMT
X-AspNet-Version: 4.0.30319
X-AspNetMvc-Version: 2.0
Cache-Control: private, max-age=0
ETag: D5k+rkf3T7SDQl8b4/Y1aQ==
Connection: Close

So no data will be transferred from the server if the data are not changed. If the data are changed, the HTTP header will be started with HTTP/1.1 200 OK and contains ETag of new modified data together with the page of data itself.

One can refresh the data of the grid either manually using "Refresh" button of the navigator or uses "Start Autorefresh" button which will execute $grid1.trigger('reloadGrid', [{ current: true}]); every second. The page looks like

I marked the most important parts on the bottom of the page with the color boxes. If one loadui: "disable" option then one don't see any changes on the grid at all during the pulling of the server. If one commented the option that one will see "Loading..." div for very short time, but no grid contain will flicker.

After starting of "Autorefreshing" one will see mostly the picture like below

If one would change some row in the first grid, the second grid will be changed in a second and one will see "Processed" text which will be changed to "Processing skipped!!!" text in one more second.

The corresponding code (I used ASP.NET MVC) on the server side is mostly the following

public JsonResult DynamicGridData(string sidx, string sord, int page, int rows,
                                  bool search, string filters)
{
    Response.Cache.SetCacheability (HttpCacheability.ServerAndPrivate);
    Response.Cache.SetMaxAge (new TimeSpan (0));

    var serializer = new JavaScriptSerializer();
    ... - do all the work and fill object var result with the data

    // calculate MD5 from the returned data and use it as ETag
    var str = serializer.Serialize (result);
    byte[] inputBytes = Encoding.ASCII.GetBytes(str);
    byte[] hash = MD5.Create().ComputeHash(inputBytes);
    string newETag = Convert.ToBase64String (hash);
    Response.Cache.SetETag (newETag);
    // compare ETag of the data which already has the client with ETag of response
    string incomingEtag = Request.Headers["If-None-Match"];
    if (String.Compare (incomingEtag, newETag, StringComparison.Ordinal) == 0) {
        // we don't need return the data which the client already have
        Response.SuppressContent = true;
        Response.StatusCode = (int)HttpStatusCode.NotModified;
        return null;
    }

    return Json (result, JsonRequestBehavior.AllowGet);
}

I hope that the main idea of the code will be clear also for people which use not only ASP.NET MVC.

You can download the project here.

UPDATED: I posted the feature request to allow beforeProcessing to break processing of the server response by returning false value. The corresponding changes are already included (see here) in the main jqGrid code. So the next release of jqGrid will include it.