My favorites | Sign in
Google
Project hosting will be READ-ONLY Wednesday at 8am PST due to brief network maintenance.
                
Search
for
Updated Feb 04 (5 days ago) by rjrjr@google.com
Labels: Type-FAQ
ServerPushFAQ  
Explains Server Push, sometimes known as 'comet', and how you can achieve this with GWT.

What is Server Push?

Web applications communicate using the HTTP protocol. HTTP has no support for allowing a server to notify a client; the model involves a strict request-response model where the client (webbrowser running a GWT app) makes a request to the server (your web server running a servlet, for example), which must then respond with the requested data. There doesn't seem to be any room in this protocol for allowing the server to send a notification to the client.

However, with some magic and creative implementation of the protocol you can 'fake' a notification from server to client, initiated by the server. This is called Server Push. This document will describe how to accomplish this, what the issues are, and how to integrate Server Push into GWT.

A common use case for Server Push is a web chat client. The 'event' of a new chat message needs to be communicated from the server to the client.

Polling as an alternative

Server Push is a more advanced and more efficient replacement for a technique called polling.

Polling Example: A web chat app application may make a request for the 200th line of an open chat conversation by making a request to an API: http://my.chat.app/getChatLine?idx=200.

If this line does not exist yet (because no one is chatting), the server will simply reply "Does Not Exist" and the client must now re-run the same query in a second or two, over and over again, until someone types something. This is polling. It has two significant problems:

  1. Inefficient: If this webapp polls the server every second for new chat lines, and the chat room sees on average 1 new text message very 30 seconds, 29 out of every 30 requests is just 'no new data'. Put differently, to send 1 line of chat, server and client communicate several kilobytes of HTTP content and headers. This is extremely inefficient use of bandwidth.
  1. Slow: Imagine for a moment someone types something almost immediatly after one of the clients received a 'no new messages yet' notification. This client will not see the new line for at least another second, as that's how much time will pass before the client will try again. Trying to improve efficiency by polling less often automatically means events get delayed longer.

Server Push avoids both of those problems.

The Basic mechanism of Server Push

Taking the same example of the previous section, your client makes a request for http://my.chat.app/getChatLine?idx=200. However, instead of returning 'does not exist' if this chat line hasn't been typed yet, the server intentionally doesn't respond. It effectively acts like a very slow server would. Once the 200th line is typed by someone, it 'wakes up' and responds with the line. The client doesn't have to keep asking for updates, and there is no poll delay either.

This idea is simple enough, but unfortunately HTTP, and all the tools around the HTTP protocol, such as most web servers, weren't designed with this idea in mind. As a result there are a large number of caveats.

Server Push pitfalls

No Flush

The HTTP protocol has no 'flush'. A normal HTTP connection can get very complicated. A browser might connect to an internal proxy, which connects to an ISP proxy, which ends in your server park at a reverse proxy, and finally connect to your real web server. Any proxy server in the chain, and the web server itself, is allowed to cache responses. In fact, because in HTTP you cannot send an error response once you begin answering, most servlet containers will cache your entire response just in case your code throws an Exception and a response of '500 - Server Error' is warranted.

Because there is no 'flush', this can cause problems. Imagine this chat app. Instead of asking for the '200th' line, it just asks for a live stream. Every time a new chat message arrives on the server, you push this out on the HTTP connection, never actually closing it.

When testing your application this might work fine, but the HTTP protocol does not guarantee that a byte sent out will actually move all the way to the requesting client. Anyone behind a caching proxy will likely see chat messages only in large bursts instead of line by line.

The solution is to always close the connection immediately after you emit any sort of data. In our hypothetical chat app, we would have 2 practical use cases for the http://my.chat.app/getChatLine?idx=200 request:

  1. The 200th line is available. In this case, respond with it (and optionally any other available lines) immediately and close the request. This is similar to any other normal request/response: The client asks something, and you reply with it.
  2. The 200th line is NOT available. In this case, you do not respond with anything, but you wait. Once the 200th line is available, you stop waiting, respond with the 200th line, and close the connection.

In either situation, once you reply with anything, don't freeze the connection again. It's the only practical way to avoid flush problems.

NB: There's another reason why this is useful. Internet Explorer basically never allows access to HTTP connections that haven't finished yet. Until you close it, your GWT code can't access the material you've sent so far.

Spinners

Because the browser has no way of knowing the difference between your server being slow, and your server explicitly waiting to respond because it's waiting for an event to occur, browsers usually give an indicator to the user that the page is loading. This can be in the form of a Loading... statement in the status line (Safari does this), or the so-called spinner (the browser logo that animates to indicate the page is still loading) continually spinning. This erroneous indication that the page is not finished loading may confuse some users. In practice there does not seem to be a way to avoid spinners in all browsers. However, XmlHttpRequests, the driving mechanism behind GWT's RPC mechanism, the RequestBuilder class, and the HTTPRequest class, show the least erroneous 'loading' indicators amongst GWT-compatible browsers of all the various ways to create Server Push connections.

  • Practical Result: Be aware that some users may be a bit confused about your page appearing to still be loading.

TODO: Check for each browser what actually happends and report on it. I can tell with certainty that Safari2 shows a 'loading...' statement in the status bar, which is not actually visible by default, and nothing else.

Timeouts

Because proxies, the web browser running your GWT code, and sometimes even your web server can't tell the difference between a slow or hanging bit of server-side code, and code that is simply waiting for an event, at some point they may assume you're just broken and assume the response is an error.

As a practical matter this means you should never let your Server Push connections last over a minute. As a result you need to employ a tactic called connection refresh:

Every 50 seconds or so, if no events occur during that time, your server should respond 'no events', and your GWT code, upon receiving the 'no events' flag, simply re-establishes the connection. Effectively we still poll, but we only do so once a minute instead of once a second.

2 connection limit

All GWT-compatible browsers will only make 2 connections to 1 server. If there are 3 resources to fetch from the same server, the third resource will be queued until one of the first 2 connections is completed. A Server Push connection counts as 1 of these two connections, which poses a problem. For example, two simultaneous Server Push connections to the same server completely block any further attempts by the browser to talk to this server.

One way to alleviate this problem is to make sure only 1 server push connection is open at any point in time. If you need more, multiplex all events onto the same server push channel. (This is effectively what comet and cometd is - it's a multiplex protocol and architecture. In practice Server Push is often equated with the term 'comet' but this isn't technically correct).

Another method that helps is to load as much data as you can from different servers. Not all data can be downloaded from different servers (due to the SameOriginPolicy limitations) but images, CSS, and external javascript files can all be loaded from other servers.

A google maps application could for example load the map images from img.maps.google.com instead of maps.google.com. The browser would perceive the two hosts as different and thus allow 2 connections to each, even if they both resolve to the same IP address (=go to the same physical server machine).

Server Threads

In order to 'freeze' a response, you would usually freeze the thread. In a servlet, you might for example do something like:

  synchronized ( chatMessages ) {
    while ( chatMessages.size() < 200 ) try {
      chatMessages.wait();
    } catch ( InterruptedException e ) {
      respond("Server interrupted. Chat Session Closed.");
    }
    respond(chatMessages.get(200));

where, if anyone types something, you wake any frozen threads by using:

  synchronized ( chatMessages ) {
    chatMessages.add(theNewChatLine);
    chatMessages.notifyAll();
  }

Unfortunately, this means each Server Push connections ties up a thread. Usually web servers (including Tomcat, Jetty and Apache) have a limited number of threads and once they are all in use, any new requests will be ignored because the server thinks it's very busy and can't accept more connections.

You could up the thread pool count in your webserver's configuration, but there are more robust, scalable solutions available, often called 'continuations'.

The idea behind a continuation is that you tell your webserver to freeze the connection. By employing low level Input/Output directives, the webserver can do this without tying up threads.

There is not (yet) a standard way to do this in the servlet spec, but here are the links to the documentation on continuations for the two most often used servlet containers:

Note that some other web frameworks simply do not support this. At time of writing, there is no way to avoid tying up threads in PHP, Ruby on Rails, Django (Python), Turbogears (Python), and many others. Web Frameworks that can definitely handle continuations are anything written in Erlang, Twisted (Python), and Seaside (smalltalk).

  • Practical Result: Use continuations if your web framework supports them.

Server Push in GWT

Because of the reasons explained in previous sections, the most scalable, compatible way to implement server push is the Wait, Respond, Close, Re-Open paradigm:

You can implement this paradigm using any of the GWT server call mechanisms; GWT-RPC, RequestBuilder and HTTPRequest are all valid options. However, you must keep track of which events your GWT code has received so far. In practice this means keeping a counter and sending it along every request. If you are using GWT-RPC, you should make this counter a field in the POJO object you are sending. If you are using RequestBuilder or HTTPRequest, include this counter in the JSON or XML data that you are sending to the server. Similarly, all events that you send from server to client need to include their 'event number'.

TODO: Add some sample code here for both GWT-RPC and JSON-over-RequestBuilder that shows how to make the call, then check if the server wants us to just re-try (to avoid timeouts), stop bothering the server altogether (stop), or process incoming events and then re-try.


Comment by andres.a.testi, Sep 13, 2007

Server Push support will be standardized in Servlet 3.0. JEE6 servlet containers, will be Comet enabled thanks to JSR 315: http://jcp.org/en/jsr/detail?id=315

Comment by giovannicaputo86, Nov 13, 2007

when add some sample code of GWT-RPC for server push??

Comment by miroslav.pokorny, Nov 19, 2007

Or if you wish to use a working comet implementation today built ontop of a regular servlet you can try rocket-gwt Comet feature. Refer to the wiki for further details. Wiki: http://code.google.com/p/rocket-gwt/wiki/Comet Download: http://code.google.com/p/rocket-gwt/downloads/list

Comment by achimabe...@web.de, Nov 21, 2007

http://groups.google.com/group/Google-Web-Toolkit/browse_thread/thread/5e851d1b960e303e/4f58a36d88f8eaca#4f58a36d88f8eaca

shows a solution for server push with GWT and the Tomcat CometProcessor?.

The development of the code has stopped for the moment, but it worked at least once.

Comment by aharsani, May 15, 2008

just finishing PHP based web server with "continuations" support, Gwt-Php framework will be released by end of this year. (www.gwtphp.com currently in construction)

Comment by eliasbalasis, May 16, 2008

how about integrating GWT with 3rd party server push libraries/frameworks like DWR? Check http://code.google.com/p/dwr4gwt

Comment by Rvanlaak, Jun 01, 2009

The GWTEventService also is working great! It doesn't need just Jetty or Tomcat, but works on both webservers independently.

http://code.google.com/p/gwteventservice/

Comment by rich...@zschech.net, Aug 24, 2009

Here is a GWT comet implementation I've built:

http://code.google.com/p/gwt-comet/

The library implements Comet by streaming messages over long lived HTTP requests to minimise latency and bandwidth requirements and maximise the throughput. This is opposed to many other implementations which use polling or long polling techniques.

Comment by dade.watson, Sep 23, 2009

Isn't this similar to BOSH, also known as httpbind?

Comment by streamhubteam, Oct 01, 2009

FYI: StreamHub? has an open source GWT server push adapter:

http://www.stream-hub.com/

Comment by nenchev.mariyan, Nov 05, 2009

stream hub is paid

Comment by dentonator, Dec 05, 2009

can you do server push on Google App Engine? Is there a good example project that shows how it can be done?

Comment by nenchev.mariyan, Dec 10, 2009

not yet

Comment by northrup.james, Feb 02, 2010

I have this solution working using blockingDequeue, but i've not yet tested abusive scenarios... this serves a need for a lowl latency gui update on a localhost kiosk servlet

client package

public interface KioskEvent extends Serializable{
}
public class PopupEvent implements KioskEvent {
    private String message;
 
    public PopupEvent(String message) {
        this.message = message;
    }
 
    public PopupEvent() {
    }
 
    public String getMessage() {
        return message;
    }
}

    Timer parkTimer= new Timer() {
                    @Override
                    public void run() {
 
                        park();
 
                    }
                };

    public void onModuleLoad() {
 [....] 
       park();
 
    }
 
    public void park() {
    PushProvider.App.getInstance().eventWait(new AsyncCallback<Collection<KioskEvent>>() {
            @Override
            public void onFailure(Throwable caught) {
                parkTimer.schedule(5000);
            }
 
            @Override
            public void onSuccess(Collection<KioskEvent> result) {
                for (Iterator<KioskEvent> iterator = result.iterator(); iterator.hasNext();) {
                    KioskEvent kioskEvent = iterator.next();
                    iterator.remove();
                    if (kioskEvent instanceof PopupEvent) {
                        PopupEvent popupEvent = (PopupEvent) kioskEvent;
                        Window.alert(popupEvent.getMessage());
                    }else if(kioskEvent instance of [...]) {[...]}
                }
                parkTimer.schedule(1);
            }
        });
    }

RPCServlet

public class PushProviderImpl extends RpcServlet implements PushProvider, PopupHandler {
 

    {
        final java.util.Timer timer = new Timer() ;

        timer.scheduleAtFixedRate(new TimerTask() {
            @Override
            public void run() {
                popup("test");
            }
        }, 5000, 60000);
    }

    BlockingDeque<KioskEvent> dq = new LinkedBlockingDeque(3);
 
    @Override
    public Collection<KioskEvent> eventWait() {
        final LinkedList<KioskEvent> list = new LinkedList<KioskEvent>();
        do {
            try {
                list.add(dq.takeFirst());
            } catch (InterruptedException e) {
 
            }
        } while (!dq.isEmpty());
        return list;  //To change body of implemented methods use File | Settings | File Templates.
    }
 

    @Override
    public void popup(String message) {
        dq.add(new PopupEvent(message));
    }
}

Sign in to add a comment