...Speculation and hypothesis

To turn an arbitrary animated 2D scene into one that uses layer compositing, you have to segment it into layers. A layer is a part of the scene for which it's possible to draw an image that can substitute for the part.

State partitioning

First, collect drawn objects into a list by drawing order. Then segment the list. If an object has a different blend mode from the previous, start a new segment. If an object has changed during the last N frames, start a new segment. If a segment has been static for M frames and it takes longer than X milliseconds to draw, cache it into a layer.

Now we have the hard layer limits, but each layer is the size of the whole screen, which is suboptimal. Optimally, each layer would be the size of its bounding box, which brings us to...

Spatial partitioning

Compute the screen-aligned bounding boxes of each object in the layer. Then go through the bounding boxes and merge all intersecting bounding boxes. Now we have the disjoint bounding boxes of the layer, or spatial sublayers.

Cache each sublayer to an offscreen image, along with a hash of the objects it depicts.

Scene compression, maybe?

If the draw list contains segments with the same segment hash, they can be drawn using the same layer image (draw largest version, use possibly scaled down version for others.) The segment hash should be incremental to make it possible to discard non-matching segments as soon as possible.

The above is from:

As CPU renderers are usually fill-rate limited (CPUs suck at SIMD and memory bandwidth), they tend to segment the scene spatially into dirty areas, and redraw only those. After all, if you're only highlighting a button, it's a waste to redraw the whole scene. But a scene with an animated background and translucent foreground objects has effectively the whole drawing area as its dirty area (e.g. animated wallpaper, transparent windows in front.)

The usual way the speed up full-screen animation is to segment the scene into layers and composite the layers using a GPU. Compositing the layers will be very fast since GPUs have massive fillrate, and you only have to redraw changed layers. Compiz, Aero and Quartz Extreme all use this method - the compositor draws all window images to screen on every frame, but window images are only updated when the window contents have changed.

For a window manager, doing layer compositing is easy, because the layers are pre-defined. For an arbitrary 2D scene, it's more difficult.

I would think this should be too hard... i might try to work on a patch