• Overview
• Features
• Dependencies
• Webservice API
• Crop an Image
• Generate a Thumbnail
• Resize an Image
• Reformat an Image
• Reduce the quality of an Image
• Combine operations in a single call
• Apache Configuration
• Configuration Parameters
• Available Log Metrics
• Cache Configuration
• DIMS Server Status
• Developer Information
• Extending mod_dims
• Troubleshooting/Errors
• Timeout Error Format
• General Error Format
• Common libcurl Error Messages
• Common mod_dims Error Messages
• Common Error Messages
• Other more serious errors

Overview

DIMS is a webservice that allows for dynamic image manipulation. It allows for easy image resizing and thumbnail generation directly in HTML pages, eliminating the need to pre-compute different image sizes and store their locations for use on a page.

Features

• Faster and smaller byte size thumbnail generation.
• Overall performance improvements.
• Extended API (and more modular for future additions).
• Ability to reference files locally as well as remotely.
• Ability to specify order of operations.
• More advanced geometry specification.
• Crop operation now accepts x, y offsets.
• Improved timeout resolution (now on both downloads and imagemagick operations).
• Improved resource constraints. (can set max memory imagemagick will use).
• Improved log messages.
• Cache header forwarding from source image.

Dependencies

ImageMagick 6.4.x: This is the engine used for image manipulation.

libcurl 7.18.x: This is used to fetch remote images.

Webservice API

All calls to the new API are made by specifying an operation along with it's arguments in the URL, for example: /thumbnail/78x100/. Operations should be specified in the order they should be performed. Some operations, such as setting quality or format, can be placed anywhere.

The base URL is now http://<dims_host>/dims3/.

Example remote URL:

http://<dims-host>/dims3/12AB/resize/100x100/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Using mod_dims you can also directly manipulate images on the local filesystem. The API call is still the same but instead of referencing the remote image at the end of the URL you specify the local image at the beginning.

Here is how the same call above would look if mod_dims was installed on an origin server:

http://<dims-host>/archive/2009/03/17/DSC_D300_1872.web.jpg/12AB/resize/100x100/

The image to the right was used as the sample image for the following examples. Click on the image to see it full size.

Crop an Image

Region can be specified as widthxheight{+-}x{+-}y{%}

Example

Crop an image to 100x100 offset 10 pixels from the top left corner.

http://dims-host/dims3/12AB/crop/100x100+10+10/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Generate a Thumbnail

Thumbnail size can be specified as widthxheight{%} {@} {!} {<} {>}.

By default, the width and height are maximum values. That is, the image is expanded or contracted to fit the width and height value while maintaining the aspect ratio of the image. Append an exclamation point to the geometry to force the image size to exactly the size you specify. For example, if you specify 640x480! the image width is set to 640 pixels and height to 480.

If only the width is specified, the width assumes the value and the height is chosen to maintain the aspect ratio of the image. Similarly, if only the height is specified (e.g., /thumbnail/x256/, the width is chosen to maintain the aspect ratio.)

To specify a percentage width or height instead, append %. The image size is multiplied by the width and height percentages to obtain the final image dimensions. To increase the size of an image, use a value greater than 100 (e.g. 125%). To decrease an image's size, use a percentage less than 100. Be sure to properly encode the % value as %25, for example 50%25

Use @ to specify the maximum area in pixels of an image.

Use ^{to set a minimum image size limit. The geometry 640x480}, for example, means the image width will not be less than 640 and the image height will not be less than 480 pixels after the resize. One of those dimensions will match the requested size, but the image will likely overflow the space requested to preserve its aspect ratio.

Example

Generate a 78x110 thumbnail of an image, maintaining the original aspect ratio but cropping down to size.

http://dims-host/dims3/12AB/thumbnail/78x110/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Resize an Image

New image size can be specified as widthxheight{%} {@} {!} {<} {>} {^}.

See thumbnail information for more details on geometry parameters.

Example

Resize an image down to 200x200 maintaining the aspect ratio.

http://dims-host/dims3/12AB/resize/200x200/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Reformat an Image

Valid formats are ''png'', ''jpg'' and ''gif''.

Example

Reformat a JPG image to a PNG.

http://dims-host/dims3/12AB/format/png/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Reduce the quality of an Image

Valid quality range is 0 to 100. This has the main affect of reducing the size of the final image.

Example

Reduce quality to 25.

http://dims-host/dims3/12AB/quality/25/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Combine operations in a single call

Example

Reformat JPG to PNG, generate a thumbnail at 78x110 and reduce the quality to 50.

http://dims-host/dims3/12AB/thumbnail/78x110/format/png/quality/50/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Apache Configuration

All configuration for DIMS should be in the ''dims.conf'' configuration file.

To load mod_dims add the following to your Apache server configuration:

<IfModule !mod_dims.c>
    LoadModule dims_module /path/to/mod_dims.so
</IfModule>

AddHandler dims-local .gif .jpg .png

<Location /dims/>
    SetHandler dims
</Location>

<Location /dims3/>
    SetHandler dims3
</Location>

<Location /dims-status/>
    SetHandler dims-status
</Location>

You will want to add configuration parameters detailed below.

It is also useful to enable core dumps using CoreDumpDirectory and setting ulimit -c unlimited.

Configuration Parameters

DimsDefaultImageURL: The image to display in case of errors or invalid requests. It should be a URL to an image. It can be a file:/// URL.

DimsDefaultImageURL http://o.aolcdn.com/art/ch_music2/albumartmissing DimsDefaultImageURL file:///path/to/no_image.gif

DimsDownloadTimeout: The length of time, in milliseconds, to wait for a remote image source to be retreived.

DimsDownloadTimeout 5000

DimsImagemagickTimeout: The length of time, in milliseconds, to allow ImageMagick to perform manipulations.

DimsImageMagickTimeout 25000

DimsCacheExpire: The default length of time, in seconds, to set the Cache-Control, Edge-Control and Expires headers.

DimsCacheExpire 86400

DimsNoImageCacheExpire: The default length of time, in seconds, to set the Cache-Control, Edge-Control and Expires headers when a NOIMAGE is returned.

DimsNoImageCacheExpire 60

DimsAddClient: Add a client to the list of valid clients. Arguments are {{{client_id noimage_url max_age downstream_ttl trust_source min_src_cache max_src_cache secret_key}}}. The max_age and downstream_ttl arguments control the times returned for the Cache-Control and Edge-Control HTTP headers, respectively. The trust_source parameter expects the single value "trust" indicating that dims should forward the Cache-Control header from the source image (if it exists) into the returned Cache-Control AND Edge-Control HTTP headers. If it is missing, the values specified in max_age and downstream_ttl will be returned. The last two arguments min_src_cache and max_src_cache specify the range of allowable values to be forwarded from the source image. If the Cache-Control header is not within range, max_age and downstream_ttl will be returned. Only client_id is required. If you do not want to set an argument, put '-' as its value. If max_age is not set the default will be used. If downstream_ttl is not set it will not be returned in the headers. The secret_key is used with /dims4/ prefix to provide secure image manipulation of any image without using whitelisting.

DimsAddClient PGMC http://o.aolcdn.com/art/ch_music2/albumartmissing 86400 86400 trust 3600 86400 s3cr3t

DimsAddClient TEST1 http://o.aolcdn.com/art/ch_music2/albumartmissing 86400 86400 trust

DimsAddClient TEST2 http://o.aolcdn.com/art/ch_music2/albumartmissing 86400 - trust

DimsAddClient TEST3 http://o.aolcdn.com/art/ch_music2/albumartmissing 86400 86400

DimsAddClient TEST4 http://o.aolcdn.com/art/ch_music2/albumartmissing 86400

DimsAddWhitelist: Add a hostname to the whitelist for retrieving remote images. Can be specified as a prefix glob *.aolcdn.com or an exact host, o.aolcdn.com. Multiple hostnames can be added per entry separated by a space or multiple entries can be used.

DimsAddWhitelist *.aolcdn.com cdn.aol.com

Available Log Metrics

A few metrics are available for logging by modifying the Apache LogFormat. To add these metrics to the Apache log add %{<metric_name>}n to the LogFormat.

DIMS_STATUS: This will contain an number indicating the status of the request.

Possible status code values are:

• 0 - DIMS_SUCCESS
• 1 - DIMS_FAILURE
• 2 - DIMS_DOWNLOAD_TIMEOUT
• 4 - DIMS_IMAGEMAGICK_TIMEOUT
• 8 - DIMS_BAD_CLIENT
• 16 - DIMS_BAD_URL
• 32 - DIMS_HOSTNAME_NOT_IN_WHITELIST

DIMS_ORIG_BYTES: The size, in bytes, of the original image before applying operations to it.

DIMS_DL_TIME: The time, in milliseconds, it took to download the original image. Always 0 for local image requests.

DIMS_IM_TIME: The time, in milliseconds, it took ImageMagick to apply the request operations. If ImageMagick operations timed out this will contain a -.

DIMS_TOTAL_TIME: The total time for this request, in milliseconds.

Cache Configuration

Even though DIMS sits behind Akamai there may still be a significant number of requests for the same image in a short period of time while Akamai fills it's cache. Because of this it may be beneficial to enable mod_cache in Apache.

It is probably easiest to use the mod_mem_cache variant (not to be confused with memcached) of mod_cache as the DIMS by itself uses very little memory leaving a lot of memory for caching.

Here is an example configuration, note that CacheMaxExpire should to be set otherwise mod_cache will attempt to use the Expires header which may be much longer.

LoadModule cache_module modules/mod_cache.so
LoadModule mem_cache_module modules/mod_mem_cache.so

<IfModule mod_cache.c>
    CacheEnable mem /
    CacheDisable /dims-status/

    CacheMaxExpire 120
    CacheDefaultExpire 120

    # mod_mem_cache (128MB cache size with max of 1M per object and max of 1024 objects)
    MCacheSize 131072
    MCacheMinObjectSize 1024
    MCacheMaxObjectSize 1048576
    MCacheMaxObjectCount 100
    MCacheRemovalAlgorithm LRU
</IfModule>

More information on this module can be found in the Apache Caching Guide.

DIMS Server Status

To see general information about a running DIMS server view the /dims-status/ URL. This URL contains information about the version of DIMS, the libraries it's linked against and counts of successes/failures.

Example:

ALIVE

Uptime:  21 hours 21 minutes 29 seconds
Restart time: Sunday, 17-Aug-2008 12:25:31 EDT

mod_dims version: 3.0 ($Revision: 52209 $)
ImageMagick version: ImageMagick 6.4.2 08/16/08 Q8 http://www.imagemagick.org
libcurl version: libcurl/7.18.2 zlib/1.2.3

Details
-------
Successful requests: 2156243
Failed requests: 16

Download timeouts: 14
Imagemagick Timeouts: 2

Developer Information

Extending mod_dims

Adding new ImageMagick commands to mod_dims is very straighforward. Every mod_dims image manipulation operation is implemented as a dims_operation_func function. This function signature is defined as:

Function Signature

apr_status_t (dims_operation_func) (dims_request_rec *d, char *args, char **err)

Arguments

dims_request_rec *d

The current DIMS request information. This contains the MagickWand, request_rec and other information that may be useful. See mod_dims.h for a complete definition of this struct.

char *args

The args provided for this command. For example if /thumbnail/78x100/ was our operation/arguments, *args would be a string containing 78x100

char **err

This is used to pass back a static string error message in case of an error.

To extend mod_dims implement a new dims_operation_func in mod_dims_ops.c, declare it in mod_dims.h and finally register it in the dims_init function in mod_dims.c.

The dims_operation_func implementation should return one of the DIMS_ status codes defined mod_dims.h. If an error occurred the **err pointer can be set to a static error message that will be logged. Returning anything other than DIMS_SUCCESS will result in the NOIMAGE image being returned to the user.

Here is an example:

First implement our new function. For this example we will add the blur command.

apr_status_t
dims_blur_operation (dims_request_rec *d, char *args, char **err) {
    MagickStatusType flags;
    GeometryInfo geometry;

    flags = ParseGeometry(args, &geometry);
    if ((flags & RhoValue) == 0) {
        *err = "Parsing blur arguments failed";
        return DIMS_FAILURE;
    }                
        
    MAGICK_CHECK(MagickBlurImage(d->wand, geometry.rho, 3), d);

    return DIMS_SUCCESS;
}

The MAGICK_CHECK macro will check to make sure the operation was successful and did not timeout. If either of those occur it will return an appropriate status code to force a NOIMAGE image to be returned to the user.

Now declare this new function in mod_dims.h by adding it to the list of dim_operation_func.

dims_operation_func 
    dims_resize_operation,
    .
    .
    . 
    dims_blur_operation;

Finally register this new function in dims_init in the mod_dims.c file. This lets the DIMS handler know to call the new dims_blur_operation function when it encounters the blur operation in the URL.

ops = apr_hash_make(p);
apr_hash_set(ops, "resize", APR_HASH_KEY_STRING, dims_resize_operation);
.
.
.
apr_hash_set(ops, "blur", APR_HASH_KEY_STRING, dims_blur_operation);

That's it. Now the new operation blur can be used by a DIMS webservice call. For example:

http://127.0.0.1:8001/dims3/TEST/blur/25/http://media.parker.beetlebug.org/archive/2009/03/17/DSC_D300_1872.web.jpg

Troubleshooting/Errors

There are three classes of errors in mod_dims;

• Errors caused during downloading of a source image. These come directly from libcurl and are logged as-is.
• Errors caused during an ImageMagick operation. These come directly from ImageMagik and are logged as-is.
• Errors caused during processing of a request by mod_dims. These fall into the category of bad input checking, bad config, etc.

ImageMagick Timeout Error Format

[client <client ip address> Imagemagick operation, '<operation>', timed out after 4 ms

<operation>

Example:

[client 127.0.0.1] Imagemagick operation, 'Resize/Image', timed out after 45773 ms. (max: 20000), on request: /03-slipknot-082707.jpg/PGMC/resize/91300x9883!/

General Error Format

Errors will be in the following format in Apache's error log:

[client <client ip address>] <source> error, '<source error message>', on request: <request uri>

Examples:

[client 10.181.182.244] Imagemagick error, 'no decode delegate for this image format `'', on request: /20080803WI55426251_WI.jpg/TEST/thumbnail/78x100/

[client 10.181.182.244] mod_dims error, 'Parsing resize geometry failed.', on reques: /test.jpg/resize/100/

[client 10.181.182.244] libcurl error, 'Timeout was reached', on request: http://o.aolcdn.com/feedgallery/music/i/s/slipknot/03-slipknot-082707.jpg

Common libcurl Error Messages

These messages are usually self explanatory. The URL that failed will be logged along with this message.

"Couldn't connect to server"

"Couldn't resolve DNS"

"Timeout was reached"

Common mod_dims Error Messages

"Requested URL has hostname that is not in the whitelist. (aaolcdn.com)"

The hostname of a remote URL did not match any hostnames or prefix globs in the whitelist.

The client id for the request was not found in the list of clients.

Request URL has an invalid geometry (e.g. 78x1a00 instead of 78x100).

Request URL has invalid geometry.

This occurs if ImageMagick had trouble reading the image. Check to make sure MAGICK_HOME is exported in the Apache start script.

This occurs when a local request is unable to find the image to resize.

Common ImageMagick Error Messages

"Memory allocation failed"

This should rarely occur, if ever, but usually when it does it's the result of an ImageMagick timeout. It does not necessary mean the process has run out of memory. In low numbers this error can safely be ignored.

"no decode delegate for this image format"

This happens when ImageMagick doesn't know how to read a source image. If this happens a lot the referrer should be checked and if necessary notify the owner of that page that they are using an invalid format. If the format appears valid check that the actual image really is that format by downloading it and verifying it in an image editor. If it is valid notify the developers of mod_dims.

This may occur if there was a failure to download the source image or potentially if the source image was zero bytes. If this happens a lot the owner of the page making the request should be notified.

This may occur if the image is corrupted. The "0x03" may be different depending on the corruption.

Other more serious errors

Any errors that have Assertion failed are results of bugs in the code and can be considered serious.

Assertion failed: (wand->signature == WandSignature), function MagickGetImageFormat, file wand/magick-image.c, line 4137.