Introduction

Welcome to Open Shading Language!

Open Shading Language (OSL) is a small but rich language for programmable shading in advanced renderers and other applications, ideal for describing materials, lights, displacement, and pattern generation.

OSL was developed by Sony Pictures Imageworks for use in its in-house renderer used for feature film animation and visual effects. The language specification was developed with input by other visual effects and animation studios who also wish to use it.

OSL is distributed under the "New BSD" license (see the "LICENSE" file that comes with the distribution). In short, you are free to use it in your own applications, whether they are free or commercial, open or proprietary, as well as to modify the OSL code as you desire, provided that you retain the original copyright notices as described in the license.

How OSL is different

OSL has syntax similar to C, as well as other shading languages. However, it is specifically designed for advanced rendering algorithms and has features such as radiance closures, BSDFs, and deferred ray tracing as first-class concepts.

OSL has several unique characteristics not found in other shading languages (certainly not all together). Here are some things you will find are different in OSL compared to other languages:

Surface and volume shaders compute radiance closures, not final colors.

OSL's surface and volume shaders compute an explicit symbolic description, called a "closure", of the way a surface or volume scatters light, in units of radiance. These radiance closures may be evaluated in particular directions, sampled to find important directions, or saved for later evaluation and re-evaluation. This new approach is ideal for a physically-based renderer that supports ray tracing and global illumination.

In contrast, other shading languages usually compute just a surface color as visible from a particular direction. These old shaders are "black boxes" that a renderer can do little with but execute to find this one piece of information (for example, there is no effective way to discover from them which directions are important to sample). Furthermore, the physical units of lights and surfaces are often underspecified, making it very difficult to ensure that shaders are behaving in a physically correct manner.

Surface and volume shaders do not loop over lights or shoot rays.

There are no "light loops" or explicitly traced rays in OSL surface shaders. Instead, surface shaders compute a radiance closure describing how the surface scatters light, and a part of the renderer called an "integrator" evaluates the closures for a particular set of light sources and determines in which directions rays should be traced. Effects that would ordinarily require explicit ray tracing, such as reflection and refraction, are simply part of the radiance closure and look like any other BSDF.

Advantages of this approach include that integration and sampling may be batched or re-ordered to increase ray coherence; a "ray budget" can be allocated to optimally sample the BSDF; the closures may be used by for bidirectional ray tracing or Metropolis light transport; and the closures may be rapidly re-evaluated with new lighting without having to re-run the shaders.

Surface and light shaders are the same thing.

OSL does not have a separate kind of shader for light sources. Lights are simply surfaces that are emissive, and all lights are area lights.

Transparency is just another kind of illumination.

You don't need to explicitly set transparency/opacity variables in the shader. Transparency is just another way for light to interact with a surface, and is included in the main radiance closure computed by a surface shader.

Renderer outputs (AOV's) are specified using "light path expressions."

Sometimes it is desirable to output images containing individual lighting components such as specular, diffuse, reflection, individual lights, etc. In other languages, this is usually accomplished by adding a plethora of "output variables" to the shaders that collect these individual quantities.

OSL shaders need not be cluttered with any code or output variables to accomplish this. Instead, there is a regular-expression-based notation for describing which light paths should contribute to which outputs. This is all done on the renderer side (though supported by the OSL implementation). If you desire a new output, there is no need to modify the shaders at all; you only need to tell the renderer the new light path expression.

Shaders are organized into networks.

OSL shaders are not monolithic, but rather can be organized into networks of shaders (sometimes called a shader group, graph, or DAG), with named outputs of some nodes being connected to named inputs of other nodes within the network. These connections may be done dynamically at render time, and do not affect compilation of individual shader nodes. Furthermore, the individual nodes are evaluated lazily, only their outputs are "pulled" from the later nodes that depend on them (shader writers may remain blissfully unaware of these details, and write shaders as if everything is evaluated normally).

No "uniform" and "varying" keywords in the language.

OSL shaders are evaluated in "SIMD" fashion, executing shaders on many points at once, but there is no need to burden shader writers with declaring which variables need to be uniform or varying. In OSL, this is done both automatically and dynamically, meaning that a variable can switch back and forth between uniform and varying, on an instruction-by-instruction basis, depending on what is assigned to it.

Arbitrary derivatives without grids or extra shading points.

In OSL, you can take derivatives of any computed quantity in a shader, and use arbitrary quantities as texture coordinates and expect correct filtering. This does not require that shaded points be arranged in a rectangular grid, or have any particular connectivity, or that any "extra points" be shaded. This is because derivatives are not computed by finite differences with neighboring points, but rather by "automatic differentiation", computing partial differentials for the variables that lead to derivatives, without any intervention required by the shader writer.

What OSL consists of

The OSL open source distribution consists of the following components:

• oslc, a standalone compiler that translates OSL source code into an assembly-like intermediate code (in the form of .oso files).

• liboslc, a library that implements the OSLCompiler class, which contains the guts of the shader compiler, in case anybody needs to embed it into other applications and does not desire for the compiler to be a separate executable.

• liboslquery, a library that implements the OSLQuery class, which allows applications to query information about compiled shaders, including a full list of its parameters, their types, and any metadata associated with them.

• oslinfo, a command-line program that uses liboslquery to print to the console all the relevant information about a shader and its parameters.

• liboslexec, a library that implements the ShadingSystem class, which allows compiled shaders to be executed within an application. Currently, it implements a SIMD-like interpreter of the OSL bytecodes.

• testshade, a program that lets you execute a shader (or connected shader network) on a rectangular array of points, and save any of its outputs as images. This allows for verification of shaders (and the shading system) without needing to be integrated into a fully functional renderer, and is the basis for most of our testsuite verification. Also note that the source code for testshade is currently the best example of how to call the OSL libraries.

• A few sample shaders.

• Documentation -- at this point consisting of the OSL language specification (useful for shader writers), but in the future will have detailed documentation about how to integrate the OSL libraries into renderers.

Current state of the project and road map

This is not a final production-ready release.

OK, here's the straight dope:

At Sony Pictures Imageworks, OSL has been integrated into our renderer and pipeline, the Shading department has recoded nearly all the facility shaders in OSL and is approaching full functionality of the shader library, and multiple shows are now doing early look development with OSL.

But we are NOT yet ready for full lighting and rendering production. In addition to some missing (very minor) functionality in the language itself, OSL's shading performance is still much slower than our previous C-language shaders (we're aiming for approximate parity with our old shaders). We are making rapid progress and hope to hit our performance targets in the next 3-4 weeks. In fact, we HAVE to, since our own shows will need full performance as they get into full production within the next few months.

We thought that even in this state, it was worth making the code available. The performance shouldn't get in your way -- even if you start integrating OSL into your renderer right away, by the time you are ready to do serious production, we will be at full performance.

What's missing? Here are a number of things that we wish had been included in the first public release of the code but are not ready yet, we promise that we are working on them and within a few weeks they will no longer be issues:

• More documentation, in particular the "Integration Guide" that documents all the public APIs of the OSL libraries that you use when integrating into a renderer.

• Currently an application executing shaders with liboslexec needs to access a few data structures and APIs that ought to be hidden. Expect a minor overhaul of the public ShadingSystem APIs so that invoking shaders from an application is easier, cleaner, and needs less access to the internal details of the library.

• The 'testshade' program runs shaders on an unwrapped flat rectangular array of points, and does not have any good way to test whether lights are working properly. We will eventually have another test program that will render a "lit ball" that will be more indicative of how the integration and lighting work.

• Currently, you have to write your own "integrator" from scratch when incorporating into a renderer application, which is less than ideal. We will eventually provide at least one sample integrator which may be used in a renderer, and outline what you need to do for more sophisticated integration strategies.

• Our set of sample shaders is quite anemic. We will eventually have a more extensive set of useful shaders.

• The code that implements "light path expressions" is currently missing from the OSL libraries. We naively thought at first that this was a renderer feature and implemented it on the wrong side of the boundary between our renderer and OSL. We will rectify that, moving the code into the public OSL library so that other renderers can use the same light expressions without needing to re-implement it.

• Performance issues -- OSL shaders currently execute 2-10 times slower than the equivalent shader written in optimized C (our goal is to achieve parity with our old C shaders). This is simply due to having just started optimization; We're making rapid progress -- a month ago we were 100-200x slower than C -- and there are multiple obvious optimizations we are working on that we are confident will achieve our performance goals within the next several weeks.

That's the short term -- all these should be completed within the next few months (over the course of the first several months of 2010).

In the longer term, there are a number of projects we hope to get to leading to a 2.x or 3.x cut of the language and library. Among our long-term goals:

• Currently "closure primitives" are implemented in C++ in the OSL library, but we would like a future spec of the language to allow new closure primitives to be implemented in OSL itself.

• Similarly, integrators are now implemented in the renderer, but we want a future OSL release to allow new integrators to be implemented in OSL itself.

• We would like to implement alternate "back ends" that would allow translation of OSL shaders (and shader networks) into code that can run on GPUs or other exotic hardware (at least for the biggest subset of OSL that can be expressed on such hardware). This would, for example, allow you to view close approximations to your OSL shaders in realtime preview windows in a modeling system or lighting tool.

• We would like to experiment with LLVM or other dynamic compilation technologies to see if there is a significant benefit to translating OSL shader networks all the way to native machine code, rather than interpreting byte codes at runtime.

We (the renderer development team at Sony Pictures Imageworks) probably can't do these all right away (in fact, probably can't do ALL of them in any time range). But we hope that as an open source project, other users and developers will step up to help us explore more future development avenues for OSL than we would be able to do alone.