In O3D, skinning refers to the process of associating mesh data with weighted Transforms for each vertex in the mesh. The Transforms, also referred to as the "bones" or "joints," define a basic network of how the pieces of a shape's "skeleton" move in relation to each other. Once this basic Transform structure has been defined, it can be reused on multiple sets of primitives, or skins that "cover" this skeleton.
Usually, you'll use a digital content creation tool such as Maya, 3ds Max, or Blender, which has skinning features that produce the vertex and transform data described here. Then you'll provide this additional data as input to O3D so that it can use the skinning data to compute the final set of rendered vertices.
If you're interested in understanding more details about skinning, the following papers provide useful background on this topic:
As a very simple example, consider this 2D shape, which is composed of 8 vertices:
Figure 1. A simple mesh, composed of 8 vertices, and a set of 3 Transforms (or bones).
Three Transforms affect how this jointed mesh can move. The skin describes (1) which Transforms affect which vertices in the mesh and (2) the amount of relative influence (or weight) each Transform has on the affected vertex. In this example, Vertex1 (V1) and Vertex2 (V2) are influenced by Transform1 (T1). This relationship can be written as:
V1skinned = V1 * T1 * 1.0 V2skinned = V2 * T1 * 1.0
where 1.0 is the weight (or influence) the Transform has on this vertex. Vertex3 (V3) is influenced by T1 and T2, so its skin entry would be as follows:
V3skinned = (V3 * T1 * W1) + (V3 * T2 * W2)
The following O3D example shows how this mesh is modified when the Transforms change:
To view the code for this example, click here.
Figure 2. A simple skinned animation, with 8 vertices and 3 Transforms.
The SkinEval object operates on three sets of input data:
|
ParamArray, which references the Transforms from the Transform graph, in indexed formSourceBuffer, which contains the original set of geometric data (positions and normals for the primitive)Skin, which contains the detailed breakdown of how each vertex is influenced by one or more Transforms
Figure 3. Evaluating skinning data.
This section describes the Skinning example in more detail. Figure 3 shows the different components that are set up to create the "inputs" to the SkinEval: the ParamArray of Transforms, the source buffer, and the skin. The SkinEval object computes the transformed vertices, which are then sent to the stream bank and vertex buffer for rendering.
Conceptually, the basic steps for skinning a simple shape are as follows:
ParamArray, which is used to index the Transforms. Transforms from the Transform graph are bound to this array.sourceBuffer to hold the original set of vertex data..Skin, which associates the Transform matrices with the vertices in the mesh and specifies the influence of each Transform.SkinEval object. SkinEval, which fills in their values.The following paragraphs describe each step in more detail, using code from the Skinning example. The code snippets are arranged conceptually; in some cases, code shown here as one step appears in separate sections of the actual code example.
This example uses the o3djs utility library function primitives.createCylinderVertices() to create the cylinder. The cylinder has a radius of 40 units and a height of 200 units, and the matrix passed in causes the entire cylinder to be created above the origin (the default is to center the cylinder around the origin, with half the cylinder below the XZ plane).
// Create a cylinder.
var vertexInfo = o3djs.primitives.createCylinderVertices(
40, 200, 12, 20,
[[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 100, 0, 1]]);
var shape = vertexInfo.createShape(g_pack, material);
This code creates a ParamArray object named matrices and sizes it to hold the 11 Transform matrices.
// Create a ParamArray to hold matrices for skinning.
var matrices = g_pack.createObject('ParamArray');
.
.
.
// Create 11 matrices on our ParamArray.
matrices.resize(11, 'o3d.ParamMatrix4');
The cylinder example creates 11 Transforms and adds them to the Transform graph. It also binds the Transforms to the ParamArray and specifies their InverseBindPose.
// Create 11 transforms for the bones and parent them into a chain.
for (var ii = 0; ii <= 10; ++ii) {
var transform = g_pack.createObject('Transform');
g_transforms[ii] = transform;
if (ii > 0) {
transform.translate(0, 20, 0);
}
transform.parent = ii == 0 ? g_client.root : g_transforms[ii - 1];
// Bind the world matrix of the transform to the corresponding matrix in the
// ParamArray.
matrices.getParam(ii).bind(transform.getParam('worldMatrix'));
// Store the inverse world matrix of each transform as the bind pose for the
// skin.
skin.setInverseBindPoseMatrix(ii, g_math.inverse(transform.worldMatrix));
}
// Add the cylinder to the root transform.
g_transforms[0].addShape(shape);
Note that, for each transform, the skin.setInverseBindPoseMatrix() function is called:
skin.setInverseBindPoseMatrix(ii, g_math.inverse(transform.worldMatrix));
This function records the value of the transformation matrix (the inverse of it, actually) of the bone at the time the skin is bound. You will see below how this value is used during skinning computations to isolate the effects of the bone movement.
For example, the mesh shown in Figure 1 is created with this code:
var vertexInfo = o3djs.primitives.createPlaneVertices(
10, 40, 1, 3,
g_math.matrix4.mul(
g_math.matrix4.mul(
g_math.matrix4.rotationX(Math.PI / 2),
g_math.matrix4.rotationY(Math.PI / 2)),
g_math.matrix4.translation([0, 20, 0])));
Here, T3 is translated 40 units above the origin. V7 and V8 are also 40 units above the origin. The following calculation
V7' = V7 * T3
would put V7skinned at 80 units above the origin because the transformation is applied twice. The solution is to apply a transformation that reflects the difference between the value of T3 when the skin was bound (the bind pose) and its current value. That effect is achieved by multiplying with the inverse of world transform matrix that was recorded with the skin.setInverseBindPoseMatrix() call. So, the position of V7 after the skinning operation is actually calculated as follows:
V7skinned = V7 * IBP3 * T3 * TS-1 * W
where
The sourceBuffer contains a copy of the POSITION and NORMAL streams created by the createCylinderVertices() function. Here is the code that copies the vertex data into the sourceBuffer so that it can be sent to the SkinEval object:
// Create a SourceBuffer for the Skin and set the vertices on it.
var sourceBuffer = g_pack.createObject('SourceBuffer');
var positionField = sourceBuffer.createField('FloatField', 3);
var normalField = sourceBuffer.createField('FloatField', 3);
sourceBuffer.allocateElements(numVertices);
positionField.setAt(0, positionStream.elements);
normalField.setAt(0, normalStream.elements);
This code creates the skin object and determines the influence (weight) of each Transform on each vertex based on the proximity of the vertex to each Transform (bone):
// Create a Skin to hold the influences and bind pose.
var skin = g_pack.createObject('Skin');
.
.
.
// Make our source data for skinning and setup the influences for each bone.
// We only need the position and normals.
var positions = [];
var normals = [];
var positionStream = vertexInfo.findStream(g_o3d.Stream.POSITION);
var normalStream = vertexInfo.findStream(g_o3d.Stream.NORMAL);
var numVertices = positionStream.numElements();
for (var ii = 0; ii < numVertices; ++ii) {
// Choose the bones to influence the vertex based on its height.
// boneArea will be a fractional value between 2 bones based on the Y
// position of the vertex. In other words, if the y Position of the vertex
// is 63 and the bones are 20 units apart, then boneArea will = 6.15, meaning
// it's between bones 6 and 7.
var boneArea = positionStream.getElementVector(ii)[1] / BONE_SPACING;
// Pull out the fractional part of boneArea
var influence = boneArea % 1;
// Compute the bone indices
var bone1 = Math.floor(boneArea);
var bone2 = bone1 + 1;
if (bone2 >= NUM_BONES) {
bone2 = NUM_BONES - 1;
}
// Now make each vertex be influenced by these two bones. In the example
// above where boneArea was 6.15 we let bone1 influence the vertex by
// (1 - 0.15) or 0.85 and bone2 influence it by 0.15.
skin.setVertexInfluences(ii, [bone1, 1 - influence, bone2, influence]);
}
Here is how you create the SkinEval object and set its matrices and skin properties, and tell it how to use the SourceBuffer (that is, set up the three sets of input to be evaluated).
// Create a SkinEval to use the Skin.
var skinEval = g_pack.createObject('SkinEval');
// Tell the SkinEval which matrices and Skin to use.
skinEval.matrices = matrices;
skinEval.skin = skin;
.
.
.
// Skinning happens in world space so bind the transform of the shape
// to the SkinEval so it can put our skin in object space.
skinEval.getParam('base').bind(g_transforms[0].getParam('worldMatrix'));
.
.
.
// Tell the skinEval how to use the SourceBuffer
skinEval.setVertexStream(g_o3d.Stream.POSITION,
0,
positionField,
0);
skinEval.setVertexStream(g_o3d.Stream.NORMAL,
0,
normalField,
0);
In the "ordinary case," where there is no skinning data, the vertex data is stored directly in the vertex buffer, as described in Shapes. This buffer is ready to be rendered by the GPU. When skinning data is used, the vertex data is first stored in a source buffer, which is regular memory accessed by the CPU. The vertex data from the source buffer is set as vertex streams on the SkinEval object, where it is transformed using the ParamArray and skin data, then output to the stream bank, as shown in Figure 2.
This code shows the final step, in which the POSITION and NORMAL streambanks are bound to the output of the SkinEval, and the transformed shape is rendered.
// Bind the Primitive's position and normal streams // to the SkinEval so the SkinEval will fill them in each frame. var streamBank = shape.elements[0].streamBank; streamBank.bindStream(skinEval, g_o3d.Stream.POSITION, 0); streamBank.bindStream(skinEval, g_o3d.Stream.NORMAL, 0);