 |
Data in UsdSkel is stored in modular, reusable, instanceable parts, adhering to the following design principles:
All posing and animation data is vectorized. This encoding is concise and ideal for scalability and speed of data consumption in the USD APIs.
Although vectorization may seem unnecessary to the client attempting to interchange individual models only, it is an important consideration for achieving a scalable encoding of crowds in VFX pipelines.
Following these principles, UsdSkel defines skeletal data using the following schema types:
Identifies where in a scene graph skeletal processing takes place, in addition to providing bounds for skinned models.
Defines the topology of the skeleton, as well as storing a bind pose. Skeletons are animated by attaching a SkelAnimation.
Stores joint and blend shape animations, which may be applied to any Skeleton.
Stores an individual blend shape, and optional in-betweens, for a geometric primitive.
API schema used to describe which skeletons affect which primitives, as well as basic skinning properties like joint influences. The BindingAPI is also used to attach a SkelAnimation to a Skeleton.
Joint data in UsdSkel is stored in vectorized arrays. Throughout UsdSkel, these arrays may be ordered in slightly different ways. For instance, a skeleton has its own explicit joint ordering, but an animation may have another order, representing either a different ordering of joints, or a sparse subset of joints.
These data orderings are referred to as a primitive's joint order, and are defined using token arrays.
The tokens in a joint array are given as Sdf-style paths, where parent-child relationships in the given paths establish a joint hierarchy. Note that these paths do not need to refer to actual primitives in the scene. They are used as a way to name and order joints in vectorized data. For example:
Note that in both example primitives above, the given paths do not reference any real primitives. Also note that each primitive has its own joint ordering, and that those orders need not be identical.
The purpose of encoding orderings in this manner is to allow for the creation of self-contained assets. For example, it is possible to construct a skel animation primitive independently from any skeleton definitions, which will remain valid even if a skeleton that the animation is mapped to has a slightly different ordering.
Part of the motivation for this is to allow for more robust, composed assets. When orderings are required to remain fixed, maintenance difficulties may arise. For example, if animation assets are produced through a process independent from the process that produces a shared definition of a skeleton asset, data can easily get out of sync.
On top of allowing for more robust assets, these flexible joint orderings allow for additional features like sparse authoring of animation data.
Data can be remapped from one joint ordering to another using a UsdSkelAnimMapper, and is done so by matching joint names.
The UsdSkelRoot schema is used to encapsulate primitives with skeletal skinning behaviors, and is required when authoring skeletal data.
A SkelRoot is a boundable primitive, and provides a place to encode the extents of all skinned primitives beneath it. It is neither expected nor required the descendent primitives that are being skinned will encode their own skinned extents. These extents are made available to renders to enable operations like out-of-camera culling, without requiring that skinned geometry be computed first.
A SkelRoot additionally gives DCC apps a way of identifying which parts of a scene graph require skeletal processing, so that they can take different code paths, as is often required to consume skeletal data.
It is possible to override a primitive's composed type in USD, in order to either enable or disable skeletal processing. For example:
The UsdSkelSkeleton schema describes a skeleton. This schema is responsible both for establishing the topology of a skeleton, as well as for identifying a bind pose.
A skeleton itself provides only structure. Meshes are posed with a skeleton by way of skeletal bindings inside of the "geometry hierarchy" of primitives.
The joints of a Skeleton are defined by the joints attribute. This attribute encodes joint paths – each of which is a valid SdfPath strings. The parent-child relationships that these paths describe are used to define the parent-child relationships of the joints themselves.
For example:
In the above example, there are four joints:
A: A root joint (no parent)A/B: A joint parented beneath joint AC: Another root joint (no parent). There can be any number of root joints.C/D/E: A joint which, since C/D has not been defined, will be parented beneath joint C.In addition to providing parent-child relationships, this token array also establishes the joint order of the skeleton.
The targets of the joints attribute are required to be authored such that all parent joints come before any of their children in the array. This is a requirement because it simplifies some computations without necessitating the creation of additional data structures. This ordering can most easily be achieved by simply sorting the array.
It's possible to test whether or not an array of joint paths defines a valid topology as follows:
Schema describing skeletal animation, within which joint animations and blend shapes are stored in a vecorized form. A SkelAnimation encodes the animation of a Skeleton as a separate primitive to facilitate instancing workflows.
Both as a storage optimization and to allow for value interpolation that preserves transform orthogonality, joint transforms are encoded as separate translate, rotate and scale components, given in joint local space. Transforms are constructed from components using an order of scale-rotate-translate. Client code may make use of the transform composition utilities for converting transforms to and from this component form.
Joint data is stored in arrays, using the joint order specified by the joints attribute. This ordering may be different from the Skeletons that the animation maps to, and may also only identify a sparse subset of the joints in a skeleton. When an animation provides sparse data, fallback values are taken from the rest pose on the UsdSkelSkeleton primitive to which they apply.
An animation source is only valid if its translation, rotation, and scale components are all authored, storing arrays size to the same size as the authored joints array. The effect of a skel animation prim may also be directly nullified by either deactivating the primitive, or by blocking the component attributes.
In addition to providing joint animations, a SkelAnimation may also provide blend shape weight animations. Blend shape weights are specified in a vectorized form using the blendShapeWeights attribute. The blendShapes attribute holds a token array which, for every element authored in blendShapeWeights, identifies which blend shape each weight value applies to.
The point of this encoding is to decouple the blendshape weight animation from the description of how that animation maps to different skinnable shapes. Refer to the BindingAPI: Binding Blend Shapes 'BindingAPI: Blend Shapes' documentation for information on how these weights are mapped to skinnable primitives.
A SkelAnimation is made to affect a Skeleton by binding the animation to the skeleton, using the skel:animationSource property of UsdSkelBindingAPI schema. For example:
When a skel:animationSource property is defined, it is "inherited" down namespace, onto any Skeleton primitive beneath it. So in the above example, </Model/Skel> will be affected by the SkelAnimation at </Model/Anim>.
There are a couple reasons why data is separated in this manner:
Schema describing a single blend shape.
Blend shapes specify a target shape, in terms of point offsets, for a point-based primitive. The offsets may hold a direct correspondance with the points of the point-based primitive to which they are meant to apply, or the points may specify a shape using a sparse subset of points. The mapping in the latter case is described using the pointIndices attribute. If a blend shape defines any in-between shapes, the same pointIndices* mapping additionaly applies to all in-between shapes.
When a mesh is skinned in UsdSkel, blend shape application precedes the effect of skinning using joint transformations. If no in-between shapes are defined, a set of blend shapes is applied against an input point as follows:
In other words, the position offsets are multiplied against the corresponding weight value, and added against the input positions. Note that the above equation only describes application of blend shapes in the absence of in-between shapes.
Each blend shape prim may define in-between shapes, which specify explicit corrective shape to apply when the blend shape is resolved at different weights.
For example, suppose that a blend shape defines a 'smile' shape. At a weight of 1, we apply the full effect of the shape. At in-between weight values, we linearly interpolate the shape back towards a neutral pose to derive an in-between shape. Using in-betweens, we can instead specify those in-between shapes explicitly. So instead of linearly interpolating to derive a 'half-smile' pose, we might provide an explicit shape corresponding to a weight of 0.5.
Scene description for the scenario described above is as follows:
While an equivalent animation could be defined without in-betweens by animating the weights of two blend shapes separately, using in-betweens provides a more convenient encoding that allows us to decouple the specification of in-betweens from the primary weight animation.
In-between shapes can be created via UsdSkelBlend::CreateInbetween(), and are manipulated and introspected using UsdSkelInbetweenShape objects. The in-between shapes themselves are encoded as attributes in the 'inbetweens:' namespace, using the 'weight' metadata field to indicate the weight value at which they apply.
The in-between shapes corresponding to weight = 0 and weight = 1 are implicitly defined on a blend shape. The former defines the null shape, for which all shape offset are zero, while the latter is the shape given by the offsets property of the UsdSkelBlendShape primitive. Because the existence of those in-betweens is implied, it is considered an authoring error for any in-betweens to specify a weight of 0 or 1.
We must also consider the behavior when multiple in-betweens have matching weight values. Although it would be possible to adopt an interpretation that allows all shapes to have meaning – such as averaging across all shapes – this introduces some ambiguity for interchange: If the in-betweens are averaged together, it's unclear what the name should be given to the resulting shape. To avoid this ambiguous interpretation, in addition to treating in-betweens with a weight value of 0 or 1 as an authoring error, it is also considered an error for any two in-betweens of a blend shape to have the same weight value. Typical run-time behavior for these malformed in-betweens is to produce an error, but continue to process the blend shape while ignoring the malformed in-betweens.
If we consider only the set of valid in-betweens, a set of blend shapes containing in-betweens are applied to a point as follows:
For example, given in-between weights of [0.25, 0.5], if a blendshape weight is greater than 0.5, we interpolate between the shape at 0.5 and the implicit primary shape (weight = 1), adding the result to the base shape. If the weight is in the [0.25,0.5] interval, we interpolate between the shapes at 0.25 and 0.5, adding the result to the base shape. Finally, if the weight is less than 0.25, we interpolate between the implicit null shape (weight = 0) and the shape at 0.25. Note also that this interpolation is unbounded, so if the input weight is, say, -0.25, then we have:
So at a weight value of -0.25, the shape at 0.25 is applied with a weight of -1.
The UsdSkelBindingAPI schema provides a means of binding a Skeleton into a geometry hierarchy. This schema is also responsible for defining joint influences, as well as an optional geomBindTransform property to define the space for skinning.
Since a UsdSkelRoot primitive provides encapsulation of skeletal data, bindings defined on any primitives that do not have a UsdSkelRoot primitive as one of their ancestors have no meaning, and should be ignored.
Meshes are skinned based on skinning transforms, computed from a UsdSkelSkeletonQuery, which is bound hierarchically using the skel:skeleton relationship. The transforms for a bound skeleton are driven from an animation source, such as a skel animation, which is also bound hierarchically, via the skel:animationSource binding relationship.
By defining bindings hierarchically, it is possible to define models that each have unique skeletal animations, while also being instanced. For example:
Each skel:skeleton binding may be thought of as identifying a Skeleton Instance, animated according to whichever inherited skel:animationSource is defined at the location at which the skel:skeleton is bound.
Although skel:animationSource bindings are inherited, they only apply when a skel:skeleton is resolved. For example:
The above example includes two separate skeleton instances, with the same animation being applied to each instance. Contrast that with the following case:
In the above example, there is only a single skeleton instance (at </A>), which has no animation, because a skel:animationSource binding is not considered to have any effect until the next skel:skeleton binding at or beneath the binding location in namespace.
One final example may clarify behavior:
In this case, there are two skeleton instances: One at </A>, referencing the animation at </Anim1>, and one at </A/C>, referencing the animation at </Anim2>. The skel:animationSource relationship at </A/B> has no effect.
Joint influences for skinning points are defined by setting the primvars:skel:jointIndices and primvars:skel:jointWeights primvars, using the UsdSkelBindingAPI. The jointIndices primvar provides an array giving the joint index of an influence, while the jointWeights primvar provides a weight value corresponding to each of those indices.
Above, since </MeshA> does not specify a skel:joints ordering of its own, the joint indices refer to the ordering of the bound Skeleton, and so the joint indices refer to joints A/B and A, respectively. However, </MeshB> specifies a skel:joints property that gives an alternate joint ordering. Using that, the indices of </MeshB> refer to joints A/B/C and A/B, in that order.
In the common case, the joint influence primvars are configured with vertex interpolation, and define a fixed number of contiguous influences per point. The primvar's elementSize defines the fixed number of influences for each point.
elementSize of the primvar. I.e.,
If a point has fewer influences than are needed for other points, the unused array elements of that point should be filled with 0, both for joint indices and for weights.
See UsdGeomPrimvar for more information on the meaning of primvar interpolation and elementSize.
No restrictions are placed on the elementSize when defining joint influences. Because such a limit may vary among different DCC applications, we feel that it is inappropriate to hard-code any such restrictions at the file storage level. Instead, for clients that require a strict limit on the number of influences per point, UsdSkel provides helper method, UsdSkelResizeInfluences, that can be used to enforce such limits within each client, as necessary.
See the skinning coding examples for an example of how resolved joint influences may be read.
The interpolation, element size and the size of the stored arrays must be consistent across both the joindIndices and jointWeights primvars.
Aside from vertex interpolation, the only other valid primvar interpolation for joint influences is constant interpolation, which is used in defining rigid deformations. It is considered an error to apply any other type of interpolation.
In addition to defining influences that vary per point, joint influences may also be defined with constant interpolation. In this form, the jointIndices and jointWeights arrays define a set of influences that apply the same way to all points. Constant joint influences are supported both as a form of storage optimization, as well as for the sake of encoding rigid deformations. By identifying rigid deformations, some clients may be able to retain instancing properties that would otherwise be lost by skinning, by affecting only the transform of an instance rather than the deforming points of that instance.
For example:
In the example above, </MeshA> can be seen as being ridigly constrained to infuence 0. </MeshB> is rigidly deformed, but it's also important to note that even though it is rigidly deformed, it is still influenced by more than one joint.
It is not expected that all applications know how to deal with rigid deformations. For applications that don't understand them, UsdSkel provides the helper methods UsdSkelSkinningQuery::ComputeVaryingJointInfluences and UsdSkelExpandConstantInfluencesToVarying, which both provide a means of querying rigid deformations as if they had been encoded with per-point influences.
To ensure correct skinning, joint influences should be normalized when they are written using the UsdSkelBindingAPI. UsdSkel does not automatically normalize weights when reading data. Clients should instead use the UsdSkelNormalizeWeights helper method to normalize weights prior to storing them.
Additionally, as part of our best practices, we suggest that joint influences be sorted from largest weight value to smallest. The UsdSkelSortInfluences utility can be used to do so. UsdSkel does not currently require sorted joint influences, but may later add such restrictions, since they enable some skinning optimizations, such as allowing a skinning kernel to exit from a point-deformation loop early.
For skinning to apply correctly, the constant primvars:skel:geomBindTransform primvar should be authored on each skinnable primitive, using the UsdSkelBindingAPI. This primvar stores the world space transform of a skinned primitive at bind time. When skinning is applied, points of each skinnable primitive are trasnformed by their geomBindTransform into bind space, after which joint transforms are applied, placing the result in either skeleton space or world space.
When a primitive is skinned, any transform on the prim authored by way of the typical UsdGeomXformable schema has no effect on the rendered results. Skinned geometry primitives are rendered in skeleton space, rather than being transformed back into local gprim space. This avoids double-transform problems that can otherwise occur, for example, if an ancestor of both a skinnable primitive and its driving Skeleton is moved.
UsdSkel decouples the encoding of blend shape animations from the description of how those animations map to skinnable primitives. This has numerous advantages, such as allowing the set of primitives skinned by a particular Skeleton Instance to be swapped out – via variants or other composition features – with a different set of skinnable primitives, holding different blend shapes. This decoupling also means that an existing blend shape animation is not necessarily invalidated by asset changes that alter the set of shapes defined for a character.
Given a skinnable primitive, the skel:blendShapes and skel:blendShapeTargets properties of the UsdSkelBindingAPI specify a mapping from the animation of the bound Skeleton Instance to different skinnable primitives.
Note that because blend shapes are usually tightly coupled with individual geometric primitives, whereas other properties of UsdSkelBindingAPI are hierarchically inherited, these blendshape-related properties are not inherited from ancestor primitives in namespace, and are relevant only when specified directly on primitives.
To bind blend shapes, the skel:blendShapeTargets rel should be created, and set to target UsdSkelBlendShape primitives, which define each shape. Additionally, the skel:blendShapes array should also be defined, providing a unique token per bound blend shape. For example:
In the above example, note that the </Anim> primitive specifies more blend shapes than are actually used on </Mesh>. This is because an animation source is providing animation data for a complete model (I.e., multiple geometric primitives). The skel:blendShape attribute, as defined on </Mesh>, provides a mapping between the blendShapeWeights animation data on </Anim> – which holds weight animation for multiple geometric primitives. The set of blend shape targets specified by the skel:blendShapeTargets rel identifies, for each mapped blend shape token in skel:blendShapes, the target UsdSkelBlendShape primitive to which animation data should apply.
From the example above, shape </Mesh/Foo> is using shapes `['B', 'A']` (in that order). First we map the blendShapeWeights from the ordering specified by </Anim.blendShapes> to the ordering on </Mesh.blendShapes>, which gives local weight values of [0.75,1.0]. For each token in </Mesh.blendShapes>, the </Mesh.blendShapeTargets> rel identifies which UsdSkelBlendShape primitive that token maps to. So B = </Mesh/Foo> and A = </Mesh/Bar>. From this, we find that we our final blend shapes and weights, as pairs, are [(</Mesh/Foo>, 0.75), (</Mesh/Bar>, 1.0).
1.8.5