**Skinning Schemas for USD**
This document motivates the desire for, and scope of, core support for
skeletal deformation and animation blending in USD, and proposes a set of
cooperating schemas to encode and decode such data in keeping with the goals
of the USD project. This document does not explore Blend Shape animation, as
schema ideas are still in development, and we think skinning is useful enough to
deploy on its own, in advance; we are confident that including support for
blend shapes will be purely additive with respect to encoding, thus not
invalidating any of the encodings proposed herein.
# Goals, Requirements, and Philosophy
While the initial goal of introducing skinning schemas into
USD is to provide a robust means of interchange for clients like game studios
that have pipelines whose terminus is real-time-engine authoring environments
such as UE4 and Unity, an equally important goal for film/VFX studios is
building performant, scalable crowds systems that feed undeformed geometry
and blended skeletal animation directly to preview and final-quality render
engines from USD, pulling from and sequencing libraries of animation. From
these goals, we derive the following requirements:
- We must provide clear authoring APIs that are not completely foreign to
the way that common authoring environments think about the relevant data.
- We must provide an encoding that is amenable to publishing modular,
reusable pieces, separating structural data from animation.
- We must allow for animations authored to a particular armature to be blended
with animations authored for different but "compatible" armatures, and
use the result with geometries bound to yet another armature.
- We must provide an encoding that is weighted towards efficient UsdStage
data extraction, should that come into conflict with sparse overridability
or one-to-one matching with popular authoring environments' data models.
- We must provide efficient GPU and CPU computation functions for the
behaviors. Concretely, this means
[UsdImaging](https://github.com/PixarAnimationStudios/USD/tree/master/pxr/usdImaging/lib/usdImaging)
will model (and efficiently extract) the data from which both GPU and CPU
based Hydra "backend renderers" will be able to generate posed geometry
from compact, schema-derived data.
## Primary Concern: Scope of the Schema
Our primary concern about adding such features to core USD is that their
inclusion not signal an intent of the USD core to support general rigging and
execution behaviors, as we strongly believe such features are beyond the
scope of USD's core concerns.
To that end, we adopt a name for the new schema module and classes that
signal a limited scope: **UsdSkel**, as the techniques it provides are
geared towards application of skeletal animation.
# Concepts and Definitions
We think it important to decompose the encoding into four primary related
concepts, to cleanly separate the concerns of:
- **Skeletal Topologies** - A *Skeleton* encapsulates the topological aspects
of skinning, so that they can be re-used and provide a basis for
identifying compatible animations. Skeletons define their topology as a
(ordered) hierarchy of *Joint* prims.
- **Geometry Binding**
- Encodes a binding of a geometry hierarchy to a *Skeleton*, and a
specification of how to deform the geometry to a posed *Skeleton*.
- Uses [primvar data](http://openusd.org/docs/api/class_usd_geom_primvar.html#details)
on gprims to specify:
- the subset of skeletal joints that drive each gprim
- the joint weights for each vertex in each gprim
- the geometry-to-skeleton binding transformation
- Encodes a binding of geometry to an *Animation Source* that will pose
the bound *Skeleton* over time.
- **Animations** - An *Animation* is an "atomic" *Animation Source* that
provides time-varying overrides for *Joints* and "the roots" of geometries
bound to a *Skeleton*. Animations must also be encapsulatable so that they
can be referenced and reused, multiply-applied, and sequenced.
- **Animation Blenders** that organize, weight, and blend multiple
*Animation Sources* together, so that they can be layered, and/or
sequenced/transitioned.
We also define **skeleton space** as the "object space of an animated
skeleton". Skeleton space does not contain the transformation that positions
the root of the *Skeleton* (and bound geometry) in the world, and therefore
serves as the canonical space in which to apply blended joint animations.
Whereas, for example, Maya APIs assume World space for many computed
quantities, we require data to be encoded in no "higher" than skeleton space,
so that animations will not break if transformations are altered on ancestor
prims of the skinned geometry.
# Primary Schemas - High-level Description
## UsdSkelJoint
At the outset, the *Joint* schema will be nothing more than an identified type
on which we may key imaging behaviors in certain display modes
(e.g. connecting joints with bones for fast animation preview). In the
future, *Joint* may provide a container for specifying end-effector locators
that deform along with animated geometry.
## UsdSkelSkeleton
A *Skeleton* defines an ordered, hierarchical topology of joints
- A *Skeleton* is modular, amenable to referencing and instancing, and provides
the target onto which *Animations* (which may move only a subset of the
*Skeleton*'s Joints) are mapped.
- A *Skeleton* holds the "rest pose" that provides a "fallback" for any *Joint*
that is not animated, and is useful to some blending algorithms, and of
course also provides one half of the binding transformation. The rest pose
is recorded in **joint local space**.
- The rest pose is vectorized according to the defined ordering of its
*Joints*, not defined by transformations on the *Joint* prims.
## UsdSkelAnimation, UsdSkelPackedJointAnimation
An *Animation* provides time-varying opinions for skeletal joints. The
skinning schemas support attaching both "simple", single animations, and
arbitrary combinations of linearly-combined ("blended") animations.
Initially, we will support a single encoding of each type. For simple
animations, we provide the *PackedJointAnimation* schema, with the following
properties:
- A *PackedJointAnimation* names (via a [UsdRelationship](http://openusd.org/docs/USD-Glossary.html#USDGlossary-Relationship)) and orders the *Joints* for
which it encodes joint-local animation. This allows
*PackedJointAnimations* to be encoded independently of any *Skeleton*, as
its data can be mapped to any *Skeleton* that possesses *Joints* whose
paths (i.e. hierarchical names) match those encoded by the
*PackedJointAnimation*.
- A *PackedJointAnimation* consists of:
- Joint transformations that describe the positioning of each joint in
**joint local space**.
- A "root transformation" that affinely moves all bound geometry. It is
encoded using the regular
[UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details)
schema which implies that *PackedJointAnimation* **IsA** UsdGeomXformable.
- The joint animation uses a **vectorized encoding: translation, rotation,
and scale vectors** that aggregate all affected *Joints*' values, just as a
*Skeleton*'s rest-pose vectorizes *Joint* transformations.
## UsdSkelAnimationBlender
An *AnimationBlender* is a type of *UsdSkelAnimation* that provides a concrete
recipe for weighted blending of other *UsdSkelAnimation* prims. Because an
*AnimationBlender* IsA *Animation*, it can blend not just atomic
*PackedJointAnimation* prims, but also other *AnimationBlenders*, which
means we can create *blend trees*, so that we can, for example, easily
describe and edit transitions between groups of layered animations, similarly
to [Unity's implementation](https://docs.unity3d.com/Manual/class-BlendTree.html).
- We propose initially a single concrete *AnimationBlender* type:
**UsdSkelLinearBlender**, which produces a linear combination of the input
animation data (SLERP for quaternion rotations).
- In the future, this abstraction gives us the opportunity to deploy other
*UsdSkelAnimationBlender* schemas with different blending algorithms, such
as [dual
quaternions](https://www.scss.tcd.ie/publications/tech-reports/reports.06/TCD-CS-2006-46.pdf)
for transformation blending.
## UsdSkelBindingAPI
The *BindingAPI* schema encodes all of the required data to bind geometry to
a *Skeleton* and to an animation-providing *SkeletonBlender*. Unlike the
other, [typed, "IsA"
schemas](http://openusd.org/docs/USD-Glossary.html#USDGlossary-IsASchema) in
UsdSkel, binding is presented as an [API
schema](http://openusd.org/docs/USD-Glossary.html#USDGlossary-APISchema)
because its purpose is to **add data onto geometry prims** from the UsdGeom
family of typed schemas, i.e. Xforms, Meshes, Curves, NurbsPatches, etc. Its
primary concerns are encoding:
- The *skinning method* that applies *Skeleton*-mapped animation to deform
geometry, which initially is limited to [Linear Blend
Skinning](http://skinning.org/direct-methods.pdf), and therefore not
explicitly encoded as data in a scene.
- The binding of geometry to an *Animation*. Like Material bindings in the
[UsdShadeMaterial](http://openusd.org/docs/api/class_usd_shade_material.html#details)
schema, this binding relationship is meant to apply hierarchically to
animate all the geometry beneath the bound prim, and we intend that binding
typically happens at the "model level" or other suitable aggregation point,
not at the individual gprim level. Unlike hierarchical Material binding, we
propose that in the interests of simplicity and avoiding the collapse to a
general rigging system, **the highest *skel:animationSource* binding in
namespace that affects a given prim is the only binding that will affect
it.**
- The binding of geometry to a *Skeleton*. This relationship, also, is meant
to apply hierarchically, and a *skel:skeleton* binding must be present on
the same composed prim as the *skel:animationSource* binding in order to
encode a fully-skinned geometry hierarchy. It is this related *Skeleton* onto
which the *SkeletonBlender*'s animation will be mapped.
- The mapping and weighting of gprim vertices to *Joints*. This can be
encoded very efficiently in terms of the *Joint* ordering provided by the
bound *Skeleton* affecting each gprim, or joints can dynamically be named and
ordered uniquely on any gprim, which will serve as an override to the
joint ordering provided by the bound Skeleton, providing an extra level of
indirection, at some decoding cost.
- The "geometry binding" transformation used to fit each gprim to its bound
*Skeleton*.
# How the Schemas Fit Together
Following is a diagram that shows how the concepts described above map to
Prims, Properties, and namespace hierarchy for a simple application of
two blended skeletal animations. The colored backdrops suggest useful
encapsulation of separable concerns to promote reuse and efficient encoding;
for example, each of the three sections may be encoded in different
files/layers, and composed together using [USD composition arcs](http://openusd.org/docs/USD-Glossary.html#USDGlossary-CompositionArcs) .
Note, however, that in addition, each *Animation* may typically (in a VFX
pipeline) be encapsulated and published into libraries such that they can be
individually referenced into a blended animation as needed.
![Diagram that shows how the related schemas apply to skin a character
model.](http://openusd.org/images/UsdSkelPrimLayout.png)
## Explanation
### Brief summary
The diagram represents a prim hierarchy comprised of Xforms, Meshes and
other prims that define the visible meshes to be modified by the
LinearBlender.
Beginning with the colored section on the left, we have a "geometry
hierarchy" for a model named Bob. In addition to Materials (which are not a
concern for this document), Bob consists primarily of a number of Meshes and
organizing Xform prims. Of primary concern is that this group of prims
contains the data we ascribe to the **UsdSkelBindingAPI**. On Bob's root
prim, we find the relationships that bind Bob to a *Skeleton*
(i.e. *skel:skeleton*), and to an *Animation* (i.e. *skel:AnimationSource*),
which in this case, is a *LinearBlender*. On each of the
*BodyMesh*, *Eye* and *Tooth* prims, we see the three primvars that provide
the mapping and weighting of each Mesh's points to the bound *Skeleton* and
its *Joints*.
In the center section, we see the definition of the *Skeleton* to which Bob
is bound, *Human_17a*. The prim structure of *Joints* underneath a
*Skeleton* prim provides the inheritance structure for the "skeleton
package", since the *joints* relationship on the *Skeleton* prim itself only
provides joint linearization ordering (needed by *restTransforms* and
*PackedJointAnimation*), **not** parenting information. A *Skeleton* can be
described in the same file as a geometry hierarchy that is bound to it, but
in a VFX crowds pipeline, it may be published as an independent asset that is
referenced and instanced into geometry assets like Bob.
In the rightmost section we see a *LinearBlender* that blends between two
animations, *WalkCycle* and *RunCycle*. For a *Blender* that is blending
between running and walking, we would tupically expect the per-animation
*weights* and *rootWeights* properties on the *LinearBlender* to be "Joint
uniform", i.e. *weights* will be an array of length 1 meaning the weight
applies equally to all *Joint* animation in the *source*, but animated over
time, as we transition between running and walking. We briefly discuss
details of how cycled (and other) *Animations* can be sequenced, in the note on
"Animation Retiming" in the schema description of
[UsdSkelPackedJointAnimation](#usdskelpackedjointanimation).
### Organizing Principles, Encoding Decisions
Skinning data is organized according to two principles:
1. **Data stored in modular, reusable, instanceable parts:**
a. *Skeleton* "rigs" can be defined and published, to be referenced in
whenever needed. Encapsulating skeletons this way is not a requirement
for sending data from Maya directly to UE4 through USD (and in such
contexts *Skeletons* can be defined "inline"), but facilitates
scalability and maintainability when deploying directly-rendered,
skinned crowds at scale in VFX pipelines.
b. *Animations* can be published individually and referenced back into
scenes. This not only allows re-use, but also enables sequencing of
animations as [Value
Clips](http://openusd.org/docs/USD-Glossary.html#USDGlossary-ValueClips)
c. An *AnimationBlender* can encapsulate the animation that it blends,
further facilitating "animation packages" that can be
referenced in and scattered onto actors already in a scene.
d. Separating the binding of *Skeleton* and *AnimationBlender* to geometry
further aids the pattern described in (c), because we can publish
assets that already contain a "bound *Skeleton*" and *jointIndices* on
its gprims that refer to that *Skeleton*'s *Joints*, and thus the asset
is "skinning ready". After referencing in such assets into a scene, all
that remains is to bind context-specific *Animations* to them.
2. **Scalability and Performance:**
a. All posing and animation data is vectorized. This encoding is concise
and ideal for scalability and speed of data consumption in the USD
APIs. A vectorized encoding does not facilitate sparse, per-joint
overrides of animation data using the current USD facilties for sparse
overriding. If this becomes an issue in the future, there are paths
we could pursue to provide this affordance. But for initial
deployment, we believe the need for scalability and performance far
outweighs the need for per-joint overrides. Since the USD APIs do
not, at present, allow reading "slices" of arrays, this encoding also
requires us to read in a complete animation sample even if only part
of the animation applies to the target *Skeleton*; given the
performance characteristics of lazy data access in USD, we feel this
is the proper tradeoff.
The decision to vectorize a *Skeleton* rest-pose not only
significantly enhances de-serialization performance, but also allows
*Skeletons* to be
[instanced](http://openusd.org/docs/USD-Glossary.html#USDGlossary-Instancing)
for encoding scalability in USD, while still allowing the "binding" of
a *Skeleton* to geometry to override the rest pose as part of the
fitting process.
b. For skinning, we require just three spatially interpolated primvars per
mesh: *jointIndices*, *jointWeights*, and *geomBindXform*. We also
allow the *joints* indexing to be overridden per-mesh, but this is
optional and hopefully rarely necessary.
# Primary Schemas - Concrete Definitions
## UsdSkelJoint
The *Joint* schema currently has no properties - its purpose is merely
declarative as defining the joints of a skeletal hierarchy.
## UsdSkelSkeleton
A *UsdSkelSkeleton* is a typed schema that derives from
[UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details). It
parents a hierarchy of *UsdSkelJoints*. The "first joint" of a *Skeleton*
should be a child *Joint* prim of the *Skeleton* prim. A *UsdSkelSkeleton*
contains the following data:
- **rel joints** - a relationship targeting all of the descendant Joint
prims in a stable ordering, with the requirement that "parent" joints must
appear earlier than "child" joints in the order. It is the ordering of
joints in this relationship that determines the layout of the
*restTransforms* property, and to which *primvars:skel:jointIndices*
refers, should no *skel:joints* relationship be authored on a gprim.
- **matrix4d[] restTransforms** - is a matrix4d array that specifies, in the
ordering imposed by joints, the rest transformations of each joint.
## UsdSkelPackedJointAnimation
*UsdSkelAnimation*, the base for both *UsdSkelPackedJointAnimation* and
*UsdSkelAnimationBlender* derives from
[UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details)
so that it can provide root-level animation, as well as joint animation.
*UsdSkelPackedJointAnimation*, the concrete *Animation* schema, encapsulates
all of the data required to animate an enumerated, ordered set of *Joints*.
- **rel joints** : relationship that names Joints, by path **rooted at the
*Animation* prim, as if it were a *Skeleton* prim.** Relationship paths in
USD can be resolved independently of whether they target an actual concrete
object. We therefore can map *PackedJointAnimation* joints onto a target
*Skeleton* by resolving the targets, and replacing the
*PackedJointAnimation* prim's path prefix with that of the target
*Skeleton*, this relationship serves two roles:
- It allows an *Animation* to be **sparse**, affecting only a subset of
the target *Skeleton*'s joints. A *PackedJointAnimation* can provide
values for only a small number of joints in a complex skeleton by
targeting only those joints in its *joints* relationship; the
vectorized animation will then be laid out according to the ordering of
those relationship targets. This can be valuable as a data frugality
technique not only for files, but to facilitate client engines in
building minimal "joint palettes" for processing. It can also
facilitate creating "layerable" animations that affect only parts of a
model.
- It provides a level of protection against trivial changes to a
published *Skeleton*. For example, an animation with properly encoded
*joints* will not "fall off" if the referenced *Skeleton* is modified
to insert a *Joint* at the end of some chain, because we can compare
the *Joint* paths encoded in the *PackedJointAnimation* to the *Joint*
paths in the *Skeleton*, at runtime.
- **float3[] translations** : 32-bit joint-local translations of all affected
joints. Length of the array should match the size of the *joints*
relationship. The USD API will perform linear interpolation between
samples.
- **quath[] rotations** : 16-bit joint-local **unit quaternion** rotations of
all affected joints. Length of the array should match the size of the
*joints* relationship. The USD API will perform SLERPing between samples.
- **half3[] scales** : 16-bit joint-local scaling of all affected joints.
Length of the array should match the size of the *joints* relationship.
The USD API will perform linear interpolation between samples.
**Notes**
- **Root transform animation.** Each *PackedJointAnimation* can also provide
[UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details)
data to define a root transformation. This allows *PackedJointAnimations*
to express a positioning of the skinned geometry "as a whole."
- **Joint transform ops.** The decomposition of animated Joint
transformations into translations, rotations, and scales does limit the
kinds of animations we can encode (e.g. no shears). However, it provides
two key advantages:
- Data economy, as the three separate ops consume less space than a
single matrix4f even if all three are in 32-bit floating point, and we
further advocate that the rotations and scales can suffice with 16-bit
half floating-point precision.
- Interpolating the independent op quantities (as done by native USD
value resolution) yields far better results than
interpolating matrices. This can be of value for any blending schemes
that can operate on the raw data.
- **Animation retiming.** How do we want to control animation timing for
purposes of blending? Given that each contributing animation is a unique
prim, referenced *Animations* can be retimed affinely with a
[SdfLayerOffset](http://openusd.org/docs/USD-Glossary.html#USDGlossary-LayerOffset),
or if more sophisticated retiming is necessary (e.g. in order to align a
walk with a run such that foot plants occur simultaneously across all
blends), one can additionally apply a sequence of [Value
Clips](http://openusd.org/docs/USD-Glossary.html#USDGlossary-ValueClips),
and use the built-in retiming capabilities of that feature. If such an
encoding proves too awkward, we can instead add explicit timing curves,
per-animation, as part of the *LinearBlender* schema, or possibly as a
common feature for all *Animation* types.
## UsdSkelLinearBlender
Note we are only defining the concrete schema *UsdSkelLinearBlender* since we
have not yet identified any properties that may be common to other, eventual
*AnimationBlender* implementations. The abstract base schema class will serve
primarily as an interface class for defining computations that clients would
require from any implementation.
A *UsdSkelLinearBlender* contains properties created dynamically in namespaced
groups, in the **anim:** namespace. In the following, each property would
appear in each of many namespaces, such as **anim:walkAnim:**,
**anim:runAnim:**, for example. The content of the "inner" namespace, in these
examples, is entirely user-controlled, specifiable at the time of authoring
the *LinearBlender* data.
- **rel source** : relationship that targets an *Animation* prim.
- **float[] weights** : specifies an overall scaling of the joint animation
to be applied when computing the blended aggregate animation. *weights* can
either be of length one, in which case a constant weighting is applied to
all Joint animation in the target *Animation*, or it can be of the same
size as the target *Animation*s *joints* relationship, in which case the
*weights* are **per joint**. For each *Joint* of the target *Skeleton*, we
will normalize the weight contributions of each *Animation*, and if the sum
is less than one, fill in the remainder from the *Skeleton*'s
*restTransforms*.
- **float rootWeight** : specifies a scaling to be applied to the
*Animation*'s contribution to the root transformation, which is encoded
separately from the joint animations. Typically, if there are multiple,
layered animations, we will want the root transformation of only the "base"
animation, so this weighting would be zero for all of the secondary, layered
animations.
**Notes**
- **Skinning root transformation.** After all Joint-based posing has been
applied to a gprim, bringing it into *skeleton space*, we then apply the
root transformation computed by the *LinearBlender* as the weighted sum of
*Animation* root transforms at the evaluation time. Finally, the
local-to-world transformation of the bound/skinned prim is applied
(i.e. the highest prim in namespace to which a *LinearBlender* (or
*PackedJointAnimation) and *Skeleton* have been bound).
- **Evaluating a Blended Animation.** When asked to sample, weight, and blend
its *Animations* at a given time, a *LinearBlender* requires as an input
the *Skeleton* onto which it should map the resulting animation. This
allows it to arrange the Joint data into the vectorized ordering in which
it will be consumed by the bound geometry.
## UsdSkelBindingAPI
To make use of skeletal skinning, a geometry hierarchy must encode several
properties on one or more prims to:
- bind *Animations*
- bind *Skeletons*
- position gprims in skeleton-space, via a mesh-binding (gprim-binding)
transformation
- attach and weight gprims to *Joints*
The *UsdSkelBindingAPI* presents a single interface that can be applied to
any geometry prim to encode and interrogate the above properties.
- **rel skel:animationSource** : relationship that binds a
*UsdSkelAnimation*. All descendant prims of the "binding prim" will be
animated by the bound *Animation*, provided they are mapped and weighted to
a *Skeleton*. If multiple *skel:animationSource* relationships are authored
in a namespace hierarchy, **the highest such binding wins** (i.e. the one
with the shortest path).
- **rel skel:skeleton** : relationship that binds a *UsdSkelSkeleton* and its
assocatiated *Joint* hierarchy to a geometry prim. For a geometry
hierarchy to be succesfully skinned, both *skel:skeleton* and
*skel:animationSource* must be bound on the same composed prim (although in USD
composition terms, the two bindings can be expressed in different "layers",
and we expect that to commonly be the case in VFX pipelines). To
facilitate creation of "skinned aggregates" from smaller, "skinnable
components", we allow *skel:skeleton* bindings to be expressed on
child/descendant prims of "the binding prim", with the interpretation that
we will use the "nested Skeleton" as a way to remap Joint bindings on the
affected geometry to the *Skeleton* of the top-level "binding prim."
- **int[] primvars:skel:jointIndices** and **int[]
primvars:skel:jointWeights** : For each gprim to be skinned (Meshes, but
also Curves, Patches, etc) we must be able to compute the number of joints
that affect its geometry, and weights for each of those joints,
**per-vertex**. Both quantities can be encoded *directly* on each gprim
using the *primvars:skel:jointIndices* and *primvars:skel:jointWeights*
primvars. The number of weighted joints must be the same for all points in
each primitive, and is encoded directly as *elementSize* metadata on the
two primvars - the length of each as well as their interpolation and
elementSize, must match.
Because we cast these data as
[UsdGeomPrimvars](http://openusd.org/docs/api/class_usd_geom_primvar.html#details),
we can powerfully compress the data encoding of rigidly deforming gprims by
using a constant interpolation with a single set of weights applied
uniformly to all the gprim's points. For example, in the diagram above, all
of the "teeth" meshes could have a constant-interpolation "float[]
primvars:skel:jointWeights = [1.0]" primvar, also weighting just a single
jaw joint, as the teeth move rigidly with the jaw.
We can be even more succinct in expression of weighting by establishing a
joint weighting at an ancestor prim of leaf gprims, with the expected
consumer behavior that the two joint primvars *will inherit down
namespace.* Note this means that in accordance with
typical attribute inheritance rules, any gprim leaf is able to overide the
inherited weighting and establish its own. This is a useful behavior,
because we might author weights on "interior prims" not only for purposes
of concise expression of rigid binding, but also because we might have
spatial data *on the interior prims* to which we wish to apply the skinning
algorithm via provided APIs, such as constraint targets that must pose
with the Skeleton and animation.
Note that uniform and faceVarying interpolation do not make sense for joint
indices and weights.
- **matrix4d primvars:skel:geomBindTransform** : Encodes, for each gprim to be
skinned, the **inverse of the transformation** that positions the gprim
into the "skeleton space" in which it is bound to a *Skeleton*. If
unauthored and no value inherits from an ancestor prim, *geomBindTransform* is
assumed to be identity.
- **rel skel:joints** : Sometimes it is desirable to encode the
*jointIndices* to a localized set of Joints, rather than in reference to a
*Skeleton* that is assumed to be bound on some ancestor prim. Though it
comes at some performance cost, *skel:joints* allows us to declare, on any
given gprim, an ordered list of Joint paths - *rooted to the gprim, i.e. as
if the gprim were a Skeleton* - to which the gprim's *jointIndices* will
apply. The skinning APIs will use this data to remap the joints onto
those of the eventually-bound *Skeleton* such that the correct animation is
applied to the gprim. Just as with the primvars, this data should *inherit*
down namespace, to define the *jointIndices* ordering for all prims beneath
the prim that specifies *skel:joints*.
**Notes**
- The "binding root" provides the "root transformation" (localToWorld
transformation) of the skinned hierarchy, and is applied after (less
locally) the root transformation computed by the bound *Animation.*
- Because the binding root can be (as in the example) expressed on a "model
root" prim, we can instance any model that is being skeletally deformed! At
the scale of hundreds or thousands of skinned characters with substantial
geometry and shading hierarchies, the Usd-core and I/O bandwidth savings of
applying this pattern can be substantial. Note that binding "outside" the
model is not required - it is perfectly acceptable to embed *Blenders*
and *Animations* inside a model's namespace and bind
on a "lower" common prim, as well (such as < /Bob/Geom> in the example,
relocating < /BobSkinning> to < /Bob/Skinning>.
# Examples
## Single File, Single Animation Example
```cpp
#usda 1.0
(
upAxis = "Y"
)
def Skeleton "Skel1"
(
# by default, we expect Skeleton's to be instanceable
instanceable = true
)
{
rel joints = [
</Skel1/joint1>,
</Skel1/joint1/joint2>,
</Skel1/joint1/joint2/joint3>,
</Skel1/joint1/joint2/joint3/joint4>,
]
uniform matrix4d[] restTransforms = [( (0, 1, 0, 0), (0, 0, 1, 0), (1, 0, 0, 0), (0, -1, 0, 1) ), ( (0, 1, 0, 0), (0, 0, 1, 0), (1, 0, 0, 0), (0, 0, 0, 1) ), ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 1, 0, 1) ), ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 1, 0, 1) )]
def Joint "joint1"
{
def Joint "joint2"
{
def Joint "joint3"
{
def Joint "joint4"
{
}
}
}
}
}
def PackedJointAnimation "SkelAnim1"
{
rel joints = [
</SkelAnim1/joint1>,
</SkelAnim1/joint1/joint2>,
</SkelAnim1/joint1/joint2/joint3>,
]
float3[] translations =[(0, -1, 0), (1, 0, 0), (1, 0, 0)]
quath[] rotations.timeSamples = {
1: [(0.5, 0.5, 0.5, 0.5), (1, 0, 0, 0), (0.5, -0.5, -0.5, -0.5)],
2: [(0.4912109375, 0.5, 0.5087890625, 0.5), (1, 0, 0, 0), (0.4912109375, -0.5, -0.5, -0.5087890625)]
}
half3[] scales = [(1, 1, 1), (1, 1, 1), (1, 1, 1)]
}
def Mesh "SkinnedMesh"
{
int[] faceVertexCounts = [4, 4, 4, 4, 4, 4, 4, 4, 4, 4]
int[] faceVertexIndices = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39]
normal3f[] normals = [(0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 1, 0), (0, 1, 0), (0, 1, 0), (0, 1, 0), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, -1, 0), (0, -1, 0), (0, -1, 0), (0, -1, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0)]
point3f[] points = [(-0.5, -1, 0.5), (0.5, -1, 0.5), (0.5, 0, 0.5), (-0.5, 0, 0.5), (-0.5, 0, 0.5), (0.5, 0, 0.5), (0.5, 1, 0.5), (-0.5, 1, 0.5), (-0.5, 1, 0.5), (0.5, 1, 0.5), (0.5, 1, -0.5), (-0.5, 1, -0.5), (-0.5, 1, -0.5), (0.5, 1, -0.5), (0.5, 0, -0.5), (-0.5, 0, -0.5), (-0.5, 0, -0.5), (0.5, 0, -0.5), (0.5, -1, -0.5), (-0.5, -1, -0.5), (-0.5, -1, -0.5), (0.5, -1, -0.5), (0.5, -1, 0.5), (-0.5, -1, 0.5), (0.5, -1, 0.5), (0.5, -1, -0.5), (0.5, 0, -0.5), (0.5, 0, 0.5), (0.5, 0, 0.5), (0.5, 0, -0.5), (0.5, 1, -0.5), (0.5, 1, 0.5), (-0.5, -1, -0.5), (-0.5, -1, 0.5), (-0.5, 0, 0.5), (-0.5, 0, -0.5), (-0.5, 0, -0.5), (-0.5, 0, 0.5), (-0.5, 1, 0.5), (-0.5, 1, -0.5)]
float2[] primvars:Texture_uv = [(0.375, 0), (0.625, 0), (0.625, 0.125), (0.375, 0.125), (0.375, 0.125), (0.625, 0.125), (0.625, 0.25), (0.375, 0.25), (0.375, 0.25), (0.625, 0.25), (0.625, 0.5), (0.375, 0.5), (0.375, 0.5), (0.625, 0.5), (0.625, 0.625), (0.375, 0.625), (0.375, 0.625), (0.625, 0.625), (0.625, 0.75), (0.375, 0.75), (0.375, 0.75), (0.625, 0.75), (0.625, 1), (0.375, 1), (0.625, 0), (0.875, 0), (0.875, 0.125), (0.625, 0.125), (0.625, 0.125), (0.875, 0.125), (0.875, 0.25), (0.625, 0.25), (0.125, 0), (0.375, 0), (0.375, 0.125), (0.125, 0.125), (0.125, 0.125), (0.375, 0.125), (0.375, 0.25), (0.125, 0.25)] (
interpolation = "vertex"
)
uniform token subdivisionScheme = "none"
# elementSize = 3 means we are weighting 3 joints per vertex. Even
# though the Skeleton has only three joints, encoding jointIndices as an
# "indexed primvar" means that we need only examine a three-element array
# to determine all the joints weighted by this mesh, rather than forty
int[] primvars:skel:jointIndices = [0, 1, 2]
int[] primvars:skel:jointIndices:indices = [0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 1, 2, 0, 0, 1, 2, 0, 1, 2, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 1, 2, 0, 1, 2, 0, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 1, 2, 0] (
elementSize = 3
interpolation = "vertex"
)
float[] primvars:skel:jointWeights = [1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0, 1, 0, 1, 0, 0] (
elementSize = 3
interpolation = "vertex"
)
# Do not need to specify primvars:skel:geomBindTransform since it is identity
rel skel:animationSource = </SkelAnim1>
rel skel:skeleton = </Skel1>
}
```
## A "Ready to Skeletally Animate" Asset
Following are the contents of a file, *BobAsset.usd*:
```cpp
#usda 1.0
def Xform "Bob"
{
# We expect Bob to be bound to an AnimationBlender at his root, and expect
# skel:skeleton to be found as a sibling relationship to the binding rel
rel skel:skeleton = </Bob/Skeleton>
def Xform "Geom"
{
def Mesh "Body"
{
# If the mesh has 1000 points, then the full size of each of the
# following primvars would be 5000. "jointIndices" is an
# "indexed primvar" that separately encodes the 8 unique Joint
# values consumed, from the 5000 uses of those values (in the
# jointIndices:indices attribute), which the UsdGeomPrimvar API
# can resolve.
int[] primvars:skel:jointIndices = [ 0, 1, 2, 3, 15, 16, 17, 18 ]
(
elementSize = 5
interpolation = "vertex"
)
int[] primvars:skel:jointIndices:indices = [ too many to enumerate ]
float[] primvars:skel:jointWeights = [ 3, 4, 5, 6, 7, ... ]
(
elementSize = 5
interpolation = "vertex"
)
# inverse of transform that takes Body into Bob's
# object space, which defines "skeleton space"
uniform matrix4d primvars:skel:geomBindTransform = [ ... ]
}
def Xform "Head"
{
def Xform "REye"
{
def Mesh "Schlera"
{
# This represents a very simple/stupid mesh binding
# in which the entire face moves rigidly with the neck.
# Because the eyes move rigidly, we can demonstrate
# "constant" weighting
int[] primvars:skel:jointIndices = [ 47 ]
int[] primvars:skel:jointIndices:indices = [ 0 ]
(
interpolation = "constant"
)
float[] primvars:skel:jointWeights = [ 1.0 ]
(
interpolation = "constant"
)
# inverse of transform that takes Schlera into Bob's
# object space, which defines "skeleton space"
matrix4d primvars:skel:geomBindTransform = [ ... ]
}
}
}
}
# Embedding the "source" Skeleton in the asset guarantees that we can
# map the asset's jointIndices to their Joints robustly, and also gives
# us the place to specify the skeleton binding pose for the asset.
def "Skeleton" (
instanceable = true
# It may make sense to lock this reference to a particular version
# if Biped17a is independently churning, since the joint mappings
# and rest-pose specified herein are strongly dependent on the
# topology of the Skeleton
references = @Biped17a/Skeleton.usd@
)
{
# The skeletal pose to which Bob's mesh bindings apply.
# This is an override to the restTransform published with Biped17a
uniform matrix4d[] restTransforms = [ ... ]
}
}
```
# Conclusion
This document proposes an object model for supporting skinned, blended,
skeletal animation in USD as a core, supported feature for interchange and
visualization in the Hydra renderer. The first deployment of these schemas
appears in the 0.7.6 USD release, but is very minimal, containing only the
most basic schema definitions required to interchange simple skeletal
animations. In the interests of delivering useful functionality
incrementally, the proposal does not yet cover:
- What the associated **Hydra data model** will be. We expect this to follow
the schemas' deployment shortly.
- Schema and support for **AnimationBlenders**
- **Blend Shapes.** We expect the *BindingAPI* schemas to be extensible to
support the concerns of blend shapes.
- Animations are required to be encoded in a vectorized SRT form. This is
ideal for data compactness and data extraction speed, but limited for
the goal of **interchanging the source Joint animation curves** between
packages. When USD supports a notion of anim curves, we can
consider extending the *Animation* schema to allow for animation curves to be
included, at least as advisory data, if not an alternate means of driving
the core data extraction and blending APIs.
- **APIs.** We expect to iterate on useful computation APIs for some time
following initial schema deployment and plugin development.