**Skinning Schemas for USD** This document motivates the desire for, and scope of, core support for skeletal deformation and animation blending in USD, and proposes a set of cooperating schemas to encode and decode such data in keeping with the goals of the USD project. This document does not explore Blend Shape animation, as schema ideas are still in development, and we think skinning is useful enough to deploy on its own, in advance; we are confident that including support for blend shapes will be purely additive with respect to encoding, thus not invalidating any of the encodings proposed herein. # Goals, Requirements, and Philosophy While the initial goal of introducing skinning schemas into USD is to provide a robust means of interchange for clients like game studios that have pipelines whose terminus is real-time-engine authoring environments such as UE4 and Unity, an equally important goal for film/VFX studios is building performant, scalable crowds systems that feed undeformed geometry and blended skeletal animation directly to preview and final-quality render engines from USD, pulling from and sequencing libraries of animation. From these goals, we derive the following requirements: - We must provide clear authoring APIs that are not completely foreign to the way that common authoring environments think about the relevant data. - We must provide an encoding that is amenable to publishing modular, reusable pieces, separating structural data from animation. - We must allow for animations authored to a particular armature to be blended with animations authored for different but "compatible" armatures, and use the result with geometries bound to yet another armature. - We must provide an encoding that is weighted towards efficient UsdStage data extraction, should that come into conflict with sparse overridability or one-to-one matching with popular authoring environments' data models. - We must provide efficient GPU and CPU computation functions for the behaviors. Concretely, this means [UsdImaging](https://github.com/PixarAnimationStudios/USD/tree/master/pxr/usdImaging/lib/usdImaging) will model (and efficiently extract) the data from which both GPU and CPU based Hydra "backend renderers" will be able to generate posed geometry from compact, schema-derived data. ## Primary Concern: Scope of the Schema Our primary concern about adding such features to core USD is that their inclusion not signal an intent of the USD core to support general rigging and execution behaviors, as we strongly believe such features are beyond the scope of USD's core concerns. To that end, we adopt a name for the new schema module and classes that signal a limited scope: **UsdSkel**, as the techniques it provides are geared towards application of skeletal animation. # Concepts and Definitions We think it important to decompose the encoding into four primary related concepts, to cleanly separate the concerns of: - **Skeletal Topologies** - A *Skeleton* encapsulates the topological aspects of skinning, so that they can be re-used and provide a basis for identifying compatible animations. Skeletons define their topology as a (ordered) hierarchy of *Joint* prims. - **Geometry Binding** - Encodes a binding of a geometry hierarchy to a *Skeleton*, and a specification of how to deform the geometry to a posed *Skeleton*. - Uses [primvar data](http://openusd.org/docs/api/class_usd_geom_primvar.html#details) on gprims to specify: - the subset of skeletal joints that drive each gprim - the joint weights for each vertex in each gprim - the geometry-to-skeleton binding transformation - Encodes a binding of geometry to an *Animation Source* that will pose the bound *Skeleton* over time. - **Animations** - An *Animation* is an "atomic" *Animation Source* that provides time-varying overrides for *Joints* and "the roots" of geometries bound to a *Skeleton*. Animations must also be encapsulatable so that they can be referenced and reused, multiply-applied, and sequenced. - **Animation Blenders** that organize, weight, and blend multiple *Animation Sources* together, so that they can be layered, and/or sequenced/transitioned. We also define **skeleton space** as the "object space of an animated skeleton". Skeleton space does not contain the transformation that positions the root of the *Skeleton* (and bound geometry) in the world, and therefore serves as the canonical space in which to apply blended joint animations. Whereas, for example, Maya APIs assume World space for many computed quantities, we require data to be encoded in no "higher" than skeleton space, so that animations will not break if transformations are altered on ancestor prims of the skinned geometry. # Primary Schemas - High-level Description ## UsdSkelJoint At the outset, the *Joint* schema will be nothing more than an identified type on which we may key imaging behaviors in certain display modes (e.g. connecting joints with bones for fast animation preview). In the future, *Joint* may provide a container for specifying end-effector locators that deform along with animated geometry. ## UsdSkelSkeleton A *Skeleton* defines an ordered, hierarchical topology of joints - A *Skeleton* is modular, amenable to referencing and instancing, and provides the target onto which *Animations* (which may move only a subset of the *Skeleton*'s Joints) are mapped. - A *Skeleton* holds the "rest pose" that provides a "fallback" for any *Joint* that is not animated, and is useful to some blending algorithms, and of course also provides one half of the binding transformation. The rest pose is recorded in **joint local space**. - The rest pose is vectorized according to the defined ordering of its *Joints*, not defined by transformations on the *Joint* prims. ## UsdSkelAnimation, UsdSkelPackedJointAnimation An *Animation* provides time-varying opinions for skeletal joints. The skinning schemas support attaching both "simple", single animations, and arbitrary combinations of linearly-combined ("blended") animations. Initially, we will support a single encoding of each type. For simple animations, we provide the *PackedJointAnimation* schema, with the following properties: - A *PackedJointAnimation* names (via a [UsdRelationship](http://openusd.org/docs/USD-Glossary.html#USDGlossary-Relationship)) and orders the *Joints* for which it encodes joint-local animation. This allows *PackedJointAnimations* to be encoded independently of any *Skeleton*, as its data can be mapped to any *Skeleton* that possesses *Joints* whose paths (i.e. hierarchical names) match those encoded by the *PackedJointAnimation*. - A *PackedJointAnimation* consists of: - Joint transformations that describe the positioning of each joint in **joint local space**. - A "root transformation" that affinely moves all bound geometry. It is encoded using the regular [UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details) schema which implies that *PackedJointAnimation* **IsA** UsdGeomXformable. - The joint animation uses a **vectorized encoding: translation, rotation, and scale vectors** that aggregate all affected *Joints*' values, just as a *Skeleton*'s rest-pose vectorizes *Joint* transformations. ## UsdSkelAnimationBlender An *AnimationBlender* is a type of *UsdSkelAnimation* that provides a concrete recipe for weighted blending of other *UsdSkelAnimation* prims. Because an *AnimationBlender* IsA *Animation*, it can blend not just atomic *PackedJointAnimation* prims, but also other *AnimationBlenders*, which means we can create *blend trees*, so that we can, for example, easily describe and edit transitions between groups of layered animations, similarly to [Unity's implementation](https://docs.unity3d.com/Manual/class-BlendTree.html). - We propose initially a single concrete *AnimationBlender* type: **UsdSkelLinearBlender**, which produces a linear combination of the input animation data (SLERP for quaternion rotations). - In the future, this abstraction gives us the opportunity to deploy other *UsdSkelAnimationBlender* schemas with different blending algorithms, such as [dual quaternions](https://www.scss.tcd.ie/publications/tech-reports/reports.06/TCD-CS-2006-46.pdf) for transformation blending. ## UsdSkelBindingAPI The *BindingAPI* schema encodes all of the required data to bind geometry to a *Skeleton* and to an animation-providing *SkeletonBlender*. Unlike the other, [typed, "IsA" schemas](http://openusd.org/docs/USD-Glossary.html#USDGlossary-IsASchema) in UsdSkel, binding is presented as an [API schema](http://openusd.org/docs/USD-Glossary.html#USDGlossary-APISchema) because its purpose is to **add data onto geometry prims** from the UsdGeom family of typed schemas, i.e. Xforms, Meshes, Curves, NurbsPatches, etc. Its primary concerns are encoding: - The *skinning method* that applies *Skeleton*-mapped animation to deform geometry, which initially is limited to [Linear Blend Skinning](http://skinning.org/direct-methods.pdf), and therefore not explicitly encoded as data in a scene. - The binding of geometry to an *Animation*. Like Material bindings in the [UsdShadeMaterial](http://openusd.org/docs/api/class_usd_shade_material.html#details) schema, this binding relationship is meant to apply hierarchically to animate all the geometry beneath the bound prim, and we intend that binding typically happens at the "model level" or other suitable aggregation point, not at the individual gprim level. Unlike hierarchical Material binding, we propose that in the interests of simplicity and avoiding the collapse to a general rigging system, **the highest *skel:animationSource* binding in namespace that affects a given prim is the only binding that will affect it.** - The binding of geometry to a *Skeleton*. This relationship, also, is meant to apply hierarchically, and a *skel:skeleton* binding must be present on the same composed prim as the *skel:animationSource* binding in order to encode a fully-skinned geometry hierarchy. It is this related *Skeleton* onto which the *SkeletonBlender*'s animation will be mapped. - The mapping and weighting of gprim vertices to *Joints*. This can be encoded very efficiently in terms of the *Joint* ordering provided by the bound *Skeleton* affecting each gprim, or joints can dynamically be named and ordered uniquely on any gprim, which will serve as an override to the joint ordering provided by the bound Skeleton, providing an extra level of indirection, at some decoding cost. - The "geometry binding" transformation used to fit each gprim to its bound *Skeleton*. # How the Schemas Fit Together Following is a diagram that shows how the concepts described above map to Prims, Properties, and namespace hierarchy for a simple application of two blended skeletal animations. The colored backdrops suggest useful encapsulation of separable concerns to promote reuse and efficient encoding; for example, each of the three sections may be encoded in different files/layers, and composed together using [USD composition arcs](http://openusd.org/docs/USD-Glossary.html#USDGlossary-CompositionArcs) . Note, however, that in addition, each *Animation* may typically (in a VFX pipeline) be encapsulated and published into libraries such that they can be individually referenced into a blended animation as needed. ![Diagram that shows how the related schemas apply to skin a character model.](http://openusd.org/images/UsdSkelPrimLayout.png) ## Explanation ### Brief summary The diagram represents a prim hierarchy comprised of Xforms, Meshes and other prims that define the visible meshes to be modified by the LinearBlender. Beginning with the colored section on the left, we have a "geometry hierarchy" for a model named Bob. In addition to Materials (which are not a concern for this document), Bob consists primarily of a number of Meshes and organizing Xform prims. Of primary concern is that this group of prims contains the data we ascribe to the **UsdSkelBindingAPI**. On Bob's root prim, we find the relationships that bind Bob to a *Skeleton* (i.e. *skel:skeleton*), and to an *Animation* (i.e. *skel:AnimationSource*), which in this case, is a *LinearBlender*. On each of the *BodyMesh*, *Eye* and *Tooth* prims, we see the three primvars that provide the mapping and weighting of each Mesh's points to the bound *Skeleton* and its *Joints*. In the center section, we see the definition of the *Skeleton* to which Bob is bound, *Human_17a*. The prim structure of *Joints* underneath a *Skeleton* prim provides the inheritance structure for the "skeleton package", since the *joints* relationship on the *Skeleton* prim itself only provides joint linearization ordering (needed by *restTransforms* and *PackedJointAnimation*), **not** parenting information. A *Skeleton* can be described in the same file as a geometry hierarchy that is bound to it, but in a VFX crowds pipeline, it may be published as an independent asset that is referenced and instanced into geometry assets like Bob. In the rightmost section we see a *LinearBlender* that blends between two animations, *WalkCycle* and *RunCycle*. For a *Blender* that is blending between running and walking, we would tupically expect the per-animation *weights* and *rootWeights* properties on the *LinearBlender* to be "Joint uniform", i.e. *weights* will be an array of length 1 meaning the weight applies equally to all *Joint* animation in the *source*, but animated over time, as we transition between running and walking. We briefly discuss details of how cycled (and other) *Animations* can be sequenced, in the note on "Animation Retiming" in the schema description of [UsdSkelPackedJointAnimation](#usdskelpackedjointanimation). ### Organizing Principles, Encoding Decisions Skinning data is organized according to two principles: 1. **Data stored in modular, reusable, instanceable parts:** a. *Skeleton* "rigs" can be defined and published, to be referenced in whenever needed. Encapsulating skeletons this way is not a requirement for sending data from Maya directly to UE4 through USD (and in such contexts *Skeletons* can be defined "inline"), but facilitates scalability and maintainability when deploying directly-rendered, skinned crowds at scale in VFX pipelines. b. *Animations* can be published individually and referenced back into scenes. This not only allows re-use, but also enables sequencing of animations as [Value Clips](http://openusd.org/docs/USD-Glossary.html#USDGlossary-ValueClips) c. An *AnimationBlender* can encapsulate the animation that it blends, further facilitating "animation packages" that can be referenced in and scattered onto actors already in a scene. d. Separating the binding of *Skeleton* and *AnimationBlender* to geometry further aids the pattern described in (c), because we can publish assets that already contain a "bound *Skeleton*" and *jointIndices* on its gprims that refer to that *Skeleton*'s *Joints*, and thus the asset is "skinning ready". After referencing in such assets into a scene, all that remains is to bind context-specific *Animations* to them. 2. **Scalability and Performance:** a. All posing and animation data is vectorized. This encoding is concise and ideal for scalability and speed of data consumption in the USD APIs. A vectorized encoding does not facilitate sparse, per-joint overrides of animation data using the current USD facilties for sparse overriding. If this becomes an issue in the future, there are paths we could pursue to provide this affordance. But for initial deployment, we believe the need for scalability and performance far outweighs the need for per-joint overrides. Since the USD APIs do not, at present, allow reading "slices" of arrays, this encoding also requires us to read in a complete animation sample even if only part of the animation applies to the target *Skeleton*; given the performance characteristics of lazy data access in USD, we feel this is the proper tradeoff. The decision to vectorize a *Skeleton* rest-pose not only significantly enhances de-serialization performance, but also allows *Skeletons* to be [instanced](http://openusd.org/docs/USD-Glossary.html#USDGlossary-Instancing) for encoding scalability in USD, while still allowing the "binding" of a *Skeleton* to geometry to override the rest pose as part of the fitting process. b. For skinning, we require just three spatially interpolated primvars per mesh: *jointIndices*, *jointWeights*, and *geomBindXform*. We also allow the *joints* indexing to be overridden per-mesh, but this is optional and hopefully rarely necessary. # Primary Schemas - Concrete Definitions ## UsdSkelJoint The *Joint* schema currently has no properties - its purpose is merely declarative as defining the joints of a skeletal hierarchy. ## UsdSkelSkeleton A *UsdSkelSkeleton* is a typed schema that derives from [UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details). It parents a hierarchy of *UsdSkelJoints*. The "first joint" of a *Skeleton* should be a child *Joint* prim of the *Skeleton* prim. A *UsdSkelSkeleton* contains the following data: - **rel joints** - a relationship targeting all of the descendant Joint prims in a stable ordering, with the requirement that "parent" joints must appear earlier than "child" joints in the order. It is the ordering of joints in this relationship that determines the layout of the *restTransforms* property, and to which *primvars:skel:jointIndices* refers, should no *skel:joints* relationship be authored on a gprim. - **matrix4d[] restTransforms** - is a matrix4d array that specifies, in the ordering imposed by joints, the rest transformations of each joint. ## UsdSkelPackedJointAnimation *UsdSkelAnimation*, the base for both *UsdSkelPackedJointAnimation* and *UsdSkelAnimationBlender* derives from [UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details) so that it can provide root-level animation, as well as joint animation. *UsdSkelPackedJointAnimation*, the concrete *Animation* schema, encapsulates all of the data required to animate an enumerated, ordered set of *Joints*. - **rel joints** : relationship that names Joints, by path **rooted at the *Animation* prim, as if it were a *Skeleton* prim.** Relationship paths in USD can be resolved independently of whether they target an actual concrete object. We therefore can map *PackedJointAnimation* joints onto a target *Skeleton* by resolving the targets, and replacing the *PackedJointAnimation* prim's path prefix with that of the target *Skeleton*, this relationship serves two roles: - It allows an *Animation* to be **sparse**, affecting only a subset of the target *Skeleton*'s joints. A *PackedJointAnimation* can provide values for only a small number of joints in a complex skeleton by targeting only those joints in its *joints* relationship; the vectorized animation will then be laid out according to the ordering of those relationship targets. This can be valuable as a data frugality technique not only for files, but to facilitate client engines in building minimal "joint palettes" for processing. It can also facilitate creating "layerable" animations that affect only parts of a model. - It provides a level of protection against trivial changes to a published *Skeleton*. For example, an animation with properly encoded *joints* will not "fall off" if the referenced *Skeleton* is modified to insert a *Joint* at the end of some chain, because we can compare the *Joint* paths encoded in the *PackedJointAnimation* to the *Joint* paths in the *Skeleton*, at runtime. - **float3[] translations** : 32-bit joint-local translations of all affected joints. Length of the array should match the size of the *joints* relationship. The USD API will perform linear interpolation between samples. - **quath[] rotations** : 16-bit joint-local **unit quaternion** rotations of all affected joints. Length of the array should match the size of the *joints* relationship. The USD API will perform SLERPing between samples. - **half3[] scales** : 16-bit joint-local scaling of all affected joints. Length of the array should match the size of the *joints* relationship. The USD API will perform linear interpolation between samples. **Notes** - **Root transform animation.** Each *PackedJointAnimation* can also provide [UsdGeomXformable](http://openusd.org/docs/api/class_usd_geom_xformable.html#details) data to define a root transformation. This allows *PackedJointAnimations* to express a positioning of the skinned geometry "as a whole." - **Joint transform ops.** The decomposition of animated Joint transformations into translations, rotations, and scales does limit the kinds of animations we can encode (e.g. no shears). However, it provides two key advantages: - Data economy, as the three separate ops consume less space than a single matrix4f even if all three are in 32-bit floating point, and we further advocate that the rotations and scales can suffice with 16-bit half floating-point precision. - Interpolating the independent op quantities (as done by native USD value resolution) yields far better results than interpolating matrices. This can be of value for any blending schemes that can operate on the raw data. - **Animation retiming.** How do we want to control animation timing for purposes of blending? Given that each contributing animation is a unique prim, referenced *Animations* can be retimed affinely with a [SdfLayerOffset](http://openusd.org/docs/USD-Glossary.html#USDGlossary-LayerOffset), or if more sophisticated retiming is necessary (e.g. in order to align a walk with a run such that foot plants occur simultaneously across all blends), one can additionally apply a sequence of [Value Clips](http://openusd.org/docs/USD-Glossary.html#USDGlossary-ValueClips), and use the built-in retiming capabilities of that feature. If such an encoding proves too awkward, we can instead add explicit timing curves, per-animation, as part of the *LinearBlender* schema, or possibly as a common feature for all *Animation* types. ## UsdSkelLinearBlender Note we are only defining the concrete schema *UsdSkelLinearBlender* since we have not yet identified any properties that may be common to other, eventual *AnimationBlender* implementations. The abstract base schema class will serve primarily as an interface class for defining computations that clients would require from any implementation. A *UsdSkelLinearBlender* contains properties created dynamically in namespaced groups, in the **anim:** namespace. In the following, each property would appear in each of many namespaces, such as **anim:walkAnim:**, **anim:runAnim:**, for example. The content of the "inner" namespace, in these examples, is entirely user-controlled, specifiable at the time of authoring the *LinearBlender* data. - **rel source** : relationship that targets an *Animation* prim. - **float[] weights** : specifies an overall scaling of the joint animation to be applied when computing the blended aggregate animation. *weights* can either be of length one, in which case a constant weighting is applied to all Joint animation in the target *Animation*, or it can be of the same size as the target *Animation*s *joints* relationship, in which case the *weights* are **per joint**. For each *Joint* of the target *Skeleton*, we will normalize the weight contributions of each *Animation*, and if the sum is less than one, fill in the remainder from the *Skeleton*'s *restTransforms*. - **float rootWeight** : specifies a scaling to be applied to the *Animation*'s contribution to the root transformation, which is encoded separately from the joint animations. Typically, if there are multiple, layered animations, we will want the root transformation of only the "base" animation, so this weighting would be zero for all of the secondary, layered animations. **Notes** - **Skinning root transformation.** After all Joint-based posing has been applied to a gprim, bringing it into *skeleton space*, we then apply the root transformation computed by the *LinearBlender* as the weighted sum of *Animation* root transforms at the evaluation time. Finally, the local-to-world transformation of the bound/skinned prim is applied (i.e. the highest prim in namespace to which a *LinearBlender* (or *PackedJointAnimation) and *Skeleton* have been bound). - **Evaluating a Blended Animation.** When asked to sample, weight, and blend its *Animations* at a given time, a *LinearBlender* requires as an input the *Skeleton* onto which it should map the resulting animation. This allows it to arrange the Joint data into the vectorized ordering in which it will be consumed by the bound geometry. ## UsdSkelBindingAPI To make use of skeletal skinning, a geometry hierarchy must encode several properties on one or more prims to: - bind *Animations* - bind *Skeletons* - position gprims in skeleton-space, via a mesh-binding (gprim-binding) transformation - attach and weight gprims to *Joints* The *UsdSkelBindingAPI* presents a single interface that can be applied to any geometry prim to encode and interrogate the above properties. - **rel skel:animationSource** : relationship that binds a *UsdSkelAnimation*. All descendant prims of the "binding prim" will be animated by the bound *Animation*, provided they are mapped and weighted to a *Skeleton*. If multiple *skel:animationSource* relationships are authored in a namespace hierarchy, **the highest such binding wins** (i.e. the one with the shortest path). - **rel skel:skeleton** : relationship that binds a *UsdSkelSkeleton* and its assocatiated *Joint* hierarchy to a geometry prim. For a geometry hierarchy to be succesfully skinned, both *skel:skeleton* and *skel:animationSource* must be bound on the same composed prim (although in USD composition terms, the two bindings can be expressed in different "layers", and we expect that to commonly be the case in VFX pipelines). To facilitate creation of "skinned aggregates" from smaller, "skinnable components", we allow *skel:skeleton* bindings to be expressed on child/descendant prims of "the binding prim", with the interpretation that we will use the "nested Skeleton" as a way to remap Joint bindings on the affected geometry to the *Skeleton* of the top-level "binding prim." - **int[] primvars:skel:jointIndices** and **int[] primvars:skel:jointWeights** : For each gprim to be skinned (Meshes, but also Curves, Patches, etc) we must be able to compute the number of joints that affect its geometry, and weights for each of those joints, **per-vertex**. Both quantities can be encoded *directly* on each gprim using the *primvars:skel:jointIndices* and *primvars:skel:jointWeights* primvars. The number of weighted joints must be the same for all points in each primitive, and is encoded directly as *elementSize* metadata on the two primvars - the length of each as well as their interpolation and elementSize, must match. Because we cast these data as [UsdGeomPrimvars](http://openusd.org/docs/api/class_usd_geom_primvar.html#details), we can powerfully compress the data encoding of rigidly deforming gprims by using a constant interpolation with a single set of weights applied uniformly to all the gprim's points. For example, in the diagram above, all of the "teeth" meshes could have a constant-interpolation "float[] primvars:skel:jointWeights = [1.0]" primvar, also weighting just a single jaw joint, as the teeth move rigidly with the jaw. We can be even more succinct in expression of weighting by establishing a joint weighting at an ancestor prim of leaf gprims, with the expected consumer behavior that the two joint primvars *will inherit down namespace.* Note this means that in accordance with typical attribute inheritance rules, any gprim leaf is able to overide the inherited weighting and establish its own. This is a useful behavior, because we might author weights on "interior prims" not only for purposes of concise expression of rigid binding, but also because we might have spatial data *on the interior prims* to which we wish to apply the skinning algorithm via provided APIs, such as constraint targets that must pose with the Skeleton and animation. Note that uniform and faceVarying interpolation do not make sense for joint indices and weights. - **matrix4d primvars:skel:geomBindTransform** : Encodes, for each gprim to be skinned, the **inverse of the transformation** that positions the gprim into the "skeleton space" in which it is bound to a *Skeleton*. If unauthored and no value inherits from an ancestor prim, *geomBindTransform* is assumed to be identity. - **rel skel:joints** : Sometimes it is desirable to encode the *jointIndices* to a localized set of Joints, rather than in reference to a *Skeleton* that is assumed to be bound on some ancestor prim. Though it comes at some performance cost, *skel:joints* allows us to declare, on any given gprim, an ordered list of Joint paths - *rooted to the gprim, i.e. as if the gprim were a Skeleton* - to which the gprim's *jointIndices* will apply. The skinning APIs will use this data to remap the joints onto those of the eventually-bound *Skeleton* such that the correct animation is applied to the gprim. Just as with the primvars, this data should *inherit* down namespace, to define the *jointIndices* ordering for all prims beneath the prim that specifies *skel:joints*. **Notes** - The "binding root" provides the "root transformation" (localToWorld transformation) of the skinned hierarchy, and is applied after (less locally) the root transformation computed by the bound *Animation.* - Because the binding root can be (as in the example) expressed on a "model root" prim, we can instance any model that is being skeletally deformed! At the scale of hundreds or thousands of skinned characters with substantial geometry and shading hierarchies, the Usd-core and I/O bandwidth savings of applying this pattern can be substantial. Note that binding "outside" the model is not required - it is perfectly acceptable to embed *Blenders* and *Animations* inside a model's namespace and bind on a "lower" common prim, as well (such as < /Bob/Geom> in the example, relocating < /BobSkinning> to < /Bob/Skinning>. # Examples ## Single File, Single Animation Example ```cpp #usda 1.0 ( upAxis = "Y" ) def Skeleton "Skel1" ( # by default, we expect Skeleton's to be instanceable instanceable = true ) { rel joints = [ </Skel1/joint1>, </Skel1/joint1/joint2>, </Skel1/joint1/joint2/joint3>, </Skel1/joint1/joint2/joint3/joint4>, ] uniform matrix4d[] restTransforms = [( (0, 1, 0, 0), (0, 0, 1, 0), (1, 0, 0, 0), (0, -1, 0, 1) ), ( (0, 1, 0, 0), (0, 0, 1, 0), (1, 0, 0, 0), (0, 0, 0, 1) ), ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 1, 0, 1) ), ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 1, 0, 1) )] def Joint "joint1" { def Joint "joint2" { def Joint "joint3" { def Joint "joint4" { } } } } } def PackedJointAnimation "SkelAnim1" { rel joints = [ </SkelAnim1/joint1>, </SkelAnim1/joint1/joint2>, </SkelAnim1/joint1/joint2/joint3>, ] float3[] translations =[(0, -1, 0), (1, 0, 0), (1, 0, 0)] quath[] rotations.timeSamples = { 1: [(0.5, 0.5, 0.5, 0.5), (1, 0, 0, 0), (0.5, -0.5, -0.5, -0.5)], 2: [(0.4912109375, 0.5, 0.5087890625, 0.5), (1, 0, 0, 0), (0.4912109375, -0.5, -0.5, -0.5087890625)] } half3[] scales = [(1, 1, 1), (1, 1, 1), (1, 1, 1)] } def Mesh "SkinnedMesh" { int[] faceVertexCounts = [4, 4, 4, 4, 4, 4, 4, 4, 4, 4] int[] faceVertexIndices = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39] normal3f[] normals = [(0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 0, 1), (0, 1, 0), (0, 1, 0), (0, 1, 0), (0, 1, 0), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, 0, -1), (0, -1, 0), (0, -1, 0), (0, -1, 0), (0, -1, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0), (-1, 0, 0)] point3f[] points = [(-0.5, -1, 0.5), (0.5, -1, 0.5), (0.5, 0, 0.5), (-0.5, 0, 0.5), (-0.5, 0, 0.5), (0.5, 0, 0.5), (0.5, 1, 0.5), (-0.5, 1, 0.5), (-0.5, 1, 0.5), (0.5, 1, 0.5), (0.5, 1, -0.5), (-0.5, 1, -0.5), (-0.5, 1, -0.5), (0.5, 1, -0.5), (0.5, 0, -0.5), (-0.5, 0, -0.5), (-0.5, 0, -0.5), (0.5, 0, -0.5), (0.5, -1, -0.5), (-0.5, -1, -0.5), (-0.5, -1, -0.5), (0.5, -1, -0.5), (0.5, -1, 0.5), (-0.5, -1, 0.5), (0.5, -1, 0.5), (0.5, -1, -0.5), (0.5, 0, -0.5), (0.5, 0, 0.5), (0.5, 0, 0.5), (0.5, 0, -0.5), (0.5, 1, -0.5), (0.5, 1, 0.5), (-0.5, -1, -0.5), (-0.5, -1, 0.5), (-0.5, 0, 0.5), (-0.5, 0, -0.5), (-0.5, 0, -0.5), (-0.5, 0, 0.5), (-0.5, 1, 0.5), (-0.5, 1, -0.5)] float2[] primvars:Texture_uv = [(0.375, 0), (0.625, 0), (0.625, 0.125), (0.375, 0.125), (0.375, 0.125), (0.625, 0.125), (0.625, 0.25), (0.375, 0.25), (0.375, 0.25), (0.625, 0.25), (0.625, 0.5), (0.375, 0.5), (0.375, 0.5), (0.625, 0.5), (0.625, 0.625), (0.375, 0.625), (0.375, 0.625), (0.625, 0.625), (0.625, 0.75), (0.375, 0.75), (0.375, 0.75), (0.625, 0.75), (0.625, 1), (0.375, 1), (0.625, 0), (0.875, 0), (0.875, 0.125), (0.625, 0.125), (0.625, 0.125), (0.875, 0.125), (0.875, 0.25), (0.625, 0.25), (0.125, 0), (0.375, 0), (0.375, 0.125), (0.125, 0.125), (0.125, 0.125), (0.375, 0.125), (0.375, 0.25), (0.125, 0.25)] ( interpolation = "vertex" ) uniform token subdivisionScheme = "none" # elementSize = 3 means we are weighting 3 joints per vertex. Even # though the Skeleton has only three joints, encoding jointIndices as an # "indexed primvar" means that we need only examine a three-element array # to determine all the joints weighted by this mesh, rather than forty int[] primvars:skel:jointIndices = [0, 1, 2] int[] primvars:skel:jointIndices:indices = [0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 1, 2, 0, 0, 1, 2, 0, 1, 2, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 1, 2, 0, 1, 2, 0, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2, 1, 2, 0] ( elementSize = 3 interpolation = "vertex" ) float[] primvars:skel:jointWeights = [1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0, 0, 1, 0, 1, 0, 0] ( elementSize = 3 interpolation = "vertex" ) # Do not need to specify primvars:skel:geomBindTransform since it is identity rel skel:animationSource = </SkelAnim1> rel skel:skeleton = </Skel1> } ``` ## A "Ready to Skeletally Animate" Asset Following are the contents of a file, *BobAsset.usd*: ```cpp #usda 1.0 def Xform "Bob" { # We expect Bob to be bound to an AnimationBlender at his root, and expect # skel:skeleton to be found as a sibling relationship to the binding rel rel skel:skeleton = </Bob/Skeleton> def Xform "Geom" { def Mesh "Body" { # If the mesh has 1000 points, then the full size of each of the # following primvars would be 5000. "jointIndices" is an # "indexed primvar" that separately encodes the 8 unique Joint # values consumed, from the 5000 uses of those values (in the # jointIndices:indices attribute), which the UsdGeomPrimvar API # can resolve. int[] primvars:skel:jointIndices = [ 0, 1, 2, 3, 15, 16, 17, 18 ] ( elementSize = 5 interpolation = "vertex" ) int[] primvars:skel:jointIndices:indices = [ too many to enumerate ] float[] primvars:skel:jointWeights = [ 3, 4, 5, 6, 7, ... ] ( elementSize = 5 interpolation = "vertex" ) # inverse of transform that takes Body into Bob's # object space, which defines "skeleton space" uniform matrix4d primvars:skel:geomBindTransform = [ ... ] } def Xform "Head" { def Xform "REye" { def Mesh "Schlera" { # This represents a very simple/stupid mesh binding # in which the entire face moves rigidly with the neck. # Because the eyes move rigidly, we can demonstrate # "constant" weighting int[] primvars:skel:jointIndices = [ 47 ] int[] primvars:skel:jointIndices:indices = [ 0 ] ( interpolation = "constant" ) float[] primvars:skel:jointWeights = [ 1.0 ] ( interpolation = "constant" ) # inverse of transform that takes Schlera into Bob's # object space, which defines "skeleton space" matrix4d primvars:skel:geomBindTransform = [ ... ] } } } } # Embedding the "source" Skeleton in the asset guarantees that we can # map the asset's jointIndices to their Joints robustly, and also gives # us the place to specify the skeleton binding pose for the asset. def "Skeleton" ( instanceable = true # It may make sense to lock this reference to a particular version # if Biped17a is independently churning, since the joint mappings # and rest-pose specified herein are strongly dependent on the # topology of the Skeleton references = @Biped17a/Skeleton.usd@ ) { # The skeletal pose to which Bob's mesh bindings apply. # This is an override to the restTransform published with Biped17a uniform matrix4d[] restTransforms = [ ... ] } } ``` # Conclusion This document proposes an object model for supporting skinned, blended, skeletal animation in USD as a core, supported feature for interchange and visualization in the Hydra renderer. The first deployment of these schemas appears in the 0.7.6 USD release, but is very minimal, containing only the most basic schema definitions required to interchange simple skeletal animations. In the interests of delivering useful functionality incrementally, the proposal does not yet cover: - What the associated **Hydra data model** will be. We expect this to follow the schemas' deployment shortly. - Schema and support for **AnimationBlenders** - **Blend Shapes.** We expect the *BindingAPI* schemas to be extensible to support the concerns of blend shapes. - Animations are required to be encoded in a vectorized SRT form. This is ideal for data compactness and data extraction speed, but limited for the goal of **interchanging the source Joint animation curves** between packages. When USD supports a notion of anim curves, we can consider extending the *Animation* schema to allow for animation curves to be included, at least as advisory data, if not an alternate means of driving the core data extraction and blending APIs. - **APIs.** We expect to iterate on useful computation APIs for some time following initial schema deployment and plugin development.