Render Settings in USD Proposal
Copyright (C) 2019, Pixar Animation Studios Revision 1.2
- Purpose and Scope
- Overall Design and Concerns
- Concrete Schemas
- Prim and Scene Organization
- Workflow Considerations
- Discussion and Questions
Purpose and Scope
As of release 19.05, USD schemas provide for encoding geometry, lights, cameras, and shading suitable for interchange between DCC's, and direct consumption by high-quality renderers. However, it takes more than a description of the scene elements to produce a rendered image useful in a VFX pipeline - we also require a set of instructions/configuration to the renderer to specify the camera(s) through which it should render, and potentially numerous other settings to configure the products the renderer must produce. While a user will often select such options in a DCC, there is value in being able to record these settings in USD, so that we can maintain a separation of concerns between DCC's and renderers; further, because USD provides composition mechanisms for constructing and varying prim configurations, there is intrinsic value in pipeline complexity management to encoding "Render Settings" in USD, and allowing them to travel with scenes.
The purpose of this document is to propose encoding and organization of Render Settings in USD, such that a properly-encoded USD file contains all the information necessary to invoke a render of the scene in a single "scene configuration" resulting in one or more render products. Some of this will be schemas; some will be namespace conventions (facilitated by schemas) aimed at making it robust to organize Render Settings for fast and easy discovery and composability.
We acknowledge that Render Settings are insufficient specification for producing a VFX "final frame", which often requires multiple render passes that each produce "elements" of the frame that are then composited together... and performing all of this work efficiently often requires explicit job structuring. These concerns are, however, beyond the scope of this document, though we are likely to visit them in subsequent phases of an umbrella project of "Rendering from USD". As a step in that direction, we expect that a given render pass would produce a specific render product (and possibly more than one).
Overall Design and Concerns
We concern ourselves with all of the inputs necessary to configure a renderer to produce a specified set of outputs, since a single render can in general produce multiple outputs. We organize these inputs into three different prim schemas in USD, as part of a "UsdRender" module:
- RenderSettings provides the "global renderer configuration" options, also identifying the camera(s) through which to render, and identifying custom outputs (RenderProducts).
- RenderVar specifies a custom variable computed by the renderer and/or shaders, providing data that can be organized and formatted into an artifact called a RenderProduct.
- RenderProduct creates one or more artifacts (typically files on a filesystem, but more generally specified by output drivers) populated by a set of RenderVars. RenderProducts are generally (multi-channel) images, but can also be 3D encodings such as point-clouds or volumes. Additionally, each RenderProduct can override most settings established on its owning RenderSettings.
We will examine the three schemas in more detail, and then explore surrounding issues such as the organization of these prims in scenes, and workflow considerations.
The usdRender domain defines a set of schemas. They are closely informed by precedent set by RenderMan and Katana.
While we hope to capture the majority of useful render configuration in these core schemas to enhance USD compliance between renderers, we understand that each renderer will likely have some configuration unique to it. As with our UsdLux lighting schemas, our expectation is that plugins that provide render delegates will also include any required additional schemas, as applied API schemas that can be added to RenderSettings and RenderProduct prims.
Prim and Scene Organization
Discovering Render Settings
We have found that some pipeline processes want to operate purely on RenderSettings data, and it therefore is quite valuable to be able to load and discover Settings rapidly, without paying costs proportional to the size of a scene. We therefore propose to enforce a naming convention whereby all UsdRenderSettings prims must be located under the path </Render>. By requiring all render-related specification to be located under a unique root-level prim, we gain several advantages:
- We can leverage UsdStage::OpenMasked() to compose just the rendering-related prims under </Render>, making the extraction of this data very fast, regardless of scene complexity.
- This constrained organization facilitates sharing of rendering data within a production pipeline, as every "shot" can sublayer in sequence, production, and facility-level layers that each contain rendering data, and we are ensured consistent composition (and ease of discovery) of all the data.
This does, in one sense, divorce a scene from its rendering specification, necessitating further management if you need to combine scenes. For example, should you want to reference one scene into another, you will need to reference in the root of the scene's geometry (</World> in Pixar's pipeline), but then the scene's render specification gets left behind. The settings can be recaptured by separately referencing the scene's </Render> prim onto the referencing scene's </Render> prim, although some path corrections may be required. We think these situations arise infrequently enough that they do not outweigh the benefits of Render data always being in a known location. In practice, while we do organize and compose together scene geometry, and we also organize and compose together render settings, these are usually independent concerns that are not often tightly coupled.
Selecting and Combining Render Settings
All RenderSettings prims must live under </Render>, but we stipulate, for simplicity, that any single render can consume only a single RenderSettings prim. We propose a new piece of string-valued SdfLayer metadata, renderSettingsPrimPath that identifies the full path to the active/current RenderSettings prim, though we think it may also be useful for applications (including API in usdImaging) to be able to specify an override to this embedded-in-the-scene value, so that multiple, differently-configured renders can happen simultaneously without mutating the UsdStage (which is not threadsafe).
Artist workflows often involve juggling variations of common settings to quickly iterate on images or to analyze renders. It is useful for common settings to have well-known names, as bookmarks that are easy to refer to, and which can be combined. An example is wanting a quick render by requesting a half-resolution image, with displacement and motion blur disabled. Our opening position on dynamically combining settings is that we would require the application that wants to support it to express it in terms of USD composition arcs, i.e., given an ordered list of paths-to-RenderSettings prims, the application would create a new RenderSettings prim in the session layer, and reference the input RenderSettings prims, in order. This keeps the schema and renderer-interface simpler, though does have several consequences worth mentioning:
- Such "just in time" authoring will cause recomposition; however, it will be tightly scoped to the single, newly introduced prim, and therefore should not be expensive.
- But because combining does require scene authoring, we cannot perform it for two different renders simultaneously from the same stage - a single thread would need to prep RenderSettings for both renders before splitting for the actual renders.
- If combining arbitrary RenderSettings prims, the specific "list operators" used in specifying the products relationship become more important - for example if two RenderSettings each create an explicit list of products, then combining the two prims by reference will select only the products of the stronger, whereas if they each prepend or append products, they will combine reasonably.
Grouping RenderVars and RenderProducts
Because all RenderProducts and RenderVars are identified by relationship, they can robustly live anywhere within the </Render> tree. We leave it as a pipeline decision as to whether further organization is worthwhile (e.g. all RenderVars live under </Render/Vars>, all RenderSettings under </Render/Settings>, etc), and stipulate applications should make no assumptions about where under </Render> any of the Render prims should live.
There may be value, in some pipelines, to consider RenderProducts to belong to a given RenderSettings prim; therefore, if we were to require them to be grouped under a RenderSettings prim, then if we use USD composition to combine RenderSettings prims, as suggested in the previous section, then we get the desirable property that the RenderProducts come along, uniquely, as well, which means that their member properties can be overridden uniquely for the combined/composed result RenderSettings prim.
Although not a part of this proposal, we may provide derived RenderVar types for common types such as P, N, Nn, Z, so that no configuration is required to consume these basic render variables.
Interpretation of RenderVar Prims in Hydra
For the majority of prim types in USD schemas, there will be one or more rprim, sprim, or bprim objects created in Hydra, associated with the scene prim’s path, and at first one might assume here would be a single bprim (buffer prim) associated with each RenderVar prim in a UsdStage. However, because we have opted to allow RenderVars to be shared among multiple active RenderProduct prims, this association does not hold. A RenderVar targeted by two different RenderProducts (which may each have different cameras and resolutions) serves only as a prescriptive input to the RenderProduct to create bprim buffers unique to itself, with path identifiers scoped to the RenderProduct.
Discovering All Potential Shader-based RenderVars
For fast discoverablilty by render-prepping software, we have stipulated that the RenderVars consumed by the configured RenderSettings must be located under the root-level </Render> location. However, typically, available shader outputs will be published within assets, which may be scattered throughout large scenes. We do not consider it practical/robust to require that </Render/Vars> (or however we choose to organize RenderVars) be perpetually kept in sync as a complete manifest of the RenderVar contents of the entire scene. Therefore, any artist or process that needs to browse the available RenderVars for a scene is faced with a more expensive discovery process.
Pixar's experience is that so much pipeline setup is required to expose a shader output for useful consumption that we typically only have a handful that are well-known to artists, and therefore a discovery process is not needed. If other studios have different experience, we can consider further schema to aid in discovery, for example requiring available shader outputs to be identified at the UsdShadeMaterial-level, so that traversing shader networks is not required.
Interactive vs. Batch Rendering
UsdRender is primarily aimed at describing a batch render process which produces image files in a filesystem.
However, it does not describe the broader context about how those files are managed within the filesystem or in a distributed computing environment. The output files are always assumed, for present discussion purposes, to be "local" to a renderer's working directory.
It is a later task to describe the broader details of batch processing of images and how the produced images are shipped around a network.
Similarly, we assume an interactive rendering system will, where sensible, be able to redirect rendering results to in-memory buffers or viewports as needed.
Discussion and Questions
ID passes/variables in renders are quite common and enormously useful. However, we do not seem to yet have standardized way to describe them, and therefore they tend to be renderer and pipeline specific. If we can standardize on a description and encoding, we might be able to tackle the problem of providing a standard RenderVar for ID information. There are a few issues to consider:
- how to resolve multiple samples per pixel; could use deep images, or could only return closest value
- how to map values in the ID AOV image to objects in the scene; whether image metadata is needed to store the mapping
- how to handle instancing (including nested instancing)
- how to handle ID's in the presence of participating media and transparent surfaces (glass, water)
Cryptomatte offers a compelling option for several of these issues.
As long as the eyes are not allowed to see different things (i.e. no differences in visibility), then stereo output from a single render can be encoded as two RenderProducts each specifying different cameras.
Camera Exposure Curves
Marquee renderers often allow clients to control the exposure of an image between the camera's shutterOpen and shutterClose, typically with a curve. Since these controls currently tend to be highly renderer-specific, we expect them to be encoded in applied API schemas, either on a UsdGeomCamera prim (since that is where shutter information already lives), or on a RenderSettings prim.
Denoising, Color Correction, and Tasks
We stipulated in the introduction that we are not, at this time, addressing the needs of compositing with these schemas. However, such "filtering" tasks as denoising and color correction seem both smaller and more requisite for producing a usable element than general compositing. Theoretically, one could encode these behaviors in a RenderProduct's driver:parameters, but that does not represent an interchangeable solution. Addressing these needs is a likely next step in the evolution of these schemas.
Why Locality of Overrides is Valuable
One goal of this schema proposal is that a USD scene be able to contain multiple, alternative collections of settings from which different renders can be initiated without requiring any stage mutations. A second goal, as discussed in Selecting and Combining Render Settings, is to make it easy to produce variations and combinations of Settings through composition (e.g. in a stage's session layer). Both of these goals are facilitated by keeping all required overrides for different configurations on render prims, because requiring overrides on renderable scene objects (even cameras) limits the number of differing render configurations we can encode in a scene.
Image-Mapping-Related Options on Camera or RenderSettings?
Several observers have wondered whether concerns such as screen window conformance, and possibly even image resolution might better be expressed as concerns of the UsdGeomCamera prim. This would mean we would use regular USD overrides on camera prims to specializeconformance properties. If we want to preserve the desirable locality of overrides for RenderSettings, we would create a new Camera prim (under the RenderSettings or RenderProduct that uses it) that references the "original" camera, thus preserving the ability to combine RenderSettings themselves via composition arcs.
Doing so, however, creates confusion for scene consumers, because we have essentially duplicated any cameras used in rendering. Which should a client import? How should a client manage overrides on the original vs duplicated cameras inside a DCC when exporting the results to USD? Further, we believe there actually is a logical separation between camera and imaging concerns.
- A camera defines a view frustum, describes lens properties and exposure, and (for cameras used for rendering) are in the purview of a layout/photography department under a director of photography.
- Producing an image as viewed through a camera requires further specification of image resolution, and how to map the screen window of the camera's view frustum to that image. These are generally concerns of rendering and/or lighting departments, and need to be varied for different renders, whereas the cameras concerns, as outlined here, do not.
We conclude that image-related properties, therefore, best reside in RenderSettings, not on Camera prims. The single concession for usability we make to "overlapping concerns" is allowing a RenderSetings prim to override a camera's exposure to be instantaneous, as this is a very common per-render adjustment.
Crop Windows and Region-of-Interest
Some rendering toolsets provide separate notions of a crop window and a region-of-interest. The former is a pipeline- or rendering-centric way to configure the renderer, and the latter is an interactive user affordance to temporarily steer render compute resources to a subset of the image. Given that they have the same effect of truncating the computed data window, and the cumulative effect is just to take the the intersection of these restricted regions, we have decided with UsdRender to only provide a single cropWindow attribute and leave any interactive region-of-interest concerns as a toolset-side concern. Specifically, we expect an interactive rendering tool could add a Session-layer override to the crop window to intersect any further region-of-interest known to the application.
We intend UsdRender to provide utilities to support this and related calculations.