Digital compositing

<h2 id="mathematics">Mathematics</h2>
<p>The basic operation used in digital compositing is known as <a href="/facts/Alpha_blending/7uUBtjrt">alpha blending</a>, where an <a href="/facts/Opacity_(optics)/l90q0sAV">opacity</a> value, 'α', is used to control the proportions of two input <a href="/facts/Pixel/UzwBvXBV">pixel</a> values that end up a single output pixel.
</p><p>As a simple example, suppose two images of the same size are available and they are to be composited. The input images are referred to as the foreground image and the background image. Each image consists of the same number of pixels. Compositing is performed by mathematically combining information from the corresponding pixels from the two input images and recording the result in a third image, which is called the composited image.
</p><p>Consider three pixels;
</p>
<ul><li>a foreground pixel, f</li>
<li>a background pixel, b</li>
<li>a composited adetpixel, c</li></ul>
<p>and 
</p>
<ul><li>α, the opacity value of the foreground pixel. (α = 1 for an opaque foreground, α = 0 for a completely transparent foreground). A monochrome <a href="/facts/Raster_image/HkXNc8oI">raster image</a> where the pixel values are to be interpreted as alpha values is known as a <a href="/facts/Matte_(filmmaking)/oSja3keR">matte</a>.</li></ul>
<p>Then, considering all three <a href="/facts/Colour_channel/eqn5Q5V4">colour channels</a>, and assuming that the colour channels are expressed in a γ=1 colour space (that is to say, the measured values are proportional to light intensity), we have:
</p>
cr = α fr + (1 − α) br
cg = α fg + (1 − α) bg
cb = α fb + (1 − α) bb
<p>Note that if the operations are performed in a colour space where γ is not equal to 1 then the operation will lead to non-linear effects which can potentially be seen as <a href="/facts/Aliasing/Pzg84HL0">aliasing</a> artifacts (or "<a href="/facts/Jaggies/J72S809e">jaggies</a>") along sharp edges in the matte.  More generally, nonlinear compositing can have effects such as "halos" around composited objects, because the influence of the alpha channel is non-linear.  It is possible for a compositing artist to compensate for the effects of compositing in non-linear space.
</p><p>Performing alpha blending is an expensive operation if performed on an entire image or 3D scene. If this operation has to be done in real time video games, there is an easy trick to boost performance.
</p>
cout = α fin + (1 − α) bin
cout = α fin + bin − α bin
cout = bin + α (fin − bin)
<p>By simply rewriting the mathematical expression, one can save 50% of the multiplications required.
</p>
<h3>Algebraic properties</h3>
<p>When many partially transparent layers need to be composited together, it is worthwhile to consider the algebraic properties of compositing operators used.  Specifically, the associativity and commutativity determine when repeated calculation can or cannot be avoided.
</p><p>Consider the case when we have four layers to blend to produce the final image: F=A*(B*(C*D)) where A, B, C, D are partially transparent image layers and "*" denotes a compositing operator (with the left layer on top of the right layer).  If only layer C changes, we should find a way to avoid re-blending all of the layers when computing F.  Without any special considerations, four full-image blends would need to occur.  For compositing operators that are <a href="/facts/Commutative/WaYxJLxd">commutative</a>, such as <a href="/facts/Additive_blending/8lK8znNX">additive blending</a>, it is safe to re-order the blending operations.  In this case, we might compute T=A*(B*D) only once and simply blend T*C to produce F, a single operation.  Unfortunately, most operators are not commutative.  However, many are <a href="/facts/Associative/EQX8lV6r">associative</a>, suggesting it is safe to re-group operations to F=(A*B)*(C*D), i.e. without changing their order.  In this case, we may compute S:=A*B once and save this result.  To form F with an associative operator, we need only do two additional compositing operations to integrate the new layer S, by computing F:=S*(C*D).  This expression indicates compositing C with all of the layers below it in one step, and then blending all of the layers on top of it with the previous result to produce the final image in the second step.
</p><p>If all layers of an image change regularly but many layers still need to be composited (such as in <a href="/facts/Distributed_rendering/0wMWTncz">distributed rendering</a>), the commutativity of a compositing operator can still be exploited to speed up computation through <a href="/facts/Parallelism_(computing)/nQzcpzQt">parallelism</a> even when there is no gain from pre-computation.  Again, consider the image F=A*(B*(C*D)).  Each compositing operation in this expression depends on the next, leading to serial computation.  However, associativity can allow us to rewrite F=(A*B)*(C*D) where there are clearly two operations that do not depend on each other that may be executed in parallel.  In general, we can build a tree of pair-wise compositing operations with a height that is logarithmic in the number of layers.
</p>
<h2 id="software">Software</h2>
<p>The most historically significant nonlinear compositing system was the <a href="/facts/Cineon/9E0uSeNn">Cineon</a>, which operated in a logarithmic color space, which more closely mimics the natural light response of film emulsions (the Cineon System, made by <a href="/facts/Kodak/zMnK74Cp">Kodak</a>, is no longer in production). Due to the limitations of processing speed and memory, compositing artists did not usually have the luxury of having the system make intermediate conversions to linear space for the compositing steps.  Over time, the limitations have become much less significant, and now most compositing is done in a linear color space, even in cases where the source imagery is in a logarithmic color space.
</p><p>Compositing often also includes <a href="/facts/Image_scaling/qNoeujzm">scaling</a>, <a href="/facts/Photograph_manipulation/HfFJFmFg">retouching</a> and <a href="/facts/Colour_correction/qjec3KIG">colour correction</a> of images.
</p>
<h3>Node-based and layer-based compositing</h3>
<p>There are two radically different digital compositing workflows: node-based compositing and layer-based compositing.
</p><p>Node-based compositing represents an entire composite as a <a href="/facts/Directed_acyclic_graph/k6zq1os9">directed acyclic graph</a>, linking media objects and effects in a procedural map, intuitively laying out the progression from source input to final output, and is in fact the way all compositing applications internally handle composites. This type of compositing interface allows great flexibility, including the ability to modify the parameters of an earlier image processing step "in context" (while viewing the final <a href="/facts/Compositing/xGdHadIj">composite</a>). Node-based compositing packages often handle <a href="/facts/Keyframing/SeHhpINL">keyframing</a> and time effects poorly, as their workflow does not stem directly from a timeline, as do layer-based compositing packages. Software which incorporates a node based interface include <a href="/facts/Natron_(software)/yb5aT7aJ">Natron</a>, <a href="/facts/Apple_Shake/F1mRBheu">Shake</a>, <a href="/facts/Blender_(software)/yzuvB0fL">Blender</a>, <a href="/facts/Blackmagic_Fusion/SYD7qp3j">Blackmagic Fusion</a>, and <a href="/facts/Nuke_(software)/60htq9oq">Nuke</a>. Nodes are a great way to organize the effects in a complex shot while maintaining a grip on the smaller details.
</p><p>Layer-based compositing represents each media object in a composite as a separate layer within a timeline, each with its own time bounds, effects, and keyframes. All the layers are stacked, one above the next, in any desired order; and the bottom layer is usually <a href="/facts/Rendering_(computer_graphics)/YYxfnmAq">rendered</a> as a base in the resultant image, with each higher layer being progressively rendered on top of the previously composited of layers, moving upward until all layers have been rendered into the final composite. Layer-based compositing is very well suited for rapid <a href="/facts/2D_computer_graphics/j6dBxDpX">2D</a> and limited <a href="/facts/3D_computer_graphics/zlWW6eaF">3D</a> effects, such as in motion graphics, but becomes awkward for more complex composites entailing numerous layers. A partial solution to this is some programs' ability to view the composite-order of elements (such as images, effects, or other attributes) with a visual diagram called a <a href="/facts/Flowchart/BtNXBfUG">flowchart</a> to nest compositions, or "comps," directly into other compositions, thereby adding complexity to the render-order by first compositing layers in the beginning composition, then combining that resultant image with the layered images from the proceeding composition, and so on.
</p>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Broadcast_designer/4RehSjI9">Broadcast designer</a></li>
<li><a href="/facts/Chroma_key/PJ2XcxFR">Chroma key</a></li>
<li><a href="/facts/Digital_asset/PmW2dpkn">Digital asset</a></li>
<li><a href="/facts/Digital_cinema/fc312V6y">Digital cinema</a></li>
<li><a href="/facts/Digital_on-screen_graphic/aknrwAPU">Digital on-screen graphic</a></li>
<li><a href="/facts/Gamma_correction/cApBLTYj">Gamma correction</a></li>
<li><a href="/facts/Graphics_coordinator/PGcprIRr">Graphics coordinator</a></li>
<li><a href="/facts/Motion_graphic/vvRyrCIc">Motion graphic</a></li>
<li><a href="/facts/Motion_graphic_design/tqYk5UQN">Motion graphic design</a></li></ul>
<h2 id="external-links">External links</h2>
<ul><li><a href="https://compositor.digital">Digital Compositor</a> Simplified online digital compositor to create a photomontage.</li></ul>
<h2 id="further-reading">Further reading</h2>
<ul><li>Mansi Sharma; Santanu Chaudhury; Brejesh Lall (2014). <a href="http://dl.acm.org/citation.cfm?id=2683555"><i>Content-aware seamless stereoscopic 3D compositing</i></a>. Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image Processing, ACM New York, NY, USA. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1145%2F2683483.2683555">10.1145/2683483.2683555</a>.</li>
<li><a href="/facts/Tom_Porter_(computer_scientist)/P4FV6UAS">T. Porter</a> and <a href="/facts/Tom_Duff/rCd1cKvB">T. Duff</a>, "Compositing Digital Images", Proceedings of <a href="/facts/SIGGRAPH/gmRKkIlG">SIGGRAPH</a> '84, 18 (1984).</li>
<li>The Art and Science of Digital Compositing (<a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-12-133960-2)</li>
<li>Samuel Collett, "<a href="https://unmaskingvfx.com/?p=146">Node Based Compositing Explained(Video)"</a> (2023)</li></ul>

Digital compositing open-in-new

Digital compositing