Companies like Pixar are surprisingly open about their craft. Indeed, they publish several papers each year laying out the technologies they’ve developed for their feature films as part of their Renderman renderer. One such paper recently caught my eye: Into the Voyd: Teleportation of Light Transport in Incredibles 2.
Anyone that’s watched Incredibles 2 will remember Voyd, an up-and-coming superhero who can open portals, connecting different areas of space; just like in the Portal game franchise if you’ve ever played those.
A portal pair is made of two parts: a ‘here’ opening and a ‘there’ opening. If you were to place them back to back, then you would see nothing happen. No teleportation occurs. Place them either side of a plane and you get a window. But if you place them in completely different places, you get a CCTV-style view of the other location. These portals allowed for interesting effects in Incredibles 2, where you could look into one portal and you see out of the other – as if your eye were teleportedacross the scene.
Which, it turns out, is pretty much how Pixar, and now Gaia, manages it.
Most of Pixar’s paper is focused on how to make this technology usable by animators, allowing them to visualise the effect inside modelling programs like Maya. This is useful in a production environment, but is ultimately useless for Gaia. Gaia can’t connect to such programs – scenes are coded in by hand – and so all we care about from the paper is how to make the effect work at all.
My first guess at how to approach this method was to simply teleport any intersecting rays on one portal to their relative position on the other, which upon further reading turned out to be exactly how Pixar approached the problem as well.
When a ray intersects with a ‘here’ opening, Gaia calculates the point of intersection and the angle of incidence in the local coordinate system of the opening. On a textured object, these are referred to as UV-coordinates or UVs. Using UVs allow us to ensure that any light rays entering ‘here’ will leave ‘there’ in the same relative position.
The same is true with the angle of the ray. By default, Gaia calculates ray paths using a global coordinate system using two three-vector variables: the origin and the direction. Both are simply positions in 3D space, with Gaia then drawing a line between them – a vector – and looking for any intersections along that line.
By describing these points in the local coordinate system of the opening, we can move the vector to the other opening, convert back to the global coordinate system and continue the ray in the correct direction., as if there was nothing even there.
To do this, we need a a local coordinate system for each opening. Gaia creates one using the surface normal of the shape, creating two more perpendicular vectors to act as axes for our new coordinate system – a set of orthonormal basis vectors. To convert a global coordinate to this local system, we multiply the vector’s components by the basis vectors. For a global position (x,y,z), the same position in a local coordinate system defined by basis vectors {A, B, C} is given by:
$latex \begin{pmatrix} A_1 & B_1 & C_1 \\ A_2 & B_2 & C_2 \\ A_3 & B_3 & C_3 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \vec{A} x + \vec{B}y + \vec{C}z$
By using the inverse of this matrix, we can transform local coordinates into global ones.
All this occurs inside the portal material class, meaning a portal can (in theory) be applied to any primitive, so long as there are two of them linked together properly. The final effect is pretty cool, although in the future I would like to try and implement edge effects similar to those used in Incredibles 2 and Portal.
The portal also has an albedo property, allowing them to tint the scene through the portal: you really can now see the world through rose-tinted glasses.
This isn’t to say there aren’t problems though. The portals cause havoc with the shadow rays, so copies of the main light are added to the light list, but shifted so as to be in the correct location after rays have passed through each portal (as if there were duplicate rooms beyond the portal). And even then there are still some lighting artefacts. More work is needed here for sure.
On the whole, implementing these portals was fun – it was fairly easy to do and provided impressive, easy to see results. It also showed how useful the heat map can be for debugging – there were several bugs that were easily solved by looking at the heat map.
Next stop I think is textures – now that Gaia can handle UV transformations this should be fairly easy to do.