I'm not really a fan of the new album, ISAM -- I find it too noodly, like late-era Autechre -- but it was fine background music to have accompanying some of the most mind-blowing visuals I've ever seen.
The set was a tetrissy 12x12x10 grid of blocks, with Tobin embedded inside the central cube. Video was projected on it from the front, but the trick was, they were projecting a 3D rendering of the object onto the object itself, so that textures appeared to be attached to the faces regardless of the angle of the object. It was a mind-blowing effect. Often the structure would appear transparent, with smoke or other objects and particle systems flowing around inside its volume. There were many effects that made it look like the set itself was deforming. The illusions were just fantastic.
It's hard to believe that they pulled this off with just one projector! It helped that the structure was set back on the stage, but in principle something like this could even work in the round, with multiple projectors: you'd just need each one to render with a corresponding observer position.
You should definitely go see this show.
Peter: The show previewing system allows you to see what it looks like in 3D before the set is made. It is a re-projection that takes the final show visual output and puts it through a Light component with a projector map used to tweak the video clips that are rendered to the structure from a specific viewpoint. This enables the team to see how badly the perspective illusion breaks as the observer's position moves up and down and from side to side. It allows for tweaking, trading off depth illusion vs the left/right position of the observer.
It's a balancing act as a few of the pieces are traditional surface mapping and 50% use the "illusion-creating" technique, the "holographic" look while putting content on the faces simultaneously. It's extremely effective considering that we're only working with one projector and really just shooting straight-on.
Matt: Peter made a system with a joystick so you could position yourself in the crowd which allowed us to see how things would look and potential problems we'd have from the crowd's perspective. It enabled us to adapt to having the projector in any position.
Peter: The Show Rendering Unit uses a virtual model of the structure to perform the final projection mapping of the content. It recasts the view point with the perspectival illusion effects from a specific point of view back out to the virtual projector which takes a picture of that. This image is then composited with another part of the rendering piece taking flat HD from frames and mapping them onto a UV grid which does the majority of the face mount, and then it combines those 2 images.
