By Ian Failes @ fxguide.com
Star Wars: The Force Awakens has been nothing but triumphant at the box office since its release in December last year. Delicately straddling the balance between conjuring up the nostalgia of the Original Trilogy while also bringing new characters and relationships into the fore, J.J. Abrams’ venture into the Star Wars universe is the first of many to come. And continuing its tour de force in that world is of course Industrial Light + Magic (ILM). Along with partner studios Virtuos, Hybride and Base FX, ILM - itself working between San Francisco, London, Vancouver and Singapore - delivered around 2,100 visual effects shots for The Force Awakens, many of them seamlessly blended into practical sets and creatures and alongside special effects, all crafted together into a film that has reignited enormous interest into the franchise. We also have a bonus interview with stereo conversion house Stereo D.
“People clearly wanted some kind of return to the DNA of those first three movies,” suggests overall visual effects supervisor Roger Guyett. “If you think about the environment those films were shot in, they were clearly shot at a time where it was harder to do visual effects. So invariably they would do things like build sets. We really set out to build as much as we could in-camera and to go to as many locations as we could - and photograph as much in-camera. But you’re still obviously doing a Star Wars movie. It would have been foolish to ignore the contemporary technology that’s available to a modern filmmaker.”
With so much to cover, in this fxguide article we explore just three major visual effects aspects of The Force Awakens: BB-8, the Millennium Falcon chase on Jakku and the battle on Takodana. You can see more coverage in our fxguidetv episode featuring interviews with several ILM artists and supervisors, plus we’ll have exclusive fxinsider stories. And you can listen to fxguide's recent vfxshow which analyzes the visual effects from the film.
BB-8 rocks the galaxy
If there was one effect in The Force Awakens that epitomizes Guyett’s point about capturing what the filmmakers could in-camera while also taking advantage of contemporary technology, it’s BB-8. The endearing - and crowd-pleasing - spherical droid was both a practical on-set puppet crafted by creature effects supervisor Neal Scanlan’s team and a CG character by ILM. “By having a puppeteer operate the droid meant that he had a personality,” says Guyett. “So by building it and giving it a personality and a character and allowing the actors to react to it in-camera means that they understand what it is you’re talking about and understand BB-8’s performance.”
“But it doesn’t mean that all of the work was done in-camera, it just means he has a singular personality,” notes the visual effects supervisor. “Once that was set, we scanned him and took photos for reference, and we built our version of it which probably represents a third of the shots you’ll see BB-8 in. Even I would look at the shots and couldn’t remember which ones were real and which ones weren’t. That really is the benefit of having gone through that process, because you just sort of establish this personality. You go through all the processes of building the real thing and by doing that you define the character so well.”
Getting emotions out of a character that is literally just a ball with a dome top, however, was still a challenge for both the puppeteers and ILM. What helped, suggests ILM animation supervisor Paul Kavanagh, was the combined efforts of Scanlan and ILM’s team in some test shots. “They built BB-8 as a CAD model so they could actually mill the parts for the puppet itself,” explains Kavanagh. “I asked if I could grab that CAD model and I chopped it up in Maya and rigged it. A lot of ideas came out of that - simple animation tests. I only had a day to do something. One thing that came out was the little antenna that wiggles. I decided that it would be kind of fun if his antennae were flexible and wiggled a lot.”
“One thing I didn’t do,” admits Kavanagh, “was make him weave as much as he does at the beginning. He really weaves from left to right. J.J. thought it looked wooden when he moved from A to B so he thought let’s have him move around. That was a good addition that went into the puppet. The other thing is he never stays still. He always rocks around on the ball.”
On set, the puppeteers had several versions of BB-8 with which to work. There was a remote control droid, another that could be pushed along with ‘control’ sticks and a BB-8 that was more static. The droid’s emotions were conveyed using detailed sound design, of course, but also the position of the dome on the ball - tending to droop forward when he was sad and propping back when excited. “I went on set and I was lucky enough to be standing inside the Falcon talking to the puppeteer,” relays Kavanagh, “and we were going through all the different character traits of BB-8. It was a great download for us when we came back from the set to start mocking up BB-8 in the computer. We didn’t want to do anything that he couldn’t really do.”
Flight of the Falcon
The Millennium Falcon. It’s one of the most iconic spaceships in the Star Wars universe, and perhaps in all science fiction. That’s why the revered craft, brought to life in the Original Trilogy via full-size builds and miniatures, was treated with the utmost respect by the visual effects crew. In the new film it plays a key role in Rey, Finn and BB-8’s escape from Jakku before being reunited with its true owner Han Solo, who then later pilots the ship to infiltrate Starkiller base.
With such a large presence in The Force Awakens and in other forms of entertainment (see below), and given its expedited status in Star Wars-lore, ILM spent significant time crafting a digital Millennium Falcon. “We started almost two and a half years ago and one of the first things we started building was the Falcon, which we knew would be in the picture, even before the script was finalized,” outlines asset build supervisor Dave Fogler. “One of the things about is that we think we know it so well. But if you really sit people down in a room and talk about what it looks like it’s really a collective hive mind. There’s a five foot model from Episode IV, a three foot model from Episode V. There was a full scale build for all three original movies and our Episode VII. So when it’s your job to build a representation of that vehicle, there isn’t actually 'a' Falcon.”
“It led to lots of conversations you wouldn’t believe,” adds Fogler. “What should the gray value be of the Falcon? What we’re talking about is miniatures of the Falcon shot on stages. So even if you look at the miniature of the Falcon and the white paint that has a certain value, well, on stage it was lit differently shot to shot. So we would start doing test renders of our Falcon and you show people, and they would say ‘Welllll, it looks too white’. Or ‘it’s too gray’. Dialing in that collective opinion of what it should look like took two years, really, of fiddling.”
ILM decided that its ‘bible’ Falcon would be the five foot model from Episode IV after being able to view and photograph and scan the model at the Lucasfilm archives. “This model is one of the most beautiful things you’ve ever seen,” says Fogler. “It’s a real high point in effects industrial design. It’s really dynamite. We wanted our Falcon to look like that. We scanned it. We photographed it. And then you take that data. Our needs were, not only did we need it to look like that, but we get really close to it in the film. We also acknowledged that in a VR experience, someone might walk up to and look at a bolt on a door. So there was a lot of talk about how far we detailed it. We went pretty far - it’s a remarkably detailed model, our Falcon.”
Interestingly, ILM found that details on the miniature Falcon worked differently than say textures on a digital equivalent. “When you’re filming an object on a stage with lights it’s intentionally fairly gestural with paint and bold with paint,” outlines Fogler. “The motion blur can create the look you need. A miniature which might have a rust drip which is really applied like a finger sometimes. You step back and squint at it and it’s gorgeous. We quite often take a photograph of an actual rust drip and use that in our texture patterns for the digital versions of our models.”
Ultimately the five foot miniature was scanned to ascertain its main proportions and then a modeler began the complicated task of detailing the Falcon’s many sections based on the miniature itself, the other miniatures and the full-scale builds. Says Fogler: “We looked at photography of the miniature and reference of what they were building on set and made decisions about say the trenches on the side of the Falcon that have all that mechanical detail. Interestingly, the choice we made for the trench what we opted for was, if you stepped back and squinted at it, we wanted it to look like the miniature, but if you went up and looked close we used parts they had been using on location.”
ILM’s digital model of the Falcon was dense but a new methodology allowed for efficient building. “The big thing we did on this show was adopt edge creasing in modeling,” explains Fogler. “It’s a methodology that allows you - if you have an edge on a model you can define the curve that’s created either with points and lines telling the renderer how to sub-divide it, or you can add data to that line. So say .4 line will round in a certain way. You can represent forms with fewer points and lines and have just some data there to tell it how to render. So we adopted edge creasing which really cuts our model weight down by more than a third.”
For texturing, ILM again looked to the photographic reference. Artists noticed that the Falcon had been juiced up with decals from miniature car kits and stickers. “A lot of the small text and things that make you believe in the scale are from actual vehicles, or are little oil stickers and highway signs,” states Fogler. “We photographed all those hi-res and put them on there and painted our own elements.”
The studio also undertook an initiative called Unified Assets which looked at modeling, texturing and shading practices that would allow that data to be more easily shared amongst facilities for related films, games, VR experiences and other entertainment uses. "It was an attempt to build a storehouse of digital assets that could be used in the future," says Fogler. "Some of it is just ensuring any methodology we use in building an asset isn’t unique to our facility or the room we’re working in. We set a certain amount of standards, even geometry naming standards, edge creasing, and our shading network was locked down and then all those pieces were packed up into a file system that could be more easily read by different facilities."
The Falcon is first seen on Jakku where Rey and Finn seconder it to escape some Tie Fighters, leading to a dramatic chase along the sand dunes and through a graveyard of downed Imperial Star Destroyers and other ships. The animation of the Falcon would of course be informed by what had been done in the Original Trilogy, while also bringing it to a new level. “When we first put the Falcon together,” says Fogler, “I think there were some assumptions that, now that we have a digital Falcon and can do whatever we want with it, we could do things like actuate the little things that are by the engines - the foils that are always there. We played around with it in some animation tests and then realized that anything that moved on the model would make you startled. For 30 years we’ve looked around at that and nothing moved on it, they didn’t actuate anything on there. It was another lesson in how far we could evolve these things. We only went as far as having the dish on the Falcon wiggle a little bit if it bumps into anything.”
Choreographing the chase began with previs and also an assessment of what plates might be able to be acquired in-camera. Scenes on the surface of Jakku were filmed in Abu Dhabi and so an effort to capture desert and dune reference for the Falcon/Tie Fighter sequence was also begun. “You’re trying to make this chase as exciting and as visceral as you can,” notes Guyett. “It doesn’t matter how much previs you’ve done, you’ve got to figure out how all these shots will be in play and how you’re going to do it, and you cannot get the speed out of a helicopter to shoot in real-time all the time. So if I knew I needed a plate of the camera following the Falcon, I’d find a location that I thought would play for that moment and I’d do some version of that plate. The thing is, the Falcon is traveling at 500 miles an hour, and a helicopter probably travels at 70 or 80 miles an hour, max.”
With those limitations in mind, most of the sequence became a CG build, although scans and photographic reference of the desert would become the clear source material. “By doing that we would always have points of reference and we could go back,” says Guyett. "A very small percentage of that isn’t real - 90 per cent is CG. But by getting all that reference, you just get this really amazing ability to understand some of the nuances and strange things you need to build in to make it real.”
A large part of the chase sequence involved implementing the fallen Imperial star ships - huge constructs that had also previously been realized as miniatures. “We wanted to keep every shot as iconic as possible,” recalls environments supervisor Susumu Yukihiro. “When we put it the way we thought was iconic, well now we had compositions, but it doesn’t look aged or weathered. Sand is always blasting and it might be rusting. But every time we added things we’d get a comment from J.J. that it looked too rusted, too brown, or with too much sand on top and it was hard to recognize. We had to find a balance of how much can we age it and still recognize it. We also had to damage it but if we damage it too much we don’t recognize it.”
Having visited the Lucasfilm archives to see the miniature ships, ILM could also adapt ideas about the chase choreography in surprising ways. “When the Falcon goes into the back of the Super Star Destroyer engine, originally that shot was going into a smaller Star Destroyer,” notes Yukihiro. “But we figured based on how fast the Falcon is flying it would just go through so fast if it was a regular Star Destroyer that is only a mile long. And the Falcon flies at 500 miles an hour. So we thought a better solution for a better story would be to make it fly through the super star destroyer which is actually 12 miles long - over 10 times longer.”
Another benefit from viewing the actual miniatures came from details seen in the models. “With the Super Star Destroyer, the design of the top is not as interesting as the bottom side,” says Yukihiro. “So we actually flopped the ship and placed it that way in the sand.”
As noted, previs helped serve as a guide to the chase, although Guyett enjoyed using it as just a launching point. “For example,” he says, “there’s a moment in the chase where you go over the edge of a star destroyer and go back down towards the desert. If you’ve ever gone on a roller coaster ride there’s always that moment when you do the zero-G thing and you go over the edge very sharply. Those sorts of things we didn’t have in the original previs. So by just changing the dialogue between the characters, different timings and things emerge.”
For Falcon cockpit shots, the actors were filmed inside a set positioned on a gimbal (orchestrated by special effects supervisor Chris Corbould). “We filmed that in England on one of the few days where we had really hard sun,” outlines Guyett. “When you watch the movie, the benefit is seeing how hard the sun is in relation to a movie light. Just seeing that effect on someone - when Daisy’s reacting in the cockpit she just says one line and you see the shadow lines of the cockpit traveling across her.”
The final animation of the Falcon swerving, ducking, weaving and evading the Tie Fighters was crafted by ILM by treating the ship as another character in the film. “It’s such a beloved character from the Original Trilogy that we didn’t want to mess with it,” acknowledges Kavanagh. “We would look to see how many frames it took for the Falcon to roll through the asteroid field from one side to the other. Back then they were really conscious of how big that ship is - it’s not a small ship. It has living quarters, it has cargo. It’s hard to actually make something that looks dynamic and maneuverable but keep the weight. They did that amazingly well back in the day - some very talented animators and motion control artists - amazing work. We just wanted to faithfully represent the Falcon in the computer.”
Kavanagh says one challenge was to faithfully re-create the Falcon’s speed. “We were always saying it’s just moving too fast. We had to keep it always within what we thought was fighter jet speeds. We’d always map how fast it was moving so we had a miles per hour printed on the Falcon so we could see how fast it was moving. If you want to speed it up that’s OK if you’ve got a super long lens shot and you’ve got things coming towards you, but another way of doing that is if you move the camera towards it. We played a lot with the camera as well as the Falcon to try and make the shots a little bit more dynamic but not cheat the Falcon and what it’s doing.”
“In the graveyard sequence we were taking a few more liberties with it,” admits Kavanagh. “It’s more of a car chase - it’s The French Connection. We wanted to make it very, very cool and feel very dynamic like that. So we were taking it and sliding it and skidding it around corners. The tail would come out and slide a lot of the time, before getting momentum and heading off somewhere else. There’s a huge shot in the sequence where the Falcon comes round and up to camera and you see the tail of the Falcon go right by the camera lens and you look right down the engines, and then it kicks into gear and shoots off into the super star destroyer.”
“In the original trilogy they were confined by the length of the stage and the length of the track of the motion control cameras,” adds Kavanagh. “That had an effect on what lens they were using to shoot the ships. So we thought about that and didn’t want to do shots that couldn’t be in the original trilogy. We don’t want to do too many crazy shots. The only real crazy shot is the loop de loop shot - it’s fun and dynamic. That was good fun to work out and do.”
Battle on Takodana
On Takodana, Han Solo and Chewbacca take Rey, Finn and BB-8 to meet Maz Kanata in her cantina bar in the hope of getting a new ship to take BB-8 to the Resistance base. The First Order attacks, but is prevented from obtaining the droid by the arrival of the Resistance X-wing fighters. ILM made use of a partial set build of the ruins of Maz’s castle to construct the battle; extensive R&D to produce scenes of the X-wing’s arriving across the water; and performance capture techniques to bring the completely digital Maz to life.
Han, Rey and Finn meet Maz in her cantina. The character was completely synthetic but her performance came from actress Lupita Nyong’o. To help realize the character, Nyong’o could rehearse with ILM’s facial cam system that is similar to Faceshift. On set, a small person who was the correct height would stand-in for Maz, while Nyong’o also did run-throughs to work out timings and physical action. Then ILM relied on the Medusa Performance Capture system, developed by Disney Research, to acquire and reconstruct Nyong’o’s facial expressions and transplant them to Maz. “We put all the markers on her face and we have the stereo head rig, and it captures her performance,” explains Kavanagh, “and then basically that information is solved into our shape library and comes through with the creature in Maya and you get a whole bunch sliders for all the different shapes which are being driven by that solve. The great thing about it is that you can tweak all those curves and you can push an expression or change the eye direction.” ILM’s proprietary facial capture Snap Solver technology was then used to capture the performance itself, with animators working in keyframe animation to help portray the final character.
“It doesn’t always work one to one,” adds Kavanagh. “You have Lupita’s beautiful face - she’s got a perfectly proportioned face, and you put that into a CG character that’s stylized and has big eyes and a smaller mouth and the distances are different. It’s just not going to work right out of the box. There’s going to be some massaging that needs to go on. Hopefully you’re going to get pretty close. That’s why we spent so much time developing this software because you really want to get something out of the box where you go wow, that looks pretty good. We still have to go in there to massage it to make sure her performance comes out 100 per cent through all of that data and reference.”
The First Order attack leaves Maz’s castle lying in ruins. These were in fact the only things shot on set on the backlot at Pinewood, from which ILM was able to continue to build the scene. “We had pieces of the castle that people were acting around and J.J. was actually staging the action in front of certain pieces,” explains visual effects supervisor Patrick Tubach. “Part of J.J.’s process is to always be fluid about that sort of thing, and to maximize the set. I can’t tell you how few pieces they had and how big they made it look, just by changing angles and lenses and your perspective on the scene. We managed to make a story out of the sequence - taking the time to make a story out of the landscape and figure out where these pieces came from and where the big tower stood and where Kylo’s ship lands.”
Pieces of the debris and ruins were scanned and photographed, while HDRIs and other measurements were also taken. “When you’re actually shooting things for real, conditions change quite a lot,” adds Tubach. “In that scene in particular there was a lot of light changing and it felt very different in the morning as opposed to the afternoon. There were a lot of scenes shot in the rain - nobody will have any idea but we were all soaked. That’s English weather for you!”
The arrival of the X-wings brings hope - and the opportunity for some incredible aerial dog fights. Before they occur, however, the X-wings skim across the water with incredible rooster tail plumes of water behind them - a signature image established early on. “It was such a striking image,” comments Tubach. “The thought of an X-wing in a daytime atmosphere, which we hadn’t ever really seen in a Star Wars film before, Having those huge rooster tails of water. We looked at everything for reference - speed boats and also the Blue Angels who practice right here near ILM. We had a really good idea of what we wanted it to look like, but once you have that reference you need to get that behavior out of the water. We worked a lot with Yanick Dusseault, our VFX art director, who came up with a visual look, and we were playing exactly to his artwork for those first early teaser shots. Then as we got closer and closer to the movie it became more technically correct.”
As the battle continues, aerial dog fights ensue. “That was a fun, fun sequence to do,” says Kavanagh. “The X-wings are outnumbered by the Tie Fighters. Tie Fighters are really maneuverable if you watch the original movies - they’re not a slow craft, they’re really fast. But the X-wing pilots are the best of the best. We sent a couple of X-wings into that fight but they were outnumbered by 60 to 70 Tie Fighters. We had to show how great and professional these X-wing pilots were, these men and women, and not trying to make it look easy. There was a lot of WWII footage we were looking at, to see those gun camera-type shots where pieces of the plane they’re shooting would come off and suddenly pieces of the wing would snap off and the thing would barrel roll down and crash - that kind of amazing real-world photography.”
A gimbal set-up similar to the Falcon shots was established in order to acquire realistic movement and natural light. “They built this massive gimbal that made everyone get sick!” recalls Kavanagh. I think Oscar Isaac is the only person who really enjoyed it. They built the front half of the X-wing, pilots were one at a time going in. They changed out the droid of course with its own name. We had to worry about making sure it was the right droid with the right pilot. It was a like a roller coaster ride where they acted out what was happening. It was great because they threw that gimbal around and you could see them getting pushed around in the cockpits and the lighting change on them. Our job was to take that and try to match move that plate and take dynamic moves. A lot of animators would animate to the background but I would say let’s not worry about that right now - let’s make some dynamic moves to match the cut.”
Unlike the Falcon, much of the environment behind the X-wing pilots is visible in the cockpit shots. “We took advantage of that several times,” notes Guyett, “by actually having the camera outside the X-wing looking back at the pilot like you’re on a mount of an aircraft. We also did some shots where we replaced everything around the pilot, say if you’re looking at the pilot and don’t have all the lights you need. We actually reconstructed the interior of the cockpit so we could have all of the interactive light play on the interior of the cockpit and the pilot - a CG version of the pilot, but the real person there too. All these things are designed to root you in the moment and give you the interactivity which I think is absolutely necessary.”
All images and clips copyright © 2015 Lucasfilm.