Scanline brings crowds - and horses - to Game of Thrones with Massive

When HBO’s Game of Thrones finished earlier this year - with a rousing season 8 - the show had already cemented its place in the history books of visual effects. This latest season was no exception, with vast CG environments, digital creatures and untold destruction created by an army of VFX artists.

Among those were artists from Scanline VFX, who were responsible for several sequences of destruction and its aftermath in King’s Landing. Although partial sets and various pieces of live-action could be filmed to realize this part of the Game of Thrones story, much of it also had to be digitally invented

In one scene in episode six of the final season, for example, Daenerys Targaryen delivers a speech to her followers, the Dothraki and the Unsullied, in the ruins of the city. The followers here were mostly all synthetic, crafted in Massive as CG crowds by Scanline. The studio also relied on Massive for shots in episode five where the dragon Drogon attacks King’s Landing, and various inhabitants and soldiers flee.

The Massive workflow at Scanline

The use of Massive is well-entrenched at Scanline. From a pipeline point of view, Scanline has written its own standalone plugin that replaces the V-Ray plugin provided with Massive. This plugin takes data from Massive standalone and plugs that into the V-Ray renderer as a direct bridge. The studio has also written a set of tools to visualize Massive in 3ds Max, which fits into their pipeline more conveniently than Massive for Max.

“As well as that,” adds Scanline visual effects supervisor Mohsen Mousavi, “we have developed some automatic cloth simulation that runs on top of a Massive simulation. We do our simulation based on the infrastructure and the pipeline that we have set up, and then we'll run that through automatic cloth simulation based on predefined cloth rigs in Maya that are set up for every agent. So if we have a simulation of a thousand agents combined of different type of agents all packed into one simulation, we can handle that with ease.”

Mousavi identifies how this worked in Game of Thrones. “Say we have a shot of the Unsullied running and Lannisters running and also civilians running,” he says. “That’s one big Massive simulation and we can basically send that to the farm after the Massive simulation is done to run a very high-end Maya cloth simulation automatically on every agent in Massive based on the type of rig that was used to generate that agent. Then that simulation, once it is done, that gets converted automatically to our V-Ray pipeline to be able to render that in Max directly in V-Ray.”

This is extended even further to allow for Massive agents to interact with effects, for example, if one of the agents is on fire and running away. “We have built all of that pipeline to be able to take a Massive agent and bring it into our Flowline proprietary fluid simulation toolset and be able to interact with thousands and thousands of Massive-generated characters,” says Mousavi.

Acquiring motion of people, and horses

Scanline utilized its in-house motion capture facility to acquire animations, actions and behaviors for the different ‘factions’ of people required in Game of Thrones. The studio also used the Massive horse agent. “Early on, we did some research to find out what was out there that could help us stay ahead of the game,” recalls Mousavi. “We didn’t want to spend too much time developing a horse agent from scratch because that is quite a big task.”

“We looked at everything that was available in the market and the one that really stood out was Massive,” continues Mousavi. “So we contacted Massive and they sent us a test build. We took that set up and plugged it in. Massive is very open and all the formats are easy to maintain and develop so it was flexible enough for us to take the agent that they had developed, and replace their riders with the Dothraki agent, maintaining all the brain and logic relation. The same goes with the horses.”

For shots in episode six where the Dothraki and others are situated in a large plaza in front of Daenerys, there were approximately 4000 horses that needed to be inserted. “We were almost done with the shots,” relates Mousavi, “when we got a note from the producers that the level of the cheering and the level of excitement in the crowd was not enough, that it didn’t respond to what Daenerys was saying.”

“That came really late in the game. But it was pretty straightforward to take our set-up and be able to completely change the look of the crowd and make them be more active. We were also able to use our motion capture facility do do several new motions that matched more closely to what was in the practical footage and plug that back into the horse agent.”

In episode five, there was a shot where the character Harry Strickland is killed as the King’s Landing gate explodes from dragon fire. It was full 360 degree camera turn with horses running towards camera. Some aspects of the scene were live-action but much of it had digital horses.

“So that was another scenario where we were able to build a custom hero setup using some of the components of the massive horse agent,” explains Mousavi. “We took the motions of those horses and plugged them into our hero muscle driven horse rig and animate on top to make sure the performance matched the practical horses from the raw plate.”

"There were 300 or 400 horses that were developed with muscle set-ups," adds Mousavi". "Once the simulation of the muscles and the skin was done, we took all of that data, we then piped it back into our Massive system to be able to allow for the rendering and instancing to happen automatically.”

What Massive lets Scanline do

Mousavi says Scanline uses several crowd tools, but has found that other ‘out of the box’ tools do not necessarily allow for full control or integration into their pipeline. “If you're bringing something new into the infrastructure, it needs to be ‘plugible’ everywhere and that's where Massive truly shines,” suggests Mousavi.

He also identifies the concept of fuzzy logic inside Massive as something that makes the tool stand alone from others. “You can get a black box of say 20 different behaviors from other tools that you can use - really good ones - but you're limited to that. In Massive, however, while there’s only a few node options, once you combine them based on the fuzzy logic rules, you can get almost any kind of behavior. You’re not bound to a set of particular behaviors at all.”