Production Pipeline Fundamentals for Film and Games

Renee Dunlop

• Publisher: Routledge
• Year: 2014
• ISBN:
  9781317936220
• Edition: 1
• Topic: Informatica
• Subtopic: Media digitali

Every production is built on the backbone of the pipeline. While a functional and flexible pipeline can't assure a successful project, a weak pipeline can guarantee its demise. A solid pipeline produces a superior product in less time and with happier artists who can remain creative throughout the grueling production schedule.

Walk through the foundational layers of the production pipeline, including IT infrastructure, software development practices and deployment policies, asset management, shot management, and rendering management. Production Pipeline Fundamentals for Film and Games will teach you how to direct limited resources to the right technological initiatives, getting the most for every dollar spent.

• 
  
  

• Learn how to prepare for and manage all aspects of the pipeline with this entirely unique, one-of-a-kind guide.

• 
  
  

• Expand your knowledge with real-world pipeline secrets handed to you by a stellar group of professionals from across the globe.

• 
  
  

• Visit the companion website for even further resources on the pipeline.

1

INTRODUCTION

Tim Green; Matt Hoesterey; Hannes Ricklefs; Mark Streatfield; Steve Theodore; Laurent M. Abecassis: Di-O-Matic; Ben Carter: Game Developer and Author, Heavy Spectrum Entertainment Labs; Ben Cole: Head of Software, MPC Vancouver; Dave Stephens: FX Supervisor

Section 1.1 Production Pipeline Fundamentals for Film and Games

There’s no question that modern computer graphics (CG) are an art form, but an art form far removed from the traditional image of the solitary artist hunkered in an attic. In today’s film and games industry, art production is a communal enterprise. Every CG element that the audience sees (or doesn’t realize that they see) in a theater, and every character or environment that a player meets in a video game, is the result of the collective effort of many artists and technicians.

Pipeline developers are the dark matter that holds that constellation of artists and technicians together. Without them, even nontechnical staff would be forced to take on coding tasks, files would proliferate unmanageably, and production schedules spiral out of control. In short, pipeline developers are the unsung heroes of the digital entertainment industry: while they shoulder some of the most difficult tasks, they are rarely noticed until something breaks.

This book is an attempt to shine some light on that dark matter. An introduction to the complex series of technical operations and human relationships that make modern graphics production possible, it brings together the knowledge of experts with just short of a cumulative 700 years of experience in the film, animation and games industries.

In it, we will attempt to provide information that will appeal to a wide range of readers, from students to artists, technical directors, developers, production coordinators and directors. In addition to providing a better understanding of the role pipeline management plays in production, we aim to outline the key challenges faced by today’s pipeline developers, and help you to identify the questions you should address before building a pipeline of your own.

Section 1.2 How This Book Will Help You

For a student, understanding how production pipelines work increases your chances of employment. One of the most common failings of interviewees is a lack of understanding of the production process. Regardless of how good you may be at your particular craft, a prospective employer needs to be certain that you will be able to integrate your talents with those of the other staff working on a project—which may run to many hundreds of people, at studios all around the world. This doesn’t mean becoming an expert in all those different disciplines, but it does mean becoming aware of them. While a programmer doesn’t need to know how to rig a model, they do need to understand the impact a code change that necessitates rerigging all the models used on a project will have on its production schedule.

Established artists looking to transfer from film to games also need to understand how production pipelines differ between the two industries. Although game cinematics are similar to film, in-game models have strict polygon and texture budgets, while the complexity of in-game animations and effects is limited by the need to render every frame in fractions of a second. Conversely, those moving from games to film should understand the need to create highly detailed assets for a medium that often requires photo-realistic output. This book should help to highlight the differences between two industries that share parallel pipelines up to a point, but then suddenly and radically diverge to meet their different goals.

Those working at smaller or newer studios will discover tricks, techniques and insights into how to optimize their existing production pipelines. The book also provides an overview of fields that may be new to you, from audio work to LIDAR (Light Detection and Ranging) scans of environments. Understanding the needs of your coworkers in other disciplines not only helps you to support them better, but to communicate your own needs more effectively, ensuring that the work you create can be integrated more successfully.

Directors also benefit tremendously from understanding the production pipelines they work within, and the impact their decisions have on other departments. It is vital to know why one particular script or scene change will be easy to implement, while another will completely derail a production. Understanding how a pipeline works should help you to make decisions at the appropriate time, avoiding the budgetary or time crises that occur far too often in production.

Finally, a better understanding of game pipelines enables department directors to eliminate dependencies on other teams that slow down production or restrict their artists’ ability to see their assets in-game. Doing so should reduce the integration bottlenecks that plague many games productions.

Section 1.3 What is a Pipeline?

The pipeline is the glue that holds together the work of each artist involved in a production. In this, a pipeline is much the same as an assembly line, in which each worker performs their task before handing off their completed work to the next. The main difference between creating art and creating factory goods is that the creative process incorporates review cycles through which the final product can be refined—and, if necessary, the pipeline itself modified.

It has been said that working on pipelines feels like standing next to a dam and trying to predict where the water will break through. Anticipating problems before they arise is a key responsibility. In that spirit, a pipeline can be defined by the questions it answers. How many shots are there? How many tasks does that break down into, and how many artists are needed to tackle them? Who is working on what? Where is their work being stored? Answering such questions will help you to determine what resources, both virtual and physical, should be allocated to a job: virtual, in the sense that files need server space; and physical, in the sense that artists need desks.

A pipeline has no reason to exist unless it has users. All pipelines have users feeding in data at the start and users consuming data at the end. But a closer look reveals that the flow of data is not linear. Most pipelines are made up of a network of smaller pipelines, forking and reconnecting. At each join, someone (or some process) is consuming data from one pipe and feeding it into the next.

Visualizing a pipeline as a flowchart enables us view patterns of data flow. This is handy for identifying trouble spots: if the flowchart looks like a plate of spaghetti, the underlying production process may be set up awkwardly. Analyzing the flow of work also helps to identify production bottlenecks. A typical example would be for an animation department to have too much work going to a single person for approval, preventing them from completing their own work and causing delays downstream.

As we noted earlier, graphics production is not only a technical process but a creative one. As such, the pipeline does not just enable data to move efficiently “downstream” from concept to execution: it also enables the team to see work in a form that can be evaluated and, if necessary, tweaked for better results. Where possible, time-consuming processes should not be placed in the middle of such creative loops. Instead, the focus should be on the cyclic nature of the pipeline, on iterative processes that are cheaply and reliably repeatable—and on the process that breaks the cycle: creative approval.

Constant iteration is a great way to achieve a high standard of quality, but requires a great deal of reworking. To reduce wasted effort, most pipelines make use of placeholder assets. These are low-quality assets that stand in for the finished versions during these iterative processes. In this way, artists can defer more time-consuming tasks, such as building high-quality assets, until later in production when we can be more confident that major changes are no longer required.

Figure 1.1 An evaluation loop from a game development pipeline. Creative feedback from staff downstream feeds back to those upstream, generating iterative refinements.

Figure 1.2 A more complex evaluation loop. Having to wait for finished models would delay the processes downstream that rely on them, such as level design. In order to gather feedback more quickly, placeholder versions are used early in production, then replaced by the real assets later.

As data is modified, either within creative iterations or as it passes downstream, a further problem arises: how can upstream changes be communicated through the dependency chain without losing or invalidating the modifications already made by other departments? For this reason, successful pipelines must be bidirectional: there must be some mechanism by which updates made at earlier stages of production can propagate to tasks that have already passed beyond them, and to alert the production team to potential problems.

The ideal pipeline is one that enables revisions to take place at any stage of production. No one ever got an effects shot or a level of a game right the first time. Instead, the pipeline must be flexible enough to enable assets to be tweaked over and over again. The faster each user can make changes, the more changes that can be made when the situation demands it.

Of course, realizing these high-level goals—avoiding bottlenecks, supporting iteration and maximizing efficiency—is a complex undertaking. The aim of this book is to provide both strategic guidance on these high-level problems and practical tips on how to implement solutions in your own productions.

Section 1.4 Differences and Similarities Between Film and Game Pipelines

Films, whether live action with visual effects (VFX) or full CG animation, rely on similar technologies to games. Both require models, textures, animation, lighting, particle effects, post-processing and audio. However, the way in which their production pipelines are structured are often quite different.

Film and video present the viewer with a fait accompli: a complete, polished package that presents the vision of the creative team. Unlike a game, the viewer has no say in how the content of a film unfolds: the action, camera angles and pacing are decided entirely by the director and production team. The end result is a linear sequence of images and audio. Short of making popcorn, there is no user interaction.

The biggest challenge in such a pipeline is managing the sheer volume of information that is required to produce photo-realistic imagery. It is not uncommon for a single creature in a VFX movie to comprise hundreds, if not thousands, of individual assets that must be assembled to generate a working render. Over the course of a production, it is often necessary to assemble terabytes of data and pass it through to the renderer or compositor.

Films are also produced under intense time pressure. Whereas games productions tend to run for between 1 and 4 years, a typical film runs from 6 months to a year. On a VFX blockbuster, this may require tens of thousands of person-days of work, which must be divided among hundreds of artists. As timescales and profit margins shrink, production processes must become more agile to enable artists to work in parallel across multiple time zones and adapt to ever-changing schedules.

During a film production, change is the norm: new directors come on board, problems on the shoot result in delays to the handover of assets to visual effect vendors, and there may be unscheduled deliveries for trailers or preview screenings. This makes it imperative that a film pipeline can not only cope with vast quantities of data, but can also do so flexibly.

Figure 1.3 A representative film production pipeline. Note the number of different processes required to create photo-realistic imagery, and their complex interdependencies.

In contrast, in video games, while the production team makes the underlying decisions about the story, visual and audio style, it is the end user who decides how that content plays out. The player controls what angle they view the action from, the order in which they see it, and often even what content they see. Taking this control out of the hands of the creative team presents a daunting challenge.

First, a game requires input data. This data comes from controllers, touch screens, webcams (some with depth information), microphones and a multitude of other input devices. Unlike a film, a game must process all of these streams of information to generate the appropriate output.

Furthermore, processing must be done in real time. All of the elements that make up a game—2D and 3D assets, lighting and physics simulations, AI and audio—must be stitched together by the game code in such a way as to create a seamless, interactive user experience.

Figure 1.4 The many types of asset that make up a modern video game. Unlike in visual effects work, the final images are assembled in real time, at run-time.

As a result, games technology is often proprietary, and constantly changing in substantial and dramatic ways. While games artists often use similar asset-creation tools to their counterparts in film, those tools must integrate with the engine the game runs on, where assets may look and behave quite differently.

The output of the component pipelines may also be quite different to those for film, often resulting in more, smaller chunks of data. Whereas a film animator will typically create long sequences of movements, a game animator will create hundreds of smaller animations—step forward, left foot down, right foot down, walk forward, walk forward with a 45° turn left, and so on—that may be triggered in response to a player’s commands. A blend tree controls how these animations are combined at run-time. Authoring such animations is not just about making them look good, but doing so in such a way that they may work alongside other data.

Figure 1.5 A simplified overview of the game-development process. Integration is the stage at which the individual assets are first assembled into a playable build of the game, and is the point at which problems often occur during production.

In the film industry, the “platform” that the end product runs on is fixed—since every film is ultimately a stream of linear video and audio data, there is no need to reinvent MPEG compression or write new DVD player software for each job. Games, on the other hand, do not have a standard output format. In addition to images and audio, it may be necessary to output rumble or force-feedback data to a controller, output network data to other computers—and, in rarer circumstances, output to some form of custom peripheral built specially for that game.

Therefore, a large part of the game development process is building the very platform the component pipelines are targeting. In film, it’s unlikely that you will discover that, say, displaying color at the correct frame rate has proven too hard, and consequently all your assets will now appear in monochrome. But in games, the runtime engineering team may well suddenly declare that the particle system everyone is using won’t work and needs replacing, or that the lighting model is being overhauled and all the scenes need to be relit.

Finally, pipelines vary according to the style of game being created. If a studio switches from a real-time strategy game (RTS) to a first-person role playing game (RPG), the existing pipeline may need to be thrown out or reworked to accommodate the new requirements. A pipeline that worked for the hundreds of equipment assets (for example, swords and shields) needed in an RTS may not work for the tens of thousands needed in an RPG. Likewise, a game with customizable characters will need its character models built, named, and organized differently from a game with pre-defined characters.

So how do these differences affect the actual production process? The best way to explain that is to take you through the production of a VFX film and compare it to that of a game, in both cases using fictitious—but hopefully representative—projects.

Section 1.5 An Overview of a Film Production

Ben Cole, Head of Software, MPC Vancouver.

On the surface, a visual effects pipeline looks simple. Think of it as a production line. Raw materials, such as plates from the live-action footage, come in and move through a series of repeatable processes that transform them into finished products. The final output of this production line is a collection of 2D image sequences, not too different to those you started with—at least, if you consider footage of a stunt double strapped into a harness and falling onto a mattress to be reasonably similar to a shot of, let’s say Tom Cruise, free jumping ...