To 'animate' is literally 'to give life to'. 'Animating' is moving something which can't move itself. Animation adds to graphics the dimension of time which vastly increases the amount of information which can be transmitted. In order to animate something, the animator has to be able to specify, either directly or indirectly, how the 'thing' is to move through time and space. The basic problem is to select or design animation tools which are expressive enough for the animator to specify what s/he wants to specify while at the same time are powerful or automatic enough that the animator doesn't have to specify the details that s/he is not interested in. Obviously, there is no one tool that is going to be right for every animator, for every animation, or even for every scene in a single animation. The appropriateness of a particular animation tool depends on the effect desired by the animator. An artistic piece of animation will probably require different tools that an animation intended to simulate reality.
There are two main categories of computer animation: computer-assisted animation and computer generated animation. Computer-assisted animation usually refers to 2D and 2 1/2 dimensional systems that computerize the traditional hand-drawn animation process. Interpolation between key shapes is typically the only algorithmic use of the computer in the production of this type of animation (in addition to the more 'non-animation' uses of the computer such as inking, implementing a virtual camera stand, shuffling paper, and managing data).
This document is mainly concerned with computer-generated animation. For discussion purposes, motion specification for computer-generated animation is divided into two categories: low level techniques (techniques that aid the animator in precisely specifying motion), and high level techniques (techniques used to describe general motion behavior) .
Low level techniques consist of techniques, such as shape interpolation algorithms (in-betweening), which help the animator fill in the details of the motion once enough information about the motion has been specified by the animator. When using low level techniques, the animator usually has a fairly specific idea of the exact motion that he or she wants.
High level techniques are typically algorithms or models used to generate a motion using a set of rules or constraints. The animator sets up the rules of the model, or chooses an appropriate algorithm, and selects initial values or boundary values. The system is then set into motion, so to speak, and the motion of the objects is controlled by the algorithm or model. The model-based/algorithmic approaches often rely on fairly sophisticated computation, such as physically based motion control. In the Appendices, the reader will find background information on various mathematical areas related to animation, such as vector algebra and numerical techniques.
This distinction is primarily for pedagogical purposes. In practice, of course, these categories are the extremes which characterize a continuum of approaches. Any technique requires a certain amount of effort from the animator and a certain amount of effort from the computer. One of the things which distinguishes animation techniques is whether its the animator or the computer which bears most of the burden. Motion specification aids are those techniques which seem to require more input from the user and fairly straightforward computation. Model-based approaches, on the other hand, require less from the animator and more computation. But the categories are really artificial. Approaches which I call motion specification aids could be discussed as algorithmic approaches which happen to require more input from the user.
Another way to characterize the difference between techniques is to look at the level of abstraction at which the animator is working. In one extreme, at a very low level of abstraction, the animator could color in every pixel individually in every frame. At the other extreme, at a very high level of abstraction, the animator could tell a computer to 'make a movie about a dog'. Presumably, the computer would whirl away while it computes such a thing. A high level of abstraction frees the animator from dealing with all of the details. A low level of abstraction allows the animator to be very precise in specifying exactly what is displayed when. In reality, animators want to be able to switch back and forth and work at various levels of abstraction. The challenge to developing animation tools is designing the tools so that animators are allowed to work at high levels of abstraction when desired, while providing them the ability to work at low levels when needed.
This document is not intended to cover the theory of animation or animation production issues (although overviews are given), but rather the general techniques and algorithmic approaches used in computer-generated animation.
When animation is recorded for later viewing, it is typcially presented in film or video formats by recording a series of still images. This is possible because the eye-brain assembles a sequence of images and interprets them as a continuous movement. Persistence of motion is created by presenting a sequence of still images at a fast enough rate to induce the sensation of continuous motion.
The receptors in the eye continually sample light in the environment. The only limitation on motion detection is the reaction time of those sensors and on certain mechanical limitations such as blinking and tracking. If an object moves fast enough, then the receptors in the eye will not be able to respond fast enough for the brain to distinguish a sharply defined, individual detail; motion blur results.
In either film or video, a sequence of images is recorded which can be played back at rates fast enough to fool the eye into interpreting them as continuous motion. Of course, in order to save resources, this rate is kept as low as possible while still maintaining the persistence of motion. Under some viewing conditions such as room lighting and viewing distance, the rate at which single images must be played back in order to maintain the perception of motion varies. The image is said to flicker when the perception of continuous motion fails to be created. The object appears as a rapid sequence of still images to the eye-brain.
There are actually two rates that are of concern. One is the number of images per second that are displayed in the viewing process. The other, is the number of different images that occur per second. The former is the playback rate; the latter is the sampling rate or update rate. For example, images are always played back at 30 images per second on a TV, but in some Saturday morning cartoons there may be only six different images per second with each image repeated five times.
Another low-tech animation piece of equipment was the flipbook. The flipbook was a tablet of paper with an individual drawing on each page so the viewer could flip through them. This was also popular in the 1800s. However, these devices were little more than parlor curiosities used for light entertainment.
While studying the early days of conventional animation is interesting in itself, the purpose for presenting an overview here is to gain an appreciation of the technological advances which drove the progress of animation in the beginning. The earliest hint of using a camera to make lifeless things appear to move was by Meleis in 1890 using simple tricks. The earliest pioneers in film animation were Emile Cohl, a Frenchman who produced several vignettes, J. Stuart Blackton, an American, who actually animated 'smoke' in a scene in 1900 and who is credited with the first animated cartoon in 1906, and the first celebrated animator, Winsor McCay, an American best known for his works Little Nemo and Gertie the Dinosaur.
Winsor McCay is considered by many to have produced the first popular 'animation'. Like many of the early animators, he was an accomplished newspaper cartoonist. He redrew each complete image on rice paper mounted on cardboard. He was also the first to experiment with color in animation. Much of his early work was incorporated into vaudeville acts in which he would 'interact' with the animated character on the screen. Similarly, early cartoons often incorporated live action with animated characters. When considering such a popular entertainment format, in order to appreciate the audience reaction the reader should keep in mind the relative naivete of the viewers at that time; they had no idea how film worked much less what hand-drawn animation was. It was, indeed, magic.
The first major technical developments in the animation process can be traced to the work (and patents) of John Bray starting in 1910. His work laid the groundwork for the use of translucent cels ( short for celluloid) in compositing multiple layers of drawings into a final image as well as the use of grey scale (as opposed to black and white) drawings. Later developments by Bray and others enhanced the overlay idea to include multiple translucent pieces of celluloid (cels), added a peg system for registration, and the drawing of the background on long sheets of paper so that panning (translating the camera parallel to the plane of the background) could be performed more easily. Out of Bray's studio came the likes of Max Fleischer (Betty Boop), Paul Terry (Terrytoons), George Stallings (?), and Walter Lantz (Woody Woodpecker). Fleischer patented rotoscoping in 1915. Rotoscoping is drawing images on cells by tracing over previously recorded live action. Bray experimented with color in one 1920 short.
During this time, animation as an art form was still struggling. The first animated character with an identifiable personality is Felix the Cat by Otto Messmer which appeared in the early 1920s in Pat Sullivan productions. In the late 1920s however, new forces had to be reckoned with: sound and Walt Disney.
Some of Disney's innovations in animation technology were the use of a storyboard to review the story, the use of pencil sketches to review motion, and the multi-plane camera stand (see Figure 2). In addition, Disney pioneered the use of sound and color in animation (although not the first to use color). Disney also studied live action sequences to create more realistic motion in his films. When he used sound for the first time in Steamboat Willie (1928), he gained an advantage over his competitors.
Camera stand animation is more powerful than you might think. A camera stand allows the parallax effect - moving of backgrounds at different rates as the observer pans across an environment to create the illusion of depth - and zooming. Each of the planes can move six directions (right, left, up, down, in, out) as well as the camera moving in and out. By keeping the camera lens open during movement, figures can be made to appear extruded into shapes of higher dimension, simulate motion blur, exhibit depth attenuation.
With regard to the art form of animation, Disney perfected the ability to impart unique, endearing personalities in his characters including Mickey Mouse, Pluto, Goofy, the three little Pigs, and the seven Dwarfs. He also developed mood pieces of animation including Skeleton Dance and Fantasia. He also promoted the idea that the mind of the character was the driving force of the action and that a key to believable animated motion was the analysis of real life motion.
In this type of animation, a physical object is manipulated, the camera takes an image of it, the object is manipulated again, another image is taken of it, and the process repeats to produce the animated sequence. Willis O'Brian of King Kong fame is usually considered the dean of stop-action animation. His understudy who went on to do impressive work in his own right was Ray Harihousen (Mighty Joe Young, Jason and the Argonauts).
The National Film Board of Canada is worthy of a special mention for their animation efforts - pinhead, sand, clay, etc. Also, worthy of mention is Zagreb (?) Studios in Yugoslavia. Because of computer animation's dependence on video technology, it is somewhat related to video art, although most of the time the relationship is viewed in the realm of post-production techniques.
The terms used to discuss the subparts of the animation production comes from film production in general. The overall animation goes by various names. Here, we will refer to it as the presentation. This refers to the entire piece under production. The major parts of the presentation are the acts. An act is a major episode within the animation and is usually identified by an associated staging area; a presentation usually consists of one to a dozen acts. An act is broken down into several scenes; a scene identifies one venue of continuous action. A scene is broken down in one or more shots. A shot is a continuous camera recording. A shot is broken down into the individual frames of film. So we have the following hierarchy:
The production of animation typically follows the following pattern. First, a preliminary story is decided on and a story board is developed which lays out the action scenes by sketching representative frames and writing down an explanation of what is taking place. This is used to review and critique the action. Then the detailed story is worked out which identifies the actions involved in more detail. Key frames are then identified and produced by master animators to aid in confirmation of character development and image quality. A test shot is a short sequence rendered in full color to further test the rendering and motion techniques. To completely check the motion, a pencil test may be shot which is a shot of full-motion but low quality images such as pencil sketches. Associate and assistant animators are responsible for producing the frames between the key; this is called in-betweening. Inking refers to the process of transferring the penciled frames to cels. Finally, coloring is applied to these cels.
Computer animation production has borrowed most of the ideas from conventional animation production including the use of a story board, test shots, and pencil testing. The use of key frames and in-betweening have also been adopted in certain computer animation systems.
Story boards have pretty much translated directly over to computer animation production although they may be kept on a computer. They still hold the same functional place in the animation process.
In computer animation there is usually a strict distinction between the creation of the models, the specification of their motion, and the rendering process which is applied to those models. In conventional animation, the model building, motion specification, and rendering are really all the same thing. In computer animation, speed-quality tradeoffs can be made in each of the three stages during the trial and error process that characterizes much of animation.
A test shot in computer animation is usually a high quality rendering of a highly detailed model to see a single frame of the final product. Pencil testing can be performed either by simplifying the sophistication of the models used, or by using low quality and/or low resolution renderings, or by using simple motion control algorithms. Place-holder cubes can be rendered in wire frame to present the gross motion of rigid bodies in space and to see spacial and temporal relationships among objects. This may also provide real-time calculation and playback of the animation. Similarly, high-quality rendering of low complexity models or low-quality renderings of highly detailed models, or some intermediate levels of both the model and the renderer can be used to give the animator clues to the finished product's quality without committing to the final computation. For example, a hardware z-buffer display can provide good turn-around time with decent quality images before going to a ray traced image. Solids of revolution objects lend themselves quite well to allowing for three, four or five levels of detail for a given model. Also, smooth shading, texture mapping, specular reflection and solid texturing can all be options presented to the animator for a given run. To simplify motion control, for example, simple interpolation between poses may be used instead of inverse dynamics.
Because of the discrete nature of frames, there may be some ambiguity when saying, for example, a span ends exactly when another span begins. In the discrete world, we will take that to mean that a span ends on the frame before the frame that the other span begins on. Two time spans can have thirteen different types of relationships (ignoring spans of zero length):
|span A begins||span A ends|
|before B||before span B begins|
|when span B begins|
|exactly when B ends|
|after B ends|
|exactly when B begins||during B|
|exactly when B ends|
|after B ends|
|during B||during B|
|exactly when B ends|
|after B ends|
|exactly when B ends||after B ends|
|after B ends||after B ends|
Squash & stretch , timing and secondary actions establish the physical basis of objects in the scene. A given object possesses some degree of rigidity and intuitively has some amount of mass. This is reflected in the distortion (squash and stretch) of its shape during an action. The animation must support these notions consistently for a given object throughout the animation. Timing has to do with how actions are spaced according to the weight, size, and personality of an object or character. Secondary actions support the main action, possibly supplying physically-based reactions to what just went on.
Slow in & slow out, and arcs have to do with how things move. They ease into and ease out of actions, typically. And they don't usually move in straight lines. These movements also support the quality of physical realism with respect to such principles involving inertia and gravity.
Straight ahead vs. pose to pose considers how a motion is created. Straight ahead refers to progressing from a starting point and developing the motion as you go. Physically based animation could be considered a form of this. Pose to pose refers to the approach of identifying key frames and then interpolating intermediate frames between them.
Anticipation and staging have to do with how an action is presented to the audience. Anticipation dictates that an upcoming action is set up so that the audience knows it (or something) is coming. Staging follows up on this notion of presenting an action so that it's not missed by the audience. Timing is also involved here, to the extent that an action has to be given the appropriate duration for the intended effect to reach the audience.
Follow through/overlapping action, exaggeration and appeal are principles which address the design of an action or actions. Exaggerate it so it can't be missed or so that it gets a point across and make it enjoyable to watch. In addition, actions should flow into one another to make the entire scene flow together.
Also produced from the effort of the University of Utah were some early films on a walking and talking figure, an animated hand and an animated face by such folks as Ed Catmull, Frank Crow, Fred Parke, and Jim Blinn. It has only been recently that this early work in animation has actually been eclipsed.
In the late sixties Chuck Csuri was doing some pioneering work in computer animation in the Computer Graphics Research Group at The Ohio State University. A major feature of the animation that came out of Ohio State in the mid-seventies was real-time video playback from a digital disk. The hardware that took the digital information, uncompressed it on the fly and converted it into a video signal was developed at North Carolina State University under John Staudhammer. In the early 80s, the research group became the Advanced Computing Center for Art and Design and continues to produce computer animation.
Norm Badler , at the University of Pennsylvania, was one of the first researchers addressing the problem of human figure animation. He has continued this research and has established the Center for Human Modeling and Simulation . Jack is a software package developed at the Center which supports the positioning and animation of anthropometrically valid human figures in a virtual world.
Also in the mid-seventies, at Cornell University , the Program for Computer Graphics , under the direction of Don Greenberg , generated some architectural walk-throughs of Cornell's campus.
In 1974, the first computer animated film called Hunger was produced by Rene Jodoin, and directed and animated by Peter Foldes. This effort was a 2 1/2 D system that depended heavily on object interpolation techniques. This is discussed further in the chapter that covers object interpolation algorithms (Chapter IV). Canada has been active in computer animation (and animation in general) due in large part to the National Film Board of Canada and the National Resource Council Canada .
NYIT got into graphics in a big way in the late 70s. They produced some incredible animation. Many of the people from the Utah effort contributed to the NYIT lab. The Works represents some of the early impressive pieces to come out of NYIT.
Some of the current activity centers are:
Current computer animation companies (who also contribute significantly to research in the area are: Pixar, Industrial Light and Magic (ILM), Pacific Data Images (PDI), Disney, Xaos, Rhythm & Hues, Digital Domain, Lamb & Company, Metrolight Studios, Boss Film Studios, deGraf/Wahrman, R/Greenberg Associates, Blue Sky Productions, Sony Pictures, Cinesite, Imageworks, and Apple . (One might also include Silicon Graphics. in that list since their equipment is used so extensively for computer animation.)
Of special note, are the award winning animations which Pixar has produced, including:
Early on, Computer Graphics (CG) appeared in a variety of movies in which it was used as computer graphics (that is, the CG was not intended to fool the audience into thinking it was anything other than CG). For example, Future World (1976) and Star Wars (1977, Image West) fall into this category. More recently, Lawnmower Man (1992, Xaos, Angel Studios) which has a segment of Hollywood's view of Virtual Reality in it, used CG in the same role, although a more sophisticated example.
Tron (1982, MAGI) was a little different. The CG was still supposed to be computer-like because the action takes place inside a computer. But in this case, it was an integral part of the environment that the actions (and actors) were taking place in. The CG was used throughout the movie. It integrated computer animation with live action, but, since the action took place in a computer, the CG didn't have to look realistic (and didn't). This was the first time CG was used as an integral part of a movie.
Along the same lines of Tron in using CG to create an 'inside the computer' environment is Reboot (1995, Limelight Ltd./BLT Productions). Reboot deserves special mention as the first Saturday morning cartoon that is full three-dimensional computer-generated animation. The action takes place inside a computer so they don't have to go for 'realism'. Still, there are several human-like main characters and, overall, The Reboot series is very impressive.
One main use of CG has been to replace physical models. In this case, CG is used to create realistic elements which are intermixed with the live action. The Last Star Fighter (1984, Gray Demos' Digital Productions, even the Cray X-MP was in the credits) used computer animation instead of building models for special effects. The action takes place in space as well as on planets; CG was used for the scenes in space and physical models were used for the scenes on a planet. It's not hard to tell when the movie switched between CG models and physical models. There were probably 20 minutes of CG used in the movie. This was the first time CG was used as part of the live action in which it wasn't supposed to look computer generated. More recently, Apollo 13 (1995, Digital Domain) used CG models of the return vehicle from the mission. In TV-land, special note should go to Babylon 5 (1995, Newtek). Babylon 5 is the first TV show to routinely use CG models as regular features of it's sci-fi show - and it's Amiga-based.
CG is also used to create 'alien' creatures. Creatures which are supposed to be realistic, but don't have to match anything that the audience is familiar with. The Abyss (1989, ILM) is one such movie in which CG is used to effect an alien creature which is integrated with the rest of the live action. Some of the CG in Terminator II served a similar purpose as well as Casper (1995, ILM), Species (1995, Boss Film Studios), Mighty Morphin' Power Rangers (VIFX), Sexy Robot (TV Commercial, Abel)
More challenging is the use of CG to create realistic models of creatures that are familiar to the audience. Jurassic Park (1993, ILM) was the first to completely integrate use of CG character animation in which the graphics were designed so as to blend in with the live action so that it was difficult to tell what was computer generated and what wasn't. Jumanji (1995, ILM) does the same thing with it's incredible modeling of animals. To a lesser extent, Batman Returns (1995, Digital Domain) also does the same thing by providing 'stunt doubles' of Batman in a few scenes. CG was used to create the face of RoboCop 2 (1990, deGraf/Wahrman) and animated skeletons in Total Recall (Metrolight) as well.
Another popular CG technique for special effects is the use of particle systems. One of the best examples is in Star Trek II: The Wrath of Khan (LucasFilm computer division, later ILM) in which a wall of fire sweeps over the surface of a planet. Another example is Lawnmower Man in which a character disintegrates into a swirl of small balls. A more recent example from television is in the opening sequence of Star Trek: Deep Space Nine (1995) to model a comet's tail. Twister also uses particle systems to simulate a tornado.
Of course, one use of computer animation is simply to 'do animation.' By that I mean computer animation is used to produce animated pieces which would otherwise be done by more traditional means - essentially 3D cartoons (although the term cheapens the idea somewhat). The Pixar animations have been previously mentioned. Technological Threat was an early animation that combined computer animation with hand-drawn animation to produce an entertaining piece. TV commercials, for the most part, fall into this category. Some popular ones come to mind: Listerine, Shell dancing cars, and LifeSavers. Toy Story (1995, Pixar, Disney), the first full length fully computer-generated 3D animation, would fall into a category of 3D cartoon. See the paper on the making of Toy Story . Absolute realism is not the objective as much as doing a computer-generated version of what would normally be done by traditional animation means. The previously mentioned 2 1/2 D Hunger would fall into this category, as would the TV episode of The Simpsons (1995, PDI) in which Homer turned into a 3D CG character and the TV special Incredible Crash Dummies (Lamb & Company).
Although I'd like to restrict my topic to 3D computer animation, I suppose that morphing should be mentioned. This is essentially a 2D procedure which warps control points (or feature lines) of one image into the control points (feature lines) of another image while the images themselves are blended. In Star Trek IV, one of the first commercial morphs was provided by ILM in the back in time dream sequence In Willow (1988, ILM), ILM provided the morph of several animals. This technique was also used by ILM in Indiana Jones and the Last Crusade (1989) and Terminator 2 . PDI is known for its use of morphing in various commercials including a Plymouth Voyager commercial and an Exxon commercial in which a car changes into a tiger. Of course, morphing has gone on to become another Energizer Bunny of TV commercials - it keeps going and going and going... Full 3D morphing has yet to make it out of the research labs and into any production environment.
There is another class of movies in which CG plays a role - that of 'hidden special effect' (for lack of a better term). CG can be used to cover up mechanical special effects or to enhance the scene to integrate a mechanical special effect more completely. For the most part, this resides in the 2D realm and, as such, will not be the focus of this document. However, with the onset of digital techniques for 2D compositing, sequences will be routinely digitally available making them susceptible to a variety of digital post-processing techniques. The first digital blue screen matte extraction was in Willow (ILM). The first wire removal was in Howard the Duck (ILM). In True Lies (1994, Digital Domain), CG was used to erase support wires from suspended actors. In Forest Gump (1994, Digital Domain), CG was used to insert a ping pong ball in a sequence showing an extremely fast action game. In Babe (1995, Rythm & Hues), CG was used to move the mouths of animals and fill in the background uncovered by the movement. In Interview with a Vampire (1994, Digital Domain), CG was used to curl the hair of a woman during the transformation into a vampire. In this case, some of the effect was created using 3D graphics and then integrated into the scene by 2D compositing.