The idea of pursuing PhD in the related field of digital humans originated from my desire to create believable digital characters.

In my childhood, I liked to draw pictures as realistically as possible with picture references. One day I drew a picture of an animal (and I was proud of how close it was to a reference picture), and showed it to my father. And his response was a bit unexpected one “why don’t you take a picture?”. Ever since something between the drawing and photograph has captured my fascination.

How can the drawing be better if it is just copying a photograph?

To explore my interests, I worked on a survey project during my undergraduate on how CG characters can overcome the Uncanny Valley (when I was majoring in Environmental Design), which founded my interests in pursuing PhD in a relevant field. One of my interests has been how a CG character can be better than an actual photograph, which has some significance to the Uncanny Valley problem. To that end, I always wanted to be involved in a digital human creation project during my PhD in addition to do pure academic research.

More than ten years later after the conversation with my father, I was admitted to USC’s PhD program and (literally) my research has been using modern photography to capture real-world objects. A consistent objective employed in the thesis is that we minimize errors with photography as a reference, and thus the result is “photoreal”. However as there are many styles to represent digital characters, “photorealism” is merely one of many ways to do so, and the “attractiveness” of the character is not necessarily on the same axis as how photoreal it is, as suggested by the Uncanny Valley.

An advantage of the capture techniques is that even if we don’t know anything about the ingredients that make the person look like him or her, we can blindly copy every single detail down to a single pore, and can achieve a photorealistic look.

My thesis has addressed three principal capture techniques for achieving multi-scale dynamic digital humans. I believe that one important purpose of the thesis has been to provide a technical solution so that if a photorealistic look is desired, a tool is already available for digital creators. The Digital Emily 2.0 project was a very unique opportunity to experiment what we can achieve in an automated and principled pipeline if we combine modern facial scanning, performance capture, and high resolution simulation.

However, the current digital character pipeline still relies on manual labors and extensive artistic inputs as just reproducing a photograph is not a true objective for the digital character creation.
What lacks in the current study is a perspective to investigate what are really the essential components that provide the impression of the person.
This is a very ill-defined problem as the answer depends on the observer. Nonetheless, experienced artists are good at extracting unique characteristics of a person, and can effectively enhance them, or if necessary can abstract unimportant details to stylize. A good example is a caricature, where person-specific characteristics are amplified to make it more like the face than the face itself. The quality of the digital character owes to the excellence in the digital artistry, as the attractive digital character is not necessarily the reproduction of the real world. This is why a computer graphics pipeline always needs artists in the loop, and this is how digital characters can go beyond a photograph.

As we are more equipped with a technical capability to create realistic digital characters, we are more able to explore the true objective for creating believable digital characters. This may involve more studies in human perceptions and psychology. More importantly, I think this can only be done through working with digital artists, who have been optimizing their approach for that objective.

May 2017

 


Koki Nagano © 2017