Rosedale: Actually, the thing I'm debating working on right now, because I'm now one of our team leads -- part of my job, because I'm back doing development more, I'm working on a map right now. We're about to release some simple speed ups to map and stuff that are kind of cool.
It turns out that I think the highest impact part of that will be for meetings in Second Life. Making them just crush video conferencing once and for all. I mean, that's what we're experimenting on.
Industry Standard: Mark [Kingdon] was also talking about this, too.
Rosedale: Upper body matching between your avatar. So, if you've got a MacBook and you've got the camera, the thing I can do -- I can show you a demo of this, if my little team chooses to work on this -- I can show you a demo of this in probably less than a couple months. It is trivial now to read that camera and not to find hands, that's hard, but to find your upper body and your eyes is very, very simple.
So, what that means is that I can easily detect head nods, maybe shoulder shrugs, definitely postural changes in the upper body, definitely lateral head sway and stuff like that.
What that means is that if you have an avatar in Second Life and you got to a business meeting and it's six months from now (my engineers hope less than that) and you just fire up Second Life, that little light's going to come on your MacBook. That's not going to be video, because everybody hates that. It's going to mean that your avatar suddenly is doing this.
So, if you think about it, of course it's fun and you're a little biased. It's delightful to come and sit in our little cool meeting room here, but we would do these kinds of interviews in Second Life then, right? I mean, it would be so easy.
It could also see gesture, in which is a little bit harder. If I can emote as an avatar like that, like if I can nod, if you can nod, if you can follow each other's conversation like that? That's huge.
Industry Standard: What's the timeline for this?
Rosedale: We could do it in two months. It can be done. There're open-source programs, there's open source code that refines your eyes and is flawless, 30 frames per second. It's literally done, it's just finished. It's just a matter of assembling the parts.
Industry Standard: What about the mouth?
Rosedale: The mouth is a lot harder. The mouth and the other 60 facial muscles is a big problem, however, I've heard -- I'm not as familiar with the latest work on that -- but everybody's pushing on it and that one is going to go the way by Moore's Law.
Because once we have your eyes, which everybody can do now -- the eyes are just unusual looking so they're very findable in the image unless there's very weird background, a bunch of eyes in the background or something, people looking over your shoulder, obviously.
But yeah, because the eyes are easy to find, the facial markers and facial morphology cannot be far behind, because, again, it's kind of an easy problem, once you find the eyes, because if you've found the eyes, you can find the mouth.
Though, the problem is that the human brain -- you've heard this before. It's this "uncanny valley" thing in filming. There's this word called the "uncanny valley." It's kind of what Pixar has so brilliantly navigated. If you present an animated figure in a movie, you have this interesting problem, which is the more it looks like a real human, the better it has to be rendered, and the more emotionally correct it has to be.
If it looks like, say, a big, furry monster, like in "Monsters, Inc.," what happens is it can kind of "Hey, George," and it can move, and we just love it. It just looks so cool to us. It's like a talking dog. It's just so amusing. It looks correct to us. That's because we have nothing to compare it to. We've never seen a talking monster. Or a talking car.
So, Pixar has built an enormous amount of success around making these amazingly rendered images, but not going into the "uncanny valley." They've got graphs on this. If you look it up online, people talk about, the "uncanny valley..."
Industry Standard: "Polar Express" is an example of that. Some people said it looked kind of creepy.
Rosedale: Yes. I actually though "Polar Express" was kind of cool, myself. But, yes, "Polar Express" is a great example, where you saw those cherub kids' faces, and you'd see them speak and your brain is kind of stuttering at it, like, "Oh. Something's wrong."
That's the challenge with avatars is, if I start moving your lips, boy. I mean, the amount of neural circuitry that we have targeted on watching somebody's mouth is gigantic. Huge parts of the neocortex. So, we are able to see the masked stuff in the way mouths move. And that's why somebody that has a slight lisp, or even very small altercations to human morphology are arrestingly disturbing to us in the real world.
So now, we're trying to render that with an avatar. Very hard problem. The way you look inside your mouth, your brain's looking for all that, like at almost hundreds of frames per second.
Industry Standard: Now, the other side of this is, of course, the cost for consumers. I mean, even if Moore's Law makes it, and is able to do this, and you guys develop the applications that can handle this, too...
Post new comment