Facebook Connect: John Carmack Keynote
Check out some current and future developments in the world of VR.
Unreal and films – highlight reel
Epic Games and Unreal Engine helped to develop the amazing virtual environments used in The Mandalorian.
As I have reviewed before, this is not done with greenscreens – but instead are projected backgrounds that are rendered realtime based on camera position. This solves almost all the visual problems created when using greenscreens.
As this highlight reel shows, their technique of combining projected, camera-tracked CG environments with live actors and props can be used for all levels of production, not just blockbusters.
I suspect we’re going to see some very interesting applications soon.
IMMERSIVE LIGHTFIELD VIDEO WITH A LAYERED MESH REPRESENTATION
I worked with a little bit of early lightfield photography back in the day. Looks like they’ve expanded and possibly found an interesting VR application. These researchers present a system for capturing, reconstructing, compressing, and rendering high quality immersive light field video.
Here’s the Siggraph paper and some more examples:
https://augmentedperception.github.io/deepviewvideo/
Live Avatar Dancing
One step closer to the end of the pop star and to the reality Macross Plus envisioned back in 1995…
Giving up on greenscreen?
CG has always had problems with realism. Eye-lines and focus distances are never perfect. Colors between live/CG elements never quite match. Reflections can be directionally incorrect, missing, or mismatched in color/intensity. Lighting color/intensity/direction is often inconsistent between the live elements and CG elements. Mattes have problems at edges. Motion tracking is usually off by just enough to cause odd movement discontinuities. All of this makes CG look cheap.
But there is a new approach using LED stages – large displays surrounding your shooting scene. It’s completely changing the game. Even more amazing, camera movement and simulation are done using the Unreal gaming engine. Even back in the mid 2000’s, I worked on a project that attempted to use a game engine for movie pre-visualization. That’s how far things have come. The amazing visuals of the Mandalorian were created using this technique – and it’s blowing green-screens away.
New Deathtrap Dungeon FMV game?
It’s no secret that I love the old Fighting Fantasy adventure gaming books. It’s a series that had the perfect mix of choose-your-own-adventure and D&D stories. It was something I discovered around 10 years old – and now have collected almost every book in the original series.
One of the best of the series was Deathtrap Dungeon. Turns out, Eddie Marsan is narrating a new FMV version of the original Deathtrap Dungeon book. Wireframe has a writeup on the new effort, and a short clip gives a teaser:
In reading that article, I found out about something equally cool. Knightmare was a British children’s adventure game show that ran on from 1987-1994. A team of four children – one who takes on the game by donning a sight-blocking helmet and the other three acting as their guide and advisers – attempting to complete a quest within a fantasy medieval environment, traversing a large dungeon and using their wits to overcome puzzles, obstacles and the unusual characters they meet along the journey.
The show is most notable for its use of blue screen chroma key to put the child into the dunngeon, use of ‘virtual reality’ interactive gameplay on television, and the high level of difficulty faced by every team.
I had no idea this show existed. I would have loved to watch it as a kid.
Update: 03/2022
The game is out and you can watch a full walkthrough here:
State of car UI systems
The UX Collective did a lot of great work looking at the upcoming UI’s for autombiles at IAA 2019 in Frankfurt. Check out what’s coming down the line at the link below.
https://uxdesign.cc/automotive-ux-highlights-from-iaa-2019-31fc0d97ef19
No more gym memberships?
With VR, are we going to need expensive gym memberships anymore?
Google hand tracking now open source
Google has made its hand detection and tracking tech open-source, giving developers the opportunity to poke around in the tech’s code and see what makes it tick.
“We hope that providing this hand perception functionality to the wider research and development community will result in an emergence of creative use cases, stimulating new applications and new research avenues,” reads a blog post from the team.
That post over on the Google AI Blog dives into exactly how the tech works, and devs interested in getting a closer look at it can find the project over on Google’s Github repository.
Fologram and building what was previously difficult
Nothing is going to be the same – not even the most menial jobs. The pace of change of what is coming, and what is happening today, is beyond even our wildest dreams.
Not even construction jobs will be the same. Such as this example of building a complex wave-like wall structure in less than a day with perfect placement of each brick.
Fologram Talks: Holographic Brickwork from Fologram on Vimeo.
Fologram combines computer-aided design with the holographic capabilities of Microsoft’s HoloLens headset to help in assembling even complex objects. The hologram can overlay exactly where each piece of the build should go, as well as an outline of the finished product.