Topics

• Virtual Reality
  • technology
  • applications
• Augmented (Mixed) Reality
  • technology
  • applications
• Standards
• That Metaverse thing
• • digital twins
• An Integration Case Study (Apple Vision Pro)

First a little history

Ivan Sutherland

(WAYYYYYYY ahead of his time)

1963

60 FREAKING YEARS AGO!!!

Jaron Lanier coined "VR"

1987

NASA Jet Propulsion Labs early pioneering (training work) 1990

What is Virtual Reality?

strap a screen to your face

Virtual Reality

Applications

• training
• therapy
• data visualization
• virtual real estate
• exercise

Virtual Reality

(significant problems)

• simulator sickness
• duration issues

Augmented Reality

"Augmented reality: because reality wasn't confusing enough on its own." - Unknown

Augmented (mixed) Reality

• Keeps you in the "real world"
• Mobile phone is intro device
  • ubiquity of phones
  • many compelling use cases
    • navigation
    • language translation
    • already highly popular (SnapChat lens)
• Interact with real objects
  • tactile doesn't need to be simulated (it's real)

Augmented (mixed) Reality

• When using an HMD (Head Mounted Display) "video pass through" used as substitute for a see-through display
  • Meta Quest 3
  • Apple Vision Pro
  • Varjo
• Virtual objects seamlessly integration with the "real world".

Augmented (mixed) Reality

Icons floating in real world space

Augmented Reality

• Visual fidelity
• Lag/virtual shaking-swimming
• Heat dissipation - power requirements
• Eye/Gaze tracking UI 
• Eye glass form factor - holy grail (coming!)

significant remaining problems

Augmented Reality

eye glasses are coming

• https://about.meta.com/en/realitylabs/projectaria/
• https://www.xreal.com/us/ar
• https://brilliant.xyz/
• https://rokid.ai/rokid-air-pro/

• https://vr-compare.com/

"Standards are like toothbrushes: everybody agrees we need them, but nobody wants to use someone else's." ... Grace Hopper

Standards

make the world interoperable

Internet - Networking Standards

that turned into something big

• TCP/IP layered protocol
•  RFC (request for comment)(https://www.ietf.org/standards/rfcs/)
• was not obvious..remember Xerox ring network or proprietary networks on SGI for multiuser interaction (flight game)

3D File Formats

• glTF www.khronos.org
  • fast optimized for real time, lots of use in wide range of companies
• USD https://openusd.org/release/index.html
  • selected by Apple and Nvidia as "metaverse" format..from Pixar 
• X3D  https://www.web3d.org/x3d/what-x3d
  • successor to VRML, ISO standard, used a lot in academia

Interaction Standards

Accessibility

• Secion 508 section508.gov
• W3C has a group WAI (Web Accessibility Initiative) https://www.w3.org/WAI/

USA Standards Strategy

• standards.gov
• https://www.nist.gov/standardsgov/usg-nss

U.S. Government National Standards Strategy for Critical and Emerging Technology

That "Metaverse" thing

“The story so far:
In the beginning the Universe was created.
This has made a lot of people very angry and been widely regarded as a bad move.”
― Douglas Adams, The Restaurant at the End of the Universe

Mike Zyda (USC)

proposed Metaverse University

https://www.youtube.com/watch?v=3zkZOBU5Pcs

That "Metaverse" thing

don't worry nobody really knows what it is

Metaverse Standards Forum: https://metaverse-standards.org/ www.metaverse

IEEE: https://metaverse.ieee.org/

ITU-T Metaverse Focus Group: https://www.itu.int/en/ITU-T/focusgroups/mv/Pages/default.aspx

Spatial version of the Internet

Meta (Facebook)

Google

Apple

Microsoft

https://youtu.be/rXgvEQrXHJc

https://www.spatial.io/blog/spatial-virtual-nft-galleries

Metaverse 

virtual real estate

Metaverse

Use Cases: digital twins

https://youtu.be/pxEgAAKQl90

Cesium blog: https://cesium.com/blog/2022/02/11/cesium-supporting-state-sized-digital-twins/

Metaverse

Podcasts

https://cesium.com/open-metaverse-podcast/

https://nextbillionseconds.com/2022/09/15/a-brief-history-of-the-metaverse-what-is-the-metaverse/

Metaverse

Integrating and Interoperable (star trek rodenburry archive)

https://home.otoy.com/roddenberryarchive_april2023/

• text to image generators
  • DallE
  • midjourney
• text to 3D objects
  • Point-e (openai)
• text to VR environment
  • blockadelabs.com
• text to video
  • runwayml.com

this will be out of date by the time you read this..the field is changing at explosive speed!!

AI image generator tools

"NFTs have single-handedly turned the internet into the world's most expensive flea market." - Seth Meyers

Cryptocurrency - Web3

Blockchain

Virtual Economies

NFTs

Cryptocurrency has only existed since approximately 2009/2010 with the creation of Bitcoin. The explosive growth of the value of a "coin" fueled surge of interest and investments. The key innovation of bitcoin is the "blockchain". A blockchain is a digital ledger. It keeps track of all transactions on the "chain"

Cryptocurrency - Web3

blockchain = distributed ledger

DAPPs

https://ethereum.org/en/dapps/

decentralized applications

digital assets

NFTs

smart contracts

https://www.rendertoken.com/

NIST blockchain report: https://www.nist.gov/blockchain

Blockchain

is a distributed ledger

a "chain" of blocks

Fungible versus Nonfungible

NFTs exist and are associate with a particular "chain". Most common is ETH (Ethereum). 

You mine it you own it (or you buy it).

Mining an NFT is really the act of digitally signing some item. 

March 2021

NFT: Non Fungible Token

provenance and scarecity

Integration Case Study

Apple Vision Pro

Apple Vision Pro

• Eye Tracking
• Hand/Gesture Tracking
• Immersive Display
• Immersive Audio
• Head Tracking

new paradigm: gaze & pinch

Apple Vision Pro

• eye fatigue: gaze is off
• arm fatigue "I pinched why don't you make the selection"
• Immersive Display - the passthrough gets fuzzy

new paradigm: gaze & pinch

Problems - It's not all rainbows and unicorns

That's All!

Note the spherical projection surface

Link:

https://slides.com/sressler/arvrmr