Virtual Reality the Metaverse and All that Reality Stuff

Sandy Ressler (NIST)

USPTO Patent Examiner Training

May 2023

DISCLAIMER: Any mention of commercial products is for informational purposes only and does not imply endorsement by NIST.

Just your typical Patent Examiner room ;-)


"Just when I discovered the meaning of life, they changed it." ― George Carlin, on constantly evolving technology


“I've come up with a set of rules that describe our reactions to technologies:

1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.

2. Anything that's invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.

3. Anything invented after you're thirty-five is against the natural order of things.”
― Douglas Adams, The Salmon of Doubt

Topics (human created)

  • Virtual Reality
    • applications
  • Augmented Reality
    • mixed reality
    • applications
  • Standards
  • Tools
  • That Metaverse thing
    • digital twins
    • Blockchain/NFTs

Topics (ChatGPT)

  • Virtual Reality (VR) and Augmented Reality (AR

  • 3D Graphics and Spatial Computing

  • Networking and Connectivity

  • Blockchain and Cryptocurrencies

  • Virtual Economies

  • Social Platforms and Interoperability

  • Gaming and Entertainment

  • User Generated Content (UGC) and Customization
  • Artificial Intelligence and Machine Learning

Not bad but it missed standards and AI and ML can be applied to anything, I'll rate it a B.


give me a bulleted list of the main components of the metaverse



First a little history

Ivan Sutherland

(WAYYYYYYY ahead of his time)



Jaron Lanier coined "VR"


NASA Jet Propulsion Labs early pioneering (training work) 1990

What is Virtual Reality?

strap a screen to your face

Virtual Reality


  • Immersive
  • Compelling
  • Isolating
  • Issues with "simulator sickness"
  • time limited (only wear that screen for so long)
  • high resolution screen
  • head tracking
  • hand tracking (sometime with extra controller devices) for interaction
  • spatial audio
  • eye tracking, also for interaction

As an aside

  • spatial audio
    • head tracked airpod
  • eye tracking
    • contact lens screens (mojo)

Virtual Reality


  • training
  • therapy
  • data visualization
  • virtual real estate
  • exercise

Virtual Reality


Virtual Reality

Physical Therapy

Virtual Reality

Treatment of Phobias

Virtual Reality

Virtual Real Estate

Virtual Reality


Virtual Reality

Exercise - Beat Saber

Virtual Reality

Fire Research: 360 degree videos

Virtual Reality

Fire Research: 360 degree videos

Virtual Reality

(significant problems)

  • simulator sickness
  • duration issues

Augmented Reality

"Augmented reality: because reality wasn't confusing enough on its own." - Unknown

Augmented (mixed) Reality

  • Keeps you in the "real world"
  • Mobile phone is intro device
    • ubiquity of phones
    • many compelling use cases
      • navigation
      • language translation
      • already highly popular (SnapChat lens)
  • Interact with real objects
    • tactile doesn't need to be simulated (it's real)

Augmented Reality

Navigation Aid

Augmented Reality


Augmented Reality

Language Translation

Augmented Reality


Immersive Reality

Data Visualization

Augmented Reality

  • Visual fidelity
  • Lag/virtual shaking-swimming
  • Heat dissipation - power requirements
  • Eye/Gaze tracking UI 
  • Eye glass form factor - holy grail (coming!)

significant remaining problems

Networking and Connectivity (Multiuser)

"The Internet: where men are men, women are men, and children are undercover FBI agents." - Anonymous

Networking and Connectivity (Multiuser)

  • Shared office

  • Concert

  • Sports Game

  • Parties


Shared Office.  Meta Horizon Workrooms



Google Project Starline






Serious Games

NPSNET demo (1995)


"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live."

- John Woods


  • General Purpose Authoring: Unreal, Unity, Omniverse
  • Web based: aframe.js
  • Specialized: LensStudio, 360 degree video
  • Photogrammetry: 3D scenes from 2D photographs

NeRF - Neural Radiance Field

In summary, NeRF is a method that uses machine learning to represent and render 3D scenes from 2D images. By training a neural network on a set of input images, NeRF can generate realistic, high-quality views of the scene from new perspectives. - ChatGPT

"Unity is the virtual reality tool that lets you create your own universe – just make sure you don't get lost in it while debugging." - Unknown

General Purpose Creation Tools

  • Unity:
  • Unreal:
  • Omniverse:

All are highly capable and robust tools - as always the choice is "it depends" on your application requirements.

Web Based: aframe.js


NeRF - NVidia Instant NeRF

SnapChat Lens Creation

  • text to image generators
    • DallE
    • midjourney
  • text to 3D objects
    • Point-e (openai)
  • text to VR environment
  • text to video

this will be out of date by the time you read this..the field is changing at explosive speed!!

AI image generator tools


Minecraft: 141 million active monthly users 2021

Fortnite: 250 million monthly active users 2023


User Generated Content

big business

virtually unlimited changes (mods)

constrained but extensive customization

"The nice thing about standards is that there are so many to choose from." ... Andrew S. Tanenbaum


3D objects - file formats

communications - Internet TCP/IP

interaction - UX


make the world interoperable

3D File Formats


Internet - Networking Standards

that turned into something big



  • TCP/IP layered protocol
  •  RFC (request for comment)(
  • was not obvious..remember Xerox ring network or proprietary networks on SGI for multiuser interaction (flight game)

Interaction Standards



  • Secion 508
  • W3C has a group WAI (Web Accessibility Initiative)

USA Standards Strategy

brand spanking new (may 2023)

U.S. Government National Standards Strategy for Critical and Emerging Technology

"NFTs have single-handedly turned the internet into the world's most expensive flea market." - Seth Meyers

Cryptocurrency - Web3



Virtual Economies


Cryptocurrency has only existed since approximately 2009/2010 with the creation of Bitcoin. The explosive growth of the value of a "coin" fueled surge of interest and investments. The key innovation of bitcoin is the "blockchain". A blockchain is a digital ledger. It keeps track of all transactions on the "chain"

Cryptocurrency - Web3

blockchain = distributed ledger


decentralized applications

Create keys (a pair is generated, one public, one private.


Public Key Cryptography

foundational technology

Virtual Economies

  • Second Life (linden dollar)
  • Game gold

"Virtual currencies: because why not trade your hard-earned real money for some pixels and in-game perks?" - Anonymous

digital assets


smart contracts

NIST blockchain report:


is a distributed ledger

a "chain" of blocks

Fungible versus Nonfungible

NFTs exist and are associate with a particular "chain". Most common is ETH (Ethereum). 

You mine it you own it (or you buy it).

Mining an NFT is really the act of digitally signing some item. 

March 2021

NFT: Non Fungible Token

provenance and scarecity

That "Metaverse" thing

“The story so far:
In the beginning the Universe was created.
This has made a lot of people very angry and been widely regarded as a bad move.”

― Douglas Adams, The Restaurant at the End of the Universe

Mike Zyda (USC)

proposed Metaverse University

That "Metaverse" thing

don't worry nobody really knows what it is

Metaverse Standards Forum: www.metaverse


ITU-T Metaverse Focus Group:

Spatial version of the Internet

Meta (Facebook)





virtual real estate


Use Cases: digital twins




Integrating and Interoperable (star trek rodenburry archive)


  • Technology is evolving FAST!
  • Virtual Reality ..applications
  • Augmented Reality .. useful applications
  • Multiuser 
  • Standards
  • Tools
  • Crypto - Blockchain - Distributed Ledger
  • Metaverse
  • This talk is obsolete already


  • Bullet Two
  • Bullet Three




  • Unity
  • Unreal
  • Omniverse


VR the Metaverse and More

By Sandy Ressler

VR the Metaverse and More

  • 350