Gerard Sans | Axiom π¬π§ PRO
Founder of Axiom Masterclass, professional trainings // Forging skills for the new era of AI. GDE in AI, Cloud & Angular. Building London's tech & art nexus @nextai_london. Speaker | MC | Trainer.
Developer Evangelist
Developer Evangelist
International Speaker
Spoken at 170Β events in 37 countries
Serverless Training
Serverless Training
What if an AI could do your job?
How long do you think it may take?
γ
AI is likely to overtake humans in the next five years
β Elon Musk
OpenAI
Greg Brockman
Elon Musk
Sam Altman
Co-Founders
Our mission is to build safe
artificial general intelligence (AGI)
that benefits all humanity.
2015
Introducing OpenIA
Initially a non-profit company,Β later on becomes "capped-profit", raisesΒ $1B.
2018
GPT is released
First Generative Pre-trained Transformer model with 117MΒ parameters.
2020
GPT-3 and OpenAI API are released
Largest model to date with 175B parameters.
2021
GitHub Copilot, DALL-E and Codex are released
All with 12B parameters.
2022
Insert & Edit released for GPT-3 and Codex. DALL-E2 is released
Iterate over existing text and refactor code via prompt. DALL-E2 has now 3.5B parameters.
NLP Models comparison
GPT-1
BERT
Parameters (Bilions)
2018
2019
2020
2021
GPT-2
RoBERTa
Codex
GPT-3
Switch
175B
1.5B
T5
11B
12B
Year
1.6T
1
10
100
1000
Gopher
280B
GPT-3
GPT-3 in action
The day started as any other day with a couple of hours of intense work before lunch.
The day started as any other day
Prompt
Hyperparameters
Response
.
π
Conversational AI
Question: Who discovered America?
Answer: Christopher Columbus.
Question: What year was that?
Answer: 3054.
Question: Who discovered America?
Answer: Christopher Columbus.
Question: What year was that?
Question: Who discovered America?
Answer: Christopher Columbus.
Question:
Question: Who discovered America?
That doesn't look right...
AI generated text may be...
Biased
Non-factual
Inaccurate
1+1= 3
GPT-3 Model Stages
Design
Predictions
Training
GPT-3
GPT-3
GPT-3
GPT-3 Model
- Released: June 2020
- Common Crawl, WebText2 (Reddit), Books1, Books2 and Wikipedia
- 175B parameters, 96 layers, 12,288 dimensions
- Total 550GB
GPT-3
GPT-3 Training DataSet
499 Billion tokens
+100 Languages
Russian
Romanian
Polish
Danish
Japanese
Swedish
Finish
French
German
Italian
Portuguese
Dutch
English
Spanish
Guten Morgen!
Training phase
Adjust model predictions using output
Wikipedia
Christopher Β is
Christopher Columbus was
Input
Output
GPT-3
Christopher Columbus discovered Β America
Christopher Columbus discovered America Β in
Christopher Columbus discovered America in Β 1492
Christopher Columbus discovered America in 1492 Β .
Christopher Columbus discovered America in 1492.
Christopher Columbus discovered America in 1492.
Christopher Columbus discovered America in 1492.
Christopher Columbus discovered America in 1492.
Christopher Columbus discovered America in 1492.
Christopher Columbus discovered America in 1492 .
Hyper-dimensional Graph
who
is
America
Columbus
discover
d_1
d_{2}
d_2
d_{12288}
d_{1}
d_{3}
Coding Tools
// Reverse a string in one line
function reverseString(str) {
return str.split('').reverse().join('');
} // Reverse a string in one lineHow did this happen!!?
GPT-3 can be used for...
Writing
Tools
Automated Summaries
Translations
Coding
Tools
Conversational
AI
Instructions
Codex
Codex Model
- Released: August 2021
- GitHub snapshot (May-2020)
- 54M public repositories
- 12B parameters, 96 layers, 12,288Β dimensions
- Total 179GB, 159GB sanitised
Codex Language Support
Perl
Go
Ruby
Python
JavaScript
TypeScript
SQL
PHP
Shell
Rust
Swift
C#
GitHub Copilot
GitHub Copilot Architecture
OpenAI
GitHub
Visual Code
Copilot Service
Codex
Technical Preview
JavaScript
TypeScript
Go
Ruby
Python
Use comments
Or cherry-pick alternatives
Insert & Edit
Refactoring
DALL-E2
Generating images
An astronaut
lounging in a tropical resort in space
in a vaporwave style
An astronaut
lounging in a tropical resort in space
as pixel art
An astronaut
lounging in a tropical resort in space
in a photorealistic style
Prompt
DALL-E2
Editing images
Prompt
DALL-E2
Add flamingo
Original
Codex Experiment
This is only the beginning
The future of AI is today
Try it!
OpenAI Playground
Copy.ai
OpenAI and the AI that may overtake humans
By Gerard Sans | Axiom π¬π§
OpenAI and the AI that may overtake humans
This is a talk on the openAI project and how close they are to creating powerful artificial general intelligence. Elon Musk said that "artificial general intelligence is likely to overtake humans in the next five years.". We'll learn about their latest iteration of their powerful language model GPT-3 and some of its incredible applications including text and code generation using their invite-only APIs. GPT3 can generate text in English language but also generate source code in JavaScript, Python or even SQL from natural language!
