UUID
The only ID you need?
ARKADIUSZ KONDAS
Lead Software Architect
@ Proget Sp. z o.o.
Zend Certified Engineer
Code Craftsman
Ultra Runner
@ ArkadiuszKondas
Zend Certified Architect
php-ai/php-ml
Genesis
Database
node (entity)
edge (relation)
Primary key
Main role: to provide a stable, indexable reference to an entity.
-
semantic / natural
extracted from the entities attributes (one or more fields)
-
technical / surrogate
completely unrelated to the fields of its entity
constructed when the entity is inserted in the DB.
Auto-increment / serial
> CREATE TABLE customer (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL
);
> INSERT INTO customer (name) VALUES ('John');
> SELECT * FROM customer;
id | name
----+------
1 | John
(1 row)
> SELECT currval('customer_id_seq'::regclass);
currval
---------
1
(1 row)
Why can be a problem?
-
Information disclosure
In url: /api/user/123
all it takes to know your user count is to create an account and look at link
-
Entity enumerations
easy to scrape all your entities (just start from 1 and ++)
-
Non uniqueness across tables
ON DELETE CASCADE
Why can be a problem?
Database round trip
$customer = new Customer('John', 'Snow');
$orm->persist($customer);
$orm->flush();
$customer->id();
Workarounds
- You can configure your sequence to start at an arbitrary point
- Have an increment bigger than one
- Share the sequence between tables
Workarounds
Obfuscation
hidden secret key is required (salt)
zackkitzmiller/tiny-php
$tiny = new \ZackKitzmiller\Tiny('5SX0...M3so9bfzZpuUenKtP74QNH6B');
echo $tiny->to(5);
// E
1 | Z4UrtW |
3456 | gPUasb |
Workarounds
Obfuscation
the salt can be guessed from about 30 hashes !!!
UUID FTW
U U ID
universally
unique
identifier
RFC 4122
Specific variant with 5 version
U U
universally
unique
Possible values: \(2^{122}\)
Collision: number of random version 4 UUIDs which need to be generated in order to have a 50% probability of at least one collision is 2.71 quintillion - generating 1 billion UUIDs per second for about 85 years
45 EB
Unsigned 128-bit integer
133187334300416338637111672389176371521
32 hexadecimal (base 16) digits
6432f291-9a6b-42ba-8e73-9e7cb327a941
Anatomy
6432f291-9a6b-42ba-8e73-9e7cb327a941
time_low
time_mid
time_hi
clock_seq_hi_and_res
node
version
variant
ramsey/uuid
Version 1: Time based
use Ramsey\Uuid\Uuid;
for ($i=0; $i<10; $i++) {
echo Uuid::uuid1()->toString() . PHP_EOL;
}
06d0af36-bfc2-11e7-a118-02424c396740
06d0f6e4-bfc2-11e7-a58f-02424c396740
06d0f7a2-bfc2-11e7-9265-02424c396740
06d0f824-bfc2-11e7-a503-02424c396740
06d0f932-bfc2-11e7-937b-02424c396740
06d0f9aa-bfc2-11e7-b47a-02424c396740
06d0fa22-bfc2-11e7-9630-02424c396740
06d0fa90-bfc2-11e7-a5d2-02424c396740
06d0fafe-bfc2-11e7-967e-02424c396740
06d0fb6c-bfc2-11e7-9e54-02424c396740
Version 3 & 5:
Name based
$namespace = Uuid::uuid3(Uuid::NAMESPACE_DNS, 'phpce.eu');
echo Uuid::uuid3($namespace, '/about');
echo Uuid::uuid3($namespace, '/contact');
da099c12-57c5-38bb-a5ca-7a9f0a6f6578
a0f6e622-41b5-3608-b858-8dc231da5c01
Hashing algorithm:
- MD5 -> Version 3
- SHA1 -> Version 5
Version 4: Random
use Ramsey\Uuid\Uuid;
for ($i=0; $i<10; $i++) {
echo Uuid::uuid4()->toString() . PHP_EOL;
}
c9cb5802-fbc2-4e63-9812-160d6b14af32
7a7e6dea-7c0d-4173-a3c6-5a9f0386e9ad
9af361b5-f0a6-4caa-af7d-ab01a8b8c0fe
95b2a611-b1f7-4496-a911-532e408b8fec
adf887f4-d07c-4149-80a9-a6adf0406490
4bd03549-de3b-49dd-a784-a32388bb777d
843ef345-1632-4960-a0d3-0967b981de99
3fa23ff3-6b80-4a65-bcc2-5a8021cf3fe0
a47b0437-8e7e-43a1-8783-c747080920b4
bdc2334e-f1cb-4c4d-a94d-4f8fe30f1d56
Version 2:
DCE Security
RFC 4122 does not explicitly define it
not implemented
ramsey/uuid-console
➜ composer global require ramsey/uuid-console
➜ uuid generate --help
Usage:
generate [options] [--] [<version>] [<namespace>] [<name>]
Arguments:
version The UUID version to generate. Supported are version "1", "3", "4" and "5". [default: 1]
namespace For version 3 or 5 UUIDs, the namespace to create a UUID for. May be either a UUID in string representation or an identifier for internally pre-defined namespace UUIDs (currently known are "ns:DNS", "ns:URL", "ns:OID", and "ns:X500").
name For version 3 or 5 UUIDs, the name to create a UUID for. The name is a string of arbitrary length.
Options:
-c, --count=COUNT Generate count UUIDs instead of just a single one. [default: 1]
--comb For version 4 UUIDs, uses the COMB strategy to generate the random data.
-g, --guid
ramsey/uuid-console
➜ uuid decode 2bdaf386-c012-11e7-8d9b-02424c396740
========= ========== ========================================
encode: STR: 2bdaf386-c012-11e7-8d9b-02424c396740
INT: 58293663835462406327300766704462292800
decode: variant: RFC 4122
version: 1 (time and node based)
content: time: 2017-11-02T21:09:54+00:00
clock: 3483 (usually random)
node: 02:42:4c:39:67:40
========= ========== ========================================
➜ uuid decode ffab8f50-4510-4c7f-af8f-e0a09f43a868
========= ========== =================================================
encode: STR: ffab8f50-4510-4c7f-af8f-e0a09f43a868
INT: 339843928432163496190993563305967331432
decode: variant: RFC 4122
version: 4 (random data based)
content: ff:ab:8f:50:45:10:4c:7f:af:8f:e0:a0:9f:43:a8:68
(no semantics: random data only)
========= ========== =================================================
Real life example
public function createAction()
{
// ...
$id = (string) UUID::generate();
$this->getContainer()->get('command_bus')->handle(
new CreateCustomer(
$id,
'Proget Sp. z o.o.',
'ul. Warszawska 28 43-430 Bielsko-Biała',
'Jan Kowalski',
'info@proget.pl',
'+48 33 822 14 85'
)
);
// ...
}
Uuid:uuid4() collisions
Uuid:uuid4() collisions #80
private static function generateBytes($length)
{
if (self::hasOpensslRandomPseudoBytes()) {
return openssl_random_pseudo_bytes($length);
}
$bytes = '';
for ($i = 1; $i <= $length; $i++) {
$bytes = chr(mt_rand(0, 255)) . $bytes;
}
return $bytes;
}
Uuid:uuid4() collisions #80
OpenSSL’s default random number generator mixes in the PID, which provides a certain degree of fork safety. However, once the PIDs wrap, new children will start to produce the same random sequence as previous children which had the same PID.
Solution
openssl_random_pseudo_bytes()
random_bytes()
>= PHP 7
< 7
paragonie/random_compat
PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
Level: advanced
COMB
- Combined UUID version 4 with timestamp
-
Replaces the least significant bytes of the node field with the current timestamp
-
Tries to compensate for the reduced clustering in database indexes
COMB
$factory = new UuidFactory();
$generator = new CombGenerator(
$factory->getRandomGenerator(),
$factory->getNumberConverter()
);
$factory->setRandomGenerator($generator);
$combUuid = $factory->uuid4();
Override Time Generator
$factory = new UuidFactory();
$factory->setTimeGenerator(
new PeclUuidTimeGenerator()
);
$uuid = $factory->uuid1();
use pecl-uuid as time generator
Override Random Generator
$randomLibFactory = new \RandomLib\Factory();
$randomLibGenerator = $randomLibFactory
->getHighStrengthGenerator();
$uuidFactory = new UuidFactory();
$uuidFactory->setRandomGenerator(
new RandomLibAdapter($randomLibGenerator)
);
$uuid = $uuidFactory->uuid4();
use ircmaxell/random-lib library as random generator
More customization
- Generators:
- RandomGeneratorInterface
- TimeGeneratorInterface
- Codecs:
- CodecInterface
- Converters:
- NumberConverterInterface
- TimeConverterInterface
- Providers:
- NodeProviderInterface
- TimeProviderInterface
UUID
in
Database
UUIDs in PostgreSQL
> select '74546161-06ea-42be-a27d-f070461557e1'::uuid;
uuid
--------------------------------------
74546161-06ea-42be-a27d-f070461557e1
(1 row)
CREATE EXTENSION "uuid-ossp";
-- or CREATE EXTENSION "pgcrypto";
CREATE TABLE customer (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name VARCHAR(255) NOT NULL
);
UUIDs in MySQL
BINARY(16)
Uuid::uuid4()->getBytes();
$factory = new UuidFactory();
$codec = new OrderedTimeCodec(
$factory->getUuidBuilder()
);
$factory->setCodec($codec);
Uuid::setFactory($factory);
Generate an optimized version 1 (time-based) UUID object
Performance
Horizontal axis – Number of inserts x 25,000
Vertical axis – Time Taken in seconds
Time
Performance
Horizontal Axis – Number of inserts x 25,000
Vertical Axis – Data Size in MB
Size
Performance
-
MySQL
https://www.percona.com/blog/2014/12/19/store-uuid-optimized-way/
-
SQL Server 200
http://www.informit.com/articles/printerfriendly/25862
-
PostgreSQL
http://www.cybertec.at/int4-vs-int8-vs-uuid-vs-numeric-performance-on-bigger-joins/
+ 10%
ramsey/uuid-doctrine
When UUID is not what you need
When UUID is not what you need
- no mutable state, no technical primary key
- strong storage constraints (2 to 4 times more space)
- you don't expose the primary key
- big database with big numer of relations
Best of Both = Integers Internal, UUIDs External
Alternatives
ulid
https://github.com/alizain/ulid
cuid
https://github.com/ericelliott/cuid
Tailor made
CREATE OR REPLACE FUNCTION insta5.next_id(OUT result bigint) AS $$
DECLARE
our_epoch bigint := 1314220021721;
seq_id bigint;
now_millis bigint;
shard_id int := 5;
BEGIN
SELECT nextval('insta5.table_id_seq') %% 1024 INTO seq_id;
SELECT FLOOR(EXTRACT(EPOCH FROM clock_timestamp()) * 1000) INTO now_millis;
result := (now_millis - our_epoch) << 23;
result := result | (shard_id <<10);
result := result | (seq_id);
END;
$$ LANGUAGE PLPGSQL;
https://engineering.instagram.com/sharding-ids-at-instagram-1cf5a71e5a5c
Summary
Summary
- Unique across every table, every database, every server
- Allows easy merging of records from different databases
- You can generate IDs anywhere, instead of having to round trip to the database
- Most replication scenarios require GUID columns anyway
- It is a whopping 4 times larger than the traditional 4-byte index value
- Cumbersome to debug "where userid='BAE7DF4-DDF-3RG-5TY3E3RF456AS10"
- The generated UUIDs should be partially sequential for best performance
Other languages
Java, Scala (native)
java.util.UUID.randomUUID
Node.js (Install uuid from npm)
var uuid = require("uuid");
uuid.v4();
Haskell (Install uuid from hackage)
uuid :: IO UUID
uuid = nextRandom
Q&A
Thanks for listening
@ ArkadiuszKondas
https://slides.com/arkadiuszkondas
https://joind.in/talk/959bf
UUID - The only ID you need?
By Arkadiusz Kondas
UUID - The only ID you need?
What is UUID and what are its successor versions? How to start using UUID in PHP? Can my database handle UUID? What is UUID suitable for, and what is not? Is UUID the only ID I need? For these and other questions I will try to answer in a presentation devoted exclusively to uniquely unique identifiers.
- 2,286