Mathias Arlaud
Co-Founder & COO @Bakslash - Co-Founder & CTO @Synegram
From Serializer to JsonEncoder
Robin Chalas
@bakslashHQ
@chalas_r
Mathias Arlaud
@bakslashHQ
@matarld
Representing data structures in a format that can be sent or persisted in order to be reconstructed later
Binary, textual
Construction pattern
Databases, flat files, APIs
Anywhere, interoperable
Serialization?
@chalas_r
@matarld
https://symfony.com/doc/current/components/serializer.html
Symfony's serializer
@chalas_r
@matarld
Symfony's serializer
{
"name":"Axel",
"color":"Red",
"age":2
}array(3) {
["name"]=> string(4) "Axel"
["color"]=> string(3) "Red"
["age"]=> int(2)
}object(Cat)#1 (3) {
["name"]=> string(4) "Axel"
["color"]=> string(3) "Red"
["age"]=> int(2)
}@chalas_r
@matarld
Some limitations
@chalas_r
@matarld
Design & BC limitations
@chalas_r
@matarld
AbstractNormalizer
allows attributes
instantiates objects
AbstractObjectNormalizer
normalization
denormalization
validation
ObjectNormalizer
extracts attributes
BC policy
🐌
Reflection is slow
Improved a lot over PHP versions, but still... Caching is needed.
The Reflection Limitation
@chalas_r
@matarld
The Groups Rabbit Hole
@chalas_r
@matarld
class Pet
{
#[ORM\Column(type: 'boolean')]
#[Groups([
'cat_list:read',
'cat:read',
'cat:write'
])]
private bool $meow = false;
#[ORM\Column(type: 'boolean')]
#[Groups([
'dog_list:read',
'dog:read',
'dog:write'
])]
private bool $bark = false;
}@chalas_r
@matarld
class Cat
{
private bool $meow = false;
}
class Dog
{
private bool $bark = false;
}
1 Resource : 1 Represensation
1 Resource : 1 Represensation
@chalas_r
@matarld
class ListCat
{
private string $name;
private bool $meow;
}
class ReadCat
{
private string $name;
private float $height;
private float $weight;
}Metadata is data
Metadata
@chalas_r
@matarld
, but about data
data
metadata
"thing"
"animal"
"cat"
"flying cat"
@chalas_r
@matarld
Metadata
data
/** @temlate T of string */
final class Rain {
/** @return list<T> */
public function content(): array {...}
}
/** @var Rain<'cat'|'dog'> $*/
$englishRain = new Rain();
$fallingStuff = $englishRain->content();type
no idea
array of something
list of something
list of cats and dogs
Possible JSON
[
[0.7]
]
{
"c": 1
}
true
{
"a": 1
}[
3.14
][
"pi"
][ true ]
[
3.14
][
"dog"
][ "cat" ]
[
"dog"
][
"dog"
]The Type limitation
@chalas_r
@matarld
/**
* @template T of Animal
*/
final class FlyingAnimal
{
public function __construct(
public object $type,
private int $speed,
) {
}
/**
* @return class-string<T>
*/
public function getTypeClass(): string
{
return $this->type::class;
}
public function setSpeed(int $speed): void
{
$this->speed = $speed;
}
}
PropertyInfo
The Type limitation
@chalas_r
@matarld
TypeInfo component
@chalas_r
@matarld
/**
* @template T of Animal
*/
final class FlyingAnimal
{
public function __construct(
public object $type,
private int $speed,
) {
}
/**
* @return class-string<T>
*/
public function getTypeClass(): string
{
return $this->type::class;
}
public function setSpeed(int $speed): void
{
$this->speed = $speed;
}
}
TypeInfo
The Type limitation
@chalas_r
@matarld
Contribute to TypeInfo!
@chalas_r
@matarld
The Memory limitation
Huge collection of objects
Super big array
array
@chalas_r
@matarld
Json streaming
Huge collection of objects
array
@chalas_r
@matarld
JsonEncoder component
@chalas_r
@matarld
JsonEncoder component
PHP
metadata
Cat::class
class Cat
{
public string $name;
public bool $flying;
}
// ...
$this->jsonEncoder->encode($cat);Possible JSON
{
"name": "any_string",
"flying: true|false
}@chalas_r
@matarld
Encoder
Possible JSON
[
{
"name": "any_string",
"flying: true|false
},
{
"name": "any_string",
"flying: true|false
}
]
return static function ($cats, $stream, $config) {
$stream->write('[');
$prefix_0 = '';
foreach ($cats as $cat) {
$stream->write($prefix_0);
$stream->write('{"name":');
$stream->write(json_encode($cat->name));
$stream->write(',"flying":');
$stream->write($cat->flying ? 'true' : 'false');
$stream->write('}');
$prefix_0 = ',';
}
$stream->write(']');
};JsonEncoder component
@chalas_r
@matarld
I want to serialize a cat!
Does an encoder exist?
/\_____/\ / o o \ ( == ^ == ) ) ( ( ) ( ( ) ( ) ) (__(__)___(__)__)
Encode the cat
Yep 👌
Compute the cat JSON shape
Generate and store the encoder
No
JsonEncoder component
@chalas_r
@matarld
Read only needed
JSON part lazily
JsonEncoder pros and cons
Blazing fast!
Flat memory usage
Stream ready
Generics ready
Simple API
Edit data on-the-fly
__ o-''|\_____/) \_/|_) ) \ __ / (_/ (_/
@chalas_r
@matarld
@chalas_r
@matarld
composer req jolicode/automapperMapper
__ o-''|\_____/) \_/|_) ) \ __ / (_/ (_/
AutoMapper
@chalas_r
@matarld
use AutoMapper\Attribute as Mapper;
final class Cat
{
#[MapTo(
target: Dog::class,
property: 'barkVolume',
)]
public int $meowVolume;
}
final class Dog
{
public int $barkVolume;
}Renaming properties
barkVolume
meowVolume
@chalas_r
@matarld
use AutoMapper\Attribute as Mapper;
final class Cat
{
#[MapTo(
target: Dog::class,
transformer: [self::class, 'toDogAge'],
)]
public int $age;
public static function toDogAge(int $value, Source $source, array $context): int
{
return floor($value * 1.3);
}
}Updating values
@chalas_r
@matarld
final class Cat
{
#[MapTo(
target: Dog::class,
transformer: CatToDogAgeTransformer::class,
)]
public int $age;
}
class CatToDogAgeTransformer implements PropertyTransformerInterface
{
public function __construct(
private AgeConverter $ageConverter,
) {}
public function transform(int $value, object|array $source, array $context): mixed
{
return $this->ageConverter->fromCatToDog($value);
}
}Updating values
use Symfony\Component\Serializer\SerializerInterface;
final readonly class MyService
{
public function __construct(
private SerializerInterface $serializer,
) {
}
}Migration level 0-1
@chalas_r
@matarld
// config/bundles.php
return [
// ...
Mtarld\JsonEncoderBundle\JsonEncoderBundle::class => ['all' => true],
AutoMapper\Bundle\AutoMapperBundle::class => ['all' => true],
TurboSerializer\TurboSerializerBundle::class => ['all' => true],
];
composer require korbeil/turbo-serializerMigration level 0-1
@chalas_r
@matarld
use Symfony\Component\Serializer\SerializerInterface;
use TurboSerializer\Serializer;
final readonly class ApiController
{
public function __construct(
private SerializerInterface $turboSerializer,
) {
}
#[Route('/api/cats')]
public function get(): Response
{
$cats = $this->getCats();
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
return new Response($this->turboSerializer->serialize(
$cats,
'json',
[Serializer::TYPE => $type]
));
}
}Serialization level 0-1
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private SerializerInterface $turboSerializer,
) {
}
#[Route('/api/create-cats')]
public function post(string $content): Response
{
$cats = $this->turboSerializer->deserialize($content, Cat::class.'[]', 'json');
}
}Deserialization level 0-1
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private SerializerInterface $turboSerializer,
) {
}
#[Route('/api/cats-but-dogs')]
public function get(): Response
{
$cats = $this->getCats();
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
return new Response($this->turboSerializer->serialize(
$cats,
'json',
[Serializer::NORMALIZED_TYPE => $type]
));
}
}Serialization level 0-2
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private SerializerInterface $turboSerializer,
) {
}
#[Route('/api/create-cats')]
public function post(string $content): Response
{
$dogsType = Type::iterable(Type::object(Dog::class), Type::int(), asList: true);
$this->turboSerializer->deserialize(
$content,
Cat::class.[],
'json',
[Serializer::NORMALIZED_TYPE => $dogsType],
);
// ...
}
}Deserialization level 0-2
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private EncoderInterface $jsonEncoder,
) {
}
#[Route('/api/cats')]
public function get(): Response
{
$cats = $this->getCats();
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
return new Response((string) $this->jsonEncoder->encode($cats, $type));
}
}Serialization level 1-1
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private DecoderInterface $jsonDecoder,
) {
}
#[Route('/api/create-cats')]
public function post(string $content): Response
{
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
$cats = $this->jsonDecoder->decode($content, $type);
}
}Deserialization level 1-1
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private EncoderInterface $jsonEncoder,
) {
}
#[Route('/api/cats')]
public function get(): Response
{
$cats = $this->getCats();
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
return new StreamedResponse(function () use ($cats, $type): void {
foreach ($this->jsonEncoder->encode($cats, $type) as $chunk) {
echo $chunk;
}
});
}
}Serialization level 1-2
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private EncoderInterface $jsonEncoder,
) {
}
#[Route('/api/cats')]
public function get(): Response
{
$cats = $this->getCats();
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
$this->jsonEncoder->encode($cats, $type, ['stream' => new OutputStream()]);
return new Response();
}
}Serialization level 1-3
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private DecoderInterface $jsonDecoder,
) {
}
#[Route('/api/create-cats')]
public function post(): Response
{
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
$cats = $this->jsonDecoder->decode(new InputStream(), $type);
foreach ($cats as $i => $cat) {
if ($i === 1000) {
dd($cat->name);
}
}
}
}Deserialization level 1-3
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private EncoderInterface $jsonEncoder,
private AutoMapperInterface $autoMapper,
) {
}
#[Route('/api/cats-but-dogs')]
public function get(): Response
{
$cats = $this->getCats();
$type = Type::iterable(Type::object(Dog::class), Type::int(), asList: true);
$this->jsonEncoder->encode($this->catsToDogs($cats), $type, ['stream' => new OutputStream()]);
return new Response();
}
private function catsToDogs(iterable $cats): iterable
{
foreach ($cats as $cat) {
yield $this->autoMapper->map($cat, Dog::class);
}
}
}Serialization level 1-4
@chalas_r
@matarld
final readonly class ApiController
{
public function __construct(
private DecoderInterface $jsonDecoder,
private AutomapperInterface $mapper,
) {
}
#[Route('/api/create-cats')]
public function post(): Response
{
$type = Type::iterable(Type::object(Cat::class), Type::int(), asList: true);
$this->catsToDogs($this->jsonDecoder->decode(new InputStream(), $type));
}
private function catsToDogs(iterable $cats): iterable
{
foreach ($cats as $cat) {
yield $this->autoMapper->map($cat, Dog::class);
}
}
}Deserialization level 1-4
@chalas_r
@matarld
Benchmark
Deserialization
Serializer
JsonEncoder
JsonEncoder
+ Automapper
Serialization
JsonEncoder
Serializer
JsonEncoder
+ Automapper
@chalas_r
@matarld
Thanks!
https://github.com/mtarld/json-encoder-bundle
https://github.com/jolicode/automapper
https://github.com/korbeil/turbo-serializer
https://github.com/korbeil/turbo-serializer-bench
Robin Chalas
@bakslashHQ
@chalas_r
Mathias Arlaud
@bakslashHQ
@matarld
From Serializer to JsonEncoder Robin Chalas @bakslashHQ @chalas_r Mathias Arlaud @bakslashHQ @matarld
[Meetup] From Serializer to JsonEncoder
By Mathias Arlaud
[Meetup] From Serializer to JsonEncoder
- 939
