A Serializer, but on steroids

Mathias Arlaud

@bakslashHQ
@matarld

Mathias Arlaud

@matarld

mtarld

@bakslahHQ

Representing data structures in a format that can be sent or persisted in order to be reconstructed later

Binary, textual

Construction pattern

Databases, flat files, APIs

Anywhere, interoperable

Serialization?

@matarld

https://symfony.com/doc/current/components/serializer.html

Symfony's serializer

@matarld

Symfony's serializer

{
  "name":"Axel",
  "color":"Red",
  "age":2
}

array(3) {
  ["name"]=> string(4) "Axel"
  ["color"]=> string(3) "Red"
  ["age"]=> int(2)
}

object(Cat)#1 (3) {
  ["name"]=> string(4) "Axel"
  ["color"]=> string(3) "Red"
  ["age"]=> int(2)
}

@matarld

Some limitations

https://github.com/symfony/symfony/issues/30818

@matarld

Design & BC limitations

AbstractNormalizer

allows attributes

instantiates objects

AbstractObjectNormalizer

normalization

denormalization

validation

ObjectNormalizer

extracts attributes

BC policy

@matarld

🐌

Reflection is slow

Improved over PHP versions, but still...

Lot of caching to improve things

The Reflection limitation

@matarld

Metadata is data

Metadata

, but about data

@matarld

data

metadata

"thing"

"animal"

"cat"

"flying cat"

Metadata

@matarld

data

/** @temlate T of string */
final class Rain {
  /** @return list<T> */
  public function content(): array {...}
}

/** @var Rain<'cat'|'dog'> $*/
$englishRain = new Rain();

$fallingStuff = $englishRain->content();

type

no idea

array of something

list of something

list of cats and dogs

Possible JSON

[
  [0.7]
]

{
  "c": 1
}


true

{
  "a": 1
}

[
  3.14
]

[
  "pi"
]

[
  true
]

[
  3.14
]

[
  "dog"
]

[
  "cat"
]

[
  "dog"
]

[
  "dog"
]

The Type limitation

@matarld

interface PropertyTypeExtractorInterface
{
    /**
     * @return Type[]|null
     */
    public function getTypes(...): ?array;
}

The Type limitation

@matarld

interface PropertyTypeExtractorInterface
{
    /**
     * @return Type[]|null
     */
    public function getTypes(...): ?array;
}

only properties

no union/intersection difference

no list/dict difference

no generics

...

The Type limitation

@matarld

TypeInfo component

https://github.com/symfony/symfony/pull/52510

@matarld

ft. @Korbeil_

TypeInfo cool stuff

/**
 * @template T of Animal
 */
final class FlyingAnimal
{
  public function __construct(
    public object $type,
    private int $speed,
  ) {
  }
  
  /**
   * @return class-string<T>
   */
  public function getTypeClass(): string
  {
    return $this->type::class;
  }
  
  public function setSpeed(int $speed): void
  {
  	$this->speed = $speed;
  }
}

PropertyInfo

TypeInfo

@matarld

TypeInfo cool stuff

@matarld

TypeInfo cool stuff

$resolver = TypeResolver::create();

$resolver->resolve('list<int>');
$resolver->resolve($reflectionClass->getProperty('isFlying'));

@matarld

TypeInfo cool stuff

Type::nullable(Type::list(Type::int()));

Type::intersection(
  Type::object(\Stringable::class),
  Type::object(\Traversable::class),
);

Type::generic(
  Type::object(Collection::class),
  Type::object(User::class),
);

@matarld

Contribute to TypeInfo!

@matarld

The Memory limitation

Huge collection of objects

Super big array

array

@matarld

Json streaming

Huge collection of objects

array

@matarld

JsonEncoder component

https://github.com/symfony/symfony/pull/51718

@matarld

JsonEncoder component

PHP

metadata

Cat::class

class Cat
{
  public string $name;
  public bool $flying;
}

// ...

$this->jsonEncoder->encode($cat);

Possible JSON

{
  "name": "any_string",
  "flying: true|false
}

@matarld

Encoder

Possible JSON

[  
  {
    "name": "any_string",
    "flying: true|false
  },

  {
    "name": "any_string",
    "flying: true|false
  }

return static function ($cats, $stream, $config) {
  $stream->write('[');
  $prefix_0 = '';
  foreach ($cats as $cat) {
    $stream->write($prefix_0);
    $stream->write('{"name":');
    $stream->write(json_encode($cat->name));
    $stream->write(',"flying":');
    $stream->write($cat->flying ? 'true' : 'false');
    $stream->write('}');
    $prefix_0 = ',';
  }
  $stream->write(']');
};

JsonEncoder component

@matarld

I want to serialize a cat!

Does an encoder exist?

    /\_____/\
   /  o   o  \
  ( ==  ^  == )
   )         (
  (           )
 ( (  )   (  ) )
(__(__)___(__)__)

Encode the cat

Yep 👌

Compute the cat JSON shape

Generate and store the encoder

JsonEncoder component

@matarld

Read only needed

JSON part lazily

JsonEncoder pros and cons

Blazing fast!

Flat memory usage

Stream ready

Generics ready

Simple API

Edit data on-the-fly

   __
o-''|\_____/)
 \_/|_)     )
    \  __  /
    (_/ (_/

@matarld

composer req jolicode/automapper

Mapper

   __
o-''|\_____/)
 \_/|_)     )
    \  __  /
    (_/ (_/

AutoMapper

@matarld

use AutoMapper\Attribute as Mapper;

final class Cat
{
  #[MapTo(
    target: Dog::class,
    property: 'barkVolume',
  )]
  public int $meowVolume;
}

final class Dog
{
  public int $barkVolume;
}

Renaming properties

barkVolume

meowVolume

@matarld

use AutoMapper\Attribute as Mapper;

final class Cat
{
  #[MapTo(
    target: Dog::class,
    transformer: [self::class, 'toDogAge'],
  )]
  public int $age;
  
  public static function toDogAge(int $value, Source $source, array $context): int
  {
    return floor($value * 1.3);
  }
}

Updating values

@matarld

Updating values

final class Cat
{
  #[MapTo(
    target: Dog::class,
    transformer: CatToDogAgeTransformer::class,
  )]
  public int $age;
}

class CatToDogAgeTransformer implements PropertyTransformerInterface
{
  public function __construct(
    private AgeConverter $ageConverter,
  ) {}

  public function transform(int $value, object|array $source, array $context): mixed
  {
    return $this->ageConverter->fromCatToDog($value);
  }
}

use Symfony\Component\Serializer\SerializerInterface;

final readonly class MyService
{
  public function __construct(
    private SerializerInterface $serializer,
  ) {
  }
}

The Serializer

@matarld

TurboSerializer

// config/bundles.php

return [
  // ...
  Mtarld\JsonEncoderBundle\JsonEncoderBundle::class => ['all' => true],
  AutoMapper\Bundle\AutoMapperBundle::class => ['all' => true],
  TurboSerializer\TurboSerializerBundle::class => ['all' => true],
];

composer require korbeil/turbo-serializer

@matarld

TurboSerializer

final readonly class ApiController
{
  public function __construct(
    private SerializerInterface $turboSerializer,
  ) {
  }

  #[Route('/api/cat-but-dogs')]
  public function get(): Response
  {
    $cats = $this->callCats();
    
    return new Response($this->turboSerializer->serialize(
      $cats, 
      'json',
      [Serializer::NORMALIZED_TYPE => Type::list(Type::object(Dog::class))]
    ));
  }
}

@matarld

JsonEncoder

final readonly class ApiController
{
  public function __construct(
    private EncoderInterface $jsonEncoder,
  ) {
  }

  #[Route('/api/cats')]
  public function get(): Response
  {
    $cats = $this->getCats();
    $type = Type::list(Type::object(Cat::class));
    
    $this->jsonEncoder->encode($cats, $type, ['stream' => new OutputStream()]);
    
    return new Response();
  }
}

@matarld

JsonEncoder

final readonly class ApiController
{
  public function __construct(
    private DecoderInterface $jsonDecoder,
  ) {
  }

  #[Route('/api/create-cats')]
  public function post(): Response
  {
    $type = Type::list(Type::object(Cat::class));
    $cats = $this->jsonDecoder->decode(new InputStream(), $type);
    foreach ($cats as $i => $cat) {
      if ($i === 1000) {
        dd($cat->name);
      }
    }
  }
}

@matarld

JsonEncoder + AutoMapper

final readonly class ApiController
{
  public function __construct(
    private EncoderInterface $jsonEncoder,
    private AutoMapperInterface $autoMapper,
  ) {
  }

  #[Route('/api/cats-but-dogs')]
  public function get(): Response
  {
    $cats = $this->getCats();
    $type = Type::list(Type::object(Dog::class));
    $this->jsonEncoder->encode($this->catsToDogs($cats), $type, ['stream' => new OutputStream()]);
    
    return new Response();
  }
  
  private function catsToDogs(iterable $cats): iterable
  {
    foreach ($cats as $cat) {
      yield $this->autoMapper->map($cat, Dog::class);
    }
  }
}

@matarld

JsonEncoder + AutoMapper

final readonly class ApiController
{
  public function __construct(
    private DecoderInterface $jsonDecoder,
    private AutomapperInterface $mapper,
  ) {
  }

  #[Route('/api/create-dogs-as-cats')]
  public function post(): Response
  {
    $type = Type::list(Type::object(Cat::class));
    $this->catsToDogs($this->jsonDecoder->decode(new InputStream(), $type));
  }
    
  private function catsToDogs(iterable $cats): iterable
  {
    foreach ($cats as $cat) {
      yield $this->autoMapper->map($cat, Dog::class);
    }
  }
}

Benchmark

@matarld

Serialization

JsonEncoder

Serializer

JsonEncoder
+ Automapper

Deserialization

Serializer

JsonEncoder

JsonEncoder
+ Automapper

Thanks!

Mathias Arlaud

@bakslashHQ
@matarld

https://github.com/mtarld/json-encoder-bundle

https://github.com/jolicode/automapper

https://github.com/korbeil/turbo-serializer

https://github.com/korbeil/turbo-serializer-bench

A Serializer, but on steroids

Mathias Arlaud

Serialization?

Symfony's serializer

Symfony's serializer

Some limitations

Design & BC limitations

🐌

The Reflection limitation

Metadata

Metadata

The Type limitation

The Type limitation

The Type limitation

TypeInfo component

TypeInfo cool stuff

TypeInfo cool stuff

TypeInfo cool stuff

TypeInfo cool stuff

Contribute to TypeInfo!

The Memory limitation

Json streaming

JsonEncoder component

JsonEncoder component

JsonEncoder component

JsonEncoder component

JsonEncoder pros and cons

Mapper

AutoMapper

Renaming properties

Updating values

Updating values

The Serializer

TurboSerializer

TurboSerializer

JsonEncoder

JsonEncoder

JsonEncoder + AutoMapper

JsonEncoder + AutoMapper

Benchmark

Thanks!

[PHPers] A Serializer on steroids

More from Mathias Arlaud