Runtime Types Validation in Typescript

@emasuriano

React Global Online Summit 2020 

Hello, I'm Ema Suriano 👋

👨‍💻 Web Developer

☮️ Open Source Enthusiast

✍️ Writer

👨‍🍳 Foodie/Cook

🌎 Traveler

Menschen using Typescript

My Application is fully typed, it's 100% bug-free

If it builds, it works in production

We're using typescript, there is no need to test

And my reaction is like ...

Hier is Why 💁‍♂️

// Typescript definition
type ExampleType = {
  name: string;
  age?: number;
  pets: {
    name: string;
    legs: number;
  }[];
};

// communicates with external API
const fetchData = (): Promise<ExampleType> => {};

const getBiped = async () => {
  const data = await fetchData();
  // { name: 'John' }
  return data.pets.find((pet) => pet.legs === 2); // Boom
};

Typescript performs Static Type Checking

Runtime Validations are outside the scope ...

Do I need to validate everything? 🤔

Simply, NO 🎉

Rule of Thumb 👍

Validate all the external sources of your Application.

  • APIs Responses
  • Content from files
  • Untyped Libraries

Commonly Used Approaches

Manual Validation

Set of conditions that check if the structure is the expected.

const validate = (data: ExampleType) => {
  if (!data.pets) return false;
  // perform more checks

  return true;
};

const getBiped = async () => {
  const data = await fetchData();
  
  if (!validate(data)) {
  	throw Error('Validation Error: structure not valid ...');
  }

  return data.pets.find((pet) => pet.legs === 2);
};

Pros

✅  No external library required


✅  Business focused

Cons

❌  Manual work


❌  Duplication of code

Using a Validation Library

Make use of the existing validation library to check for the structure.

ajv
joi
v8n
validate.js
import Ajv from 'ajv'

const ajv = new Ajv();

const validate = ajv.compile({
  properties: {
    name: {
      type: 'string',
      minLength: 3,
    },
    age: { type: 'number' },
    pets: {
      type: 'array',
      items: {
        name: {
          type: 'string',
          minLength: 3,
        },
        legs: { type: 'number' },
      },
    },
  },
  required: ['name', 'pets'],
  type: 'object',
});

const getBiped = async () => {
  const data = await fetchData();
  
  if (!validate(data)) {
    throw Error('Validation failed: ' + ajv.errorsText(validate.errors));
    // Error: Validation failed: data should have required property 'pets'
  }

  return data.pets.find((pet) => pet.legs === 2);
};

Pros

✅  A standardized way to create validators and checks

✅  Improvement of Error Reporting

Cons

❌  Introduction of new Syntax

❌  Validators and Types are not in Sync*

Dynamic Types Validator

Create validators from your types.

Before

Now

Introducing

Automatically generate a validator using JSON Schema and AJV for any TypeScript type.

typescript-json-validator

Made by @ForbesLindesay 👏

Usage 👀

// src/types/ExampleType.ts

type ExampleType = {
  name: string;
  age?: number;
  pets: {
    name: string;
    legs: number;
  }[];
};
> npx typescript-json-validator src/types/ExampleType.ts ExampleType

# ExampleType.validator.ts created!
/* tslint:disable */
// generated by typescript-json-validator
import { inspect } from "util";
import Ajv = require("ajv");
import ExampleType from "./ExampleType";

// Instance of Ajv
export const ajv = new Ajv({
  allErrors: true,
  coerceTypes: false,
  format: "fast",
  nullable: true,
  unicode: true,
  uniqueItems: true,
  useDefaults: true,
});

ajv.addMetaSchema(require("ajv/lib/refs/json-schema-draft-06.json"));

export { ExampleType };

// Definition of Schema
export const ExampleTypeSchema = {
  $schema: "http://json-schema.org/draft-07/schema#",
  defaultProperties: [],
  properties: {
    age: {
      type: "number",
    },
    name: {
      type: "string",
    },
    pets: {
      items: {
        defaultProperties: [],
        properties: {
          legs: {
            type: "number",
          },
          name: {
            type: "string",
          },
        },
        required: ["legs", "name"],
        type: "object",
      },
      type: "array",
    },
  },
  required: ["name", "pets"],
  type: "object",
};
export type ValidateFunction<T> = ((data: unknown) => data is T) &
  Pick<Ajv.ValidateFunction, "errors">;
export const isExampleType = ajv.compile(ExampleTypeSchema) as ValidateFunction<
  ExampleType
>;

// Expose validate function
export default function validate(value: unknown): ExampleType {
  if (isExampleType(value)) {
    return value;
  } else {
    throw new Error(
      ajv.errorsText(
        isExampleType.errors!.filter((e: any) => e.keyword !== "if"),
        { dataVar: "ExampleType" }
      ) +
        "\n\n" +
        inspect(value)
    );
  }
}

Implementation 💪

import validate from 'src/types/ExampleType.validator';

const getBiped = async () => {
  const data = validate(await fetchData());

  return data.pets.find((pet) => pet.legs === 2);
};

Typescript ❤️ AJV

interface ExampleType {
  /**
   * @format email
   */
  email?: string;
  /**
   * @minimum 0
   * @maximum 100
   */
  answer: number;
}
export const ExampleTypeSchema = {
  $schema: 'http://json-schema.org/draft-07/schema#',
  defaultProperties: [],
  properties: {
    answer: {
      maximum: 100,
      minimum: 0,
      type: 'number',
    },
    email: {
      format: 'email',
      type: 'string',
    },
  },
  required: ['answer'],
  type: 'object',
};

But wait for a second 🤔

pre commands 🎉
{
  "scripts": {
    "prestart": "ts-node generate-validators.ts src/types",
    "start": "react-scripts start",
    "prebuild": "ts-node generate-validators.ts src/types",
    "build": "react-scripts build",
  }
}
➜  my-project git:(master) ✗ yarn start
yarn run v1.22.5

$ ts-node generate-validators.ts src/types
Generating validators from src/types ...
 - AnotherType.validator.ts created!
 - ExampleType.validator.ts created!

$ react-scripts start
import { readdirSync, statSync } from 'fs';
import { promisify } from 'util';

const exec = promisify(require('child_process').exec);

const VALIDATOR_EXTENSION = '.validator.ts';

const [dir] = process.argv.slice(2);
if (!statSync(dir).isDirectory)
  throw new Error('Please specify a directory ...');

console.log(`Generating validators from ${dir} ...`);

const types = readdirSync(dir)
  .filter((x) => !x.endsWith(VALIDATOR_EXTENSION))
  .map((type) => type.split('.')[0]);

Promise.all(
  types.map(async (type) => {
    const command = `yarn typescript-json-validator ${dir}/${type}.ts ${type}`;
    const { stderr } = await exec(command);
    if (stderr)
      throw new Error(
        `There was an error while creating the validators: ${stderr}`,
      );

    console.log(`- ${type}${VALIDATOR_EXTENSION} created!`);
  }),
);

Basic Implementation

import { statSync } from 'fs';
import { watch } from 'chokidar';
import { basename } from 'path';
import { promisify } from 'util';

const exec = promisify(require('child_process').exec);
const readFile = promisify(require('fs').readFile);
const writeFile = promisify(require('fs').writeFile);

const VALIDATOR_EXTENSION = '.validator.ts';

const [dir] = process.argv.slice(2);
if (!statSync(dir).isDirectory)
  throw new Error('Please specify a directory ...');

console.log(`Generating validators from ${dir} ...`);

const generateValidator = async (path: string) => {
  const { stderr } = await exec(
    `yarn typescript-json-validator ${path} ${basename(path, '.ts')}`,
  );
  if (stderr)
    throw new Error(
      `There was an error while creating the validators: ${stderr}`,
    );

  console.log(`- ${path.replace('.ts', VALIDATOR_EXTENSION)} created!`);
};

watch(dir, {
  ignored: `**/*${VALIDATOR_EXTENSION}`,
})
  .on('add', generateValidator)
  .on('change', generateValidator);

With Watch 👀

My Experience with this Approach 🛠

Dashboard to provide a quick overview of Open Source projects in Github

oss-project-dashboard

Get Gist Information

Fetch Projects Information

Compare Threshold against Projects

Application Flow 

External Sources

  1. Content from GitHub Gist
  2. Data coming from the GitHub API

Relevant Types

// Settings.ts

export type Threshold = {
  pullRequests?: number;
  issues?: number;
};

export type Settings = {
  projects: string[];
  threshold?: Threshold;
};

export default Settings;

// Project.ts

export type Stargazer = {
  id: string;
  name: string | null;
  avatarUrl: string;
};

export type Countable = {
  totalCount: number;
};

export type Project = {
  id: string;
  url: string;
  name: string;
  pullRequests: Countable;
  vulnerabilityAlerts: Countable;
  issues: Countable;
  stargazers: Countable & {
    nodes: Stargazer[];
  };
};

export default Project;

1. Validate Gist Info

import { isQueryReady } from '../utils/queries';
import { useQuery } from '@apollo/react-hooks';
import { QueryResult } from '@apollo/react-common';
import { ApolloError } from 'apollo-boost';
import { EMPTY_SETTINGS } from '../utils/constant';
import { INVALID_SETTINGS_ERROR } from '../utils/error';
import { Settings } from '../types/Settings';
import { Query } from '../queries/ProjectQuery';
import validateSettings from '../types/Settings.validator';

type SettingsQueryResult = QueryResult<QueryData> & { output: Settings };

const useSettingsQuery = (gistName: string): SettingsQueryResult => {
  const projectsQuery = useQuery(Query, {
    variables: { name: gistName },
    skip: !gistName,
  });

  if (isQueryReady(projectsQuery)) {
    const { viewer } = projectsQuery.data!;
    const gistFile = viewer.gist.files[0];

    try {
      const gistContent = JSON.parse(gistFile.text);
      const settings = validateSettings(gistContent);
      return {
        ...projectsQuery,
        output: settings,
      };
    } catch (error) {
      return {
        ...projectsQuery,
        output: EMPTY_SETTINGS,
        error: new ApolloError({
          errorMessage: INVALID_SETTINGS_ERROR,
          extraInfo: error,
        }),
      };
    }
  }

  return {
    ...projectsQuery,
    output: EMPTY_SETTINGS,
  };
};

export default useSettingsQuery;

2. Validate Projects

import { isQueryReady } from '../utils/queries';
import { useQuery } from '@apollo/react-hooks';
import { Query, QueryData } from '../queries/RepositoriesQuery';
import { QueryResult } from '@apollo/react-common';
import Project from '../types/Project';
import validateProject from '../types/Project.validator';
import { projectNameToParts } from '../utils/string';

type ProjectDataQueryResult = QueryResult<QueryData> & { output: Project[] };

const DEFAULT: string[] = [];

const useProjectDataQuery = (projects = DEFAULT): ProjectDataQueryResult => {
  const repositoriesQuery = useQuery<QueryData>(Query(projects), {
    skip: projects.length === 0,
  });

  if (isQueryReady(repositoriesQuery)) {
    try {
      const projectsWithData = projects
        .map((projectName) => {
          const { key } = projectNameToParts(projectName);
          return repositoriesQuery.data![key];
        })
        .filter(Boolean)
        .map(validateProject);

      return {
        ...repositoriesQuery,
        output: projectsWithData,
      };
    } catch (error) {
      return {
        ...repositoriesQuery,
        error: new ApolloError({
          errorMessage: INVALID_PROJECTS_ERROR,
          extraInfo: error,
        }),
        output: [],
      };
    }
  }

  return {
    ...repositoriesQuery,
    output: [],
  };
};

export default useProjectDataQuery;

Summary 🧳

 

Approach

 

Manual

 

Library

 

Dynamic Types

No New Syntax

 

✅

 

❌

 

✅

Validators and Types in Sync

 

❌

 

❓

 

✅


Standardization

 

❌

 

✅

 

✅

Validate more,
with less 👍

Thanks for listening! 🙌

slides.com/emasuriano/runtime-types-validation-in-typescript

References 📖

Questions? 🗣

Runtime Types Validation in Typescript

By Ema Suriano

Made with Slides.com

Runtime Types Validation in Typescript

Sometimes using types inside our application, it's not enough to make sure that it won't break on runtime. Because of that, we see ourselves forced to write validation for our entities manually. In this talk, I want to present a way that you can generate validators for all your types in your application automatically, and how to integrate this into your development workflow. Recording link: https://www.youtube.com/watch?v=VI-HhBTB7cc

  • 1,906

More from Ema Suriano