Facial Recognition

by Sunny Cao

Can you recognize this face?

You can probably recognize this face immediately (unless you have prosopagnosia). It seems quite simple for you to recognize faces, but for computers, this is a difficult challenge to figure out.

When you look at these images, you can easily recognize that it depicts the same person, even though the image may be squashed, stretched, or otherwise edited. When a computer sees these images, however, it is much more difficult for it to come to the same conclusion as you.

Why is it so much more difficult for computers to recognize what humans can understand in a glance?

1. Humans recognize patterns instead of scrutinizing pixels

  • humans can recognize faces from holistic patterns (eyes, nose, mouth, spacing, symmetry)
  • computers start with pixel values of an image
  • even small changes, like differences in lighting, angle, expression, accessories, etc. will be read differently by a machine, which can lead to a completely different understanding of what the image shows

2. Faces constantly change

  • humans can account for variations in hairstyle, hair color, accessories (like glasses and hats) different expressions, etc. because we understand what parts of a face remains stable
  • computers have to learn which features remain the same, and which features can change

3. Perspective and lighting

  • humans can recognize faces from any angle or lighting
  • changes in pose (e.g. side profile vs. front view) and lighting can confuse machines, who rely on each pixel value
  • machines need many, perhaps thousands, of training examples to handle these variations

4. Humans have context

  • humans can use context clues (such as where you see the person, familiar clothing, etc.) to recognize faces
  • since computers usually don't have context, they can only rely on the image itself

The process goes like this:
​face detection -> feature extraction -> feature representation -> database comparison -> matching and recognition -> decision making

So....how do machines even recognize faces? Why is it so difficult??

Face Detection

  • First step: locate and isolate faces in images or video frames

  • Techniques:

    • Haar Cascades: uses simple patterns (like edges) for quick detection

    • HOG (Histogram of Oriented Gradients): captures shape and structure of faces

    • CNNs (Convolutional Neural Networks): deep learning-based, highly accurate, can detect faces under varying conditions

Feature Extraction

  • Extract key facial features: eyes, nose, mouth, jawline, face contours

  • Convert these features into a mathematical representation called a feature vector

  • Techniques:

    • PCA (Principal Component Analysis): reduces dimensionality while keeping main features

    • LBP (Local Binary Patterns): captures local texture patterns

    • Deep Learning Networks: automatically learn complex facial features

Feature Representation

  • Feature vectors encode unique characteristics of a face into numbers

  • Enables comparison between faces in a consistent format

  • Each person’s face gets a unique embedding in high-dimensional space

  • A feature vector is like a "face fingerprint," no two vectors are exactly the same for different people

Database Comparison

  • Detected face vectors are compared with a database of known faces

  • Database stores:

    • Feature vectors

    • Corresponding identities (names, IDs, metadata)

  • Large databases require efficient search algorithms for speed

Matching and Recognition

  • System calculates similarity or distance between the detected face vector and stored vectors

  • If similarity exceeds a threshold, the system considers it a match

  • Types:

    • Verification (1:1): Confirming if the person is who they claim to be

    • Identification (1:N): Determining the person’s identity from a large set of known faces

Decision Making

  • Based on matching, the system decides the identity of the person

  • Actions can include:

    • Unlocking devices

    • Granting access to restricted areas

    • Tagging people in photos

  • Accuracy depends on detection, extraction, and database quality

THIS IS DAN RYAN

NOW YOU KNOW HOW A COMPUTER RECOGNIZES FACES!!!