Model Serving with FastAPI

Error Handling & HTTP Status Codes

Learning Outcome

4

Apply safe error responses without leaking internals.

3

Recognize retryable vs non-retryable errors.

2

Identify when to use HTTPException vs a custom handler.

1

Understand which status code fits each API situation

5

Connect retry and backoff to calling LLM APIs.

Status Codes: Client Errors (4xx)

400

401

403

404

422

Bad Request

Unauthorized

Forbidden

Not Found

Unprocessable Entity

429

Too Many Requests

Passed the schema but breaks a business rule, e.g., a prediction date in the past.

API key missing, wrong, or expired.

 Identified but not allowed, e.g., free-tier key calling a premium model.

Unknown model name or prediction ID.

Failed Pydantic validation; automatic.

Over the request limit for the time window.

4xx = the client must fix the request.

Status Codes: Success and Server Errors (2xx, 5xx)

 

200 OK

a type only, e.g., tenure_months: int.

201 Created

coupon_code: str | None = None.

500 Internal Server Error

Literal restricts values

to a listed set.

503 Service Unavailable

Running but can't serve right now, e.g., model still loading or maintenance.

Never return 200 with an error inside the body. Clients, monitors, and load balancers judge success by the status code.

Raising Errors with HTTPException

Code

raise HTTPException(status_code=404,
                    detail="Model 'churn-v9' not found")

Detail can be a dict for structure, e.g., {"error": "model_not_found", "available": [...]}. Headers can add Retry-After for 429.

Detect the problem inside the endpoint, e.g., a model name not in the registry.

Raise HTTPException; execution stops immediately.

FastAPI responds with that status and a JSON detail body.

Custom Exceptions and Exception Handlers

Raise them in the logic

Prediction code never mentions status codes.

Code

@app.exception_handler(ModelNotLoadedError)
def model_not_loaded(request, exc):
    return JSONResponse(status_code=503,
                        content={"error": "model_not_loaded"})

Map once with a handler

 One function converts the error into a consistent JSONResponse.

Define domain errors

ModelNotLoadedError, FeatureOutOfRangeError describe problems in your app's terms.

Handling Unexpected Errors Safely

Fail fast when the model is missing

Return 503 from /predict and /health, not a confusing 500 on every request.

4

No stack traces to clients

They reveal file paths, library versions, and code structure. Full traceback goes to server logs only.

1

Return a request ID

The client reports it; the team finds the exact log entry instantly.

2

One error format

404, 422, and 500 share the same JSON structure.

3

Retryable vs Non-Retryable Errors (Core Mechanism)

Step 1:

Check the status code familyContent generation and assistance

Step 2 : 400, 401, 403, 404, 422

Don't retry; fix the request first.

Step 3 : 429

Wait for the Retry-After time, then retry.

Step 4: 503 and some 5xx

Retry with backoff: 1 s, 2 s, 4 s, instead of hammering the server.

LLM provider APIs regularly return 429 on rate limits and 5xx (some use 529) when overloaded, so every production GenAI app needs retry with backoff.

​HTTPException vs Custom Exception Handler (Comparison)

AspectHTTPExceptionCustom Exception + Handler
        Where rules liveDirectly in the endpointAnywhere in business or prediction logic
     Knows about HTTP?Yes, status set where raisedNo; the handler decides the status
    ConsistencyDepends on each developerOne handler, one format everywhere
       Reuse outside APINot suited to scripts or batch jobsSame logic works in scripts and jobs
    Typical useSimple checks, e.g., missing ID in one endpointDomain errors, e.g., model not loaded, quota exceeded
  AnalogyOne ATM printing its own messageBank-wide standard screen for every ATM

Think About This: Spotting the Difference

A client requests a model name that isn't in the registry

Client asked for something that doesn't exist.

The server restarted and the model is still loading

 request is fine; server is temporarily unable.

Too frequent; slow down.

A client sends 300 requests a minute on a 100/minute plan

404

503

429

4xx says you fix it; 5xx says we'll fix it.

 Real-World Applications

Load balancers

Route traffic away from instances whose /health returns 503.

Support desks

A user quotes the request_id, and the exact log entry is found in seconds.

Monitoring dashboards

Error rates are counted by status code, so honest codes keep alerts accurate.

GenAI apps

Retry 429 and 5xx from LLM providers with backoff; 401 means fix the API key.

Summary

5

Tracebacks in logs, request ID to client.

4

Retry 429 and 503 with backoff.

3

HTTPException inline; handlers for domain errors.

2

Never return 200 for a failure.

1

4xx: client fixes. 5xx: server fixes.

Quiz

The server just restarted and the model is still loading when a valid prediction request arrives. What should the API return?

A.503 Service Unavailable

B.500 Internal Server Error

C.422 Unprocessable Entity

D.200 OK with {"error": "model loading"}

Quiz-Answer

The server just restarted and the model is still loading when a valid prediction request arrives. What should the API return?

A.503 Service Unavailable

B.500 Internal Server Error

C.422 Unprocessable Entity

D.200 OK with {"error": "model loading"}

7.5 Error Handling & HTTP Status Codes

By Content ITV

7.5 Error Handling & HTTP Status Codes

  • 123