Error handling in RESTful API's

Mon 30 May 2016 - PyCon US 2016 Open Spaces

John Roa

Technical team Lead

LendingFront

Error and exception handling

Two error handling approaches

Look Before You Leap

(LBYL)

Easier to Ask for Forgiveness than Permission

(EAFP)

Look Before You Leap

(LBYL)

Lots of if statements

 

You always check to see if something can be done before doing it

https://www.jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/

Look Before You Leap

(LBYL)

https://www.jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/

def print_object(some_object):
    # Check if the object is printable...
    if isinstance(some_object, str):
        print(some_object)
    elif isinstance(some_object, dict):
        print(some_object)
    elif isinstance(some_object, list):
        print(some_object)
    # 97 elifs later...
    else:
        print("unprintable object")

Easier to Ask for Forgiveness than Permission

(EAFP)

https://www.jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/

 you just do the thing

 

If it turns out that wasn't possible, shrug "my bad", and deal with it.

Easier to Ask for Forgiveness than Permission

(EAFP)

try:
    # try to execute some code here
except Exception:
    # if the initial execution fails, execute this one

Easier to Ask for Forgiveness than Permission

(EAFP)

https://www.jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/

def print_object(some_object):
    # Check if the object is printable...
    try:
        printable = str(some_object)
        print(printable)
    except TypeError:
        print("unprintable object")

CONTEXT

WEB API

UI: web pages, mobile apps

HTTP

Web APIs

  • programmatic interface 
  • one or more publicly exposed endpoints
  • defined request-response message system

REST

REpresentational State Transfer

Is an style of software architecture for distributed software

REST

Not only about using GET, POST, PUT, DELETE methods

REST

Six Constraints:

1. Uniform Interface

2. Stateless Interactions

3. Cacheable

4. Client-Server

5. Layered System

6. Code on Demand

More Info: 

https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm

UI: web pages, mobile apps

HTTP

RESTful APIs

  • programmatic interface 
  • one or more publicly exposed endpoints
  • defined request-response message system
  • Implements REST architecture principles

let's cut to the chase

Error handling in RESTful API's

do's and don'ts

don't

Forget/Avoid error handling

class LambdaView(View):

    @staticmethod
    def get(request, object_id):
        return HttpResponse(LambdaFunction.objects.get(id=object_id).name)

DON'T!  :'(

try / except pass

        try:
            ...
            ...
            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except Exception :
            pass

http://www.slideshare.net/pydanny/python-worst-practices

https://realpython.com/blog/python/the-most-diabolical-python-antipattern/

don't

Pokemon exception handling

don't

Pokemon exception handling: "gotta catch 'em all"

try:
    # try to do something
except Exception as e:
    # Gotta catch 'em all!
    return HttpResponse('Sorry, something is broken. We will look into that.',
        status=status.HTTP_500_INTERNAL_SERVER_ERROR)

do

class LambdaEditView(View):
    @staticmethod
    def get(request, object_id):
        try:
            ...
            ...
            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except LambdaFunction.DoesNotExist:
            return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
        except Exception as e:
            return HttpResponse('Sorry, something is broken. We will look into that.', 
                                status=status.HTTP_500_INTERNAL_SERVER_ERROR)

Error handling through multiple exceptions :)

don't

class LambdaEditView(View):
    @staticmethod
    def get(request, object_id):
        try:
            ...
            ...
            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except LambdaFunction.DoesNotExist:
            return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
        except Exception as e:
            return HttpResponse('Sorry, something is broken. We will look into that.', 
                                status=status.HTTP_500_INTERNAL_SERVER_ERROR)

Error handling ONLY through exceptions

do

class LambdaEditView(View):
    @staticmethod
    def get(request, object_id):
        try:

            token = request.GET.get('token')

            if not token:
                return HttpResponse('Invalid token.', status=status.HTTP_400_BAD_REQUEST)

            ...
            ...

            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except LambdaFunction.DoesNotExist:
            return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
        except Exception as e:
            return HttpResponse('Sorry, something is broken. We will look into that.', 
                                status=status.HTTP_500_INTERNAL_SERVER_ERROR)

Error handling through exceptions and if's

do

Error handling through exceptions and if's

No golden hammer here

 

Make sure of understanding the concepts and apply them depending on your particular case.

don't

Use a 200 status code for every response

HTTP/1.1 200 OK
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
    "success": false,
    "data": {
        "name": "Not Found Exception",
        "message": "The requested resource was not found."
    }
}

do

HTTP/1.1 404 Not Found
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
    "name": "Not Found Exception",
    "message": "The requested resource was not found."
}

don't

Using a 500 status code for all errors

HTTP/1.1 500: Internal server error
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
    "success": false,
    "data": {
        "name": "Invalid parameter",
        "message": "The given parameter is not valid."
    }
}

do

HTTP/1.1 400: Bad request
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
    "success": false,
    "data": {
        "name": "Invalid parameter",
        "message": "The given parameter is not valid."
    }
}

How do I choose the code?

How do I choose the code?

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

1xx: Informational - Request received, continuing process

How do I choose the code?

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

2xx: Success - The action was successfully received, understood, and accepted

How do I choose the code?

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

3xx: Redirection - Further action must be taken in order to complete the request

How do I choose the code?

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

4xx: Client Error - The request contains bad syntax or cannot be fulfilled

How do I choose the code?

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

4xx: Client Error - The request contains bad syntax or cannot be fulfilled

How do I choose the code?

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

5xx: Server Error - The server failed to fulfill an apparently valid request

What?? that's a lot of codes!!

It's a matter of being practical...

Be practical ;)

2xx Success

200 OK​

 201 Created

 204 No Content

3xx Redirection

 304 Not Modified​

4xx Client Error

400 Bad Request

401 Unauthorized

 403 Forbidden

 404 Not Found

 409 Conflict

5xx Server Error

500 Internal Server Error

Use the most common HTTP status codes

http://www.restapitutorial.com/httpstatuscodes.html

don't

Skip logging

class LambdaEditView(View):
    @staticmethod
    def get(request, object_id):
        try:
            ...
            ...
            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except LambdaFunction.DoesNotExist:
            # Where is the log? :(
            return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
        except Exception as e:
            # Where is the log? :(
            return HttpResponse('Sorry, something is broken. We will look into that.', 
                                status=status.HTTP_500_INTERNAL_SERVER_ERROR)

don't

Logging 400-level codes at ‘error’ level

class LambdaEditView(View):
    @staticmethod
    def get(request, object_id):
        try:
            ...
            ...
            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except LambdaFunction.DoesNotExist:
            log.error("Lambda with id %s was not found." % object_id)
            return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
        except Exception as e:
            log.error("Unexpected error. Error detail: %s" % str(e))
            return HttpResponse('Sorry, something is broken. We will look into that.', 
                                status=status.HTTP_500_INTERNAL_SERVER_ERROR)

http://www.slideshare.net/pydanny/python-worst-practices

do

Logging 400-level codes at ‘info’ level

class LambdaEditView(View):
    @staticmethod
    def get(request, object_id):
        try:
            ...
            ...
            return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
        except LambdaFunction.DoesNotExist:
            log.info("Lambda with id %s was not found." % object_id)
            return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
        except Exception as e:
            log.error("Unexpected error. Error detail: %s" % str(e))
            return HttpResponse('Sorry, something is broken. We will look into that.', 
                                status=status.HTTP_500_INTERNAL_SERVER_ERROR)

don't

Expose the “guts” to a client: full stack trace, environment variables’ values, etc. Too implementation-specific or “low-level” error messages

File "/home/taiga/.virtualenvs/taiga/lib/python3.4/site-packages/django/db/models/fields/related.py", line 572, in __get__
    rel_obj = qs.get()
  File "/home/taiga/.virtualenvs/taiga/lib/python3.4/site-packages/django/db/models/query.py", line 357, in get
    self.model._meta.object_name)
taiga.projects.userstories.models.DoesNotExist: UserStory matching query does not exist.

http://scriptin.github.io/2015-02-08/restful-http-error-handling.html

https://github.com/taigaio/taiga-back/issues/478

do

Use domain/API-specific exceptions with direct, precise, implementation-agnostic error messages written in plain language.

    try:
        question = Question.objects.get(pk=question_id)
    except Question.DoesNotExist:
        raise Http404("Question does not exist")

http://scriptin.github.io/2015-02-08/restful-http-error-handling.html

https://docs.djangoproject.com/en/1.9/intro/tutorial03/

do

Use domain/API-specific exceptions with direct, precise, implementation-agnostic error messages written in plain language.

...
...
except forms.ValidationError:
    return HttpResponse('Invalid email format', status=status.HTTP_400_BAD_REQUEST)
except Client.DoesNotExist:
    return HttpResponse('Client does not exists', status=status.HTTP_400_BAD_REQUEST)
except BadSignature:
    return HttpResponse('Invalid token.', status=status.HTTP_400_BAD_REQUEST)
except Exception as e:
    return HttpResponse('Sorry, something is broken. We will look into that.', 
                        status=status.HTTP_500_INTERNAL_SERVER_ERROR)

do

Handle custom codes

HTTP/1.1 400 Bad Request
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
    "name": "Invalid username",
    "message": "The username is not valid.",
    "code": 40001
}
HTTP/1.1 400 Bad Request
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
    "name": "Invalid email",
    "message": "The email is not valid.",
    "code": 40002
}

do

Handle custom codes: twitter


HTTP/1.1 403 Forbidden
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
  "errors": [
    {
      "code": 64,
      "message": "Your account is suspended and is not permitted to access this feature"
    }
  ]
}

https://dev.twitter.com/overview/api/response-codes

do

Handle custom codes: twitter

https://dev.twitter.com/overview/api/response-codes

do

Handle custom codes: facebook


HTTP/1.1 403 Forbidden
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8

{
	"error": {
		"message": "Message describing the error", 
		"type": "OAuthException", 
		"code": 190,
		"error_subcode": 460,
		"error_user_title": "A title",
		"error_user_msg": "A message"
		"fbtrace_id": "EJplcsCHuLu"
	}
}

https://developers.facebook.com/docs/graph-api/using-graph-api#errors

do

Handle custom codes: facebook

https://developers.facebook.com/docs/graph-api/using-graph-api#errors

do

Handle custom codes: Salesforce marketing cloud

https://code.exacttarget.com/apis-sdks/rest-api/error-handling.html

do

Provide a Help URL

https://www.box.com/blog/get-developer-hugs-with-rich-error-handling-in-your-api/

[shell]HTTP/1.1 400 Bad Request

 

{
  "type": "error",
  "status": 400,
  "code": "bad_request",
  "context_info": {
    "errors": [
      {
        "reason": "expiration_not_allowed",
        "name": "unshared_at",
        "message": "Expiration cannot be set on a shared link with 'collaborators' access",
        "location": "entity-body"
      }
    ]
  },
  "help_url": "https://box-content.readme.io/reference#errors",
  "message": "Bad Request",
  "request_id": "13628284550db80756c518"
}

do

Provide a Help URL: box

https://box-content.readme.io/reference#errors​

do

Provide a Help URL: twilio

https://www.twilio.com/docs/api/errors/20003

Resources

http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#eafp-vs-lbyl

https://www.servage.net/blog/2013/04/08/rest-principles-explained/

http://www.slideshare.net/pydanny/python-worst-practices

https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm

http://www.yiiframework.com/doc-2.0/guide-rest-error-handling.html

http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

http://scriptin.github.io/2015-02-08/restful-http-error-handling.html

http://caines.ca/blog/2013/04/21/3-terrible-anti-patterns-for-error-handling-in-rest-apis/

http://www.soa-probe.com/2013/02/layer-7-on-rest-api-error-handling.html

http://www.restapitutorial.com/httpstatuscodes.html

https://realpython.com/blog/python/the-most-diabolical-python-antipattern/

http://www.codingpedia.org/ama/resources-on-how-to-design-error-handling-in-a-rest-api/

https://www.box.com/blog/get-developer-hugs-with-rich-error-handling-in-your-api/

http://blog.cloud-elements.com/error-handling-restful-api-design-part-iii

https://developers.google.com/drive/v3/web/handle-errors

https://www.jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/

http://tutorials.jenkov.com/java-exception-handling/validation-throw-exception-or-return-false.html

http://www.wiredmonk.me/error-handling-and-logging-in-flask-restful.html

http://www.lopakalogic.com/articles/web-services-articles/advanced-error-handling-rest-based-web-service/

http://apigee.com/about/blog/technology/restful-api-design-what-about-errors

Social

 

John Roa

@jhonjairoroa87

 

www.linkedin.com/in/johnroa27

 

Thanks!

Q & A

Made with Slides.com