Error handling in RESTful API's
Mon 30 May 2016 - PyCon US 2016 Open Spaces
John Roa
Technical team Lead
Error and exception handling
Two error handling approaches
Look Before You Leap
(LBYL)
Easier to Ask for Forgiveness than Permission
(EAFP)
Look Before You Leap
(LBYL)
Lots of if statements
You always check to see if something can be done before doing it
Look Before You Leap
(LBYL)
def print_object(some_object):
# Check if the object is printable...
if isinstance(some_object, str):
print(some_object)
elif isinstance(some_object, dict):
print(some_object)
elif isinstance(some_object, list):
print(some_object)
# 97 elifs later...
else:
print("unprintable object")Easier to Ask for Forgiveness than Permission
(EAFP)
you just do the thing
If it turns out that wasn't possible, shrug "my bad", and deal with it.
Easier to Ask for Forgiveness than Permission
(EAFP)
try:
# try to execute some code here
except Exception:
# if the initial execution fails, execute this one
Easier to Ask for Forgiveness than Permission
(EAFP)
def print_object(some_object):
# Check if the object is printable...
try:
printable = str(some_object)
print(printable)
except TypeError:
print("unprintable object")
CONTEXT
WEB API
UI: web pages, mobile apps
HTTP
Web APIs
- programmatic interface
- one or more publicly exposed endpoints
- defined request-response message system
REST
REpresentational State Transfer
Is an style of software architecture for distributed software
REST
Not only about using GET, POST, PUT, DELETE methods
REST
Six Constraints:
1. Uniform Interface
2. Stateless Interactions
3. Cacheable
4. Client-Server
5. Layered System
6. Code on Demand
UI: web pages, mobile apps
HTTP
RESTful APIs
- programmatic interface
- one or more publicly exposed endpoints
- defined request-response message system
- Implements REST architecture principles
let's cut to the chase
Error handling in RESTful API's
do's and don'ts
don't
Forget/Avoid error handling
class LambdaView(View):
@staticmethod
def get(request, object_id):
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)DON'T! :'(
try / except pass
try:
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except Exception :
passdon't
Pokemon exception handling
don't
Pokemon exception handling: "gotta catch 'em all"
try:
# try to do something
except Exception as e:
# Gotta catch 'em all!
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)
do
class LambdaEditView(View):
@staticmethod
def get(request, object_id):
try:
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except LambdaFunction.DoesNotExist:
return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
except Exception as e:
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)Error handling through multiple exceptions :)
don't
class LambdaEditView(View):
@staticmethod
def get(request, object_id):
try:
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except LambdaFunction.DoesNotExist:
return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
except Exception as e:
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)Error handling ONLY through exceptions
do
class LambdaEditView(View):
@staticmethod
def get(request, object_id):
try:
token = request.GET.get('token')
if not token:
return HttpResponse('Invalid token.', status=status.HTTP_400_BAD_REQUEST)
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except LambdaFunction.DoesNotExist:
return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
except Exception as e:
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)Error handling through exceptions and if's
do
Error handling through exceptions and if's
No golden hammer here
Make sure of understanding the concepts and apply them depending on your particular case.
don't
Use a 200 status code for every response
HTTP/1.1 200 OK
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"success": false,
"data": {
"name": "Not Found Exception",
"message": "The requested resource was not found."
}
}do
HTTP/1.1 404 Not Found
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"name": "Not Found Exception",
"message": "The requested resource was not found."
}don't
Using a 500 status code for all errors
HTTP/1.1 500: Internal server error
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"success": false,
"data": {
"name": "Invalid parameter",
"message": "The given parameter is not valid."
}
}do
HTTP/1.1 400: Bad request
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"success": false,
"data": {
"name": "Invalid parameter",
"message": "The given parameter is not valid."
}
}How do I choose the code?
How do I choose the code?
1xx: Informational - Request received, continuing process
How do I choose the code?
2xx: Success - The action was successfully received, understood, and accepted
How do I choose the code?
3xx: Redirection - Further action must be taken in order to complete the request
How do I choose the code?
4xx: Client Error - The request contains bad syntax or cannot be fulfilled
How do I choose the code?
4xx: Client Error - The request contains bad syntax or cannot be fulfilled
How do I choose the code?
5xx: Server Error - The server failed to fulfill an apparently valid request
What?? that's a lot of codes!!
It's a matter of being practical...
Be practical ;)
2xx Success
200 OK
201 Created
204 No Content
3xx Redirection
304 Not Modified
4xx Client Error
400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found
409 Conflict
5xx Server Error
500 Internal Server Error
Use the most common HTTP status codes
don't
Skip logging
class LambdaEditView(View):
@staticmethod
def get(request, object_id):
try:
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except LambdaFunction.DoesNotExist:
# Where is the log? :(
return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
except Exception as e:
# Where is the log? :(
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)don't
Logging 400-level codes at ‘error’ level
class LambdaEditView(View):
@staticmethod
def get(request, object_id):
try:
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except LambdaFunction.DoesNotExist:
log.error("Lambda with id %s was not found." % object_id)
return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
except Exception as e:
log.error("Unexpected error. Error detail: %s" % str(e))
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)do
Logging 400-level codes at ‘info’ level
class LambdaEditView(View):
@staticmethod
def get(request, object_id):
try:
...
...
return HttpResponse(LambdaFunction.objects.get(id=object_id).name)
except LambdaFunction.DoesNotExist:
log.info("Lambda with id %s was not found." % object_id)
return HttpResponse("Lambda not found.", status=status.HTTP_404_NOT_FOUND)
except Exception as e:
log.error("Unexpected error. Error detail: %s" % str(e))
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)don't
Expose the “guts” to a client: full stack trace, environment variables’ values, etc. Too implementation-specific or “low-level” error messages
File "/home/taiga/.virtualenvs/taiga/lib/python3.4/site-packages/django/db/models/fields/related.py", line 572, in __get__
rel_obj = qs.get()
File "/home/taiga/.virtualenvs/taiga/lib/python3.4/site-packages/django/db/models/query.py", line 357, in get
self.model._meta.object_name)
taiga.projects.userstories.models.DoesNotExist: UserStory matching query does not exist.
do
Use domain/API-specific exceptions with direct, precise, implementation-agnostic error messages written in plain language.
try:
question = Question.objects.get(pk=question_id)
except Question.DoesNotExist:
raise Http404("Question does not exist")do
Use domain/API-specific exceptions with direct, precise, implementation-agnostic error messages written in plain language.
...
...
except forms.ValidationError:
return HttpResponse('Invalid email format', status=status.HTTP_400_BAD_REQUEST)
except Client.DoesNotExist:
return HttpResponse('Client does not exists', status=status.HTTP_400_BAD_REQUEST)
except BadSignature:
return HttpResponse('Invalid token.', status=status.HTTP_400_BAD_REQUEST)
except Exception as e:
return HttpResponse('Sorry, something is broken. We will look into that.',
status=status.HTTP_500_INTERNAL_SERVER_ERROR)do
Handle custom codes
HTTP/1.1 400 Bad Request
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"name": "Invalid username",
"message": "The username is not valid.",
"code": 40001
}HTTP/1.1 400 Bad Request
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"name": "Invalid email",
"message": "The email is not valid.",
"code": 40002
}do
Handle custom codes: twitter
HTTP/1.1 403 Forbidden
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"errors": [
{
"code": 64,
"message": "Your account is suspended and is not permitted to access this feature"
}
]
}do
Handle custom codes: twitter
do
Handle custom codes: facebook
HTTP/1.1 403 Forbidden
Date: Sun, 02 Mar 2014 05:31:43 GMT
Server: Apache/2.2.26 (Unix) DAV/2 PHP/5.4.20 mod_ssl/2.2.26 OpenSSL/0.9.8y
Content-Type: application/json; charset=UTF-8
{
"error": {
"message": "Message describing the error",
"type": "OAuthException",
"code": 190,
"error_subcode": 460,
"error_user_title": "A title",
"error_user_msg": "A message"
"fbtrace_id": "EJplcsCHuLu"
}
}do
Handle custom codes: facebook
do
Handle custom codes: Salesforce marketing cloud
do
Provide a Help URL
[shell]HTTP/1.1 400 Bad Request
{
"type": "error",
"status": 400,
"code": "bad_request",
"context_info": {
"errors": [
{
"reason": "expiration_not_allowed",
"name": "unshared_at",
"message": "Expiration cannot be set on a shared link with 'collaborators' access",
"location": "entity-body"
}
]
},
"help_url": "https://box-content.readme.io/reference#errors",
"message": "Bad Request",
"request_id": "13628284550db80756c518"
}
do
Provide a Help URL: box
do
Provide a Help URL: twilio
Resources
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#eafp-vs-lbyl
https://www.servage.net/blog/2013/04/08/rest-principles-explained/
http://www.slideshare.net/pydanny/python-worst-practices
https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm
http://www.yiiframework.com/doc-2.0/guide-rest-error-handling.html
http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
http://scriptin.github.io/2015-02-08/restful-http-error-handling.html
http://caines.ca/blog/2013/04/21/3-terrible-anti-patterns-for-error-handling-in-rest-apis/
http://www.soa-probe.com/2013/02/layer-7-on-rest-api-error-handling.html
http://www.restapitutorial.com/httpstatuscodes.html
https://realpython.com/blog/python/the-most-diabolical-python-antipattern/
http://www.codingpedia.org/ama/resources-on-how-to-design-error-handling-in-a-rest-api/
https://www.box.com/blog/get-developer-hugs-with-rich-error-handling-in-your-api/
http://blog.cloud-elements.com/error-handling-restful-api-design-part-iii
https://developers.google.com/drive/v3/web/handle-errors
https://www.jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/
http://tutorials.jenkov.com/java-exception-handling/validation-throw-exception-or-return-false.html
http://www.wiredmonk.me/error-handling-and-logging-in-flask-restful.html
http://apigee.com/about/blog/technology/restful-api-design-what-about-errors
Social
Thanks!
Q & A
Error handling in RESTful API's Mon 30 May 2016 - PyCon US 2016 Open Spaces
Error handling in RESTful API's
By Jhon Jairo Roa
Error handling in RESTful API's
Set of do's and don'ts when dealing with errors in RESTful API's
- 2,883
