Web Application and DevOps

Kracekumar - @kracetheking

Chapters

  • How does a web application work?
  • Different components of a web application
  • Managing the server
  • Why DevOps?

1. How does a web application work?

What is a web application?

The web application is a client-server software in which client runs a web browser.

Examples

How does a web browser communicate?

1. The web browser requests a page using HTTP.

2. The web server responds HTML response.

krace@hotbox /m/u/c/t/src> curl -v http://httpbin.org/html
* Hostname was NOT found in DNS cache
*   Trying 54.225.186.255...
* Connected to httpbin.org (54.225.186.255) port 80 (#0)
> GET /html HTTP/1.1
> User-Agent: curl/7.35.0
> Host: httpbin.org
> Accept: */*
> 
< HTTP/1.1 200 OK
< Connection: keep-alive
* Server gunicorn/19.7.0 is not blacklisted
< Server: gunicorn/19.7.0
< Date: Mon, 20 Mar 2017 19:03:54 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 3741
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< Via: 1.1 vegur
< 
<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
      <h1>Herman Melville - Moby-Dick</h1>

      <div>
        <p>
          Availing himself of the mild, summer-cool weather that now reigned in these latitudes, and in preparation for the peculiarly active pursuits shortly to be anticipated, Perth, the begrimed, blistered old blacksmith, had not removed his portable forge to the hold again, after concluding his contributory work for Ahab's leg, but still retained it on deck, fast lashed to ringbolts by the foremast; being now almost incessantly invoked by the headsmen, and harpooneers, and bowsmen to do some little job for them; altering, or repairing, or new shaping their various weapons and boat furniture. Often he would be surrounded by an eager circle, all waiting to be served; holding boat-spades, pike-heads, harpoons, and lances, and jealously watching his every sooty movement, as he toiled. Nevertheless, this old man's was a patient hammer wielded by a patient arm. No murmur, no impatience, no petulance did come from him. Silent, slow, and solemn; bowing over still further his chronically broken back, he toiled away, as if toil were life itself, and the heavy beating of his hammer the heavy beating of his heart. And so it was.—Most miserable! A peculiar walk in this old man, a certain slight but painful appearing yawing in his gait, had at an early period of the voyage excited the curiosity of the mariners. And to the importunity of their persisted questionings he had finally given in; and so it came to pass that every one now knew the shameful story of his wretched fate. Belated, and not innocently, one bitter winter's midnight, on the road running between two country towns, the blacksmith half-stupidly felt the deadly numbness stealing over him, and sought refuge in a leaning, dilapidated barn. The issue was, the loss of the extremities of both feet. Out of this revelation, part by part, at last came out the four acts of the gladness, and the one long, and as yet uncatastrophied fifth act of the grief of his life's drama. He was an old man, who, at the age of nearly sixty, had postponedly encountered that thing in sorrow's technicals called ruin. He had been an artisan of famed excellence, and with plenty to do; owned a house and garden; embraced a youthful, daughter-like, loving wife, and three blithe, ruddy children; every Sunday went to a cheerful-looking church, planted in a grove. But one night, under cover of darkness, and further concealed in a most cunning disguisement, a desperate burglar slid into his happy home, and robbed them all of everything. And darker yet to tell, the blacksmith himself did ignorantly conduct this burglar into his family's heart. It was the Bottle Conjuror! Upon the opening of that fatal cork, forth flew the fiend, and shrivelled up his home. Now, for prudent, most wise, and economic reasons, the blacksmith's shop was in the basement of his dwelling, but with a separate entrance to it; so that always had the young and loving healthy wife listened with no unhappy nervousness, but with vigorous pleasure, to the stout ringing of her young-armed old husband's hammer; whose reverberations, muffled by passing through the floors and walls, came up to her, not unsweetly, in her nursery; and so, to stout Labor's iron lullaby, the blacksmith's infants were rocked to slumber. Oh, woe on woe! Oh, Death, why canst thou not sometimes be timely? Hadst thou taken this old blacksmith to thyself ere his full ruin came upon him, then had the young widow had a delicious grief, and her orphans a truly venerable, legendary sire to dream of in their after years; and all of them a care-killing competency.
        </p>
      </div>
  </body>
* Connection #0 to host httpbin.org left intact
</html>⏎

How does web server handle a web request?

2. Components

Demo App

Input

Input

Detail Page

Comments

Components

  • Web Application - Python/Django
  • Database - Postgres
  • Queue - RabbitMQ
  • Worker - Celery

Source Code

@login_required
@require_http_methods(['GET', 'POST'])
def create_proposal(request, conference_slug):
    conference = get_object_or_404(Conference, slug=conference_slug)
    if request.method == 'GET':
        return handle_proposal_get_request(conference, request)

    # POST Workflow
    form = ProposalForm(conference, data=request.POST, action="create")

    if not form.is_valid():
        return render(request, 'proposals/create.html',
                      {'form': form,
                       'conference': conference,
                       'errors': form.errors})

    # Valid Form - Store to DB
    proposal = Proposal.objects.create(
        author=request.user,
        conference=conference,
        title=form.cleaned_data['title'],
        description=form.cleaned_data['description'],
        ...)
    host = '{}://{}'.format(settings.SITE_PROTOCOL, request.META.get('HTTP_HOST'))
    send_mail_for_new_proposal(proposal, host)

    return HttpResponseRedirect(reverse('proposal-detail',
                                        args=[conference.slug, proposal.slug]))

3. Installation

 

Server Setup

  • Web Server - Nginx
  • WSGI Accelerator - uWSGI
  • Web App - Django + some extra dependencies
  • Database - Postgres
  • Queue - RabbitMQ
  • Worker - Celery

Dependencies

  • System dependencies
  • Application dependencies

System Dependencies

Dependencies installed from OS repository.

E.g: http://packages.ubuntu.com/

sudo apt-get install <package>

Example

  • Nginx
  • uWSGI
  • Postgres server
  • Postgres client
  • libpq-dev
  • git
  • python
  • python-dev
  • lib*
  • ntp
  • python-pip

What is app dependencies?

  1. Dependencies required for running web application.
  2. Installed from language specific repositories. E.g: pypi.python.org
  3. pip install Django==1.8.1

Example

Bridge

  • Some maintainer publishes their package to both language and OS specific repositories. Eg: Django, uWSGI
  • Some packages require system dependencies. Eg: Postgres python driver depends on `psycopg2`.
  • It's advised first to install system dependencies.

Deployment

  • SSH into the machine
  • Pull the latest code
  • Make DB changes
  • Restart all the services like uWSGI, Nginx

Growth!

What was started as a hobby project, now became a full-time business! After constant effort in six months, fifty conferences uses the web app. What a feat! From one developer the team grew to five developers. Developers are churning features every day and deploying many times daily. The sys admin wants to spin up multiple machines. More work for sys admin :-(

What are the problems now?

  • Sys admin has to spin up new machines manually.
  • Setting up a new machine can take a significant amount of time.
  • Deploying serially to each machine is painful.
  • Upgrading/Patching security updates are hard.
  • Manual setup error 

Solution

4. DevOps

What is DevOps?

DevOps is a combination of software development and operations - Patrick Debois

Advantages of DevOps

  • Continuous Delivery
  • Ease of management
  • Faster resolution for problems
  • Happy team :-)

What is continuous delivery?

  • Build Software in short cycle
  • Build, Test, Feedback, Fix, Test, Deploy

Ease of management

  • Deploy to all machines with single command 
  • Spin up new machine in few minutes with the latest code
  • Spin down new machine at any time

Faster resolution

  • Buggy code can be rolled back instantaneously
  • Faulty server can be removed in no time with replacement

Configuration Management



sudo salt 'web-app-*' state.highstate

Example setup

base:
  '*':
    - base
  'web-app-*': 
    - postgresql
    - nginx
    - lib/uwsgi_common
    - junction

base

base_installs:
  pkg.installed:
    - names:
      - git
      - htop
      - curl
      - zsh
      - vim-nox

nginx

nginx:
  pkgrepo.managed:
    - ppa: nginx/stable
  pkg.installed:
    - refresh: True
  service.running:
    - require:
      - pkg:
          nginx
    - reload: True
    - watch:
      - file: /etc/nginx/nginx.conf
      - file: /etc/nginx/sites-available/*
      - file: /etc/nginx/sites-enabled/*

nginx_config_folders:
  file.directory:
    - names:
        - /etc/nginx/sites-enabled/
        - /etc/nginx/sites-available/
        - /etc/nginx/sites-available/upstreams
    - require:
        - pkg: nginx

Q&A

Why DevOps?

By Kracekumar

Made with Slides.com

Why DevOps?

The talk covers different components of the web application, how they interact, what are their dependencies, what happens in production and why you need to automate.

  • 1,178

More from Kracekumar