Self-contained Java services

Johnathan Gilday
Next Century Corporation

2016-07-14

Some motivation

• At Next Century, we build a lot of services for our customers, often in Java
• Our services are getting trimmer and more plentiful
• Our development teams continue to become more involved with the infrastructure management and production operations

We need to optimize the way we build and deploy these services. Abandoning the shared application container is one way

What do I mean by container?

• Also called "application server"
• One process
• Manages one or more archives (WAR, EAR)
• Examples: Apache Tomcat, Wildfly, Glassfish

To deploy application archives to running containers:

1. Install container
2. Build WAR
3. Push WAR to container

Containers Manage your app's lifecycle

@WebListener
public class AppInitializer implements ServletContextListener {
    
    @Override
    public void contextInitialized(final ServletContextEvent sce) {
        // container calls this method to initialize app
    }

    @Override
    public void contextDestroyed(final ServletContextEvent sce) {
        // container calls this method to shutdown app
    }
}

@WebServlet(name = "my-servlet", urlPatterns = "/my-servlet")
public class MyServlet extends HttpServlet {

    @Override
    public void doGet(final HttpServletRequest req, final HttpServletResponse resp) {
        // container calls this method on incoming request
    }
}

When are containers helpful?

Development Team

• Delivers a packaged application with dependencies outside of those provided by the container

• Delivers a configuration files and documentation for running the service

Operations Team

• Configures the operating system
• Configures the container
• May run multiple applications on the container

Containers work best when there is a clear separation of responsibilities between teams developing and maintaining the application

What if there's just one team?

Discrepancies between application and container cause headaches

• Logging
• Configuration
• Dependencies (classpath)
• Test environments (mvn jetty:run vs container)

Environment discrepancies cause headaches

• Different versions of dependencies on the classpath
• Example: container provides a different version of servlet than the one used in testing
• Containers use varying strategies for classpath loading

Proposed Solution:

Deliver self-contained applications

Self-contained apps

• Live in their own process
• Embed their own web server
• Embed all their own dependencies
• Define their own means of configuration

What does developing a self-contained app look like?

Runs in its own process

/**
 * Point of entry. Configure and start app
 */
public class App {

    public static void main(final String[] args) {
        // configure application
        // register signal handlers if desired
        // run web server until app terminates
    }
}

• Embraces Unix process model: use of environment variables, command-line args, signals, STDOUT, STDERR
• Easy to run
• Development and Production parity

Embeds its own server

public static void main(final String[] args) {
    // start jetty
    logger.info("listening on port {}", port);
    final ResoureConfig rc = ResourceConfig.forApplication(app);
    final Server server = JettyHttpContainerFactory.createServer(baseUri, rc);
    try {
        server.start();
        server.join();
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

• "Don’t deploy your application in Jetty, deploy Jetty in your application!"
• Integrates well with Jersey via JettyHttpContainerFactory
• Jetty is not the only solution, but it’s mature and lightweight

Bundles its dependencies

$ gradle shadowJar


$ java -jar build/libs/my-app.jar
listening on port 8000

• Sometimes called "fat jar", "uber jar", or "shadow jar"
• Prefer all dependencies bundled together: we don't want dependencies to change after building
• Easier than using shell scripts to build up a classpath argument for java
• Include main class in jar manifest

Logs to STDOUT

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%-5level %logger{5} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>

• Easy debugging
• One stream of log statements with flexible downstream processing
• Let production environment worry about filtering, rotation

What does running a self-contained app look like?

systemd managed daemon

[Unit]
Description=my-app service

[Service]
EnvironmentFile=-/etc/sysconfig/my-app
ExecStart=/usr/bin/java \$JVM_OPTS -jar /opt/my-app/my-app-${version}-all.jar
User=my-app
Restart=on-failure

[Install]
WantedBy=multi-user.target

• Much easier than init scripts
• systemctl for start, stop, status, restart
• Automatic start on system boot
• Note: same command to start as used in development

don't forget journald

[vagrant@localhost ~]$ journalctl -fu how-to-microservice
-- Logs begin at Thu 2016-07-14 00:07:10 UTC. --
Jul 14 00:08:44 localhost.localdomain systemd[1]: Started how-to-microservice service.
Jul 14 00:08:44 localhost.localdomain systemd[1]: Starting how-to-microservice service...
Jul 14 00:08:44 localhost.localdomain java[11726]: 2016-07-14T00:08:44,718Z INFO  c.j.App - listening on port 8000

• Captures STDOUT, STDERR from systemd managed daemons
• Easy to tail and grep your app's logs
• Highly configurable: log rotation, format, forwarding

What does deploying a self-contained app look like?

RPM

$ rpm -qlp build/distributions/how-to-microservice-0.0.4-1.e7.noarch.rpm
/etc/how-to-microservice
/etc/how-to-microservice/settings.conf
/etc/sysconfig
/etc/sysconfig/how-to-microservice
/etc/systemd
/etc/systemd/system
/etc/systemd/system/how-to-microservice.service
/opt/how-to-microservice
/opt/how-to-microservice/how-to-microservice-0.0.4-SNAPSHOT-all.jar
/opt/how-to-microservice/settings.conf

• Packages user install, config file template, systemd unit file, binaries, package dependencies (java)
• Host your own yum repository for easy updates (jenkins, Nexus)
• Establishes a common means for deploying your container-less apps

Build RPM with Gradle

task rpm(type: Rpm) {
    it.dependsOn shadowJar

    packageName = project.name
    version = '0.0.4'
    release = '1.e7'
    os = LINUX

    requires('java-1.8.0-openjdk')

    ...

• Netflix OSS nebula-ospackage-plugin gradle plugin
• Builds deb and rpm
• Host your own yum repository with Nexus, deploy with Gradle

Test RPM with Vagrant

Vagrant.configure(2) do |config|
  config.vm.box = "geerlingguy/centos7"
  ...
  config.vm.provision "shell", inline: <<-SHELL
    sync_dir=/vagrant
    rpm=($sync_dir/build/distributions/*.rpm)
    if [ ! -f $rpm ]; then
      echo "how-to-microservice RPM not found"
      exit 1
    fi
    sudo yum erase -y how-to-microservice
    sudo yum install -y $rpm
    sudo systemctl restart how-to-microservice
  SHELL
end

• Vagrant defines development virtual machines with Ruby DSL
• Vagrant shell provisioner installs RPM in the Vagrant virtual machine
• Rebuild virtual machine:

  vagrant destroy -f && vagrant up

What does dockerizing a self-contained app look like?

Dockerfile

FROM java:8
MAINTAINER Johnathan Gilday

COPY ./build/libs/how-to-microservice.jar /opt/how-to-microservice/
EXPOSE 8000
WORKDIR /opt/how-to-microservice

CMD ["java", "-jar", "how-to-microservice.jar"]

• Manage a docker container instead of a systemd service
• Manage logs with docker log driver instead of journald
• Deploy a docker image instead of an RPM

Thanks!

Sample self-contained jersey service
https://github.com/gilday/how-to-microservice

Gradle plugin for building "fat jars"
https://github.com/johnrengelman/shadow

Netflix OSS Gradle plugins (including gradle-os-package)
https://nebula-plugins.github.io/

Nexus yum repository hosting
https://books.sonatype.com/nexus-book/reference/yum-configuration.html

Vagrant shell provisioner
https://www.vagrantup.com/docs/provisioning/shell.html