How to Get Started with Apache Mesos, Marathon, and ZooKeeper

Cloud Computing

One of the larger problems companies have when deciding to Dockerize their applications is that they often continue to think of managing their compute, memory, and CPU resources in a traditional Hypervisor-centric model.

The benefits of Docker in production can be more fully realized when the speed of infrastructure provisioning matches or exceeds the speed of application development, application containerization, and application release. In short, coupling the rapid application spin-up time of a Docker container or cluster of Docker services should be easily abstracted through a similarly rapid and simple datacenter interaction.

Imagine if you could program against your entire datacenter as if it were a single pool of resources…

Apache Mesos

…Enter Apache Mesos, a distributed systems kernel with built-in support for Docker containers, an HTTP API, a Web UI, true high availability, and fault tolerance that is designed natively for the cloud.

Apache Mesos

Following along with the Advanced Mesos Tutorial provided by Mesos, let’s experiment with a simple Vagrant VM running Mesos.

First Things First

To begin, you’ll want to Install Vagrant and VirtualBox. For ease of starting, you may download my simple GitHub starter repo. Vagrant will use the ansible_local provider to install Ansible on the guest VM and configure it with the necessary packages to run Mesos (PR’s accepted for those Ansibulls out there with better ideas for the playbook). Once you’ve cloned the repository, you can CD into it and run: $ vagrant up

Per the Mesos instructions, our Vagrantfile and Ansible playbook.yml will execute tasks which perform the following:

  • Install a simple Centos-7.1 Vagrant VM

Install vagrant

  • Configure your /etc/hosts file to have the correct hosts

Correct Hosts 1 Correct Hosts 2

  • Add the Mesosphere RPM repository

Mesosphere RPM repository

  • Install the Mesos and Marathon RPMs

Mesos and Marathon RPMs

Remember, you can use vagrant halt, vagrant destroy, and vagrant up to rebuild the VM at will.

Continuing on, the Ansible playbook installs Zookeeper, which Apache Mesos uses as the master election service.

  1. Install the necessary ZooKeeper RPMs & install it
  2. Initialize the zookeeper-server to use ID of 1
  3. Start ZooKeeper

Shell Install ZooKeeper

Next, our Ansible playbook will start the Mesos Master and Slave processes as well as starting Marathon.

Ansible playbook

Let’s run $ vagrant ssh to get into our Vagrant VM and test out a few things. $ vagrant ssh $ sudo netstat -nlp | grep mesos

Vagrant

Now, you should be able to access the Mesos user interface with your browser at 192.168.33.10:5050 Mesos UI

You can test the mesos-execute command via the following: $ export MASTER=$(mesos-resolve cat /etc/mesos/zk 2>/dev/null) $ mesos help

Mesos Execute

Using Marathon

Marathon is a production-proven Apache Mesos framework for container orchestration, providing a REST API for starting, stopping, and scaling applications. Written in Scala, Marathon can run in highly-available mode by running multiple copies. The state of running tasks gets stored in the Mesos state abstraction.

Marathon is a meta framework, meaning you can start other Mesos frameworks such as Chronos or Storm with it to ensure they survive machine failures. It can launch anything that can be launched in a standard shell. In fact, you can even start another Marathon instances via Marathon.

Mesos can launch anything that can be launched in a standard shell.

Analogous to running $ sudo service marathon start, we’ll use Ansible’s service module to enable and start marathon. If you’ve ran $ vagrant up, this task has already been completed for you.

Visit the Marathon GUI by clicking 192.168.33.10:8080/ Marathon GUI

Returning to the Vagrant VM CLI, you can test installing an application that will be run by Marathon.

$ python -m SimpleHTTPServer

Now visit 192.168.33.10:8000 and you should see: Directory

Feel free to CTRL+Z this process and $ exit out of the Vagrant VM.

Marathon GUI

Now, let’s start using the Marathon GUI located at: 192.168.33.10:8080/ui/#/apps

Click on Create Application:

Create Application

Here, we’re presented with a window in which we can name and configure an application to run in Marathon. In this case, we’ll use the same simple Python webserver:

For name: mypythonserver

For command: python -m SimpleHTTPServer $PORT

The $PORT ENV variable will dynamically assign this to a port.

Assigned to Port

Click Create Application and we’re off. We now see the status go from deploying to running:

Deploying to Running

We can discover details about this instance running and find the port to access it from a web browser:

mypythonserver mps running

In my case, the random port assigned is 31377, which I can access in a browser:

Port Access

We can now test the scaling ability with Marathon. I’ve set my instance configurations for mypythonserver to be 0.1 for CPU and 32 for Memory. Click on Scale Application, select 4 as the number of instances to scale to, and watch Marathon go to work.

Marathon Scaling Scale Application

This is just the tip of the iceberg for using Mesos to automate your datacenter. We’ll move onto more complex deployments in the next installment. Until then, happy hacking!

James Denman

James Denman

VP of Technology

James Denman is the VP of Technology at Levvel and is located in Sydney, Australia. He has over 10 years of experience working in the Defense, Diplomatic, and NGO technology sectors across more than 12 countries.