Docker-based Apps with Persistent Storage

Use the Apcera Platform to create and run apps from Docker images that leverage the Apcera Platform’s NFSv4 support for persistent storage even when the app is not running.

In this tutorial I’ll show an example of how to create an application for the Apcera Platform from a Docker image, and then how to extend the utility of that app by enhancing it with NFSv4 compliant persistent storage. Support for NFSv4 compliant storage comes from a new feature in the platform, the Apcera file system, or APCFS.

Requirements:

  • Apcera Platform is installed and running (latest version).

Note: To complete the final step in this tutorial you will need to have an Apcera Platform cluster with at least 2 Instance Managers.

1: Create an app from a Docker image.

You can easily run a Docker Image as an app in the Apcera Platform. Several popular Docker images are provided as graphical tiles for quick selection, but you can really can specify any Docker image by specifying the image details in the form fields. We’ll use the official Ubuntu image for this tutorial.

Start the Web Console using the URL provided when you installed Apcera Platform. Login as the admin user.

Click on the Launch button in the left nav panel and the select Docker Image.

Tutorial Image

Click on the Ubuntu Linux tile.

Tutorial Image

Configure the app:

  • Name: ubuntu-server
  • Namespace: <use the default>
  • Click Submit

Tutorial Image

Your ubuntu-server app is created from the Ubuntu Linux Docker image and deployed to the Apcera Platform.

Tutorial Image

To run any Docker image in Apcera, at a minimum you must have in your cluster the package file package::/apcera/pkg/os::ubuntu-14.04-apc3, which satisfies the “os.linux” dependency required for a container. If you have not bootstrapped your CE cluster or imported this base package to your EE cluster, you receive the following error:

Tutorial Image

2: Explore the Ubuntu environment.

Next we verify that the app is running by connecting to the Ubuntu Linux environment.

Connect to a secure session in the running Ubuntu Linux job using the APC command line. Switch to a terminal window and enter the apc job connect command.

apc job connect ubuntu-server

The prompt changes to reflect that you’re now in the Ubuntu Linux environment

Tutorial Image

Create a simple text file within the file system of the Ubuntu environment with the following commands:

mkdir /home/mystuff
cd /home/mystuff
vi somefile.txt

Put the vi editor into Insert mode by pressing the “i” key, then type in a line of text, such as:

This is some text data.

Tutorial Image

Press the “esc” key to stop insert mode and enter “:wq” to write your changes to the file and quit the vi editor. You should be back at the OS prompt in the Ubuntu environment.

You can output your file with the cat command:

cat somefile.txt

Tutorial Image

By design containers are ephemeral and their file systems are not persistent. If you restart the job, you lose the data. To demonstrate this, return to the Web Console and restart the job.

Tutorial Image

Once the ubuntu-server job is fully restarted, see if the text file is still there by connecting to the job and looking for the file where we created it in the file system. Return to the terminal window and enter the following:

apc job connect ubuntu-server
cat /home/mystuff/somefile.txt

The file was lost (and the subdirectory it was created in) when the Ubuntu Linux job was re-started. This demonstrates the ephemeral nature of containers and that by default there is no persistent storage for them. If the container requires persistence, you must enable it as described next.

Tutorial Image

3. Create an APCFS service and add it to the Ubuntu Linux job.

The Apcera Platform provides you with the Apcera File System (APCFS) which is a NFSv4-compliant persistent file system. In this part of the tutorial we will create a APCFS service and bind the Ubuntu Linux job to it for persistent storage.

Return to the Web Console in the Browser. Click on Services in the left nav panel. Click on the Create Service button.

Tutorial Image

Configure the service by entering the following values in the fields:

  • Name: ubuntu-docker-apcfs-service
  • Namespace: <use the default>
  • Type: nfs
  • Provider FQN: provider::/apcera/providers::apcfs

Click Submit

The new service is now in the list of services available on the cluster.

Tutorial Image

Now we can add this service to our Ubuntu Linux job. First we need to stop the job.

Select the ubuntu-server job from the Apps page.

Now stop the job and click Yes to confirm.

Tutorial Image

Now go to the NETWORKING tab and click on Add Service Binding.

From the selection list choose the service you created: service::/sandbox/admin::ubuntu-docker-apcfs-service.

Enter a suitable name for the binding, such as ubuntu-apcfs-service-binding.

Click Submit.

Tutorial Image

Click the Start button to start the job.

When the job is started, return to the terminal window and connect to it again in a secure session:

apc job connect ubuntu-server

The APCFS service provides persistent storage mounted at the file system directory /nfs/data.

Change to that directory in the container:

cd /nfs/data

As you did previously, use vi to create a simple text file:

vi somefile.txt

Put the vi editor into Insert mode by pressing the “i” key, then type in a line of text, such as:

This is some persisted data.

Press the “esc” key to stop insert mode and enter “:wq” to write your changes to the file and quit the vi editor. You should be back at the OS prompt in the Ubuntu environment.

You can output your file with the cat command:

cat somefile.txt

Tutorial Image

Restart the ubuntu-server job again by switching back to the Web Console in the browser. Click on the Apps link in the left nav panel, then click on the ubuntu-server job, and then click on Restart at the top right.

Tutorial Image

Once the ubuntu-server job has restarted, connect again to the job and verify that the text data file is still available.

apc job connect ubuntu-server
cat /nfs/data/somefile.txt

The text file is still there. This demonstrates how you can easily create persistent storage for your Docker jobs.

Tutorial Image

4. Observe Data Persistence with Workload Mobility.

In this final step of the tutorial we’ll see how persistent storage for our data is maintained even when the job is moved to run on a different host.

Return to the Web Console in the Browser. Click on Apps in the left nav panel. Click on the ubuntu-server job in the list, and then click on the SCHEDULING tab. Go to the Instances section and notice which Instance Manager the ubuntu-server job is running on.

Tutorial Image

Click on the Cluster link in the left nav panel and click on the INSTANCE MANAGERS tab

Each Instance Manager (IM) is a host in the cluster where jobs can run.

Click on the link for IM where the ubuntu-server job is running, and then click on the INSTANCES tab. Confirm that the ubuntu-server job is in the list of jobs running on that IM.

Tutorial Image

Now click your browser’s Back button two times to return to the INSTANCE MANAGERS tab of the Cluster page. Now click on the link for the IM that the ubuntu-server is not running on.

Tutorial Image

Click on the INSTANCES tab to verify that the ubuntu-server job is not running on this IM.

Tutorial Image

Click on the DETAILS tab and go to the Info section at the Tags label. Copy the host tag for this IM to the clipboard.

Tutorial Image

Move the ubuntu-server job to the other IM (the one it is not running on) by creating an Instance Manager Tag for the job requiring it to run on the other IM.

Click on Apps in the left nav panel, then click on the ubuntu-server job, and then click on the SCHEDULING tab.

Click on the Add Tag button. Paste the host tag you copied previously into the Target field and click on hard for the tag’s Type. Then click on Submit.

You are prompted to restart the job. Click on Yes, restart now.

Tutorial Image

When the job has fully restarted, notice in the Instance section that it is now running on the IM of the other host.

Again, return to the terminal window and connect to it again in a secure session

apc job connect ubuntu-server

Output the file you previously created:

cat /nfs/data/somefile.txt

The file is still accessible even though the ubuntu-server job has been restarted on a different host!

Tutorial Image

This concludes this tutorial. To recap, we started by showing how easy it is create an app from a Docker image and get it running as a job in the cluster, using the Web Console.

Then we showed how easy it is to add persistent storage to the docker app using the APCFS service provided by the platform.

Finally the job was moved from one Instance Manager to another, ie from one host to another host, within a cluster on a local machine, maintaining the connection to its persistent storage.

Of course the Instance Manager could have just as easily been in a cloud environment like AWS or in a data center with OpenStack. You can set up such hybrid clouds or multi-clouds with the Apcera Platform Enterprise Edition.