Application Staging

This section describes all aspects and components of the Apcera staging process, including:

Staging overview

You use staging to initialize, configure, and supply arguments in a uniform and repeatable way to executable components as they are ingested into your Apcera cluster. Common staging tasks include compiling source code, running unit and system tests, scanning for viruses, minifying stylesheets and Javascript, and so forth.

screenshot

Apcera provides system-level jobs for staging your applications, and you can write your own using the Stager API.

Stagers

A stager is an Apcera job that runs automatically when you upload a new package as part of an app, Docker image, or service (for example, using apc app create). Once staged, a package is ready to be used. Apps and services in Apcera can use only staged packages. A staged package is a bundle of source, dependencies, language runtime, and compiled/generated staging output, ready for execution.

A stager runs within another type of job known as the staging pipeline.

About permissions on staged files

When you run apc app create (or apc app deploy), the source files are TAR'd and uploaded, and file permissions are left as-is. Most stagers have logic to "standardize" permissions on uploaded files. Other stagers, such as the PHP stager, does not have such logic. In this case, if you have strict file permissions set on the source, the same code/files may have slightly different staging outcomes, and you will need to change the permissions to be read-friendly.

Staging pipelines

A staging pipeline is an Apcera job that encapsulates a named and ordered collection of stagers. Each staging pipeline comprises a least one stager. Apcera provides several staging pipelines, and you can create your own.

screenshot

Creating or deploying any job in Apcera requires the use of a staging pipeline. The result of the staging process is an executable job or a package that provides some dependency for jobs. If you do not specify a staging pipeline, Apcera uses information in the current working directory to infer which staging pipeline to use. The staging coordinator is a background job that performs package resolution.

A stager can appear in any number of staging pipelines. For example, a stager that performs virus scanning might be used in all staging pipelines in a cluster. Each stager consists of least one package and a job definition. It is possible to build a stager in any language or framework supported by the cluster, and each stager in a staging pipeline can be written in a different language.

The following table lists and describes each staging pipeline provided by Apcera:

Staging Pipeline Description
bash Bash scripts, most commonly used for creating new stagers
compiler Package build scripts, most commonly used for creating new packages
go Go applications
java Java applications
nodejs Node.js applications
perl Perl scripts
php PHP applications
python Python applications
ruby Ruby and Ruby on Rails applications
static-site Static websites

Automatic staging pipeline detection

You do not need to specify a staging pipeline explicitly in an apc app create command. By looking in the working directory in which you are running APC, Apcera can automatically determine the staging pipeline to run for apps written in a variety of languages and frameworks. This section describes how Apcera automatically detects, and creates stagers for, apps written in Ruby, Java, Go, Python, Node.js, PHP, Perl, Bash, and static websites.

To perform automatic staging pipeline detection, Apcera maintains and implements a set of file type detection rules. The APC client retrieves the detection rules from the API Server and then checks the data in the working directory against the detection rules. If there is a match, the system applies the detected stager to create the runtime package for the app. If the system does not detect the source code, the APC client reports that there is no stager available, in which case you need to specify the stager as described below.

Detection rules

Runtime Pattern Stager  
bash "bash_start.sh", "bash_setup.sh" Bash stager  
go "*.go" Go stager  
java "pom.xml", ".jar", ".war", "*.java" Java stager  
nodejs "package.json", "node_modules", "*.js" Nodejs stager  
perl "Makefile.PL", "Build.PL", "META.yml", "META.json" Perl stager  
php "*.php", "php.ini" php PHP stager
python "*.py","runtime.txt","manage.py" Python stager  
ruby "*.rb", "Gemfile", "Gemfile.lock" Ruby stager  
static-site "index.html" Static-site stager  

Overriding automatic staging pipeline detection

In some cases the automatically detected staging pipeline might not be the desired one. If your app satisfies the requirements for more than one application type, you should specify the desired staging pipeline explicitly using the --staging option when creating or redeploying your application. For example, the following command explicitly uses the static-site stager when deploying the my-website app.

apc app create my-website --staging /apcera::static-site

Ruby staging pipeline

The Ruby staging pipeline is automatically invoked when the working directory contains a file of one of the following types:

  • *.rb
  • Gemfile
  • Gemfile.lock

If an app has no dependencies, it should include an empty Gemfile.

Java staging pipeline

The Java staging pipeline is automatically invoked for several types of Java applications. The supported applications types, and the rules used to determine the correct application type are shown below, in the order that they are applied:

  1. Maven — A pom.xml file is present in the app deploy folder.

    The Java staging pipeline supports webapps with and without WAR packaging.

  2. WAR — A *.war file is present in the app deploy folder.

    Deployments are limited to a single WAR file which is deployed with Tomcat.

  3. Stand-alone Java app — A *.java file is present in the app deploy folder.

    • The $JAVA_OPTS environment variable is supported.
    • Apcera uses the Tomcat app server when a WAR file is present.

  4. JAR — A single *.jar file is present in the app deploy folder; if any Java source files are found, then the application will be staged as a stand-alone Java app (#3).

    • Deployments are limited to a single JAR file.
    • Custom flags can be passed to the JAR by setting the JAVA_FLAGS environment variable on the job. The JAVA_OPTS environment variables is also supported to set memory and other JVM settings.

Bash staging pipeline

The Bash staging pipeline is automatically invoked when the working directory contains either bash_setup.sh or bash_start.sh.

  • The bash_start.sh script is required and is automatically executed when you start your application via apc app start.
  • The bash_setup.sh file is optional and is executed during the application staging process. It provides an opportunity to perform any setup tasks required by bash_start.sh.

See the example-bash application for an example of using these scripts.

Go staging pipeline

The Go staging pipeline is automatically invoked when the working application source directory contains a *.go file.

Go is a compiled language, so the staging pipeline has to ensure that everything that requires source code has access to it, either arranging the order of tasks appropriately or ensuring that the source code remains available after the compilation phase. The built-in Go stager does not provide source code to subsequent stagers in the staging pipeline. Setting the RUN_TESTS environment variable to true tells the Go stager to run tests. The Go stager then ensures that everything the tests need is available when they run.

By default, the Go stager deletes your application's source files once the application is successfully compiled. You can optionally direct the stager to keep source files after compilation by passing KEEP_SOURCE=true as a package environment variable.

apc app create gowebserver --pkg-env 'KEEP_SOURCE=true'

This would be necessary, for example, if your Go application is a web server that serves up HTML and other resource files included in the source code.

Node.js staging pipeline

The Node.js staging pipeline is automatically invoked when the working application source directory contains one of the following:

  • package.json file
  • node_modules folder
  • *.js file

Perl staging pipeline

The Perl staging pipeline is automatically invoked when the working application source directory contains one of the following file types:

  • Makefile.PL
  • Build.PL
  • META.yml
  • META.json

PHP staging pipeline

The PHP staging pipeline is automatically invoked when the working application source directory contains both of the following file types:

  • *.php
  • php.ini

You can specify the required webserver (php-apache or php-fpm-nginx) in a runtime.txt file. Apache is the default web server. An empty runtime.txt file specifies Apache.

Python staging pipeline

The Python staging pipeline is automatically invoked when the working application source directory contains one of the following file types:

  • *.py
  • runtime.txt
  • manage.py

The runtime.txt file msut be formatted like python-2.7 or python-2.7.2.

An optional requirements.txt can be provided to install additional Python software packages.

  • Basic — requirements.txt does not contain django, gunicorn, or requirements.txt is missing

    Default behavior is to run start.py in the working directory

  • Gunicorn - requirements.txt contains gunicorn, but not django

    Default behavior is to run wsgi.py in the working directory

  • Django - requirements.txt contains django, but not gunicorn

    Default behavior is to run manage.py in the working directory

  • Gunicorn and Django - requirements.txt contains gunicorn and django

    Default behavior is to run manage.py in the working directory

Static website staging pipeline

The static-site staging pipeline creates an application consisting of your web site's source files (HTML, JavaScript, CSS, etc.) and an Nginx instance to serve them. You can optionally provide a custom nginx.conf file to configure the Nginx instance.

The static-site staging pipeline is automatically invoked when your application source directory contains either an index.html or index.htm file. It takes the following actions:

  • Extracts web site files into the application package's /app/www directory.
  • Adds the NGINX package as a dependency.
  • Copies a Bash script used to start Nginx to the /app directory
  • Adds the following environment variables to the application package:
    • START_COMMAND=./start_nginx
    • START_PATH=/app

See Tutorial: Deploying a Static Web Site for an example of using the static site staging pipeline.

A note on start commands: When creating apps with staging pipelines, the start command will be issued from the runner user unless specifically overridden. This user owns the /app directory in the container, but otherwise has limited permissions. If necessary, the runner user can invoke sudo without a password. Keep this in mind when defining your start commands.

Providing a custom nginx.conf

The static-site staging pipeline includes a default nginx.conf file that configures your site's Nginx instance. If the root of your application source directory contains an nginx.conf file, the static-site stager will use that file instead. This lets you, for example, define a custom 404 page or set up redirects.

For information about the Nginx configuration file see the Nginx documentation.

APC commands for staging

The -h option on an APC command displays documentation rather than executing the command. For example, apc stager create -h produces a list of options and explains what they do. It also provides examples that you can copy and paste to get the syntax right.

Stager commands

The APC commands for individual stagers have the following form:

apc stager <subcommand> [optional args]
Subcommand Purpose
create Create and deploy a stager
delete Delete a stager
export Export stagers to a single package file
from file Create a new stager from a tarball
list List your cluster's stagers
remove Remove a stager from a staging pipeline
show Show detailed information about a stager

Staging pipeline commands

The APC commands for staging pipelines have the following form:

apc staging pipeline <subcommand> [optional args]
Subcommand Purpose
append Add stagers to the end of a staging pipeline
create Create a staging pipeline from a stager
clone Clone an existing staging pipeline
prepend Add stagers to the beginning of a staging pipeline
export Export a staging pipeline to a package file
delete Delete a staging pipeline
list List your cluster's staging pipelines
remove Remove stagers from a staging pipeline

For any option, a version beginning with a single hyphen is a shortcut for the one that begins with two hyphens. For example, for the apc stager create command -sp is a shortcut for --staging. Some options (for example, --additive) have no shortcuts.

Creating a new stager

You can create a custom stager with the Ruby Stager API Library or Stager API. You deploy your stager using the apc stager create command. You can then add the stager to an existing staging pipeline; you can also create a new pipeline to contain the stager with the --pipeline flag.

The stager create command has the following form:

$ apc stager create <stager-name> [optional args]

For example, the following creates a new stager job named spell-checker from Ruby source code. The --pipeline flag creates a new staging pipeline (with the same name as the stager job) to which the newly created stager job is added.

apc stager create spell-checker \
  --path /src/spellchecker-stager \
  --start-command="./stager.rb" \
  --memory 1024 --pipeline \
  --staging=/apcera::ruby

Like the apc app create command, this command creates a corresponding package that is linked to the stager job. By default, all files and folders in the directory specified by the --path flag (or current working directory) are included in the generated package. You can exclude files or folders from the package by adding a .apcignore file to the root directory that specifies file and folder patterns to match. See Excluding files from built packages for more information.

Also see the sample stagers in the Apcera sample-apps{ target="_blank"} Github repository.

Optional arguments:

Flag Description
-p, --path PATH Path to the stager being deployed (default: current path)
--pipeline Specifies that a staging pipeline should be created as well. Its FQN will be constructed from its name. (default: false)
-s, --staging NAME Staging pipeline to use when deploying the stager. (default: do not use a staging pipeline)
-c, --cpus CPU Milliseconds of CPU time per second of physical time. May be greater than 1000ms/second in cases where time is across multiple cores. (default: 200ms/second)
-m, --memory MEM Memory allocated to the stager, in MB (default: 256MiB)
-d, --disk DISK Disk space to allocate, in MB (default: 1GiB)
-n, --netmin NET Network throughput to allocate (floor), in Mbps (default: 5Mbps)
-nm, --netmax NET Amount of network throughput to allow (ceiling), in Mbps. (default: 5Mbps)
-e, --env-set ENV=VAL Sets an environment variables on the app. Multiple values can be supplied by invoking multiple times (e.g. --env-set 'HOME=/usr/local' --env-set 'PATH=/usr/bin:/opt/local')
--package-name NAME Package name, if it should be different than the stager. Accepts PQNs.
--start-cmd CMD Command to start the stager
--additive FALSE Indicates that the stager should run within the application's runtime environment (default: false)

additive means rely on a prior stager to ensure that the app's dependencies are satisfied and its environment is set up properly. Thus, it can't be first in the staging pipeline.

Once deployed, a stager is visible in the list of Apcera jobs.

Specifying a staging pipeline

You can override the default staging pipeline to use for your app or package by explicitly declaring the staging pipeline to use. You do this by supplying the --staging parameter and value when you create or update (deploy) an application, image, or package. Specifyuing the staging pipeline may be necessary when:

  • The system cannot detect the source code
  • The automatically detected staging pipeline is not be the desired one
  • Your app satisfies the requirements for more than one application type
  • You want to use a custom or cloned staging pipeline

For example, the following command explicitly uses the static-site stager when deploying the my-website app.

apc app create my-website --staging /apcera::static-site

Cloning a staging pipeline

You can clone an existing staging pipeline using a command such as apc staging pipeline clone /apcera::static-site. Cloning a staging pipeline is often a useful technique as a starting point for creating custom staging pipelines where you want to add or remove one or more stagers from the pipeline.

Adding a stager to a staging pipeline

Apcera's multi-stager process lets you write independent stagers for your tasks, and order them into a staging pipeline. You can run this staging pipeline on packages before they are deployed.

You can append or prepend stagers to an existing staging pipeline or create a new staging pipeline by setting the --pipeline flag during stager create:

staging pipeline append appends a new stager to the end of an existing staging pipeline:

$ apc staging pipeline append <pipeline-name> <stager-names> [...]

Note that the list of stagers is space-separated, as multiple arguments.

staging pipeline prepend prepends a new stager to the beginning of an existing staging pipeline:

$ apc staging pipeline prepend <pipeline-name> <stager-names> [...]

Adding the --pipeline flag to a stager create command creates a new staging pipeline with the same name as the stager:

apc stager create <stager-name> --pipeline [other command specific options]

The system-provided staging pipelines are stored in the /apcera namespace. If you want to invoke a non-default staging pipeline, the command syntax is -sp /apcera::<staging-pipeline-name>. For example: -sp /apcera::ruby.

Updating a stager job

To update a stager job that's been deployed you have two options: replace the stager application package with the apc package replace command, or delete and recreate the stager. The first approach is recommended since it doesn't affect the functionality or availability of staging pipelines that use the stager; the second removes the deleted stager job from any staging pipelines that reference it, temporarily affecting the functionality of those pipelines.

To replace a stager's application package:

  1. Using your system's tar command-line tool or similar utility, create a .tar.gz file containing the stager's source files, for example:

    tar -zcvf package.tar.gz stager-source-files/*

  2. Run apc package replace <stager-fqn> to replace the stager's existing package, passing it the location of the .tar.gz. file as the second parameter. You must specify the same staging pipeline you used to create the stager originally (/apcera::ruby in this case):

    apc package replace /dev::my-stager package.tar.gz --staging /apcera::ruby

To delete and recreate the stager:

  1. Use the apc stager delete <stager-fqn> command with the --forcecommand flag to remove the stager from any staging pipelines it belongs to:

    apc stager delete my-stager --force

  2. Re-create the stager using the apc stager create <stager-name> command, as before.
  3. Add the stager to any staging pipelines it belongs to with the apc staging pipeline append or apc staging pipeline prepend commands (see Staging pipeline commands).

Debugging and troubleshooting staging

The Apcera built-in stagers are written to output various informational and warning messages as the staging process completes. If a fatal error occurs in staging, relevant information is provided and the stager job will terminate.

Sometimes, having access to additional stager job information is valuable for understanding the process or for troubleshooting issues. Debug messages that are normally hidden can be shown for increased stager verbosity with the following command:

apc app create <app-name> --pkg-env STAGER_DEBUG=<verbosity-level>

The different verbosity levels are as follows:

  • v will display the same stager messaging as default, but the individual output lines will be tagged with contextual severity labels.
  • vv will tag lines and introduce debug-level output to the user.
  • vvv will do all of the above in addition to performing some debriefing commands should a fatal error occur. These commands include items like showing environmental variables for the stager job.

For example: You can create an app with vv level stager debugging by running

apc app create test-app --pkg-env STAGER_DEBUG=vv

Using stager log drains

Another option for obtaining staging logs is through the use of logs drains. A log drain target can be attached to all of the staging jobs in a pipeline with the following command:

apc app create <app-name> --pkg-env STAGER_DRAIN=<drain-url>

Any drain target that is supported in standard job log drains will work with this staging option. The target for the staging logs could even be another job running a syslog server. In this case, the command to define the log drain target may look like:

apc app create test-app --pkg-env STAGER_DRAIN="syslog://10.10.10.10:50154

© 2021 Apcera Inc. Last updated: 2018-02-20 17:33:33 +0100 Feedback