Stager API
Apcera stagers let you create and manage your organization's lifecycle management processes. Stagers run as jobs, and you can write them in any language you wish. They are run via staging pipelines, which execute stagers in a specified order.
Apcera provides you with the Stager API for writing custom stagers. The Stager API is a web-services based API that provides the mechanism for creating stagers. In addition, Apcera provides a library in the Ruby language to negotiate the complexities of the Stager API and provide a simple front end.
This article describes how to use the web services-based Stager API. If you want to use the Stager API Ruby library, refer to this article.
If you are just getting started writing custom stagers, or prefer less complexity, it is recommended that you use the Ruby library for the Stager API. Anything you can do in the web service-based Stager API, you can do with the Ruby library.
How to write a custom stager using the Stager API
To interact with the staging coordinator, the stager uses the URL specified in the STAGER_URL
environment variable
The stager performs the following steps:
-
Download a package by sending a
GET
command to$STAGER_URL/data
. The package is normally a gzipped tarfile. -
Unzip and untar the package contents.
-
Perform tasks on the package; if the stager alters the package, it creates a gzipped tarfile of the altered package and uploads it with a
POST
to$STAGER_URL/data
. For each task,-
If successful, issue a
POST
to$STAGER_URL/done
. -
Otherwise, issue a
POST
to$STAGER_URL/failed
.
-
API endpoints
Operation | Endpoint | Description |
---|---|---|
GET |
$STAGER_URL/data |
Download package |
POST |
$STAGER_URL/data |
Upload package to Staging Coordinator |
GET |
$STAGER_URL/meta |
Obtain package metadata |
PUT |
$STAGER_URL/meta |
Modify package metadata |
POST |
$STAGER_URL/snapshot |
Updates the package with a snapshot of the stager's filesystem |
POST |
$STAGER_URL/done |
Marks successful completion of Stager |
POST |
$STAGER_URL/failed |
Marks failure of Stager |
POST |
$STAGER_URL/relaunch |
Relaunches Stager with new dependencies |
Status codes
You might see the following codes in response to HTTP commands sent to $STAGER_URL
:
Status code | Description |
---|---|
200 |
Successful response |
450 |
Request is missing a UUID field |
451 |
Given UUID is not valid |
452 |
UUID is not current |
445 |
Failed to marshal environment variables during package metadata download |
448 |
Specified resource not supported during package metadata alteration |
499 |
May be returned for multiple errors. Error message describes the error. |
These codes and messages are subject to change.
Uploading data
Uploading new package data can be done by making a POST
to $STAGER_URL/data
. The request body then replaces the contents of the package. When uploading, you must also supply the SHA256 checksum through the query string on the request, so you must make the request to $STAGER_URL/data?sha256={sha256}
. The SHA256 cannot use form fields for the request because stagers use the request body for the package data. The upload request must also have the Content-Length
header set to the size of the data being uploaded. Most HTTP clients automatically do this. If not, you will see a "You must provide a Content-Length when uploading" error message returned.
Snapshots
Update the package by making a POST
to $STAGER_URL/snapshot
. The snapshot functionality allows capturing the changes made to the filesystem of the stager from outside of the isolation context. Apcera uses a layered filesystem outside of the stager, so snapshot helps to capture only what has been changed within the stager and skips files that haven't been modified. This is useful in scenarios where a stager makes changes to the filesystem in numerous locations rather than in one directory.
Typically, a runtime stager puts the application in an /app
directory and expects everything to be located in that directory. However, if you write a stager to compile runtime packages, the stager may create files in many places or run scripts. There may be changes across the filesystem, so the snapshot is ideal in this scenario because it captures only the files that have changed.
The snapshot request can be filtered within the stager by passing the directory value with the request.
Metadata
Package metadata can be accessed via GET $STAGER_URL/meta
.
Sample response:
{
"dependencies": [
{"type": "os", "name": "linux"},
{"type": "package", "name": "build-essential"}
],
"provides": [
{"type": "runtime", "name": "ruby"},
{"type": "runtime", "name": "ruby-1.9"}
],
"environment": {
"PATH": "/opt/apcera/ruby-1.9.3/bin:$PATH"
},
"templates": [
{
"path": "foo",
"left_delimiter": "<<",
"right_delimiter": ">>"
},
{
"path": "bar"
}
]
}
The template uses default delimiters {{ and }}.
The Stager can alter the metadata by issuing a PUT
to $STAGER_URL/meta
. The PUT accepts the following form fields:
Request Field | Type | Description |
---|---|---|
resource |
string |
Signifies what is being altered, one of: environment , provides , dependencies , templates . |
action |
string |
One of: add , remove , for all resource form values. |
key |
string |
Specifies the property name to set under the given resource. |
value |
string |
Value associated with a key, used for add action only. |
type |
string |
Required for dependencies and provides , can be one of the following: os , runtime , file , package |
name |
string |
Required for dependencies and provides , can be any string. |
Many languages and frameworks support flipping a development/production switch. In Ruby, a curl call to set the
RAILS_ENV
variable to production would be:
$ curl -X PUT -F resource=environment -F action=add -F key=RAILS_ENV
-F value=production $STAGER_URL/meta
Special package environment variables
A stager can set environment variables on the package that have a special purpose when the package is used with a job. At run time, environment variables from the job and all the packages are merged together and layered so that the package for application has precedence over its dependencies (such as the Ruby or Node runtime and so forth).
These special environment variables are typically meant to provide a default if the equivalent value isn't set on the job itself.
-
START_COMMAND
— The default command to launch the application. Typically a stager determines the command to launch the application based on its language/framework and dependencies. -
START_PATH
— The directory in which to start the application. Typically, stagers put the application in a/app
directory.
Relaunching a Stager
The system cannot dynamically alter the filesystem of a Stager once it is running. To finish staging, a Stager should relaunch if it needs files from previously altered dependencies. Relaunch includes altered dependencies and environment variables.
To relaunch, make a POST
to $STAGER_URL/relaunch
.
- Apcera only does basic
SHA
checksum and length validation checks on packages.- When uploading a new package, Apcera expects the
tar
file to include the full path for all files. So if a file namedindex.html
is meant to be in/app
, the tarfile should have it as/app/index.html
. Apcera extracts all packages relative to the Isolation Context's root directory.- Stagers stream output from
stdout
andstderr
back to users.- Currently, Stagers will run for a maximum time limit of ten minutes. The Staging Coordinator terminates Stagers when this limit is reached, and thus fails the staging process.
Package dependencies within a stager
All of the dependencies of the package being staged are made available to the stager within its filesystem under a /stagerfs
directory. This allows the stager to make use of the package's dependencies for any steps required. Often the libraries needed by the application must be compiled or processed by the same runtime version. For instance, you need the same version of Node to install npm
packages that will be used when the application is run.
Stagers can also chroot
to the /stagerfs
directory to run commands within the package's equivalent filesystem.
For any package dependencies that are placed in the
/stagerfs
directory that define environment variables such asPATH
, the paths given are not altered, so they do not reference/stagerfs
.
RSpec stager
RSpec is a Ruby testing tool. This bash script implements an Rspec stager that does the following:
-
curl
fetches content from$STAGER_URL/data
, writes the content to$TMPDIR/raw.tar.gz
, and discards all status and error messages by sending them to/dev/null
. If the fetch fails,curl
exits, aborting the script. -
It changes to the
/app
directory and extracts content from the retrieved tarball. -
If there is no
spec/
dir, the stager makes a POST to$STAGER_URL/failed
, marking the stager as failed. -
If
spec/
exists, the stager runsbundle install
to install the application's dependencies and runrspec
to test the application. -
If the tests pass, the stager indicates completion by making a
POST
to$STAGER_URL/done
. If the tests fail, the stager indicates this by making aPOST
to$STAGER_URL/failed
.
#!/bin/bash
set -e
export TMPDIR=/tmp
# download the package to stage
echo "Downloading to /app"
curl $STAGER_URL/data > $TMPDIR/raw.tar.gz 2>/dev/null
cd /app
tar xzf $TMPDIR/raw.tar.gz --strip-components=1 app
# make sure we have rspec tests
if [ ! -d spec ]; then
echo "No spec folder found. Not a valid project."
curl -X POST $STAGER_URL/failed
exit 1
fi
# run bundler to install test dependencies
echo "-----> Running bundle install for test dependencies"
bundle install --without development --path vendor/bundle \
--binstubs vendor/bundle/bin --deployment
# run tests
echo "-----> Running tests"
set +e
bundle exec rspec
rspec_exit=$?
set -e
# check the exit code
if [[ $rspec_exit != 0 ]]; then
echo "-----> Rspec failed!"
curl -X POST $STAGER_URL/failed >/dev/null 2>&1
exit $rspec_exit
fi
# success, mark stager done
echo "-----> Rspec tests passed."
curl -X POST $STAGER_URL/done >/dev/null 2>&1
To deploy the stager to Apcera, use the following command:
$ apc stager create ruby-rspec -p rspec-stager --start-command=./rspec-stager --additive