Marathon applications normally lose their state when they terminate and are relaunched. In some contexts, for instance, if your application uses MySQL, you’ll want your application to preserve its state. You can use an external storage service, such as Amazon’s Elastic Block Store (EBS), to create a persistent volume that follows your application instance.
An external storage service enables your apps to be more fault-tolerant. If a host fails, Marathon reschedules your app on another host, along with its associated data, without user intervention.
You can specify an external volume in your Marathon app definition. Learn more about Marathon application definitions.
The cmd in this app definition appends the output of the date command to test.txt. You can verify that the external volume is being used correctly if you see that the logs of successive runs of the application show more and more lines of date output.
{
"id": "hello",
"instances": 1,
"cpus": 0.1,
"mem": 32,
"cmd": "date >> test-rexray-volume/test.txt; cat test-rexray-volume/test.txt",
"container": {
"type": "MESOS",
"volumes": [
{
"containerPath": "test-rexray-volume",
"external": {
"size": 100,
"name": "my-test-vol",
"provider": "dvdi",
"options": { "dvdi/driver": "rexray" }
},
"mode": "RW"
}
]
},
"upgradeStrategy": {
"minimumHealthCapacity": 0,
"maximumOverCapacity": 0
}
}
In the app definition above:
containerPath specifies where the volume is mounted inside the container. You can specify a relative path or an absolute container path containing forward slashes (/). Note: This path must exist on the host machine.
name is the name that your volume driver uses to look up your volume. When your task is staged on an agent, the volume driver queries the storage service for a volume with this name. If one does not exist, it is created implicitly. Otherwise, the existing volume is reused.
The external.options["dvdi/driver"] option specifies which Docker volume driver to use for storage. If you are running Marathon on DC/OS, this value is probably rexray. Learn more about REX-Ray. The synatax for this parameter is <provider-name>/<option-name>.
You can specify additional options with container.volumes[x].external.options[optionName]. The dvdi provider for Mesos containers uses dvdcli, which offers the options documented here. The availability of any option depends on your volume driver.
Create multiple volumes by adding additional items in the container.volumes array.
Volume parameters cannot be changed after you create the application.
Important: Marathon will not launch apps with external volumes if upgradeStrategy.minimumHealthCapacity is greater than 0.5, or if upgradeStrategy.maximumOverCapacity does not equal 0.
Below is a sample app definition that uses a Docker container and specifies first an external volume, second a sandbox-relative host-volume.
The cmd in this app definition appends the output of the date command to test.txt. You can verify that the external volume is being used correctly if you see that the logs of successive runs of the application show more and more lines of date output.
{
"id": "/test-docker",
"instances": 1,
"cpus": 0.1,
"mem": 32,
"cmd": "date >> /data/test-rexray-volume/test.txt; cat /data/test-rexray-volume/test.txt",
"container": {
"type": "DOCKER",
"docker": {
"image": "alpine:3.1",
"network": "HOST",
"forcePullImage": true
},
"volumes": [
{
"containerPath": "/data/test-rexray-volume",
"external": {
"name": "my-test-vol",
"provider": "dvdi",
"options": { "dvdi/driver": "rexray" }
},
"mode": "RW"
},
{
"hostPath": "var-log",
"containerPath": "/var/log/myapp",
"mode": "RW"
}
]
},
"upgradeStrategy": {
"minimumHealthCapacity": 0,
"maximumOverCapacity": 0
}
}
Important: Refer to the REX-Ray documentation to learn which versions of Docker are compatible with the REX-Ray volume driver.
Apps that use external volumes can only be scaled to a single instance because a volume can only attach to a single task at a time. This may change in a future release.
If you scale your app down to 0 instances, the volume is detached from the agent where it was mounted, but it is not deleted. If you scale your app up again, the data that was associated with it is still be available.
The default implicit volume size is 16 GB. If you are using the original Mesos containerizer or the UCR, you can modify this default for a particular volume by setting volumes[x].external.size. For the Mesos and Docker containerizers, you can modify the default size for all implicit volumes by modifying the REX-Ray configuration.
By default, external volumes can not share the same name. As there are providers that allow this, there is a flag that prevents the uniqueness check on the volume name. volumes[x].external.shared can be set to true. In this case, this volume is not included when checking if the volume name already exists. It still verifies that no other external volumes with volumes[x].external.shared=false exist, so all volumes with the same name must have this flag set.
"container": {
"type": "MESOS",
"volumes": [
{
"external": {
"size": 5,
"name": "volumename",
"provider": "dvdi",
"shared": "true",
"options": {
"dvdi/driver": "pxd",
"dvdi/shared": "true"
}
},
"mode": "RW",
"containerPath": "/mnt/nginx"
}
],
}
VOLUME entries, Docker may create anonymous external volumes. This is default Docker behavior with respect to volume management when the --volume-driver flag is passed to docker run. However, anonymous volumes are not automatically deleted and will accumulate over time unless you manually delete them. To prevent Docker from creating anonymous volumes, you can either use a Mesos container with a Docker image or follow these steps:docker inspect the app’s Docker image before running the app in Marathon and make a note of the VOLUME entries in the specification.VOLUME entry that should not map to an anonymous external volume.specify a relative hostPath for a host-volume to instruct Mesos to create the mount in the task’s sandbox ($MESOS_SANDBOX/$hostPath); see the sandbox-relative host-volume example above.
You can only assign one task per volume. Your storage provider might have other limitations.
The volumes you create are not automatically cleaned up. If you delete your cluster, you must go to your storage provider and delete the volumes you no longer need. If you’re using EBS, find them by searching by the container.volumes.external.name that you set in your Marathon app definition. This name corresponds to an EBS volume Name tag. Any anonymous volumes that docker has created on your behalf will have been assigned a UUID for their name.
Volumes are namespaced by their storage provider. If you’re using EBS, volumes created on the same AWS account share a namespace. Choose unique volume names to avoid conflicts.
If you are using Amazon’s EBS, it is possible to create clusters in different availability zones (AZs). If you create a cluster with an external volume in one AZ and subsequently destroy that cluster, a new cluster may not have access to that external volume because it could be in a different AZ.
Launch time might increase for applications that create volumes implicitly. The amount of the increase depends on several factors which include the size and type of the volume. Your storage provider’s method of handling volumes can also influence launch time for implicitly created volumes.