You can create and administer jobs for the DC/OS cluster in any of the following ways:
- interactively with the DC/OS UI
- interactively or programmatically with the DC/OS CLI
- directly through calls to the DC/OS application programming interface (API for job-related operations
The DC/OS UI only provides access to a subset of the job-related functionality provided through the dcos job
CLI and Jobs API. For more advanced job configurations and activity, use the CLI dcos job
commands or the Jobs API.
The DC/OS application programming interface (API) provides underlying functionality that you can access through the DC/OS UI and CLI. In most cases, you would only use the API directly if you are integrating the functionality with a custom program or automation script.
Managing jobs with the DC/OS UI
You can manage the most common job-related activity interactively through the DC/OS UI. For example, you can add, modify, run, and remove jobs directly from the Jobs tab in the UI.
Figure 1 - Jobs tab and jobs list
Split screen
Note that in all the Jobs UI configuration screens, you can choose to view the UI alone, or split the screen between the UI and a JSON Editor window. In the following discussion of the Jobs configuration options, we will show you the split screen so that you can see how they work together, but you may always choose to use the UI by itself. To invoke the JSON Editor, click on the JSON Editor toggle switch next to the Submit button.
Edits made to either interface will be immediately reflected in the other. For example, if you enter a value in any field in the left hand UI, it will be added to the JSON file on the right hand.
To dismiss the JSON Editor screen, click the toggle again.
Figure 2 - Split screen
Add a job
There are two ways to create a job in the DC/OS UI.
- If you have no jobs configured, the Jobs screen will contain a notice that you have no active jobs, and will display a Create a Job button.
Figure 3 - Create a Job button
- Regardless of whether you have active jobs or not, you can always click on the + sign in the upper right corner to create a new job.
Figure 4 - Click on plus sign
A configuration screen for a new job will appear. Note that this screen is the same as that used for editing jobs.
Figure 5 - Job configuration screen
Configuring a job
You will see eight tabs on the left hand side of the screen. These help you name, configure and manage your job:
Name | Description |
---|---|
General | Sets the most basic job settings, such as the job identifier, CPU, memory and disk requirements. |
Container Runtime | Specifies whether the job runs using the Universal Container Runtime or the Docker Engine. |
Schedule | Sets up a schedule for your job. You can use the cron format. |
Environment | Specifies environment variables to be attached to each instance of your job. |
Volumes | Configures a stateful job by setting up a persistent volume. |
Placement | Specifies the placement of agent nodes in regions and zones for high availability, or to expand capacity to new regions. |
Run Configuration | Advanced settings for the job. |
Secrets | Sets up a Secret Store to secure important values like private keys, API tokens, and so forth. |
Detailed information about the parameters and values for each of the configuration screens can be found in the Jobs documentation.
General
Select General to edit the most basic job settings, such as the job identifier, CPU, memory, and disk requirements.
Figure 6 - General configuration tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Container Runtime
Select Container Runtime to specify whether the container for the new job runs using the Universal Container Runtime or the Docker engine. We support the Universal Container Runtime (UCR) and the Docker Engine.
UCR
Universal Container Runtime uses the native Mesos engine and supports GPU resources. This is the recommended selection.
Figure 7 - Jobs > Container Runtime tab
Docker Engine
The Docker Engine is Docker’s container runtime. It requires an image and does not support GPU resources.
Figure 8 - Jobs > Container Runtime > Docker Engine tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Schedule
You can set up jobs with a scheduler using the cron
format. Select Schedule to specify a schedule for when the job runs.
Figure 9 - Jobs > Schedule tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
After you define the schedule, you can activate or deactivate it by selecting or deselecting the Enable Schedule option. You can also modify or remove the schedule when needed after you have submitted the new job definition.
Environment
Select Environment to configure any environment values to be attached to each instance of your job that is launched.
Figure 10 - Jobs > Environment tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Volumes
Select the Volumes tab to create a stateful job by configuring a persistent volume. Persistent volumes enable instances to be restarted without data loss.
Figure 11 - Jobs > Volumes tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Placement
You can configure the placement of agent nodes in regions and zones for high availability, or to expand capacity to new regions when necessary.
Figure 12 - Jobs > Placement tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Run Configuration
Select Run Configuration to specify advanced settings for the new job.
Figure 13 - Jobs > Run Configuration tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Secrets
Select the DC/OS Secret Store to secure important values like private keys, API tokens, and database passwords.
Figure 14 - Secrets tab
The parameters for this tab and the values allowed are found in the Jobs documentation of the UI.
Network
Networking information can be specified using the JSON tab. See the following example for how networks are defined for jobs:
For example, if the job id is mytestjob
, run:
Starting a job manually from the command-line or through the DC/OS UI triggers a new job run each time the command is executed. Jobs that are triggered manually on-demand ignore concurrency policy settings.
If a schedule is used to start a job, however, the job’s concurrency policy determines whether a new job run instance is triggered. Being able to control whether jobs run concurrently is one of the main differences between triggering a job to run manually or using a schedule.
Remove a job from the command-line
You can remove a job using the command-line program dcos job remove <job-id>
as long as the job does not have any active job instances running. If a job has any currently active running instances, you must stop all of the currently-active jobs. After you stop all running job instances, you can remove the job using the dcos job remove <job-id>
command.
To remove a job:
-
Check the status of active jobs by running a command similar to the following:
-
Stop all running job instances for the job you want to delete and remove the job by running the following command:
-
Verify that you have removed the specified job by running the following command:
Modify a job from the command line
To modify your job, update your JSON job file, then run
Modify a job’s schedule
You can update the schedule of your job in two ways, depending if your job has a schedule specified in the <job-file>.json
or if your job’s schedule is kept in a separate file.
Modify a job with a schedule
Modify the schedules
parameter of your <job-file>.json
. Then run
Modify a job with a separate schedule file
Modify <schedule-file>.json
. Then, run one of the following commands:
View job details
List all jobs:
List all previous runs of your job:
To view details about your job, run:
To view details about your job’s schedule, run:
View job logs
To view the log for your job:
To get the log for only a specific job run, use a job run ID from dcos job history <job-id>
Managing jobs with the DC/OS API
You can create and manage jobs through calls to the Jobs API endpoints. This section highlights the most common tasks you perform through job-related API calls. For more complete information about the Jobs API, see the Jobs API reference information.
Preparing to use API cals
The code examples in this section illustrate how to include Jobs API calls to perform job-related tasks with the client URL (cURL) program. For detailed information about using curl
command, see the curl
man page.
In addition, one important difference between using the DC/OS CLI or UI and the API is how you configure the job schedule. The DC/OS CLI and UI support a combined JSON format that allows you to specify a schedule in the job descriptor. To schedule a job using the Jobs API, you must use two separate calls:
- Use one call to add an unscheduled job.
- Use a second call to associate a specific schedule file (
schedule-file.json
) with the job.
Add a job using an API call
The following command adds a job called myjob.json
.
Remove a job using an API call
The following command removes a job regardless of whether the job is running:
To remove a job only if it is not running, set stopCurrentJobRuns
to False
.
Modify or view a job using an API call
The following command shows all jobs:
The following command lists job runs:
Stop a run with the following command:
Add a schedule to a job
The following command adds a schedule to a job:
Start a job using an API call
You can use the DC/OS API to start jobs programmatically. Similar to starting a job using the UI or CLI, you must specify the job identifer in the call.
To trigger a job run to start you can use a REST API call similar to the following: