Jobs

This is a Core Concept page which offers an overview of key topics essential for understanding Jobs. Start here for foundational knowledge, then explore our guide and tutorials for more in-depth learning.

The base unit of Qarnot is the Task which consists of a single computation instruction, but sometimes you need to chain tasks together to perform more complex computations. For example using the results of the task as the input for the next one. Many specialized tools like Airflow, Kubeflow,... will offer more capabilities to create complex workflows but you can use Qarnot’s integrated Jobs to perform simple Tasks chaining.

Glossary

Parent: the parent task is the task executed before the current task.
Children: the children tasks are the tasks executed after the current task.
Job Quota: the maximum number of jobs you can execute at the same time.

Core Components

Dependencies

When creating a job you need to define the parameter “Use Dependencies” as True in order to be able to chain tasks, then when creating a task you will only need to:

Specify the Job in which you want to execute the task Job UUID
Define the “dependencies” which is the parent task Additionally if you need to use the parent task results as input you need to declare it in your resources.
More information is available in the guide section.

Time to live and execution management

On top of your task execution time to live and failure/cancellation behavior you can specify Job specific parameters to ensure the whole execution is executed as planned.

Job wall Time: maxWallTime
Description: Maximum real living time of the job.
Behavior when job quota is attained: autoDeleteOnCompletion
Description: Whether the job should be deleted if completed and the job quota is reached.
Job execution Limit: completionTimeToLive
Description: Limit for the job execution time.

Pool

You can use Jobs with pool to speed up tasks boot time, simply add the Pool ID in your Job definition. You won’t need to define the pool for each task linked to this job as they will automatically be assigned to the job’s pool.

Fore more information on how to use our platform, please take a look at our guides, tutorial, and documentation article: