Home
Getting started
Bring your software with Docker
Core concepts
How-to guides
Monitor and debugging
Developer tools
Glossary
Qarnot's computing platform is well suited for embarassingly parallel computations, in which commands and data don't change across instances , only the instance ID does. On every instance, your program can access the instance ID using the INSTANCE_ID environment variable.
There are two ways of specifying the range of values that INSTANCE_ID will take across your instance:
N), in that case, INSTANCE_ID will take all the values 0, 1, 2, ..., N-1,INSTANCE_ID should take across instances, by using advanced ranges, the syntax of which is detailed below.Here is a simple distributed Hello world example to get more familiar with running code on multiple Qarnot instances.
Python
import qarnot
conn = qarnot.connection.Connection(client_token = "<<<MY_SECRET_TOKEN>>>")
task = conn.create_task('helloworld', 'docker-batch', 4)
task.constants['DOCKER_CMD'] = 'echo hello world from node ${INSTANCE_ID}!'
task.run()
print(task.stdout())
Output
1> hello world from node 1!
3> hello world from node 3!
0> hello world from node 0!
2> hello world from node 2!
What happened?
After opening a connection using your authentification token, you created a task of 4 instances that will launch ubuntu docker containers retrieved from Docker Hub. Each container will use the ${INSTANCE_ID} parameter to output a string using this parameter.
You can view that same output on Tasq.
An advanced range is formatted in a "standard" way (you probably encountered it in your printer settings for instance), as a comma-separated list of integers (e.g: 2, 3, 4) or ranges of integers (e.g: 5-9 represents 5, 6, 7, 8, 9).
For example, the range 1-6,15,20,25-30 represents the instances 1 to 6 and 15 and 20 and 25 to 30. Extensively, that makes 1, 2, 3, 4, 5, 6, 15, 20, 25, 26, 27, 28, 29, 30.
Each instance will execute on its own computing node, and every instance will have the same profile, input data and constants, to the exception of:
INSTANCE_ID constant, which is set to the instance number,INSTANCE_ID environment variable.You can read up more on Qarnot constants and environment variables.
As much as possible, the instances of your tasks will run in parallel on different nodes. If not enough compute nodes are available or if your maximum simultaneous CPU limit has been reached, the remaining instances are put in queue by the Qarnot scheduler. They will be dispatched when free compute nodes become available, or some of your other frames complete.
You can monitor the state of each compute node in real time from the Tasq dashboard using the instance state viewer.
Instances are running on different geographical locations, so running communicating applications such as MPI is not possible out of the box as they use clusters. You can refer to the running on a cluster documentation page for more information.
MPI tasks are supported but have not been made available for public use. You can contact us if you need to run this kind of tasks.
For more information on launching tasks please consult the following articles