Rémi Bouzel
Scientific Director
HPC platform
Launch compute tasks in a few lines of code or a few clicks on Tasq, our HPC platform.

Kaniko on Qarnot Cloud - documentation

November 4, 2021 - Documentation

Introduction

Kaniko is an open-source tool developed by Google which allows you to build container images from a Dockerfile inside a container.
It is currently not possible to build docker images within a docker container without the use of the --privileged flag (or --cap-add), which gives access to the docker daemon running on the host, and can therefore be a potential security issue. Kaniko doesn't depend on a Docker daemon, which enables building container images in environments that can't easily or securely run a Docker daemon, such as a docker container or a standard Kubernetes cluster.

Versions

The test case uses Kaniko version 1.9.1.

Here is the current available version on Qarnot:

Release yearVersion
20211.6.0
20221.9.1

If you are interested in another version, please send us an email at qlab@qarnot.com.

Prerequisites

Qarnot prerequisites

  • Please ensure that you have created a Qarnot account here
  • Retrieve your Qarnot authentication token here
  • Install Qarnot’s Python SDK here

Note: in addition to the Python SDK, Qarnot provides C# and Node.js SDKs and a Command Line Interface.

Docker hub prerequisites

You need to have a docker hub account to be able to push the image on it once it is built. For this, you just need to sign up on the docker hub. Once your account is created, make sure you save your docker user login/ID (name of your repository) and your docker password, they will be useful later.

Test case

The objective of this test case is to build a docker image containing the OR-Tools python package. You can download the input file (just a Dockerfile) here.

OR-Tools is an open source operations research software. It is specialized in combinatorial optimization, which seeks to find the best solution to a problem out of a very large set of possible solutions.

Launching the case

Once everything is set up, use the following script in your favorite interpreter to launch the docker image build on Qarnot.

kaniko.py

 

Make sure you have copied your authentication token in the script (instead of <<<MY_SECRET_TOKEN>>>) to be able to launch the task on Qarnot.

 

Make sure you updated the kaniko.py scripts with your docker credentials:

  • "IMAGE_NAME" corresponds to the name you want to give to your docker image (careful, it must follow the pattern D_USER/<name>)
  • "IMAGE_TAG" corresponds to the tag of the image. For instance, it can correspond to the version of the image
  • "D_USER" corresponds to your docker hub user login/ID
  • "D_PWD" corresponds to your docker hub password
  • here is an example (careful: you can’t reproduce it as you don’t have the proper credits to push an image to the qarnotlab repository):

                    task.constants["IMAGE_NAME"] = "qarnotlab/ortools"

                    task.constants["IMAGE_TAG"] = "9.0"

                    task.constants["D_USER"] = "qarnotlab"

                    task.constants["D_PWD"] = "MySecretPassword"

 

Finally, your working directory should look like this :

  • dataset-kaniko/
    • Dockerfile : Dockerfile to build the OR-Tools image
  • kaniko.py : Python script to run the computation on Qarnot

To launch this script, you just need to open a terminal in your working directory and execute python3 kaniko.py

Results

At any given time, you can monitor the status of your task on the general web interface, Console. The following figure shows a successful Kaniko demo on Qarnot’s Console.

The built image will be available in the docker hub, and can be downloaded with the command:

                    docker pull <REPO>:<TAG>

for example:

                    docker pull qarnotlab/ortools:9.0

If it is hosted on a private repository, you will need to log in to this private repository with your docker credentials, you can do this with the command:

                    docker login

Once downloaded, you can use the image the way you want. For instance, you can deploy a container with a shell interface, by running the following command:

                    docker run --it -rm <REPO>:<TAG> /bin/sh

for example:

                    docker run --it -rm qarnotlab/ortools:9.0 /bin/sh

Wrapping up

That’s it! If you have any questions, please contact qlab@qarnot.com and we will help you with pleasure!

If you are interested in knowing more about OR-Tools, find out how to run an OR-Tools payload on Qarnot on our blog.

 

Share on networks