Paul Benoit
CEO & co-founder

Distributed Monte Carlo simulations

January 19, 2017 - Finance

CC-BY / jgilhutton
 

In this article, we'll describe how to use Qarnot computing HPC platform to perform distributed Monte Carlo simulation. As a 'hello world' Monte Carlo workload, we'll use a basic $pi$ estimation.

Estimating $pi$ using Monte Carlo Simulation

Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to obtain numerical results. To estimate $pi$, we randomly place $N$ points in a square of side$1$ and count $P$, the number of points that are at a distance $d<1$ from the origin as shown in the following figure.

When $N$ is large, $P/N$ converges towards $pi/4$, the surface of the quarter circle. This being said, the following code will help you test this method.

You can notice the convergence toward $pi$ when you increase the number of samples but also the non deterministic behaviour of the piMc function.

To make the piMc function deterministic, you can seed the random generator. Then, seeds can be used to split the computation into multiple independant sampling rounds.

As you can see, the precision increase with both the number of samples per split and the number of split.

Let's use Qarnot computing HPC service

First let's save the following script in a piMc.py file.

This script simply accepts the seed and number of samples as arguments.

Now, let's use Qarnot python SDK to launch the distributed computation. You'll need to launch the following qPiMc.py script after replacing with the key you obtained by registering at Qarnot computing.

By executing this script, you'll :

  • connect to Qarnot API by using your API token
  • create a task composed of 10 frames numbered from 1 to 10
  • upload the qPiMc.py to be executed on each node
  • select the right docker container to be deployed on each node (here, we use anaconda, a sledgehammer to crack a nut...)
  • define the command to be launched within containers
  • launch the task and print the aggregated stdout when all nodes exited (approx. 1 minute from cold boot to shutdown)

During the execution, you can follow the progress on the Qarnot console. Here is an example with 100,000,000 samples per seed :

Next steps!

With Qarnot HPC services, you can go well beyond what could be described in this article. You will probably come up with many more interesting use cases. So, go ahead! Try it now, and share your feedback with the community, so that we can keep improving on this product.

[1] Monte Carlo method: wikipedia

[2] Parallel Pi Calculation using Python's multiprocessing module: github

[3] Qarnot computing Developer & API documentation: qarnot

Share on networks