.. _tutorials-example-02-python:

Example 2: Python/Conda Job
===========================

This example shows how to build a small Apptainer image for a CPU-only Python
calculation. The image starts from a public Docker Python image, installs
Miniforge, creates a Conda environment inside the image, and runs a simple
prime-number calculation through Slurm.

Use this pattern when your workflow needs Python packages but does not need a
GPU.

1. Connect to the CoSTAR cluster
--------------------------------

Open a terminal and connect to the CoSTAR login node:

.. code-block:: bash

   ssh ab1234@costar-login01

Replace ``ab1234`` with your University username.

2. Create the Apptainer definition file
---------------------------------------

Download the definition file, or create a file called
``example-02-python.def`` in your working directory and copy in the recipe
below.

* :download:`example-02-python.def <source_files/example-02-python.def>`

The ``Bootstrap: docker`` and ``From: python:3.12-slim`` lines tell Apptainer
to build the image from the public Docker Hub Python image. The ``%post``
section installs Miniforge and creates a Conda environment called
``costar-example`` inside the image.

.. dropdown:: example-02-python.def (click to collapse / view)
   :color: primary
   :open:
   :icon: file-code

   .. literalinclude:: source_files/example-02-python.def
      :language: text
      :linenos:

3. Create the Python script
---------------------------

Download the Python script, or create a file called ``example-02-python.py`` in
your working directory and copy in the script below.

* :download:`example-02-python.py <source_files/example-02-python.py>`

.. dropdown:: example-02-python.py (click to collapse / view)
   :color: primary
   :open:
   :icon: file-code

   .. literalinclude:: source_files/example-02-python.py
      :language: python
      :linenos:

4. Build the Apptainer image
----------------------------

.. note::

   On CoSTAR, Apptainer is available on compute nodes, not on the login node.
   If you want to build the image on the cluster, start an interactive Slurm
   session on a compute node first.

Start an interactive Slurm session for one hour. This does not request a GPU
and only needs minimal resources for the build, so Slurm can choose any
available node on any partition:

.. code-block:: bash

   srun -N 1 --time=00-01:00:00 --pty bash

The image is lightweight and based on ``python:3.12-slim``, so the build may
only take around 10 minutes. The one-hour allocation gives some buffer for
queueing, pulling the base image, and installing the Conda packages.

To build the image use:

.. code-block:: bash

   apptainer build example-02-python.sif example-02-python.def

When the build finishes, exit the interactive session:

.. code-block:: bash

   exit

5. Create the submission script
-------------------------------

Download the example submission script, or create a file called
``example-02-python.sh`` in your working directory and copy in the script
below.

* :download:`example-02-python.sh <source_files/example-02-python.sh>`

The job script runs ``example-02-python.py`` inside the Apptainer image built in
step 4.

.. dropdown:: example-02-python.sh (click to collapse / view)
   :color: primary
   :open:
   :icon: file-code

   .. literalinclude:: source_files/example-02-python.sh
      :language: bash
      :linenos:

This example uses ``apptainer run`` because the definition file contains a
``%runscript`` section:

.. code-block:: text

   %runscript
       exec /opt/conda/envs/costar-example/bin/python "$@"

When you run:

.. code-block:: bash

   apptainer run example-02-python.sif example-02-python.py

Apptainer executes the image's runscript and passes ``example-02-python.py`` to
it.

You can also use ``apptainer exec`` to specify the command directly:

.. code-block:: bash

   apptainer exec example-02-python.sif python example-02-python.py

Use ``apptainer run`` when the image defines a useful default command in
``%runscript``. Use ``apptainer exec`` when you want to explicitly choose the
command to run inside the image.

6. Submit your job
------------------

Submit the script to the Slurm scheduler:

.. code-block:: bash

   sbatch example-02-python.sh

You can check the status of your job by running:

.. code-block:: bash

   squeue --me

7. Check the output
-------------------

When the job finishes, Slurm writes the standard output and error streams to
files in the submission directory:

* ``slurm.<node>.<job_id>.out`` contains normal output from the calculation.
* ``slurm.<node>.<job_id>.err`` contains error messages, if any were produced.

The output file should show the Python executable inside the container, the
NumPy version, the prime search limit, the number of primes found, and the
largest prime found.

For example, the ``.out`` file should contain output similar to:

.. code-block:: text

   Hello from a Python/Conda environment inside Apptainer
   Python executable: /opt/conda/envs/costar-example/bin/python
   NumPy version: 2.4.5
   Prime search limit: 100000
   Number of primes found: 9592
   Largest prime found: 99991