.. _tutorials-advanced-apptainer: Advanced Apptainer Build Pipeline ================================= This guide shows how to build a GPU-ready Apptainer image that recreates a known working Anaconda environment, keeps pip dependencies in a separate list for easier version changes, and compiles custom CUDA kernels for Gaussian Splatting. The environment in this example is for the following codebase: https://github.com/tobias-kirschstein/gghead.git For background on Apptainer, Docker image conversion, and where containers can be built on CoSTAR, see :ref:`containers`. Quick highlights ---------------- - Recreates a working Anaconda environment from a pinned export file. - Keeps pip dependencies in a separate requirements list for easier maintenance. - Allows building Gaussian Splatting CUDA kernels without a GPU present during the build by setting ``TORCH_CUDA_ARCH_LIST``. - Installs code-server so the container is ready for Open OnDemand debugging. 1. Download the files --------------------- Download the Apptainer definition file and the two dependency lists. * :download:`guide-02-container.def ` * :download:`guide-02-env.txt ` * :download:`guide-02-pip.txt ` .. dropdown:: guide-02-container.def (click to collapse / view) :color: primary :open: :icon: file-code .. literalinclude:: source_files/guide-02-container.def :language: text :linenos: 2. Prepare a build directory and build the image ------------------------------------------------ Create a build directory, move the downloaded files into it, and rename them to match what the definition file expects: .. code-block:: bash mkdir -p ~/apptainer-builds/gghead cd ~/apptainer-builds/gghead mv ~/Downloads/guide-02-container.def gghead.def mv ~/Downloads/guide-02-env.txt gghead-env.txt mv ~/Downloads/guide-02-pip.txt gghead-pip.txt Apptainer builds should run on a compute node. Start a small interactive session and build the ``.sif`` image: .. code-block:: bash srun -N 1 --ntasks=1 --cpus-per-task=8 --mem=32G --time=04:00:00 --pty bash cd ~/apptainer-builds/gghead apptainer build gghead.sif gghead.def This should provide you with the .sif file which should be about 7.5G in size once built on the CoSTAR node. .. note:: The resulting ``.sif`` file can be large. Follow the storage guidance in :ref:`software-container` and avoid filling your home quota. 3. Apptainer build file walkthrough ----------------------------------- This definition file is organized into standard Apptainer sections. It is based on an Ubuntu 22.04 base image and uses a pinned conda export plus a separate pip list to keep versions stable and easy to update. Key sections and what they do: **Bootstrap and base image** The ``Bootstrap: docker`` and ``From: ubuntu:22.04`` lines define the base image. This keeps the build reproducible and ensures a known starting point. **%files** Copies the environment and pip requirement files into the image. The file names are standardized to ``gghead-env.txt`` and ``gghead-pip.txt`` so the build steps can rely on consistent paths. **%post** Runs the build steps inside the image: - Installs system packages required for conda, build tools, and runtime libraries. - Installs Mambaforge and recreates the conda environment from the pinned export file. Mambaforge is preferred here because it ships with ``mamba`` for faster and more reliable dependency solving compared with a vanilla Miniconda install, while still being compatible with the same environment export format. Mamba is a drop in replacement for conda but is optimized for speed by using multithreading and a more optimized dependency solver. - Installs pip dependencies from the separate requirements list. - Builds the Gaussian Splatting CUDA kernels. ``TORCH_CUDA_ARCH_LIST`` is set so the kernels can be compiled without a GPU present during the build. The value ``7.5`` corresponds to the GPU compute capability (for example, NVIDIA T4 is 7.5). To choose the right value for your hardware, check the GPU model in your environment and look up its compute capability in NVIDIA's GPU support table. Use the major.minor compute capability number (for example, ``8.0`` for A100, ``8.6`` for RTX 30xx). The compute capability version for H200 is ``9.0`` but in this example an older one is used to allow the same reproducible software environment to run on the user's desktop GPU for benchmarking purposes and live demos. Using the latest compute capability however should result in minor performance gains in this case. - Installs code-server and ``debugpy`` for Open OnDemand debugging. ``code-server`` provides the web-based VS Code interface, and ``debugpy`` enables Python debugging sessions from that interface. - Cleans conda and apt caches to reduce the final image size. **%environment and %runscript** Ensures the conda environment activates automatically for interactive shells and when the container runs. **Version pinning** The definition file pins versions for Mambaforge and code-server. Pinning tool versions is a best practice for reproducibility because it reduces unexpected build failures caused by upstream changes and makes it easier to rebuild the same image later. If you do update these versions, do so intentionally and document the change alongside any environment updates. You can adjust or re-use components of this definition file for your own projects as needed. For example, you could swap out the base image, change the Anaconda environment, or remove the code-server installation if you don't need it. 4. Use the container in Open OnDemand ------------------------------------- This image installs code-server and ``debugpy`` so it is ready for Open OnDemand debugging workflows. You can launch a session in Open OnDemand and point it at the container image, or use it in batch jobs with ``apptainer run`` or ``apptainer exec`` as needed. If you need a refresher on Apptainer run and exec syntax, see the commands and examples in :ref:`software-container`.