.. _support-faq: *** FAQ *** .. for the dropdown colour choose from "primary", "secondary", "success", "info", "warning", "danger", "light", "muted", "dark", "white", or "black". General Information =================== .. dropdown:: How to get access to the CoSTAR cluster? :color: light :class-body: shibuya-dropdown-links To obtain access to the CoSTAR cluster for a new user talk to your organisation's CoSTAR representative to get teh process started. .. dropdown:: How do I log in to the CoSTAR cluster? :color: light :class-body: shibuya-dropdown-links You can access CoSTAR through :ref:`Open OnDemand or SSH `. Open OnDemand provides a browser-based interface for files, terminals, and interactive apps. SSH is suitable for users comfortable with the command line and Slurm job submission. .. dropdown:: How do I run code on the cluster? :color: light :class-body: shibuya-dropdown-links Log in to CoSTAR and submit work through Slurm. For batch work, create a job script that requests the resources your code needs, such as CPUs, memory, GPUs, and wall time, then submit it with ``sbatch``. You can also run code interactively with ``srun``, which gives you a shell on an allocated compute node so you can test commands, debug workflows, or run short interactive tasks. See :ref:`Running Jobs ` and :ref:`Interactive jobs ` for examples. .. dropdown:: Who should I contact for technical issues or outages? :color: light :class-body: shibuya-dropdown-links Submit a request through the :ref:`CoSTAR support portal ` so the issue can be tracked and prioritised. Include job IDs, file paths, commands, error messages, and relevant times where possible. Slurm Scheduler =============== .. dropdown:: How can I attach to a running Slurm job? :color: light :class-body: shibuya-dropdown-links You can attach to a running Slurm job via the following command .. code-block:: slurm :linenos: :caption: Slurm command srun --pty --overlap --jobid MY-JOBID bash Replace ``MY-JOBID`` with your Slurm job ID. This should give you a ``bash`` shell on the first node allocated to your job. .. dropdown:: Why has my Slurm job been pending for a long time? :color: light :class-body: shibuya-dropdown-links Use ``squeue -u `` to inspect your jobs. Pending jobs show ``PD`` in the state column, and the ``NODELIST(REASON)`` column explains why the job is waiting. Common reasons include: - ``Priority``: other jobs currently have higher scheduling priority. - ``Resources``: no available node currently matches the resources you requested, such as partition, CPUs, memory, GPUs, or constraints. For more detail, run ``scontrol show job ``. See :ref:`Job Management ` and :ref:`Slurm job priority and fairshare `. .. dropdown:: How do I submit my job to the right node or partition? :color: light :class-body: shibuya-dropdown-links Request the appropriate Slurm partition and node features in your job script, for example: .. code-block:: slurm #SBATCH --partition= #SBATCH --constraint= Use ``sinfo`` to inspect available partitions and node states before submission. See :ref:`sbatch options ` and :ref:`Job Management `. .. dropdown:: How can I check output from my Slurm job? :color: light :class-body: shibuya-dropdown-links Check the files set with ``#SBATCH --output`` and ``#SBATCH --error`` in your job script. If you do not set these, Slurm writes to a default output file such as ``slurm-.out``. While a job is running, you can monitor an output file with: .. code-block:: console tail -f After a job completes, use ``sacct`` to inspect job state and resource usage. See :ref:`Job Management `. Software ======== .. dropdown:: The software I need is not installed. What should I do? :color: light :class-body: shibuya-dropdown-links On CoSTAR, software is normally provided through containers, user-managed environments, or selected Open OnDemand apps. Start with :ref:`Software ` to choose the right approach. In many cases you can: - use or build an :ref:`Apptainer container ` - install lightweight user-level tools with :ref:`Conda ` - request support for shared interactive applications through the :ref:`support portal ` If none of above works for you, you can submit a :ref:`support ticket ` to discuss this with the CoSTAR maintainer team. .. dropdown:: What should I do if I want to use Anaconda or Conda? :color: light :class-body: shibuya-dropdown-links Miniconda is recommended on CoSTAR because it is smaller than full Anaconda and uses less home directory storage. You can install Miniconda in your user space and create isolated environments for your work. See :ref:`Conda ` for setup and storage guidance. .. dropdown:: How do I create or use a custom container image on CoSTAR? :color: light :class-body: shibuya-dropdown-links CoSTAR supports Apptainer containers at runtime. Docker images can usually be converted into Apptainer images for use on the cluster. See :ref:`Containers ` for guidance on building, converting, storing, and running container images. Data Storage ============ .. dropdown:: Who can see my files? :color: light :class-body: shibuya-dropdown-links Files are separated by user in the home directory and the scratch space, so only you can see your files in these locations. However, if you have a shared project directory, other members of the project will be able to see files in that location. .. dropdown:: Which storage location should I use? :color: light :class-body: shibuya-dropdown-links Use your CoSTAR home directory for personal working files, scripts, and outputs you need to keep during active work. Use scratch space for temporary files and high-performance read/write workloads during jobs. CoSTAR local storage is not permanent research data storage and is not backed up. Important research data should be copied to an appropriate managed storage location. See :ref:`Storage Options `. .. dropdown:: How do I check how much storage I am using? :color: light :class-body: shibuya-dropdown-links You can use commands such as ``du``, ``df``, or ``ncdu`` to check storage usage. For examples, see :ref:`Checking your HPC local storage usage `. .. dropdown:: It is taking too long to transfer my data? :color: light :class-body: shibuya-dropdown-links Large transfers can take a long time and may fail if your network connection is interrupted. For repeated or large transfers, ``rsync`` is often better than ``scp`` because it can resume partial transfers and only copies changed data. See :ref:`Transferring data to/from HPC storage `. If you are still unsure, open a :ref:`support ticket `. Software Environments ===================== .. dropdown:: I have my own Docker or Apptainer image. How do I use it with the Open OnDemand interface? :color: light :class-body: shibuya-dropdown-links Docker is not run directly on CoSTAR. If you have a Docker image, convert it to an Apptainer ``.sif`` image first. Some Open OnDemand apps may provide a "Custom Apptainer" option. If that option is available, provide the full path to your Apptainer image when launching the app. The image must contain the software required by that app. For example, a custom VS Code environment must include ``code-server``. If the Open OnDemand app does not provide a custom image option, run your Apptainer image from a Slurm batch or interactive job instead. See :ref:`Open OnDemand ` and :ref:`Containers ` for more details. .. dropdown:: My Apptainer build runs out of storage. What can I do? :color: light :class-body: shibuya-dropdown-links Apptainer images and build caches can be large, so building in your home directory can quickly use your quota. Build on a compute node, not on the login node. Start an interactive Slurm session, then build the image from there. Use scratch space for temporary build files and caches, for example ``/mnt/fast/costar_nobackup/scratch4weeks/``. You can also set Apptainer cache and temporary directories before building: .. code-block:: bash export APPTAINER_CACHEDIR=/mnt/fast/costar_nobackup/scratch4weeks//apptainer-cache export APPTAINER_TMPDIR=/mnt/fast/costar_nobackup/scratch4weeks//apptainer-tmp Scratch space is not backed up and may be cleaned, so move any final ``.sif`` image you need to keep to a suitable storage location after the build. See :ref:`Containers ` and :ref:`HPC Local Storage `. .. dropdown:: My Conda environment is using too much storage. What can I do? :color: light :class-body: shibuya-dropdown-links Conda environments and package caches can use a lot of space in your home directory. You can remove unused packages and caches with ``conda clean --all``. For larger environments, consider recreating them from an ``environment.yml`` file in a more suitable storage location, or packaging the environment inside an Apptainer container. See :ref:`Conda storage guidance `. Job Performance =============== .. dropdown:: How do I know how many CPU cores or how much RAM to request? :color: light :class-body: shibuya-dropdown-links Start by checking whether your code is serial or parallel. If it is parallel, benchmark representative input data with different CPU and memory requests, then choose the smallest request that gives stable performance. Requesting more cores than your code can use will not necessarily make the job faster and may make it wait longer in the queue. See the :ref:`Benchmarking and scaling ` guidance. .. dropdown:: My code is taking too long to run. What can I do? :color: light :class-body: shibuya-dropdown-links First identify the bottleneck: CPU, GPU, memory, storage I/O, or communication between processes. Then test changes systematically rather than only requesting more resources. Useful steps include profiling your code, improving slow algorithms or I/O patterns, using optimised libraries, testing scaling behaviour, and using containers for reproducible software stacks. .. dropdown:: How do I implement checkpointing in my HPC job? :color: light :class-body: shibuya-dropdown-links Checkpointing means saving job state periodically so work can restart after an interruption, failure, or time limit. This is especially useful for long-running jobs. Common approaches include framework-native checkpoints, such as PyTorch or TensorFlow model checkpoints, or saving application state at regular intervals and restarting from the latest checkpoint in your job script. See :ref:`Checkpointing `.