FAQ

General Information

How to get access to the CoSTAR cluster?

To obtain access to the CoSTAR cluster for a new user talk to your organisation’s CoSTAR representative to get teh process started.

How do I log in to the CoSTAR cluster?

You can access CoSTAR through Open OnDemand or SSH.

Open OnDemand provides a browser-based interface for files, terminals, and interactive apps. SSH is suitable for users comfortable with the command line and Slurm job submission.

How do I run code on the cluster?

Log in to CoSTAR and submit work through Slurm. For batch work, create a job script that requests the resources your code needs, such as CPUs, memory, GPUs, and wall time, then submit it with sbatch. You can also run code interactively with srun, which gives you a shell on an allocated compute node so you can test commands, debug workflows, or run short interactive tasks.

See Running Jobs and Interactive jobs for examples.

Who should I contact for technical issues or outages?

Submit a request through the CoSTAR support portal so the issue can be tracked and prioritised. Include job IDs, file paths, commands, error messages, and relevant times where possible.

Slurm Scheduler

How can I attach to a running Slurm job?

You can attach to a running Slurm job via the following command

Slurm command
1srun --pty --overlap --jobid MY-JOBID bash

Replace MY-JOBID with your Slurm job ID. This should give you a bash shell on the first node allocated to your job.

Why has my Slurm job been pending for a long time?

Use squeue -u <username> to inspect your jobs. Pending jobs show PD in the state column, and the NODELIST(REASON) column explains why the job is waiting.

Common reasons include:

  • Priority: other jobs currently have higher scheduling priority.

  • Resources: no available node currently matches the resources you requested, such as partition, CPUs, memory, GPUs, or constraints.

For more detail, run scontrol show job <jobid>. See Job Management and Slurm job priority and fairshare.

How do I submit my job to the right node or partition?

Request the appropriate Slurm partition and node features in your job script, for example:

#SBATCH --partition=<partition_name>
#SBATCH --constraint=<feature>

Use sinfo to inspect available partitions and node states before submission. See sbatch options and Job Management.

How can I check output from my Slurm job?

Check the files set with #SBATCH --output and #SBATCH --error in your job script. If you do not set these, Slurm writes to a default output file such as slurm-<jobid>.out.

While a job is running, you can monitor an output file with:

tail -f <output_file>

After a job completes, use sacct to inspect job state and resource usage. See Job Management.

Software

The software I need is not installed. What should I do?

On CoSTAR, software is normally provided through containers, user-managed environments, or selected Open OnDemand apps. Start with Software to choose the right approach.

In many cases you can:

If none of above works for you, you can submit a support ticket to discuss this with the CoSTAR maintainer team.

What should I do if I want to use Anaconda or Conda?

Miniconda is recommended on CoSTAR because it is smaller than full Anaconda and uses less home directory storage. You can install Miniconda in your user space and create isolated environments for your work. See Conda for setup and storage guidance.

How do I create or use a custom container image on CoSTAR?

CoSTAR supports Apptainer containers at runtime. Docker images can usually be converted into Apptainer images for use on the cluster.

See Containers for guidance on building, converting, storing, and running container images.

Data Storage

Who can see my files?

Files are separated by user in the home directory and the scratch space, so only you can see your files in these locations. However, if you have a shared project directory, other members of the project will be able to see files in that location.

Which storage location should I use?

Use your CoSTAR home directory for personal working files, scripts, and outputs you need to keep during active work. Use scratch space for temporary files and high-performance read/write workloads during jobs.

CoSTAR local storage is not permanent research data storage and is not backed up. Important research data should be copied to an appropriate managed storage location. See Storage Options.

How do I check how much storage I am using?

You can use commands such as du, df, or ncdu to check storage usage. For examples, see Checking your HPC local storage usage.

It is taking too long to transfer my data?

Large transfers can take a long time and may fail if your network connection is interrupted. For repeated or large transfers, rsync is often better than scp because it can resume partial transfers and only copies changed data.

See Transferring data to/from HPC storage. If you are still unsure, open a support ticket.

Software Environments

I have my own Docker or Apptainer image. How do I use it with the Open OnDemand interface?

Docker is not run directly on CoSTAR. If you have a Docker image, convert it to an Apptainer .sif image first.

Some Open OnDemand apps may provide a “Custom Apptainer” option. If that option is available, provide the full path to your Apptainer image when launching the app. The image must contain the software required by that app. For example, a custom VS Code environment must include code-server.

If the Open OnDemand app does not provide a custom image option, run your Apptainer image from a Slurm batch or interactive job instead. See Open OnDemand and Containers for more details.

My Apptainer build runs out of storage. What can I do?

Apptainer images and build caches can be large, so building in your home directory can quickly use your quota.

Build on a compute node, not on the login node. Start an interactive Slurm session, then build the image from there. Use scratch space for temporary build files and caches, for example /mnt/fast/costar_nobackup/scratch4weeks/<username>.

You can also set Apptainer cache and temporary directories before building:

export APPTAINER_CACHEDIR=/mnt/fast/costar_nobackup/scratch4weeks/<username>/apptainer-cache
export APPTAINER_TMPDIR=/mnt/fast/costar_nobackup/scratch4weeks/<username>/apptainer-tmp

Scratch space is not backed up and may be cleaned, so move any final .sif image you need to keep to a suitable storage location after the build. See Containers and HPC Local Storage.

My Conda environment is using too much storage. What can I do?

Conda environments and package caches can use a lot of space in your home directory. You can remove unused packages and caches with conda clean --all.

For larger environments, consider recreating them from an environment.yml file in a more suitable storage location, or packaging the environment inside an Apptainer container. See Conda storage guidance.

Job Performance

How do I know how many CPU cores or how much RAM to request?

Start by checking whether your code is serial or parallel. If it is parallel, benchmark representative input data with different CPU and memory requests, then choose the smallest request that gives stable performance.

Requesting more cores than your code can use will not necessarily make the job faster and may make it wait longer in the queue. See the Benchmarking and scaling guidance.

My code is taking too long to run. What can I do?

First identify the bottleneck: CPU, GPU, memory, storage I/O, or communication between processes. Then test changes systematically rather than only requesting more resources.

Useful steps include profiling your code, improving slow algorithms or I/O patterns, using optimised libraries, testing scaling behaviour, and using containers for reproducible software stacks.

How do I implement checkpointing in my HPC job?

Checkpointing means saving job state periodically so work can restart after an interruption, failure, or time limit. This is especially useful for long-running jobs.

Common approaches include framework-native checkpoints, such as PyTorch or TensorFlow model checkpoints, or saving application state at regular intervals and restarting from the latest checkpoint in your job script. See Checkpointing.