.. _cluster-storage:

***************
Storage Options
***************

Storage on the CoSTAR cluster is designed for **active computing**, not long-term archiving. Choosing the correct location for your scripts, temporary scratch files, and massive datasets ensures your jobs run as fast as possible.

Select a topic below to learn how to manage your data effectively:

.. raw:: html

   <div style="margin-bottom: 50px;"></div>

.. grid:: 1 2 2 2
   :gutter: 3
   :margin: 1

   .. grid-item-card:: :octicon:`database` File Systems & Quotas
      :link: hpc-local-storage
      :link-type: ref

      Learn about the different storage areas (Home vs. Scratch), check your space limitations, and understand file retention policies.

   .. grid-item-card:: :octicon:`arrow-switch` Transferring Data
      :link: hpc-data-transfer
      :link-type: ref

      Learn how to move files to and from the cluster using Open OnDemand, ``scp``, ``rsync``, or SFTP tools.

   .. grid-item-card:: :octicon:`file-directory` File Management
      :link: hpc-working-with-storage
      :link-type: ref

      Practical guides on how to browse, edit, and manage your files using the terminal or graphical tools like VS Code.

   .. grid-item-card:: :octicon:`repo` Large Datasets
      :link: dataset-management
      :link-type: ref

      Best practices for handling, sharing, and storing larger datasets required for AI and machine learning workflows.


.. raw:: html

   <div style="margin-bottom: 50px;"></div>

.. important::
   **No automatic backups:**

   Cluster storage is highly performant but is generally not backed up. You are responsible for transferring your critical results and final datasets off the cluster for safe keeping.



.. toctree::
   :maxdepth: 2
   :hidden:

   local-storage
   data-transfer
   working-with-storage
   dataset-management