.. _scheduler-fairshare: ************************** Job priority and Fairshare ************************** What is Fairshare ================= On Slurm-based clusters, each user is associated with a :ref:`Slurm` account typically related to their research group, department, or company. Users belong to accounts, and accounts have **shares** associated with them. The number of shares each account has is influenced by multiple factors and can vary from cluster to cluster. The cluster is a limited resource, and *Fairshare* allows us to ensure everyone gets a fair opportunity to use it regardless of how big or small the group is. Look up your shares ------------------- To see how much your group/account has used of their *Fairshare*, use the ``sshare`` command to show a summary of this information. ``sshare -a --account=`` In the example below we have used the Chemistry account. The first line of the ``sshare`` gives the summary for the whole account, with the additional lines giving a summary per user on the account. .. code-block:: console :caption: Example output from a sshare command [abc123@costar-login01 chem_reservation]$ sshare -a --account=chemistry Account User RawShares NormShares RawUsage EffectvUsage FairShare -------------------- ---------- ---------- ----------- ----------- ------------- --------- chemistry 8423 0.112157 735603420 0.169374 0.351074 chemistry bobby 1 0.007477 0 0.011292 0.351074 chemistry tony 1 0.007477 0 0.011292 0.351074 chemistry susan 1 0.007477 203592409 0.055052 0.006076 chemistry user1 1 0.007477 0 0.011292 0.351074 chemistry user2 1 0.007477 0 0.011292 0.351074 chemistry someguy 1 0.007477 0 0.011292 0.351074 chemistry chemist 1 0.007477 38 0.011292 0.351074 chemistry user3 1 0.007477 517403742 0.122502 0.000012 :**RawShares**: Chemistry has 8423 RawShares. Each user of that lab has a RawShare of its parent, this means that all the users in chemistry pull from the total Share of the Account and do not have their own individual subShares of the account Share. Thus all users in this lab have full access to the full Share of the Account. :**NormShares**: NormShares is the chemistry account's RawShares divided by the total number of RawShares given out to all accounts on the cluster. NormShare is the fraction of the cluster the account has contributed/invested in; for the chemistry account this is about 11.21% of the CoSTAR cluster. :**RawUsage**: RawUsage is the amount of usage the account/user has used on the CoSTAR cluster. This RawUsage is also adjusted by the halflife that is set for the cluster which is 30 days. This means that usage in the last 30 days counts at full cost, from 60 days ago costs half, usage 90 days ago one fourth. So RawUsage is the aggregate of the account's past usage with this halflife weighting factor. The RawUsage for the account is the sum of the RawUsage for each user, thus, sshare is an effective way to figure out which users have contributed the most to the account's score. :**EffectvUsage**: EffectvUsage is the account's RawUsage divided by the total RawUsage for the cluster. Thus EffectvUsage is the percentage of the cluster the account has actually used. For chemistry they have used 16.9% of the cluster. :**Fairshare**: The *Fairshare* score is calculated using the following formula: ``f = 2^(-EffectvUsage/NormShares)``. From this number, we can assess how much an account is using of their contribution/investment in the CoSTAR cluster. :**1.0**: Un-used. The account has not run any jobs recently. :**1.0 > f > 0.5**: Under-utilisation. The account is under-utilising their share. For example, if the *Fairshare* score is 0.75 an account has recently underutilised their share of the resources 1:2. :**0.5**: Average utilisation. The account on average is using exactly as much as their share. :**0.5 > f > 0**: Over-utilisation. The account has overused their share. For example, if the *Fairshare* score is 0.25 an account has recently over-utilised their share of the cluster 2:1. :**0**: No share left. The account has vastly overused their share. Job priority ============ Job priority is an integer that governs the position of a job in the pending queue relative to other jobs. Individual job priorities are calculated based two inputs: A user's *FairShare* score and the age of the job. This priority accrues over time, gaining a maximum value at 7 days due to the job's age. Thus even jobs from accounts that have low priority will eventually run due to the growth in their job age priority. To view the priority for a specific job, use the ``sprio`` command. - Print the list of all pending jobs with their weighted priorities .. code-block:: console $ sprio JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65539 62664 0 51664 1000 10000 0 65540 62663 0 51663 1000 10000 0 65541 62662 0 51662 1000 10000 0 - Print the list of all pending jobs with their normalised priorities .. code-block:: console $ sprio -n JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65539 0.00001459 0.0007180 0.5166470 1.0000000 1.0000000 0.0000000 65540 0.00001459 0.0007180 0.5166370 1.0000000 1.0000000 0.0000000 65541 0.00001458 0.0007180 0.5166270 1.0000000 1.0000000 0.0000000 - Print the job priorities for specific jobs .. code-block:: console $ sprio --jobs=65548,65547 JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65547 62078 0 51078 1000 10000 0 65548 62077 0 51077 1000 10000 0 - Print the job priorities for jobs of specific users .. code-block:: console $ sprio --users=fred,sally JOBID USER PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65548 fred 62079 1 51077 1000 10000 0 65549 sally 62080 1 51078 1000 10000 0 .. important:: A low *Fairshare* score does not stop jobs from running. Bit it influences their priority over other jobs. There are no quotas or limits on how many jobs a user can submit/run on the CoSTAR cluster, and all jobs submitted will eventually run (if the request resource do not exceed cluster limitations, e.g. asking for runtimes beyond the timelimit). .. note:: This material was based on the `Harvard FAS Research Computing Fairshare Documentation `_.