Skip to content

_slurm

JobRunnerConfigSLURM

Bases: BaseModel

Common SLURM configuration.

Note: this is a common and abstract class, which gets transformed into more specific configuration objects during job execution.

Valid JSON example 

{
  "default_slurm_config": {
      "partition": "main",
      "cpus_per_task": 1
  },
  "gpu_slurm_config": {
      "partition": "gpu",
      "extra_lines": ["#SBATCH --gres=gpu:v100:1"]
  },
  "batching_config": {
      "target_cpus_per_job": 1,
      "max_cpus_per_job": 1,
      "target_mem_per_job": 200,
      "max_mem_per_job": 500,
      "target_num_jobs": 2,
      "max_num_jobs": 4
  },
  "user_local_exports": {
      "CELLPOSE_LOCAL_MODELS_PATH": "CELLPOSE_LOCAL_MODELS_PATH",
      "NAPARI_CONFIG": "napari_config.json"
  }
}

Attributes:

Name Type Description
default_slurm_config _SlurmConfigSet

Common default options for all tasks.
gpu_slurm_config _SlurmConfigSet | None

Default configuration for all GPU tasks.
batching_config _BatchingConfigSet

Configuration of the batching strategy.
user_local_exports DictStrStr | None

Key-value pairs to be included as export-ed variables in SLURM submission script, after prepending values with the user's cache directory.
Source code in fractal_server/runner/config/_slurm.py 
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
class JobRunnerConfigSLURM(BaseModel):
    """
    Common SLURM configuration.

    Note: this is a common and abstract class, which gets transformed into
    more specific configuration objects during job execution.

    Valid JSON example
    ```JSON
    {
      "default_slurm_config": {
          "partition": "main",
          "cpus_per_task": 1
      },
      "gpu_slurm_config": {
          "partition": "gpu",
          "extra_lines": ["#SBATCH --gres=gpu:v100:1"]
      },
      "batching_config": {
          "target_cpus_per_job": 1,
          "max_cpus_per_job": 1,
          "target_mem_per_job": 200,
          "max_mem_per_job": 500,
          "target_num_jobs": 2,
          "max_num_jobs": 4
      },
      "user_local_exports": {
          "CELLPOSE_LOCAL_MODELS_PATH": "CELLPOSE_LOCAL_MODELS_PATH",
          "NAPARI_CONFIG": "napari_config.json"
      }
    }
    ```

    Attributes:
        default_slurm_config:
            Common default options for all tasks.
        gpu_slurm_config:
            Default configuration for all GPU tasks.
        batching_config:
            Configuration of the batching strategy.
        user_local_exports:
            Key-value pairs to be included as `export`-ed variables in SLURM
            submission script, after prepending values with the user's cache
            directory.
    """

    model_config = ConfigDict(extra="forbid")

    default_slurm_config: _SlurmConfigSet
    gpu_slurm_config: _SlurmConfigSet | None = None
    batching_config: _BatchingConfigSet
    user_local_exports: DictStrStr | None = None

_BatchingConfigSet

Bases: BaseModel

Options to configure the batching strategy (that is, how to combine several tasks in a single SLURM job).

Attributes:

Name Type Description
target_cpus_per_job PositiveInt
max_cpus_per_job PositiveInt
target_mem_per_job MemMBType

(see _parse_mem_value for details on allowed values)
max_mem_per_job MemMBType

(see _parse_mem_value for details on allowed values)
target_num_jobs PositiveInt
max_num_jobs PositiveInt
Source code in fractal_server/runner/config/_slurm.py 
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
class _BatchingConfigSet(BaseModel):
    """
    Options to configure the batching strategy (that is, how to combine
    several tasks in a single SLURM job).

    Attributes:
        target_cpus_per_job:
        max_cpus_per_job:
        target_mem_per_job:
            (see `_parse_mem_value` for details on allowed values)
        max_mem_per_job:
            (see `_parse_mem_value` for details on allowed values)
        target_num_jobs:
        max_num_jobs:
    """

    model_config = ConfigDict(extra="forbid")

    target_num_jobs: PositiveInt
    max_num_jobs: PositiveInt
    target_cpus_per_job: PositiveInt
    max_cpus_per_job: PositiveInt
    target_mem_per_job: MemMBType
    max_mem_per_job: MemMBType

_SlurmConfigSet

Bases: BaseModel

Options for the default or gpu SLURM config.

Attributes:

Name Type Description
partition NonEmptyStr | None
cpus_per_task PositiveInt | None
mem MemMBType | None

See _parse_mem_value for details on allowed values.
constraint NonEmptyStr | None
gres NonEmptyStr | None
time NonEmptyStr | None
exclude NonEmptyStr | None
nodelist NonEmptyStr | None
account NonEmptyStr | None
extra_lines list[NonEmptyStr] | None
Source code in fractal_server/runner/config/_slurm.py 
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
class _SlurmConfigSet(BaseModel):
    """
    Options for the default or gpu SLURM config.

    Attributes:
        partition:
        cpus_per_task:
        mem:
            See `_parse_mem_value` for details on allowed values.
        constraint:
        gres:
        time:
        exclude:
        nodelist:
        account:
        extra_lines:
    """

    model_config = ConfigDict(extra="forbid")

    partition: NonEmptyStr | None = None
    cpus_per_task: PositiveInt | None = None
    mem: MemMBType | None = None
    constraint: NonEmptyStr | None = None
    gres: NonEmptyStr | None = None
    exclude: NonEmptyStr | None = None
    nodelist: NonEmptyStr | None = None
    time: NonEmptyStr | None = None
    account: NonEmptyStr | None = None
    extra_lines: list[NonEmptyStr] | None = None
    gpus: NonEmptyStr | None = None