Skip to content

_submit_setup

Submodule to define _slurm_submit_setup, which is also the reference implementation of submit_setup_call in fractal_server.app.runner.v1._common).

_slurm_submit_setup(*, wftask, workflow_dir_local, workflow_dir_remote)

Collect WorfklowTask-specific configuration parameters from different sources, and inject them for execution.

Here goes all the logic for reading attributes from the appropriate sources and transforming them into an appropriate SlurmConfig object (encoding SLURM configuration) and TaskFiles object (with details e.g. about file paths or filename prefixes).

For now, this is the reference implementation for the argument submit_setup_call of fractal_server.app.runner.v1._common.execute_tasks.

Parameters:

Name Type Description Default
wftask WorkflowTask

WorkflowTask for which the configuration is to be assembled

 required
workflow_dir_local Path

Server-owned directory to store all task-execution-related relevant files (inputs, outputs, errors, and all meta files related to the job execution). Note: users cannot write directly to this folder.

 required
workflow_dir_remote Path

User-side directory with the same scope as workflow_dir_local, and where a user can write.

 required

Returns:

Name Type Description
submit_setup_dict dict[str, object]

A dictionary that will be passed on to FractalSlurmExecutor.submit and FractalSlurmExecutor.map, so as to set extra options.
Source code in fractal_server/app/runner/v1/_slurm/_submit_setup.py 
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
def _slurm_submit_setup(
    *,
    wftask: WorkflowTask,
    workflow_dir_local: Path,
    workflow_dir_remote: Path,
) -> dict[str, object]:
    """
    Collect WorfklowTask-specific configuration parameters from different
    sources, and inject them for execution.

    Here goes all the logic for reading attributes from the appropriate sources
    and transforming them into an appropriate `SlurmConfig` object (encoding
    SLURM configuration) and `TaskFiles` object (with details e.g. about file
    paths or filename prefixes).

    For now, this is the reference implementation for the argument
    `submit_setup_call` of
    [fractal_server.app.runner.v1._common.execute_tasks][].

    Arguments:
        wftask:
            WorkflowTask for which the configuration is to be assembled
        workflow_dir_local:
            Server-owned directory to store all task-execution-related relevant
            files (inputs, outputs, errors, and all meta files related to the
            job execution). Note: users cannot write directly to this folder.
        workflow_dir_remote:
            User-side directory with the same scope as `workflow_dir_local`,
            and where a user can write.

    Returns:
        submit_setup_dict:
            A dictionary that will be passed on to
            `FractalSlurmExecutor.submit` and `FractalSlurmExecutor.map`, so
            as to set extra options.
    """

    # Get SlurmConfig object
    slurm_config = get_slurm_config(
        wftask=wftask,
        workflow_dir_local=workflow_dir_local,
        workflow_dir_remote=workflow_dir_remote,
    )

    # Get TaskFiles object
    task_files = get_task_file_paths(
        workflow_dir_local=workflow_dir_local,
        workflow_dir_remote=workflow_dir_remote,
        task_order=wftask.order,
        task_name=wftask.task.name,
    )

    # Prepare and return output dictionary
    submit_setup_dict = dict(
        slurm_config=slurm_config,
        task_files=task_files,
    )
    return submit_setup_dict