Development
The development of Fractal Server takes place on the fractal-server Github repository. To ask questions or to inform us of a bug or unexpected behavior, please feel free to open an issue.
To contribute code, please fork the repository and submit a pull request.
Set up the development environment¶
Install poetry¶
Fractal uses poetry to manage the development environment and dependencies, and to streamline the build and release operations; at least version 1.3 is recommended.
A simple way to install poetry
is
pipx install poetry==1.8.2`
Clone repository¶
You can clone the fractal-server
repository via
git clone https://github.com/fractal-analytics-platform/fractal-server.git
Install package¶
Running
poetry install
poetry run
, as in poetry run fractalctl start
.
It may sometimes be useful to use a different Python interpreter from the one installed in your system. To this end we suggest using pyenv. In the project folder, invoking
pyenv local <version>
poetry env use <version>
Update database schema¶
Whenever the models are modified (either in
app/models
or in
app/schemas
), you should
update them via a migration. To check whether this is needed, run
poetry run alembic check
If needed, the simplest procedure is to use alembic --autogenerate
to create
an incremental migration script, as in
$ export SQLITE_PATH=some-test.db
$ rm some-test.db
$ poetry run fractalctl set-db --skip-init-data
$ poetry run alembic revision --autogenerate -m "Some migration message"
# UserWarning: SQLite is partially supported but discouraged in production environment.SQLite offers partial support for ForeignKey constraints. As such, consistency of the database cannot be guaranteed.
# INFO [alembic.runtime.migration] Context impl SQLiteImpl.
# INFO [alembic.runtime.migration] Will assume non-transactional DDL.
# INFO [alembic.autogenerate.compare] Detected added column 'task.x'
# Generating /some/path/fractal_server/migrations/versions/155de544c342_.py ... done
Release¶
- Checkout to branch
main
. - Check that the current HEAD of the
main
branch passes all the tests (note: make sure that you are using the poetry-installed local package). - Update the
CHANGELOG.md
file (e.g. remove(unreleased)
from the upcoming version). - If you have modified the models, then you must also create a new migration script (note: in principle the CI will fail if you forget this step).
-
Use one of the following
to test updating the version bump.poetry run bumpver update --tag-num --tag-commit --commit --dry poetry run bumpver update --patch --tag-commit --commit --dry poetry run bumpver update --minor --tag-commit --commit --dry poetry run bumpver update --set-version X.Y.Z --tag-commit --commit --dry
-
If the previous step looks good, remove
--dry
and re-run to actually bump the version, commit and push the changes. - Approve (or have approved) the new version at Publish package to PyPI.
- After the release: If the release was a stable one (e.g.
X.Y.Z
, notX.Y.Za1
orX.Y.Zrc2
), movefractal_server/data_migrations/X_Y_Z.py
tofractal_server/data_migrations/old
.
Run tests¶
Unit and integration testing of Fractal Server uses the pytest testing framework.
To test the SLURM backend, we use a custom version of a Docker local SLURM cluster. The pytest plugin pytest-docker is then used to spin up the Docker containers for the duration of the tests.
Important: this requires docker being installed on the development system,
and the current user being in the docker
group. A simple check for this
requirement is to run a command like docker ps
, and verify that it does not
raise any permission-related error. Note that also docker-compose
must be
available, but this package is installed as a dependency of pytest-docker
(when it is installed with the extra docker-compose-v1
, as in the case of
Fractal Server).
If you installed the development dependencies, you may run the test suite by invoking
poetry run pytest
fractal-server
repository. It is sometimes
useful to specify additional arguments, e.g.
poetry run pytest -s -vvv --log-cli-level info --full-trace
Tests are also run as part of GitHub Actions Continuous
Integration
for the fractal-server
repository.
Documentation¶
The documentations is built with mkdocs and the Material theme. Whenever possible, docstrings should be formatted as in the Google Python Style Guide.
To build the documentation
- Setup a python environment and install the requirements from
docs/doc-requirements.txt
. - Run
and browse the documentation at
poetry run mkdocs serve --config-file mkdocs.yml
http://127.0.0.1:8000
.