scTimeBench was tested and supported using Python 3.10. If any other version that is 3.10+ does not work when using the benchmark, please submit an issue to this GitHub.
Important Note: this setup is needed twice. Once for the user to run the benchmark metrics, and the second time for the method itself needing to read from scTimeBench.method_utils.method_runner and other important shared constants. In short, this means we need to install scTimeBench into two separate virtual environments:
- Your normal pip installation where you'll be running the benchmark from. This will require the extra "[benchmark]" group installation (pip) or the extra group installation
--extra benchmark(uv). - For each method's virtual environment, you need to install the scTimeBench.
Due to external dependencies and a more complex setup, we have decided to package everything under uv (see: https://github.com/astral-sh/uv). To start with, you need to get all the necessary extern dependencies, which can be done either by running:
git submodule update --init extern/
If you wish to benchmark across all methods, feel free to clone the submodules for all the methods as well with:
git submodule update --init
Then, install uv and run the following:
uv sync --extra benchmark
If you're using uv under a method's virtual environment, either the pip installation or the following will suffice:
uv sync
If the external dependencies such as pypsupertime or sceptic are not used (which they are not used by default), you can install using pip as follows:
pip install -e ".[benchmark]"
to run the benchmark. For your own method, simply install without the extra benchmarking requirements with
pip install -e .
There are extra dependencies that can be found under pyproject.toml.
scTimeBench is controlled by a central configuration file which determines which datasets, methods, and metrics to run. An example of this can be found under configs/scNODE/gex.yaml.
configs/: possible yaml config files to use as a starting pointextern/: external packages that required edits for compatability such as pyPsupertime and Sceptic.methods/: the different methods that are possible to use, including defined submodules. Add your own methodology here.src/: where the scTimeBench package lies. Seesrc/ReadMe.mdfor more documentation on the modules that exist there.test/: unit tests for each method, dataset, metric, and other important modules.
The entrypoint for the benchmark is defined as scTimeBench. Run scTimeBench --help for more details, or refer to src/scTimeBench/config.py and the documentation.
Run the package with:
scTimeBench --config configs/scNODE/gex.yaml
For a full running example using scNODE, refer to our example Jupyter Notebook.
If you want to contribute, please install the dev environments with:
uv sync --extra dev --extra benchmark
or
pip install -e ".[dev, benchmark]"
To enable the autoformatting, please run:
pre-commit install
before committing.
Follow our example tutorials on adding new methods, datasets, and metrics in our documentation here: TODO-ADD-THIS.
If your change heavily modifies the architecture, please run the necessary tests under the test/ environment using pytest. Read more on the different available tests under test/ReadMe.md. See more information on the pytest documentation: https://docs.pytest.org/en/stable/. A useful flag is -s to view the entire output of the test.
@article {scTimeBench,
author = {Osakwe, Adrien and Huang, Eric Haoran and Li, Yue},
title = {scTimeBench: A streamlined benchmarking platform for single-cell time-series analysis},
elocation-id = {2026.03.16.712069},
year = {2026},
doi = {10.64898/2026.03.16.712069},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2026/03/18/2026.03.16.712069},
eprint = {https://www.biorxiv.org/content/early/2026/03/18/2026.03.16.712069.full.pdf},
journal = {bioRxiv}
}