Replace pylint + autopep8 with ruff#1300
Replace pylint + autopep8 with ruff#1300sbryngelson wants to merge 12 commits intoMFlowCode:masterfrom
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
This PR migrates the Python toolchain’s linting/formatting workflow from pylint/autopep8 to ruff, and then applies ruff-driven cleanup (imports, formatting, some small refactors) across the toolchain plus example/benchmark case files.
Changes:
- Replace pylint + autopep8 usage with ruff (new
ruff.toml, updated bootstrap scripts, updated docs/CLI help text). - Apply broad import ordering / formatting updates across toolchain modules, viz utilities, params tooling, and docs generators.
- Minor functional tweaks (e.g.,
sys.exitinstead ofexit,MFCConfig.__hash__to allow hashing).
Reviewed changes
Copilot reviewed 156 out of 158 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| toolchain/pyproject.toml | Replace pylint/autopep8 dependencies with ruff. |
| toolchain/mfc/viz/viz.py | Remove pylint pragmas; reorder imports. |
| toolchain/mfc/viz/tui.py | Remove pylint pragmas; minor refactors/formatting. |
| toolchain/mfc/viz/test_viz.py | Remove pylint pragmas; import/blank-line normalization. |
| toolchain/mfc/viz/silo_reader.py | Replace pylint pragmas with ruff noqa; formatting. |
| toolchain/mfc/viz/renderer.py | Replace pylint pragmas with ruff noqa; reorder imports. |
| toolchain/mfc/viz/reader.py | Replace pylint pragmas with ruff noqa; formatting. |
| toolchain/mfc/viz/interactive.py | Remove pylint pragmas; reorder imports; formatting. |
| toolchain/mfc/viz/_step_cache.py | Replace pylint pragmas; formatting. |
| toolchain/mfc/validate.py | Use sys.exit; reorder imports/strings. |
| toolchain/mfc/user_guide.py | Reformat/normalize strings; minor readability tweaks. |
| toolchain/mfc/state.py | Import normalization; add MFCConfig.__hash__; ruff-style noqa. |
| toolchain/mfc/sched.py | Reformat; tighten conditionals; import normalization. |
| toolchain/mfc/run/run.py | Import normalization; string normalization; formatting. |
| toolchain/mfc/run/queues.py | Import formatting; string normalization. |
| toolchain/mfc/run/input.py | Import normalization; formatting; small cleanup in clean(). |
| toolchain/mfc/run/case_dicts.py | Remove pylint pragmas; regex string normalization. |
| toolchain/mfc/printer.py | Import formatting. |
| toolchain/mfc/params/validate.py | Ensure registry population via definitions import; reorder imports. |
| toolchain/mfc/params/suggest.py | Import ordering/formatting. |
| toolchain/mfc/params/schema.py | Import ordering; remove pylint pragma. |
| toolchain/mfc/params/registry.py | Import ordering; remove pylint pragma; lazy import cleanup. |
| toolchain/mfc/params/namelist_parser.py | Remove stray blank line. |
| toolchain/mfc/params/generators/json_schema_gen.py | Ensure definitions import for registry population; reorder imports. |
| toolchain/mfc/params/generators/docs_gen.py | Ensure definitions import; reorder imports; remove pylint pragmas. |
| toolchain/mfc/params/generators/init.py | Import ordering. |
| toolchain/mfc/params/descriptions.py | Remove pylint pragmas; formatting. |
| toolchain/mfc/params/definitions.py | Import ordering; remove pylint pragmas. |
| toolchain/mfc/params/ast_analyzer.py | Import ordering; remove broad-except pylint pragmas. |
| toolchain/mfc/params/init.py | Ensure definitions import occurs early; reorder exports. |
| toolchain/mfc/params_tests/test_validate.py | Import ordering. |
| toolchain/mfc/params_tests/test_registry.py | Remove pylint pragma; formatting. |
| toolchain/mfc/params_tests/test_integration.py | Remove pylint pragma; formatting. |
| toolchain/mfc/params_tests/test_definitions.py | Import ordering. |
| toolchain/mfc/params_tests/snapshot.py | Import ordering. |
| toolchain/mfc/params_tests/runner.py | Import ordering; collapse long imports; minor output tweak. |
| toolchain/mfc/params_tests/negative_tests.py | Import ordering. |
| toolchain/mfc/params_tests/mutation_tests.py | Import ordering; minor output tweak. |
| toolchain/mfc/params_tests/inventory.py | Import ordering. |
| toolchain/mfc/params_tests/coverage.py | Import ordering. |
| toolchain/mfc/params_cmd.py | Remove pylint pragmas; import grouping; formatting. |
| toolchain/mfc/packer/tol.py | Import ordering; signature formatting. |
| toolchain/mfc/packer/packer.py | Import ordering. |
| toolchain/mfc/packer/pack.py | Import ordering. |
| toolchain/mfc/packer/errors.py | Import formatting. |
| toolchain/mfc/lock.py | Replace pylint pragmas with ruff noqa; formatting. |
| toolchain/mfc/init.py | String normalization; import ordering; formatting. |
| toolchain/mfc/ide.py | Remove pylint pragmas; string normalization; formatting. |
| toolchain/mfc/generate.py | Use sys.exit; import ordering; formatting. |
| toolchain/mfc/gen_physics_docs.py | Replace pylint pragmas with ruff noqa; formatting. |
| toolchain/mfc/gen_case_constraints_docs.py | Remove pylint pragmas; refactor long literals; formatting. |
| toolchain/mfc/count.py | Import ordering; minor variable renames; formatting. |
| toolchain/mfc/completion.py | Import ordering; minor output formatting. |
| toolchain/mfc/common.py | Import ordering; replace pylint global pragmas with ruff noqa; formatting. |
| toolchain/mfc/cli/test_cli.py | Remove pylint pragma; add tests for MFCConfig.__hash__. |
| toolchain/mfc/cli/schema.py | Remove pylint pragmas; formatting/spacing normalization. |
| toolchain/mfc/cli/docs_gen.py | Import ordering; collapse helper signature formatting. |
| toolchain/mfc/cli/commands.py | Update lint command wording (pylint→ruff); formatting/indent normalization. |
| toolchain/mfc/cli/argparse_gen.py | Remove pylint pragmas; small is not str fix; formatting. |
| toolchain/mfc/cli/init.py | Import ordering. |
| toolchain/mfc/clean.py | Import ordering. |
| toolchain/mfc/case_utils.py | String normalization; formatting. |
| toolchain/mfc/bench.py | Import ordering; minor print formatting; small refactor. |
| toolchain/mfc/args.py | Import ordering; string normalization; formatting. |
| toolchain/main.py | Remove pylint pragmas; import ordering; formatting; simplify some blocks. |
| toolchain/indenter.py | Import ordering; string normalization; formatting. |
| toolchain/bootstrap/lint.sh | Switch linting from pylint to ruff; add ruff auto-fix step; simplify test invocation. |
| toolchain/bootstrap/format.sh | Switch Python formatting to ruff check --fix-only + ruff format; remove autopep8 pipeline. |
| toolchain/bootstrap/format_python.sh | Remove autopep8-based formatter script. |
| ruff.toml | Add ruff configuration (rules, ignores, per-file ignores, isort config). |
| examples/scaling/export.py | Import ordering. |
| examples/scaling/benchmark.py | Import ordering. |
| examples/scaling/analyze.py | Import ordering. |
| examples/nD_perfect_reactor/case.py | Import ordering; minor f-string spacing. |
| examples/nD_perfect_reactor/analyze.py | Import ordering; string normalization; formatting. |
| examples/3D_turb_mixing/case.py | Import ordering. |
| examples/3D_TaylorGreenVortex/case.py | Import ordering. |
| examples/3D_TaylorGreenVortex_analytical/case.py | Import ordering. |
| examples/3D_shockdroplet/case.py | Import ordering. |
| examples/3D_shockdroplet_muscl/case.py | Import ordering. |
| examples/3D_recovering_sphere/case.py | Import ordering. |
| examples/3D_rayleigh_taylor/case.py | Import ordering. |
| examples/3D_rayleigh_taylor_muscl/case.py | Import ordering. |
| examples/3D_phasechange_bubble/case.py | Import ordering. |
| examples/3D_lagrange_shbubcollapse/case.py | Import ordering. |
| examples/3D_lagrange_bubblescreen/case.py | Import ordering. |
| examples/3D_IGR_TaylorGreenVortex/case.py | Import ordering. |
| examples/3D_IGR_TaylorGreenVortex_nvidia/case.py | Import ordering. |
| examples/3D_IGR_jet/case.py | Import ordering. |
| examples/3D_IGR_33jet/case.py | Import ordering. |
| examples/2D_zero_circ_vortex/case.py | Import ordering. |
| examples/2D_zero_circ_vortex_analytical/case.py | Import ordering. |
| examples/2D_whale_bubble_annulus/case.py | Import ordering. |
| examples/2D_viscous/case.py | Import ordering. |
| examples/2D_triple_point/case.py | Import ordering. |
| examples/2D_TaylorGreenVortex/case.py | Import ordering. |
| examples/2D_shocktube_phasechange/case.py | Import ordering. |
| examples/2D_shockdroplet/case.py | Import ordering. |
| examples/2D_shockdroplet_muscl/case.py | Import ordering. |
| examples/2D_shockbubble/case.py | Import ordering. |
| examples/2D_rayleigh_taylor/case.py | Import ordering. |
| examples/2D_phasechange_bubble/case.py | Import ordering. |
| examples/2D_mixing_artificial_Ma/case.py | Import ordering. |
| examples/2D_lungwave/case.py | Import ordering. |
| examples/2D_lungwave_horizontal/case.py | Import ordering. |
| examples/2D_laplace_pressure_jump/case.py | Import ordering. |
| examples/2D_lagrange_bubblescreen/case.py | Import ordering. |
| examples/2D_jet/case.py | Import ordering. |
| examples/2D_isentropicvortex/case.py | Import ordering; remove unnecessary f-string prefixes. |
| examples/2D_isentropicvortex_analytical/case.py | Import ordering; remove unnecessary f-string prefixes. |
| examples/2D_IGR_triple_point/case.py | Import ordering. |
| examples/2D_IGR_2fluid/case.py | Import ordering. |
| examples/2D_GreshoVortex/case.py | Import ordering. |
| examples/2D_axisym_shockwatercavity/case.py | Import ordering. |
| examples/2D_axisym_shockbubble/case.py | Import ordering. |
| examples/2D_acoustic_pulse/case.py | Import ordering. |
| examples/2D_acoustic_pulse_analytical/case.py | Import ordering. |
| examples/2D_5wave_quasi1D/case.py | Import ordering. |
| examples/1D_titarevtorro/case.py | Import ordering. |
| examples/1D_titarevtorro_analytical/case.py | Import ordering. |
| examples/1D_sodshocktube/case.py | Import ordering. |
| examples/1D_sodshocktube_muscl/case.py | Import ordering. |
| examples/1D_sodHypo/case.py | Import ordering. |
| examples/1D_shuosher_wenoz5/case.py | Import ordering. |
| examples/1D_shuosher_wenom5/case.py | Import ordering. |
| examples/1D_shuosher_wenojs5/case.py | Import ordering. |
| examples/1D_shuosher_teno7/case.py | Import ordering. |
| examples/1D_shuosher_teno5/case.py | Import ordering. |
| examples/1D_shuosher_old/case.py | Import ordering. |
| examples/1D_shuosher_analytical/case.py | Import ordering. |
| examples/1D_reactive_shocktube/case.py | Import ordering; minor f-string spacing. |
| examples/1D_qbmm/case.py | Import ordering; remove stray comment marker. |
| examples/1D_poly_bubscreen/case.py | Import ordering; remove stray comment marker. |
| examples/1D_multispecies_diffusion/case.py | Import ordering; minor f-string spacing. |
| examples/1D_laxshocktube/case.py | Import ordering. |
| examples/1D_inert_shocktube/case.py | Import ordering; minor f-string spacing. |
| examples/1D_impact/case.py | Import ordering. |
| examples/1D_hypo_2materials/case.py | Import ordering. |
| examples/1D_exp_tube_phasechange/case.py | Import ordering. |
| examples/1D_exp_bubscreen/case.py | Import ordering. |
| examples/1D_convergence/case.py | Import ordering; simplify constant strings. |
| examples/1D_bubblescreen/case.py | Import ordering; remove stray comment marker. |
| examples/0D_bubblecollapse_adap/case.py | Import ordering. |
| docs/documentation/contributing.md | Update lint gate description for ruff-based linting. |
| CLAUDE.md | Update developer instructions: ./mfc.sh lint now uses ruff. |
| benchmarks/viscous_weno5_sgb_acoustic/case.py | Import ordering. |
| benchmarks/igr/case.py | Import ordering. |
| benchmarks/ibm/case.py | Import ordering. |
| benchmarks/hypo_hll/case.py | Import ordering. |
| benchmarks/5eq_rk3_weno3_hllc/case.py | Import ordering. |
| line-length = 200 | ||
| target-version = "py39" | ||
| exclude = ["_version.py"] | ||
|
|
||
| [lint] | ||
| select = ["E", "F", "W", "I", "PL"] | ||
| ignore = [ | ||
| # Complexity thresholds (project style) | ||
| "PLR0911", # too-many-return-statements | ||
| "PLR0912", # too-many-branches | ||
| "PLR0913", # too-many-arguments | ||
| "PLR0915", # too-many-statements | ||
| "PLR2004", # magic-value-comparison | ||
| # Import patterns (project style) | ||
| "PLC0415", # import-outside-toplevel | ||
| # Global variable reads (module-level singleton pattern) | ||
| "PLW0602", # global-variable-not-assigned | ||
| ] | ||
|
|
||
| [lint.per-file-ignores] | ||
| "examples/**/*.py" = ["F401", "F403", "F405", "F811", "F821", "E402", "E722", "E741"] | ||
| "benchmarks/*/case.py" = ["F401", "F403", "F405", "F811", "F821", "E402", "E741"] | ||
|
|
||
| [lint.isort] | ||
| known-first-party = ["mfc"] |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| # Global variable reads (module-level singleton pattern) | ||
| "PLW0602", # global-variable-not-assigned | ||
| ] |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
toolchain/bootstrap/format.sh
Outdated
| if ! find $SEARCH_PATHS -type f 2>/dev/null | grep -E '\.(py)$' \ | ||
| | xargs --no-run-if-empty -L 1 -P ${JOBS:-1} $SHELL toolchain/bootstrap/format_python.sh; then | ||
| # Format Python files with ruff (auto-fix lint issues, then format) | ||
| ruff check --fix-only $SEARCH_PATHS |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
toolchain/bootstrap/format.sh
Outdated
| # Format Python files with ruff (auto-fix lint issues, then format) | ||
| ruff check --fix-only $SEARCH_PATHS | ||
| if ! ruff format $SEARCH_PATHS; then |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
- Replace autopep8 with ruff format, pylint with ruff check - Add ruff.toml config at repo root (E, F, W, I, PL rule sets) - Enable ruff check --fix in format.sh and lint.sh for safe auto-fixes - Remove all ~229 inline pylint: disable comments - Fix code violations: E731 (lambda), E741 (ambiguous var), E721 (type comparison), PLW1641 (hash), PLR1722 (sys.exit), F541 (f-string), E501 (line length), E702 (semicolons), PLW2901 (loop var overwrite) - Add ~12 noqa comments for intentional global statements (PLW0603/PLW0602) - Remove 8 zero-violation rules from ignore list to keep config minimal - Fix lint.sh symlink path resolution breaking ruff config discovery - Replace pylint and autopep8 deps with ruff in pyproject.toml Resolves MFlowCode#1281 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…-fix - Add PLW0602 (global-variable-not-assigned) to ruff.toml ignore list instead of scattering noqa comments across ~19 call sites - Change ruff check --fix to keep stderr visible (> /dev/null || true instead of > /dev/null 2>&1 || true) so config/invocation errors are not silently swallowed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use dataclasses.fields() instead of dataclasses.asdict() in MFCConfig.__hash__ to avoid issues with nested types and sorted() - Delete format_python.sh which is no longer referenced by anything Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use ruff check --fix-only instead of --fix || true so genuine ruff failures propagate while still tolerating unfixable violations - Update CLAUDE.md: Pylint -> Ruff in lint command description - Add 4 unit tests for MFCConfig.__hash__ (hash/eq contract, set/dict usage) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wrap ruff check --fix-only in error checking in format.sh so a missing or broken ruff installation does not silently succeed. Add isinstance guard to MFCConfig.__eq__ so comparisons with non-MFCConfig objects return NotImplemented instead of raising AttributeError. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Run indenter.py on all files in a single Python process instead of spawning 84 separate processes. Move convergence loop from per-file (format_file.sh) to top-level (format.sh) with md5sum comparison. Delete the now-unused format_file.sh wrapper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Run format first (modifies files), then lint, spelling, source lint, and doc refs in parallel. Skip slow matplotlib rendering tests during precheck via MFC_SKIP_RENDER_TESTS=1 (CI still runs the full suite). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Restore specific error messages (raw directives, double precision, junk patterns) lost during precheck parallelization. Clarify that --fix-only in format.sh only errors if ruff itself fails to run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude Code ReviewHead SHA: 7da4061 Files changed: 165 (selected key files below)
Summary
Findings1. # ruff.toml, line 17
"PLW0602", # global-variable-not-assigned
2. # format.sh, approx. line 50
FORTRAN_FILES=$(find $FORTRAN_DIRS -type f 2>/dev/null | ...)
3. # precheck.sh, approx. line 138
echo -e "$SOURCE_MSGS" | while read -r msg; do
[ -n "$msg" ] && error "$msg"
doneMessages are appended with 4. if ! ruff check --fix-only $PYTHON_DIRS; then
error "ruff failed to run. Check your ruff installation."
exit 1
fi
5. # examples/2D_IGR_2fluid/case.py, line 11 (post-patch)
l = 1
eps = 1e-6
No issues with
🤖 Generated with Claude Code |
Claude Code ReviewHead SHA: Files changed: 165 (toolchain infrastructure, 80+ example/benchmark case files, docs) Key files reviewed:
Summary
Findings1. Minor: "PLW0602", # global-variable-not-assigned
2. Fortran convergence check hashes all files jointly, not per-file old_hash=d41d8cd98f00b204e9800998ecf8427e -
# ... run indenter + fprettify on all files ...
new_hash=d41d8cd98f00b204e9800998ecf8427e -The old 3. FORTRAN_FILES=When using custom 4. 5. Minor observations (no action required)
Overall this is a high-quality, well-scoped migration. The mechanical changes (import ordering, quote normalisation, |
Store per-file md5 hashes in build/.cache/format/ after formatting. On subsequent runs, only re-format files whose hash changed. Combined with parallel precheck, total precommit time drops from 35s to 13s. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude Code ReviewHead SHA: Key files:
Summary
Findings1. The Fortran format cache key is derived from: cache_key="$f"Two files with paths that differ only in slash vs. underscore (e.g., 2. FORTRAN_FILES=$(find $FORTRAN_DIRS ...)When 3. # Skip slow rendering tests (matplotlib/imageio) during local precheck.
# CI runs the full suite via ./mfc.sh lint without this variable.
export MFC_SKIP_RENDER_TESTS=1This means 4. "PLW0602", # global-variable-not-assignedThis suppresses the rule for the entire codebase, not just the files with the module-level singleton pattern. A Minor / Non-blocking
Overall this is a clean, well-scoped modernization. The findings above are mostly low-severity or style; none should block merging. CI green would confirm correctness. 🤖 Generated with Claude Code |
Collapse 40,316 indexed parameter entries (e.g., patch_ib(1)%radius through patch_ib(1000)%radius) into 241 regex patterns using JSON schema patternProperties. Schema compiles in 0.1s instead of 7s. Total precommit hook time drops from ~13s to ~8-10s. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Claude Code ReviewHead SHA: ee73c1f Files changed: 165 (7,267 additions / 6,599 deletions) Selected files reviewed:
Summary
Findings1. 2. cache_key=$(echo "$f" | tr '/' '_')Paths like 3. 4. # --fix-only exits 0 even when unfixable violations remain
if ! ruff check --fix-only $PYTHON_DIRS; then
error "ruff failed to run. Check your ruff installation."The comment is correct, but the error message says "ruff failed to run" — it could also exit non-zero on a syntax error in a Python file. A more accurate message would be "ruff check --fix-only exited non-zero (syntax error or ruff not installed)." 5. 6. No blocking issues found. The migration is clean and the
|
Summary
Resolves #1281 — replaces pylint + autopep8 with ruff for all Python linting and formatting. Ruff is ~200-500x faster (under 0.2s vs 30-90s for the old pylint + autopep8 pipeline across 226 Python files).
What changed
ruff format. Per-filefind | xargsreplaced with direct directory-levelruff formatcalls.ruff checkwith rule sets E (pycodestyle), F (pyflakes), W (warnings), I (isort), PL (pylint-equivalent).ruff check --fixruns automatically in both./mfc.sh formatand./mfc.sh lintto fix safe issues (import sorting, unused f-prefixes, etc.) before checking.ruff.tomlat repo root (centralized, replaces scattered pylint/autopep8 config). Only 6 rules ignored — all high-count structural/style rules (too-many-branches, magic-value-comparison, etc.).# pylint: disable=comments. Added ~12 targeted# noqa:comments for intentionalglobalstatements.l→line!=→is notfor type comparison__hash__toMFCConfigdataclassexit()→sys.exit()fprefix from 40 f-strings without placeholderspylintandautopep8withruffinpyproject.toml.lint.shused$(pwd)/toolchain/which resolved symlinks and broke ruff config discovery; switched to relative paths.Config details (
ruff.toml)Per-file ignores for
examples/andbenchmarks/(permissive for user-facing case files).Test plan
./mfc.sh lintpasses (ruff check clean + 165 unit tests)./mfc.sh formatruns without errors./mfc.sh precheckpasses all 5 checks (formatting, spelling, toolchain lint, source lint, doc references)🤖 Generated with Claude Code