Skip to content

fix(python): avoid FileAlreadyExistsException in venv cleanup#373

Open
ruromero wants to merge 1 commit intomainfrom
fix/venv-cleanup-race
Open

fix(python): avoid FileAlreadyExistsException in venv cleanup#373
ruromero wants to merge 1 commit intomainfrom
fix/venv-cleanup-race

Conversation

@ruromero
Copy link
Collaborator

@ruromero ruromero commented Mar 26, 2026

Summary

  • Remove redundant Files.createFile() before Files.write() in PythonControllerVirtualEnv
  • Files.write() already creates the file if it doesn't exist
  • The createFile() call caused FileAlreadyExistsException when a previous analysis failed mid-way and left the requirements.txt behind in /tmp/trustify_da_env/

Test plan

  • All existing Python provider tests pass (35 tests, 0 failures)
  • Manual: trigger a Python analysis that fails (e.g. unresolvable dep), then trigger another — should no longer throw FileAlreadyExistsException

Implements TC-3894

🤖 Generated with Claude Code

Remove redundant Files.createFile() before Files.write() in
PythonControllerVirtualEnv. Files.write() already creates the file
if it doesn't exist. The createFile() call caused
FileAlreadyExistsException when a previous analysis failed mid-way
and left the requirements.txt behind in /tmp/trustify_da_env/.

Implements TC-3894

Assisted-by: Claude Code
@qodo-code-review
Copy link
Contributor

ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan

Review Summary by Qodo

Remove redundant file creation in Python venv cleanup

🐞 Bug fix

Grey Divider

Walkthroughs

Description
• Remove redundant Files.createFile() call before Files.write()Files.write() already creates file if missing
• Fixes FileAlreadyExistsException when retrying failed analyses
• Prevents stale requirements.txt from causing race conditions
Diagram
flowchart LR
  A["Previous: createFile + write"] -->|"FileAlreadyExistsException"| B["Failed on retry"]
  C["Fixed: write only"] -->|"Auto-creates file"| D["Works on retry"]
Loading

Grey Divider

File Changes

1. src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java 🐞 Bug fix +0/-1

Remove redundant file creation in venv cleanup

• Removed redundant Files.createFile(envRequirements) call
• Files.write() automatically creates the file if it doesn't exist
• Eliminates FileAlreadyExistsException when stale requirements.txt remains from failed analysis
• Simplifies code by removing unnecessary operation

src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Mar 26, 2026

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (0) 📐 Spec deviations (0)

Grey Divider


Action required

1. TOCTOU write in /tmp 🐞 Bug ⛨ Security
Description
PythonControllerVirtualEnv.cleanEnvironment() deletes requirements.txt, runs a non-trivial external
command (pip freeze), then writes requirements.txt; after removing Files.createFile(), a file
recreated during that window will now be overwritten instead of failing fast. Because the venv
directory is a fixed /tmp path and prepareEnvironment() trusts any pre-existing path, this enables
symlink/race-style clobbering and can corrupt cleanup behavior.
Code

src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[R91-96]

        Files.deleteIfExists(envRequirements);
        String freezeOutput =
            Operations.runProcessGetOutput(pythonEnvironmentDir, pipBinaryLocation, "freeze");
-        Files.createFile(envRequirements);
        Files.write(envRequirements, freezeOutput.getBytes());
        Operations.runProcessGetOutput(
            pythonEnvironmentDir, pipBinaryLocation, "uninstall", "-y", "-r", "requirements.txt");
Evidence
The virtualenv directory is hard-coded to /tmp/trustify_da_env and is only created if it doesn’t
already exist (no validation of what already exists at that path). In cleanEnvironment(false), the
code deletes requirements.txt, runs pip freeze (time gap), then writes requirements.txt; removing
the explicit createFile() step eliminates the prior behavior where a recreated path would have
triggered FileAlreadyExistsException, leaving a TOCTOU window for another process/user to recreate
the path before the write.

src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[29-54]
src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[71-101]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`cleanEnvironment(false)` deletes `/tmp/trustify_da_env/requirements.txt`, runs `pip freeze`, then writes `requirements.txt`. With the removal of `Files.createFile()`, a path recreated between delete and write will be overwritten, creating a TOCTOU window on a `/tmp`-based directory.

### Issue Context
- The environment directory is hard-coded under `/tmp` and `prepareEnvironment()` trusts any existing path at that location.
- `pip freeze` introduces a time gap between delete and write.

### Fix Focus Areas
- src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[39-43]
- src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[89-96]

### Proposed fix
1. Write freeze output to a temp file in `pythonEnvironmentDir` (e.g., `Files.createTempFile(pythonEnvironmentDir, "requirements", ".txt")`).
2. Atomically replace the final `requirements.txt` using `Files.move(temp, envRequirements, ATOMIC_MOVE, REPLACE_EXISTING)` (fallback to non-atomic move if ATOMIC_MOVE unsupported).
3. (Optional hardening) Validate `pythonEnvironmentDir` is a real directory (not a symlink) before using it, and consider creating a per-run temp venv directory instead of a fixed `/tmp/trustify_da_env` path.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Global venv path shared 🐞 Bug ⛯ Reliability
Description
PythonControllerVirtualEnv uses a single fixed directory (/tmp/trustify_da_env) for all virtualenv
operations, so overlapping runs can interfere by reading/writing the same requirements.txt and
uninstalling packages from the same environment. The changed write behavior makes such overlap more
likely to proceed (rather than erroring) and can lead to cross-run contamination.
Code

src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[R91-96]

        Files.deleteIfExists(envRequirements);
        String freezeOutput =
            Operations.runProcessGetOutput(pythonEnvironmentDir, pipBinaryLocation, "freeze");
-        Files.createFile(envRequirements);
        Files.write(envRequirements, freezeOutput.getBytes());
        Operations.runProcessGetOutput(
            pythonEnvironmentDir, pipBinaryLocation, "uninstall", "-y", "-r", "requirements.txt");
Evidence
The controller hard-codes pythonEnvironmentDir to /tmp/trustify_da_env, and the cleanup flow
writes a shared requirements.txt then executes pip uninstall -r requirements.txt inside that
same directory. Separately, PythonProvider creates a new controller instance when not injected,
meaning multiple call sites can end up operating on the same global directory (even if via distinct
controller objects).

src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[29-34]
src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[89-96]
src/main/java/io/github/guacsec/trustifyda/providers/PythonProvider.java[228-257]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
A single fixed venv directory (`/tmp/trustify_da_env`) and shared `requirements.txt` are used for all virtualenv runs. Any overlapping runs can step on each other’s environment state and cleanup (freeze/uninstall).

### Issue Context
This PR reduces one failure mode (`FileAlreadyExistsException`), but the underlying shared-state design remains and can manifest as non-deterministic cleanup/analysis outcomes when runs overlap.

### Fix Focus Areas
- src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[29-34]
- src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[71-101]
- src/main/java/io/github/guacsec/trustifyda/providers/PythonProvider.java[228-257]

### Proposed fix options
- Prefer per-analysis isolation: create a unique temp directory for each venv (e.g., `Files.createTempDirectory("trustify_da_env_")`) and clean it up afterwards.
- Or serialize access: add a lock (e.g., a filesystem lock file in the env dir) around prepare/install/cleanup so only one run mutates the shared venv at a time.
- Additionally, consider caching a single controller per provider instance if shared venv semantics are intended (and still guard with locking).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

@github-actions
Copy link
Contributor

Test Results

379 tests   379 ✅  1m 46s ⏱️
 25 suites    0 💤
 25 files      0 ❌

Results for commit 420c7e3.

Comment on lines 91 to 96
Files.deleteIfExists(envRequirements);
String freezeOutput =
Operations.runProcessGetOutput(pythonEnvironmentDir, pipBinaryLocation, "freeze");
Files.createFile(envRequirements);
Files.write(envRequirements, freezeOutput.getBytes());
Operations.runProcessGetOutput(
pythonEnvironmentDir, pipBinaryLocation, "uninstall", "-y", "-r", "requirements.txt");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Toctou write in /tmp 🐞 Bug ⛨ Security

PythonControllerVirtualEnv.cleanEnvironment() deletes requirements.txt, runs a non-trivial external
command (pip freeze), then writes requirements.txt; after removing Files.createFile(), a file
recreated during that window will now be overwritten instead of failing fast. Because the venv
directory is a fixed /tmp path and prepareEnvironment() trusts any pre-existing path, this enables
symlink/race-style clobbering and can corrupt cleanup behavior.
Agent Prompt
### Issue description
`cleanEnvironment(false)` deletes `/tmp/trustify_da_env/requirements.txt`, runs `pip freeze`, then writes `requirements.txt`. With the removal of `Files.createFile()`, a path recreated between delete and write will be overwritten, creating a TOCTOU window on a `/tmp`-based directory.

### Issue Context
- The environment directory is hard-coded under `/tmp` and `prepareEnvironment()` trusts any existing path at that location.
- `pip freeze` introduces a time gap between delete and write.

### Fix Focus Areas
- src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[39-43]
- src/main/java/io/github/guacsec/trustifyda/utils/PythonControllerVirtualEnv.java[89-96]

### Proposed fix
1. Write freeze output to a temp file in `pythonEnvironmentDir` (e.g., `Files.createTempFile(pythonEnvironmentDir, "requirements", ".txt")`).
2. Atomically replace the final `requirements.txt` using `Files.move(temp, envRequirements, ATOMIC_MOVE, REPLACE_EXISTING)` (fallback to non-atomic move if ATOMIC_MOVE unsupported).
3. (Optional hardening) Validate `pythonEnvironmentDir` is a real directory (not a symlink) before using it, and consider creating a per-run temp venv directory instead of a fixed `/tmp/trustify_da_env` path.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@qodo-code-review
Copy link
Contributor

ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: call-shared / integration-tests (windows-latest, gradle-groovy)

Failed stage: Run Integration Tests [❌]

Failed test name: ""

Failure summary:

The action failed during the integration test/validation step for stack analysis results.
- The
runner executed python -u shared-scripts/run_tests_no_runtime.py java artifact gradle-groovy and
later validated the stack analysis output.
- Validation reported a mismatch in remediation counts
for stack_analysis with provider rhtpa and source osv-github: expected 14 remediations but the
output contained 2 (❌ stack_analysis provider rhtpa source osv-github remediations mismatch:
expected 14, got 2 at log line ~15244).
- Because of this mismatch, Stack analysis validation failed
and the process exited with code 1 (log lines ~15245-15246).

Relevant error logs:
1:  ##[group]Runner Image Provisioner
2:  Hosted Compute Agent
...

281:  shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
282:  env:
283:  TRUSTIFY_DA_DEV_MODE: true
284:  TRUSTIFY_DA_BACKEND_URL: https://exhort.stage.devshift.net
285:  PYTHONIOENCODING: utf-8
286:  PYTHONUNBUFFERED: 1
287:  pythonLocation: C:\hostedtoolcache\windows\Python\3.11.9\x64
288:  PKG_CONFIG_PATH: C:\hostedtoolcache\windows\Python\3.11.9\x64/lib/pkgconfig
289:  Python_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.11.9\x64
290:  Python2_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.11.9\x64
291:  Python3_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.11.9\x64
292:  ##[endgroup]
293:  + python -u shared-scripts/run_tests_no_runtime.py java artifact gradle-groovy
294:  ---
295:  Scenario: No runtime available
296:  Description: It fails when no runtime is available
297:  Manifest: D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\scenarios\gradle-groovy\simple\build.gradle
298:  Expecting failure (no runtime available)
299:  Executing: java -jar D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\artifact\cli.jar component D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\scenarios\gradle-groovy\simple\build.gradle
300:  ✅ Command failed as expected (no runtime available)
301:  Executing: java -jar D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\artifact\cli.jar stack D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\scenarios\gradle-groovy\simple\build.gradle
302:  ✅ Command failed as expected (no runtime available)
303:  Executing: java -jar D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\artifact\cli.jar stack D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\scenarios\gradle-groovy\simple\build.gradle --summary
304:  ✅ Command failed as expected (no runtime available)
305:  Executing: java -jar D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\artifact\cli.jar stack D:\a\trustify-da-java-client\trustify-da-java-client\integration-tests\scenarios\gradle-groovy\simple\build.gradle --html
306:  ✅ Command failed as expected (no runtime available)
307:  ---
...

12508:  "name": "Apache License 2.0",
12509:  "isDeprecated": false,
12510:  "isOsiApproved": true,
12511:  "isFsfLibre": true,
12512:  "category": "PERMISSIVE"
12513:  }
12514:  ],
12515:  "expression": "Apache-2.0",
12516:  "name": "Apache License 2.0",
12517:  "category": "PERMISSIVE",
12518:  "source": "deps.dev",
12519:  "sourceUrl": "https://api.deps.dev"
12520:  }
12521:  ]
12522:  },
12523:  "pkg:maven/com.google.errorprone/error_prone_annotations@2.10.0?scope=compile": {
12524:  "concluded": {
...

15230:  "direct": 4,
15231:  "transitive": 9,
15232:  "total": 13,
15233:  "dependencies": 10,
15234:  "critical": 1,
15235:  "high": 4,
15236:  "medium": 8,
15237:  "low": 0,
15238:  "remediations": 6,
15239:  "recommendations": 119,
15240:  "unscanned": 0
15241:  }
15242:  }
15243:  }
15244:  ❌ stack_analysis provider rhtpa source osv-github remediations mismatch: expected 14, got 2
15245:  ❌ Stack analysis validation failed
15246:  ##[error]Process completed with exit code 1.
15247:  Post job cleanup.

@ruromero ruromero requested a review from soul2zimate March 26, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant