Skip to content

fix(sdk): skip evaluation when no controls apply#124

Merged
lan17 merged 3 commits intomainfrom
codex/fix-issue-123-repro-test
Mar 16, 2026
Merged

fix(sdk): skip evaluation when no controls apply#124
lan17 merged 3 commits intomainfrom
codex/fix-issue-123-repro-test

Conversation

@lan17
Copy link
Contributor

@lan17 lan17 commented Mar 13, 2026

Summary

  • Make the SDK evaluation path a true no-op when no enabled control applies to the current step/stage.
  • Prefilter both local (execution="sdk") and server (execution="server") controls using the same applicability rules before any evaluation work starts.
  • Preserve the existing fail-open behavior for malformed server controls by still deferring those to the server.
  • Add regression coverage for disabled and non-matching controls across both local and server execution.

Scope

  • User-facing/API changes:
    • If no control applies to a step invocation, the SDK does nothing: no local control engine execution and no POST /api/v1/evaluation request.
  • Internal changes:
    • Reuse the SDK control engine's applicability logic to determine whether local or server evaluation should run at all.
    • Extend local-evaluation tests to cover disabled, stage-mismatched, step-type-mismatched, step-name-mismatched, and regex-mismatched controls for both execution modes.
  • Out of scope:
    • Server-side evaluation behavior when applicable controls do exist.
    • Broader control loading or caching changes.

Risk and Rollout

  • Risk level: low
  • Rollback plan:
    • Revert this PR to restore the previous behavior where the SDK entered evaluation as long as local or server controls existed for the agent.

Testing

  • Added or updated automated tests
  • Ran make check (or explained why not)
    • Ran make test, plus targeted ruff and mypy on the touched SDK files.
  • Manually verified behavior

Checklist

@lan17 lan17 changed the title fix: skip server evaluation when no controls apply fix(sdk): skip server evaluation when no controls apply Mar 13, 2026
@lan17 lan17 changed the title fix(sdk): skip server evaluation when no controls apply fix(sdk): skip evaluation when no controls apply Mar 13, 2026
@codecov
Copy link

codecov bot commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Contributor

@namrataghadi-galileo namrataghadi-galileo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@lan17 lan17 merged commit d2cd9f9 into main Mar 16, 2026
6 checks passed
@lan17 lan17 deleted the codex/fix-issue-123-repro-test branch March 16, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip evaluation entirely when no controls apply to the current step

2 participants