ethrpc: auto-split FilterLogs on block range / result size errors#204
ethrpc: auto-split FilterLogs on block range / result size errors#204
Conversation
Add automatic range splitting to Provider.FilterLogs() so callers get transparent retry when nodes reject eth_getLogs requests for being too large (block range exceeded, too many results, etc.). Uses an AIMD strategy (multiplicative decrease /1.5 on error, additive increase +10% on success) to discover the node's true limit. Once calibrated, the known-good range is reused directly with no further probing. New options: WithFilterLogsMaxRange(n), WithAutoFilterLogsRange(bool). Feature is disabled by default — zero behavioral change unless opted in.
Reject requests with a total block range exceeding 10,000 when auto-split is enabled, to prevent hammering nodes with hundreds of sub-queries. Only applies when the feature is opted into — disabled mode is a direct pass-through as before.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 223520ca7b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if batchRange < 1 { | ||
| batchRange = 1 | ||
| } | ||
|
|
||
| continue // retry the same cursor with a smaller range |
There was a problem hiding this comment.
If a "too much data" error occurs when batchRange has already reached 1, the code clamps it back to 1 and immediately retries the exact same sub-query, so cursor never advances and FilterLogs can loop forever until the context is canceled. This is reachable when one block alone exceeds provider limits (or when the error matcher is overly broad), so auto-split can hang callers instead of returning an actionable error.
Useful? React with 👍 / 👎.
| return nil, err | ||
| } | ||
|
|
||
| end := cursor + batchRange |
There was a problem hiding this comment.
WithFilterLogsMaxRange is documented as "at most maxRange blocks", but the split window is computed as end = cursor + batchRange and both endpoints are included, so each request spans batchRange+1 blocks. For example, maxRange=1 still queries two blocks at a time, which can violate strict upstream limits and trigger avoidable retries/errors.
Useful? React with 👍 / 👎.
1 participant
Add automatic range splitting to Provider.FilterLogs() so callers get transparent retry when nodes reject eth_getLogs requests for being too large (block range exceeded, too many results, etc.).
Uses an AIMD strategy (multiplicative decrease /1.5 on error, additive increase +10% on success) to discover the node's true limit. Once calibrated, the known-good range is reused directly with no further probing.
New options: WithFilterLogsMaxRange(n), WithAutoFilterLogsRange(bool). Feature is disabled by default — zero behavioral change unless opted in.