fix(dfir_pipes): resolve_futures poll_ready blocking multiple pending futures, fix #2662#2663
Draft
MingweiSamuel wants to merge 1 commit intomainfrom
Draft
fix(dfir_pipes): resolve_futures poll_ready blocking multiple pending futures, fix #2662#2663MingweiSamuel wants to merge 1 commit intomainfrom
MingweiSamuel wants to merge 1 commit intomainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes ResolveFutures push operator readiness semantics so pending futures in “blocking mode” (no subgraph_waker) don’t cause poll_ready to return Pending, which previously prevented start_send from being called and effectively serialized the queue.
Changes:
- Update
ResolveFutures::poll_readyto opportunistically drain ready outputs but always returnDone. - Clarify the
Poll::Pendingbehavior inempty_readyas being relevant topoll_flush’s blocking semantics. - Add a regression test ensuring
poll_readyremainsDonewhile futures are pending.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Deploying hydro with
|
| Latest commit: |
5e54692
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://f94c0557.hydroflow.pages.dev |
| Branch Preview URL: | https://sandbox-ed7cc062-2e2e-4e04-8.hydroflow.pages.dev |
MingweiSamuel
changed the title
MingweiSamuel
force-pushed
the
mingwei/fix-2662
branch
3 times, most recently
from
3182f00 to
9bfd3d3
Compare
In `ResolveFutures::poll_ready`, the call to `empty_ready` would return
`Pending` when the queue contained unresolved futures in blocking mode
(no subgraph_waker). This prevented callers from ever calling `start_send`
to add new futures, effectively serializing the queue to one future at a
time.
The fix makes `poll_ready` always return `Done` after opportunistically
draining any ready futures. The `Pending` behavior is preserved in
`poll_flush`, which correctly blocks until all futures resolve before
flushing downstream.
Changes:
- `dfir_pipes/src/push/resolve_futures.rs`:
- `poll_ready` now discards the `empty_ready` result and always returns
`Done`, allowing callers to keep adding futures to the queue.
- Added regression test `poll_ready_allows_send_while_futures_pending`
that uses a two-poll future to verify `poll_ready` doesn't block.
- Updated comment on the `Pending` branch in `empty_ready` to clarify
it's only relevant for `poll_flush`.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
MingweiSamuel
force-pushed
the
mingwei/fix-2662
branch
from
9bfd3d3 to
5e54692
Compare
Member
|
We need to think about this more carefully. If the future has a side effect it is important that they run sequentially and never concurrently. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In
ResolveFutures::poll_ready, the call toempty_readywould returnPendingwhen the queue contained unresolved futures in blocking mode (no subgraph_waker). This prevented callers from ever callingstart_sendto add new futures, effectively serializing the queue to one future at a time.The fix makes
poll_readyalways returnDoneafter opportunistically draining any ready futures. ThePendingbehavior is preserved inpoll_flush, which correctly blocks until all futures resolve before flushing downstream.Changes:
dfir_pipes/src/push/resolve_futures.rs:poll_readynow discards theempty_readyresult and always returnsDone, allowing callers to keep adding futures to the queue.poll_ready_allows_send_while_futures_pendingthat uses a two-poll future to verifypoll_readydoesn't block.Pendingbranch inempty_readyto clarify it's only relevant forpoll_flush.