Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update handling of duplicate batches to avoid editing existing messages #1647

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

peterbroadhurst
Copy link
Contributor

@peterbroadhurst peterbroadhurst commented Feb 27, 2025

Proposed changes

Depends on #1649 going in first

If a FireFly node receives a batch a 2nd time, you can end up with messages that are:

  • Marked pending (without a confirmed time)
  • Have a message_confirmed event associated

Logically this should be impossible, but it happens because we overwrite the message state in the upsert logic, after the insert fails.

Easy to recreate this, by just using /operation/{opId}/retry to re-send a batch containing confirmed messages.

This PR changes the fallback logic, after the insert fails, to only insert the missing messages.


Types of changes

  • Bug fix
  • New feature added
  • Documentation Update

  • I have read the contributing guidelines.
  • I have performed a self-review of my own code or work.
  • I have commented my code, particularly in hard-to-understand areas.
  • My changes generates no new warnings.
  • My Pull Request title is in format < issue name > eg Added links in the documentation.
  • I have added tests that prove my fix is effective or that my feature works.
  • My changes have sufficient code coverage (unit, integration, e2e tests).

Signed-off-by: Peter Broadhurst <[email protected]>
@awrichar
Copy link
Contributor

Good find; thanks for fixing this.

awrichar
awrichar previously approved these changes Feb 28, 2025
Signed-off-by: Peter Broadhurst <[email protected]>
Copy link

codecov bot commented Feb 28, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.95%. Comparing base (490539e) to head (d934bcc).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1647   +/-   ##
=======================================
  Coverage   99.95%   99.95%           
=======================================
  Files         337      337           
  Lines       29607    29632   +25     
=======================================
+ Hits        29595    29620   +25     
  Misses          8        8           
  Partials        4        4           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants