Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

data consistency when running mysql sink changefeed with redo log and injecting S3 network partition #11744

Open
fubinzh opened this issue Nov 14, 2024 · 2 comments · May be fixed by #11747
Open

data consistency when running mysql sink changefeed with redo log and injecting S3 network partition #11744

fubinzh opened this issue Nov 14, 2024 · 2 comments · May be fixed by #11747
Assignees
@CharlesCheung96
Labels
affects-6.5 affects-7.1 affects-7.5 affects-8.1 affects-8.5 area/ticdc Issues or PRs related to TiCDC. severity/critical type/bug The issue is confirmed as a bug.

Comments

@fubinzh
Copy link

fubinzh commented Nov 14, 2024

What did you do?

  1. create tidb sink changefeed with redo and syncpoint on
  2. run workload
  3. inject cdc and redo log storage server network partition for 5m
  4. sleep 2m
  5. pause changefeed
  6. run redo apply
  7. run redo meta command, get resolved ts of changefeed (TS1)
  8. compare data consistency between upstream snapshot TS1 and downstream

What did you expect to see?

Downstream and upstream (snapshot TS1) should be eventually consistent

What did you see instead?

There is a low possibility (4 repreduces out of 50 executions ) that data will be inconsistent.

Versions of the cluster

CDC version:
release-version: v8.5.0-alpha-7-g64decf622
git-hash: 64decf6
@fubinzh fubinzh added area/ticdc Issues or PRs related to TiCDC. type/bug The issue is confirmed as a bug. labels Nov 14, 2024
@fubinzh
Copy link
Author

fubinzh commented Nov 14, 2024

/assign @CharlesCheung96

@fubinzh
Copy link
Author

fubinzh commented Nov 14, 2024

/severity critical

@CharlesCheung96 CharlesCheung96 linked a pull request Nov 14, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CharlesCheung96 CharlesCheung96

Labels
affects-6.5 affects-7.1 affects-7.5 affects-8.1 affects-8.5 area/ticdc Issues or PRs related to TiCDC. severity/critical type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ticdc(redo, sink): return correct error in redo writer & fix default retryer CharlesCheung96/tiflow
2 participants
@fubinzh @CharlesCheung96