You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It takes too long to sync up pindexer on a raw database: on the order of hours. The processing logic we do is not so complicated as to make this take this long, and I suspect that the architecture of processing one event at a time and touching the database for every event makes this slow.
I think we should move to have a "per batch of blocks" processing logic, only reading the database before the batch, and writing after the batch.
This also has the advantage of making certain app views simpler, because right now they would like to process all the events in a block at once, e.g. to know the timestamp for each event, but have to resort to tricks like using an in database queue instead.
The text was updated successfully, but these errors were encountered:
It takes too long to sync up pindexer on a raw database: on the order of hours. The processing logic we do is not so complicated as to make this take this long, and I suspect that the architecture of processing one event at a time and touching the database for every event makes this slow.
I think we should move to have a "per batch of blocks" processing logic, only reading the database before the batch, and writing after the batch.
This also has the advantage of making certain app views simpler, because right now they would like to process all the events in a block at once, e.g. to know the timestamp for each event, but have to resort to tricks like using an in database queue instead.
The text was updated successfully, but these errors were encountered: