Limits on collectible count

[latenssi]

When creating a distribution of 50 packs and 10 collectibles in each (500 collectibles total) with a gas limit of 9999 we get this error:

[Error Code: 1101] cadence runtime error Execution failed:
error: computation limited exceeded: 9999
   --> f3fcd2c1a78f5eee.PDS:143:16
    |
143 |                 pdsCollection.deposit(token:<-nft)

We do not get the error with 45 packs and 10 collectibles in each.

[latenssi]

If we set gas limit to 0 the above scenario works fine but if we then set the number of packs to 100 (1000 collectibles total) we get this error:

[Error Code: 1101] cadence runtime error Execution failed:
error: [Error Code: 1105] total event byte size (256146) exceeds limit (256000)
  --> 01cf0e2f2f715450.ExampleNFT:61:12
     |
  61 |             emit Deposit(id: id, to: self.owner?.address)
     |

[latenssi]

Both of these errors occur during the "settle" phase.

[latenssi]

Making the "settle" requests in batches of 10 NFTs per request, allows both of the above scenarios to work fine. Previously there was no batching at all.

[latenssi]

Raising the pack count to 1000 (10k collectibles total, with batching) and using 9999 gas limit results in this error:

[Error Code: 1101] cadence runtime error Execution failed:
error: computation limited exceeded: 9999
   --> 01cf0e2f2f715450.PackNFT:190:12
       |
       190 |             destroy oldToken
           |             ^^^^^^^^^^^^^^^^

Seems to happen in the "mint" phase (minting of Pack NFTs).

[latenssi]

Again adding batching to the "mint" requests (mint 10 packs per request) allows the above scenario to work fine.

[latenssi]

Trying 10k packs with 10 collectibles each (100k collectibles) caused the emulator to crash at block 1400 without an error. The backend service was giving this error:

error while runnig poller "handleResolved": 18446744073709551615 is greater than maximum value for Int8

[latenssi]

Enabled transaction fees on the emulator to estimate costs (1k packs, 10 collectibles, 10k total collectibles):

PDS account balance before: 100.00070000
Issuer account balance before: 100.00080000

PDS account balance after: 99.97280000 (0.0342 difference)
Issuer account balance after: 99.99040000 (0.0176 difference)

NOTE: "Temporarily, transactions on Flow are free": https://docs.onflow.org/faq/developers/#how-much-will-gas-cost-on-flow
So we might need to record the gas spent manually as we can not use the numbers above to evaluate the costs.

[latenssi]

With 10k collectibles and 1s block time, resolve, settle and mint took 19.3959264s

[latenssi]

Trying 10k packs with 10 collectibles each (100k collectibles) caused the emulator to crash at block 1400 without an error. The backend service was giving this error:

error while runnig poller "handleResolved": 18446744073709551615 is greater than maximum value for Int8

This is indirectly caused by the emulator running out of memory and crashing.

[latenssi]

Setting FLOW_PERSIST to "true" on the emulator resolved the above memory issue and allowed 100k collectibles to pass.

[latenssi]

Currently the (very rough) time estimate on emulator for 1M collectibles to settle and mint the corresponding packs is around 100 minutes. This is purely based on linear extrapolation of 100k collectibles taking 590s. Testing this with emulator was ok for around 60 minutes after which testing had to be stopped for external reasons. Emulator was cumulatively being allocated more and more memory (around 10GB when stopped) even when using the persistent storage mentioned above.

[latenssi]

With a batch size of 1k (and a gas limit of 1M) for the settlement I got this error:

[Error Code: 1105] total event byte size (256027) exceeds limit (256000)

Same with 0.9k, 0.8k and 0.6k.

Batch size of 500 was ok, settling 5000 collectibles took about 2min 20s, around 35 collectibles per second (so not much of an improvement).

Tests were done on emulator with persistence mode, blocktime of 1s and transaction-max-gas-limit=1000000.