[Feature]: replace "encoding/json" with a more performant option

[p0mvn]

Summary

The default JSON encoding library is not the most performant.

In Osmosis, we've run into this issue where JSON serialization would be one of the biggest bottlenecks in one of the off-chain services.

Problem Definition

See PR for more context: osmosis-labs/osmosis#7065

Proposed Feature

Replacing to https://github.com/json-iterator/go showed some improvement.

As a result, I'm proposing to evaluate a replacement JSON library in cosmos SDK to see if we can save on serialization in import/export genesis.

[julienrbrt]

Hopefully long term we can migrate to encoding/json/v2 instead: golang/go#63397

[tac0turtle]

in the case of genesis import/export i think a state modifier may be more performant than anything we can do with json.

In disaster recovery scenarios a chain would write a short state modifier, since the chain is halted all validators would run the state transition producing a valid state. This is similar to what we have today with store migrations. Ideally this never happens.

[Halimao]

How about bytedance/sonic, it looks like better than json-iterator/go

[conorpp]

sonic is not very cross-platform and does not support go 1.24. They have this ugly log that shows up if you run a binary with it compiled on go 1.24.

WARNING:(ast) sonic only supports go1.17~1.23, but your environment is not suitable

Also, these top performance metrics are only applicable if you provide the avx and sonic build tags, so likely no performance is actually being gained.

Can you please drop sonic in favor of a more go native solution? I'm patching in the meantime to get rid of it. I like github.com/goccy/go-json.

[conorpp]

FYI for anyone else wishing to patch this issue, my team forked github.com/goccy/go-json and changed it's module & package name just to be able to:

replace github.com/bytedance/sonic => github.com/cordialsys/go-json v0.10.5-sonic-replace

Which will remove the sonic dependency in place of go-json.

[julienrbrt]

sonic is not very cross-platform and does not support go 1.24. They have this ugly log that shows up if you run a binary with it compiled on go 1.24.

WARNING:(ast) sonic only supports go1.17~1.23, but your environment is not suitable

Also, these top performance metrics are only applicable if you provide the avx and sonic build tags, so likely no performance is actually being gained.

Can you please drop sonic in favor of a more go native solution? I'm patching in the meantime to get rid of it. I like github.com/goccy/go-json.

That is mainly due to a regression in Go 1.24 -> ref bytedance/sonic#741
Go 1.24.1 should make it work again.