Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: bump wazero version #64

Merged
merged 2 commits into from
Jun 12, 2024
Merged

chore: bump wazero version #64

merged 2 commits into from
Jun 12, 2024

Conversation

nilslice
Copy link
Member

Opening this up as a draft for now for discussion.

This PR brings wazero up to the latest version, which introduces their new optimizing compiler. I've added benchmarks from the current version we use v1.3.0, and the latest v1.7.0:

v1.3.0:

goos: linux
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1255U
BenchmarkInitialize/noop-12                  314           3610336 ns/op         2850623 B/op       2744 allocs/op
BenchmarkInitializeWithCache/noop-12                2060            549867 ns/op         1630131 B/op       1554 allocs/op
BenchmarkNoop/noop-12                             245217              4664 ns/op           26216 B/op         22 allocs/op
BenchmarkReplace/8192-12                            5328            249861 ns/op          32.79 MB/s       34408 B/op         23 allocs/op
BenchmarkReplace/16383-12                           2366            495646 ns/op          33.06 MB/s       42600 B/op         23 allocs/op
BenchmarkReplace/32768-12                           1046           1110322 ns/op          29.51 MB/s       58985 B/op         23 allocs/op
BenchmarkReplace/empty-12                         222662              4755 ns/op           26216 B/op         22 allocs/op
BenchmarkReplace/2048-12                           17350             66603 ns/op          30.75 MB/s       28264 B/op         23 allocs/op
BenchmarkReplace/4096-12                            9249            122996 ns/op          33.30 MB/s       30312 B/op         23 allocs/op

v1.7.0:

goos: linux
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1255U
BenchmarkInitialize/noop-12                   64          17807498 ns/op         5553731 B/op       8519 allocs/op
BenchmarkInitializeWithCache/noop-12                1300            798052 ns/op         1830382 B/op       1954 allocs/op
BenchmarkNoop/noop-12                              67366             17587 ns/op           70142 B/op         23 allocs/op
BenchmarkReplace/8192-12                            5043            247079 ns/op          33.16 MB/s       78329 B/op         24 allocs/op
BenchmarkReplace/16383-12                           2517            457056 ns/op          35.85 MB/s       86523 B/op         24 allocs/op
BenchmarkReplace/32768-12                           1292            940309 ns/op          34.85 MB/s      102912 B/op         24 allocs/op
BenchmarkReplace/empty-12                          67363             16843 ns/op           70137 B/op         23 allocs/op
BenchmarkReplace/2048-12                           15170             81431 ns/op          25.15 MB/s       72183 B/op         24 allocs/op
BenchmarkReplace/4096-12                            8074            134374 ns/op          30.48 MB/s       74230 B/op         24 allocs/op

@evacchi
Copy link
Contributor

evacchi commented May 28, 2024

we released v1.7.2 a few days ago with a bunch of fixes https://github.com/tetratelabs/wazero/releases/tag/v1.7.2
main includes some improvements to the compilation times as well.

@nilslice
Copy link
Member Author

This release may also see some improvements
https://github.com/tetratelabs/wazero/releases/tag/v1.7.3

Will test it out

@nilslice
Copy link
Member Author

nilslice commented Jun 11, 2024

Looks like we're using a now-removed API (1.7.0 -> 1.7.3):

go test -bench ./...
# github.com/extism/go-sdk [github.com/extism/go-sdk.test]
./extism_test.go:773:62: undefined: experimental.FunctionListenerFactoryKey
./extism_test.go:793:61: undefined: experimental.FunctionListenerFactoryKey

@nilslice
Copy link
Member Author

nilslice commented Jun 11, 2024

1.3.0:

BenchmarkInitialize/noop-14                          722           1637676 ns/op         2863588 B/op       2534 allocs/op
BenchmarkInitializeWithCache/noop-14                3314            338563 ns/op         1627090 B/op       1481 allocs/op
BenchmarkNoop/noop-14                             631005              1938 ns/op           26504 B/op         22 allocs/op
BenchmarkReplace/empty-14                         583592              2049 ns/op           26504 B/op         22 allocs/op
BenchmarkReplace/2048-14                           28004             43246 ns/op          47.36 MB/s       28552 B/op         23 allocs/op
BenchmarkReplace/4096-14                           14134             83635 ns/op          48.97 MB/s       30600 B/op         23 allocs/op
BenchmarkReplace/8192-14                            7153            166118 ns/op          49.31 MB/s       34696 B/op         23 allocs/op
BenchmarkReplace/16383-14                           3462            341831 ns/op          47.93 MB/s       42888 B/op         23 allocs/op
BenchmarkReplace/32768-14                           1543            773949 ns/op          42.34 MB/s       59273 B/op         23 allocs/op

1.7.0:

BenchmarkInitialize/noop-14                          124           9525460 ns/op         6155813 B/op       8558 allocs/op
BenchmarkInitializeWithCache/noop-14                2179            487101 ns/op         1871706 B/op       1945 allocs/op
BenchmarkNoop/noop-14                             223034              5150 ns/op           70144 B/op         23 allocs/op
BenchmarkReplace/8192-14                            8060            137262 ns/op          59.68 MB/s       78343 B/op         24 allocs/op
BenchmarkReplace/16383-14                           4262            275337 ns/op          59.51 MB/s       86539 B/op         24 allocs/op
BenchmarkReplace/32768-14                           2041            587690 ns/op          55.76 MB/s      102929 B/op         24 allocs/op
BenchmarkReplace/empty-14                         182199              5555 ns/op           70140 B/op         23 allocs/op
BenchmarkReplace/2048-14                           29769             40297 ns/op          50.82 MB/s       72198 B/op         24 allocs/op
BenchmarkReplace/4096-14                           16035             72510 ns/op          56.49 MB/s       74245 B/op         24 allocs/op

1.7.3:

BenchmarkInitialize/noop-14                          158           7486412 ns/op         5319317 B/op       6911 allocs/op
BenchmarkInitializeWithCache/noop-14                2560            473021 ns/op         1851399 B/op       1806 allocs/op
BenchmarkNoop/noop-14                             141536              7226 ns/op           70139 B/op         23 allocs/op
BenchmarkReplace/16383-14                           4167            283178 ns/op          57.86 MB/s       86535 B/op         24 allocs/op
BenchmarkReplace/32768-14                           1978            604613 ns/op          54.20 MB/s      102923 B/op         24 allocs/op
BenchmarkReplace/empty-14                         161700              6700 ns/op           70134 B/op         23 allocs/op
BenchmarkReplace/2048-14                           28716             41730 ns/op          49.08 MB/s       72199 B/op         24 allocs/op
BenchmarkReplace/4096-14                           16026             74786 ns/op          54.77 MB/s       74245 B/op         24 allocs/op
BenchmarkReplace/8192-14                            7995            144315 ns/op          56.76 MB/s       78339 B/op         24 allocs/op

@zshipko
Copy link
Contributor

zshipko commented Jun 11, 2024

There do seem to be some improvements despite there being some regressions, I think I'm in favor of updating to 1.7.3 since it probably includes some other bugfixes and improvements compared to 1.3.0

@nilslice
Copy link
Member Author

I think I'm in favor of updating to 1.7.3

I am too! I think we just need to consider what the alternative is for the experimental Listener things are which were removed. We have tests to ensure we can still call these APIs, and I think we use them in the Observe Go SDK, so there's some additional work to figure out where they went and/or how to fix/update the code.

@zshipko
Copy link
Contributor

zshipko commented Jun 12, 2024

It looks like we can just switch over to experimental.WithFunctionListenerFactory - I will try to update the test

@nilslice nilslice marked this pull request as ready for review June 12, 2024 00:09
@nilslice
Copy link
Member Author

@zshipko i can't approve, given it's my PR - but if CI passes, feel free to merge! we can do some additional testing before we release.

@zshipko zshipko merged commit 7d5e112 into main Jun 12, 2024
3 checks passed
@zshipko zshipko deleted the update-wazero branch June 12, 2024 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants