Does this library take an advantage of zero-copy upgrade in Websocket?

[WeiquanWa]

Hi,
I've reached this library across gorilla/websocket and gobwas/ws libraries because I thought this would smoothly support 1M websocket connections. Looking at the sample codes, it's appeared this library uses http upgrade mode not tcp upgrade(inspired by zero-copy upgrade). Is there any benchmark result between gorilla, gobwas and this in terms of resource utilization, speed, scaling and etc?

[lesismal]

Thank you for your attention!

zero-copy upgrade

The Upgrade func just checks some header info and responses data to the request, then finish ws protocol handshake.
So I'm a little confused about that what does zero-copy upgrade mean which may be defined by gobwas/ws.

I don't so agree with the definitions defined by gobwas/ws, and as I know, it has some problem.
I have made some comments in gobwas/ws's issue list, if interested in the problem, please just search me in the issue list.

Is there any benchmark result between gorilla, gobwas and this in terms of resource utilization, speed, scaling and etc?

For benchmark, I suggest users run the test themselves.
Because I saw that many repos show benchmark results in their README/Doc, but when I run the same benchmark code, got a much different result from the repos authors'.

Back to nbio's websocket

If use std's http.Server, and nbio's websocket Upgrade, with the std TCPConn/tls.Conn, it's just like gorilla, nbio also use 1 goroutine to handle reading, and if you set upgrader.BlockingModAsyncWrite = true, will use dynamic goroutine to handle writing.

nbhttp.Engine has different IOMod, for more details please refer to README, or here:
https://github.com/lesismal/nbio/releases/tag/v1.3.5

[lesismal]

In the latest version, nbhttp/websocket supports more Upgrade style, for example:

1. UpgradeAndTransferConnToPoller transfer a Conn(which is from std http.Server, non-tls or using llib/tls) to nbio.Engine to handle the IO things, then the Conn don't need a reading goroutine to handle reading, but will be managed by nbio's poller(in *nix system) and save lots of goroutines.
2. Upgrade a Conn(which is from std http server) without changing the blocking mod, and keep using a goroutine to handle reading.
3. Upgrade a Conn(which is from nbio.Engine, maybe blocking mod or non-blocking mod)

There are many other different styles that I don't wanna list here, but I can give some example code if you tell the scenario you want.

[lesismal]

If using nbio poller to manage connections, and there are not a huge num of connections, nbio's qps/tps may be lower than std-based frameworks. But if there're lots of connections, it saves lots of goroutine and won't STW as easily as std-based frameworks, and may performs a better qps/tps than them.

Of course, users can use IOModBlocking to gain a better performance(maybe better than gorilla), or IOModMix to gain balance between performance and cpu/mem cost at different high or low online time period.

[WeiquanWa]

@lesismal thank you for your kind reply.
Will you describe the problems that gobwas/ws has? I already went through issues posted into gobwas/ws, but not clear yet.

I don't so agree with the definitions defined by gobwas/ws, and as I know, it has some problem.

[WeiquanWa]

We're still back and forth on determining which library will be used for a high performance ws.

[lesismal]

It should be fine if using gobwas/ws with std http.Server together.
But there's blocking problem when using gobwas/ws with netpoll(such as easygo), especially on the public internet.

For more details:
gobwas/ws#143
gobwas/ws-examples#18
gorilla/websocket#481

[lesismal]

If your service is not high online, I would like to suggest to keep using gorilla or melody, or there's a new framework gws, gws should be the fastest in std-based frameworks by now.

If your service has huge amount of connections and want reduce your hardware usage, or want to keep balance between performance and hardware usage, nbio would be better.
If using nbio with std http.Server, or using IOModBlocking, nbio should performs a little bit slower than gws, but should not be worse than other std-based frameworks.
If using nibo IOModMixed/IOModNonBlocking, we can handle 1m or even higher connections in a single node and keep balance between performance and cpu/mem cost.

[WeiquanWa]

I'm a bit confused between IOModNonBlocking vs IOModBlocking vs IOModMixed. From my understanding,

• IOModNonBlocking will handle all connections by poller. no goroutine.
• IOModBlocking will handle a connection with a goroutine for BlockingModAsyncWrite=false, otherwise two goroutines.
• IOModMixed will work in a same way as IOModBlocking until the number of connection will reach MaxBlockingOnline. If there are more new connections than MaxBlockingOnline, the new connections will be handled by poller.

Please confirm if it's right.

[lesismal]

IOModNonBlocking will handle all connections by poller. no goroutine.
IOModMixed will work in a same way as IOModBlocking until the number of connection will reach MaxBlockingOnline. If there are more new connections than MaxBlockingOnline, the new connections will be handled by poller.

Yes, that's right!

IOModBlocking will handle a connection with a goroutine for BlockingModAsyncWrite=false

This is also right!

otherwise two goroutines.

That's not exactly right.
Actually, when there's data sending, the connection hold a goroutine, but when it finish sending and doesn't have data sending now, it release the goroutine.
So, most of the time, the connections cost less than 2x connection num for async writine.

For more details:
https://github.com/lesismal/nbio/blob/master/nbhttp/websocket/conn.go#L655

[WeiquanWa]

I've measured the number of goroutines running under the IOModBlocking + BlockingModAsyncWrite=true. The number of connections is incremented by 2 for each new connection. I think it intends to create two goroutines for async reading and writing from a new connection?

[lesismal]

I've measured the number of goroutines running under the IOModBlocking + BlockingModAsyncWrite=true. The number of connections is incremented for each new connection. I think it intends to create two goroutines for async reading and writing?

For 1 connection, at most 1 goroutine for async write.
If you got more than that num of goroutines, it may be a bug. Please provide me the example code that can reproduce it.

[WeiquanWa]

Version: 1.3.15
Test file: https://github.com/lesismal/nbio-examples/blob/master/websocket_1m/server_nbio/server.go

1. Changes on the test file:

svr = nbhttp.NewServer(nbhttp.Config{
		Network:                 "tcp",
		Addrs:                   addrs,
		MaxLoad:                 1000000,
		ReleaseWebsocketPayload: true,
		Handler:                 mux,
		ReadBufferSize:          1024 * 4,
		IOMod:                   nbhttp.IOModBlocking,  # this line changed
		MaxBlockingOnline:       100000,
	})

2. Changes on a source file; github.com/lesismal/nbio/nbhttp/websocket/upgrader.go:97

func NewWebsocketReader() *WebsocketReader {
	wr := &WebsocketReader{
		Engine: DefaultEngine,
		// BlockingModReadBufferSize: DefaultBlockingReadBufferSize,
		BlockingModAsyncWrite:     true,  #this line changed
	}
        ...
}

With the above changes, runtime.NumGoroutine() is incremented by 2 for each new connection.

[lesismal]

Ah I got it.

Use dynamic goroutine for async writing is supported since 1.3.16, please update to 1.3.16, I'll update nbio-examples dependecy later.

[lesismal]

After you open this issue, I create a benchmark repo to compare different frameworks, would like to invite you to try it here:
https://github.com/lesismal/go-websocket-benchmark

[WeiquanWa]

Great. I will upgrade the version and try benchmark.

I have another questions.

1. The API func (wr *WebsocketReader) SetBlockingMod(blocking bool)
   This API sets isBlockingMod. What's the difference between isBlockingMod and BlockingModAsyncWrite?
2. There is no exposed API to set BlockingModAsyncWrite?
3. In the naming such as IOModBlocking, what do you mean by blocking? Does it block anything? Maybe this question is stupid, but want to clarify.

[lesismal]

Great. I will upgrade the version and try benchmark.

I already update nbio-examples's nbio version to v1.3.16, please just pull nbio-examples.

[lesismal]

1. The API func (wr *WebsocketReader) SetBlockingMod(blocking bool)
   This API sets isBlockingMod. What's the difference between isBlockingMod and BlockingModAsyncWrite?

Please update to v1.3.16, src changed a lot.
In old version, Upgrader is WebsocketReader; While in new version, Upgrader just be used to Upgrade a websocket conn from http request, its API most like configurations.

BlockingMod means we must use a goroutine to handle reading;
Then, if the websocket.Conn is BlockingMod and BlockingModAsyncWrite is false, the WriteMessage API is also Blocking func, if there's broadcasting logic in your code, if some connection is blocking on WriteMessage, the other connections would all waiting for them in the same for-loop. So, we need to set it to async write, than WriteMessage would not block.
Of course, if there's not broadcasting logic in your code, you can just set Upgrader.BlockingModAsyncWrite=false, then never use extra goroutine to handle writing.

2. There is no exposed API to set BlockingModAsyncWrite?

In the latest version, you can set it like this:
Upgrader.BlockingModAsyncWrite = true/false

3. In the naming such as IOModBlocking, what do you mean by blocking? Does it block anything? Maybe this question is stupid, but want to clarify.

Because net.TCPConn or std tls.Conn provide only blocking API.
When we call net.TCPConn.Write, if the connections tcp send queue in kernal is full, it will block until it's writable.

[lesismal]

func broadcast() {
	for _, conn := range allConns {
		// std's TCPConn/tls.Conn.Write may block when tcp send queue in kernal is full
		// then, other conns in this for-loop would waiting too
		// it's kind of like Head-of-line blocking, HOL
		conn.WriteMessage(...)
	}
}

[WeiquanWa]

@lesismal thank you for your details again. Maybe my last question is how to implement JWT authorization with IOModBlocking mode. Any sample code would be great.

[lesismal]

how to implement JWT authorization with IOModBlocking mode

It should be the same as your implementation using std http.Server.

[WeiquanWa]

okay. is there any option to set ping/pong interval? The current websockget upgrader enables ping/pong handler by default?

[lesismal]

is there any option to set ping/pong interval?

I think the best way to check zombie connections is:

1. server SetReadDeadline everytime after receiving a message
2. client keep sending normal message or ping message within keepalive-interval

For the server-side, nbio provide a easy way:

upgrader.KeepaliveTime = YourKeepaliveInterval

Then nbio will update ReadDeadline automatically:
https://github.com/lesismal/nbio/blob/master/nbhttp/websocket/conn.go#L161

So, you just need to set upgrader.KeepaliveTime = YourKeepaliveInterval, and make sure your client do the right thing too.

The current websockget upgrader enables ping/pong handler by default?

Yes, there are some handlers set by default:
https://github.com/lesismal/nbio/blob/master/nbhttp/websocket/upgrader.go#L103

[WeiquanWa]

Then nbio will update ReadDeadline automatically:
https://github.com/lesismal/nbio/blob/master/nbhttp/websocket/conn.go#L161

This feature is very useful. By the way, based on my testing, the function func (c *Conn) handleWsMessage(opcode MessageType, data []byte) doesn't get called on receiving messages. It's only invoked when closing a connection.

[lesismal]

the function func (c *Conn) handleWsMessage(opcode MessageType, data []byte) doesn't get called on receiving messages. It's only invoked when closing a connection.

Please provide full example code, both server and client.
I need to go to bed now, I'll try it tomorrow.

[WeiquanWa]

ah, I've figured out why the handleWsMessage function doesn't get invoked. I was testing with OnDataFrame mode. After setting it to false, the function is invoked and works as expected.