Tunneling with YARP

[jsandv]

How awesome would it be if we could use YARP as a reverse proxy over WebSockets?

I have one public facing ASPNETCORE server. Then I have multiple small blazor-server applications running on multiple machines in customers infrastructure (behind firewall etc). It would be great if I could setup YARP as a reverse proxy over websockets/signlar so I can reach the sites from public facing server.

ASPNETCORE Server (domain.com):

  .AddYarpWebSocketServerEndpoint("/someSite1");
  .AddYarpWebSocketServerEndpoint("/someSite2");
  .AddYarpWebSocketServerEndpoint("/someSite3");

NETCOREAPP adding YARP on Client machine 1 proxy app:

  .AddYarpWebSocketClient("https://domain.com/someSite1", "http://192.168.1.100:5000" );
  .AddYarpWebSocketClient("https://domain.com/someSite2", "http://192.168.1.101:5000" );

NETCOREAPP adding YARP on Client machine 2 proxy app:

  .AddYarpWebSocketClient("https://domain.com/someSite3", "http://192.168.1.100:5000" );

Browser (domain.com/someSite1)->Public server (WebSocket/SignalR-server-Hub) | Client(WebSocket/SignalR) -> 192.168.1.100 (blazor server app)

This article explains it pretty well: https://dev.to/hgsgtk/reverse-http-proxy-over-websocket-in-go-part-1-13n4

[karelz]

Triage: If we understand it correctly, you use YARP to receive (and respond via) normal HTTP traffic, but you need custom transport to the destinations.
Is that correct?

You can achieve that via customization of HttpClient. You can plug it in via https://microsoft.github.io/reverse-proxy/articles/http-client-config.html#custom-iforwarderhttpclientfactory

[karelz]

@jsandv were you successful in following it via docs link above?

[jsandv]

The provided link is not enough. Yarp would have to live on the public server as part of the public app. Then when requests are coming in, they are tunneled to the allready connected proxy/agent via websockets, So the tunnel is allready established because yarp would live in the client app which will execute the actual query to the endpoint.

[samsp-msft]

@karelz - I think what @jsandv is trying to do is to use YARP as a tunnel to break through firewalls, similar to Azure Relay. This is one of the features that I have been looking at with @davidfowl.
You would have an instance of YARP in both networks. The internal network instance would create an web sockets connection to the external network instance. Routes would be configured on the external network instance to resources via the internal network instance. Requests to those routes would be tunneled from the external network instance to the internal network instance which would then make local requests within its network.

[karelz]

Triage: It would come down to YARP tunneling feature (between 2 YARP instances) - config driven.

[RoySalisbury]

This is what I actually thought YARP was .. but its not. I want something like Azure Relay that we can build into our own API's. WCF had something like this (netTcpRelay).

Roy

[davidfowl]

This would be a very cool feature and we have all of the pieces to build it. In fact @samsp-msft and I have discussed it in the past but it hasn't risen to be a priority as yet.

As an example, we actually have a azure relay server implementation for ASP.NET Core today that does this (https://github.com/Azure/azure-relay-aspnetserver).

If you're interested, it would be cool to build a prototype and plug it into YARP. We have all of the required extensibility points to build this as a plugin (I believe).

[henriksen]

Was thinking of building this exact thing today as a side-project and looking at YARP.

Thinking that this could this be solved with a custom IForwarderHttpClientFactory that returns a HttpMessageInvoker with a custom WebSocketsHandler : HttpMessageHandler. There we override SendAsync and push data down to a web socket client, which again creates a HttpMessageHandler on its side and sends it to the local host. Am I way off base here? Am I thinking too simple and naive?

This would relay traffic from the external network to the internal network, so there would be one YARP instance in the external network. Traffic originating in the internal network would just go straight out through the firewall and thus we wouldn't need a YARP on the internal network. Internal agent would just be a local agent that connect to the external instance via web sockets and forwards any requests and responses back and forth.

Would it make sense to use SignalR here instead of raw web sockets? According to docs SignalR has "no significant performance disadvantage compared to using raw WebSockets" for "most scenarios".

[samsp-msft]

I was thinking that it would nest http calls inside of a web socket connection. Assuming we have 4 nodes:

• a web browser on a device connected to the internet - we'll call this the browser. It wants to access resources hosted on...
• OnPremServer - a web server, residing on premises of a company, and protected by a firewall. It doesn't have any internet facing access.
• OnPremProxy - an instance of yarp+ that acts as a gateway for requests from CloudProxy which come through a websocket connection to CloudProxy. It is located on premises and has internal network access to OnPremServer.
• CloudProxy - an instance of yarp+ that faces the internet and acts as the endpoint for the websocket connection from OnPremProxy, and is located at a cloud provider or somewhere that is internet addressable.

At startup, OnPremProxy will make an outbound HTTPS connection to CloudProxy, stating that it's the connection from OnPremProxy, and with appropriate credentials such as a client cert. That connection is then upgraded to a websocket connection. The connection can be initiated by OnPremProxy so it can break through the firewall as an outbound http request.

CloudProxy has a special route for OnPremServer that specifies that the requests need to be routed via OnPremProxy.

The browser makes a request to CloudProxy for a resource that its route configuration says is on OnPremServer. Rather than making a direct http request, it will make the http request over the websocket connection from OnPremProxy. (HttpClient can do this by using the ConnectCallback of SocketsHttpHandler). When the request is received by OnPremProxy, it then has a route for OnPremServer and can forward the request and route back the results.

The advantage of this approach is that Browser and OnPremServer don't have to be aware at all of the special nature of how requests get routed to OnPremServer. The server stack for OnPremServer can be whatever stack is needed, and doesn't need to be .NET or have any special configuration.

Similar to "Browser", other servers in the Cloud datacenter can also make requests via CloudProxy to access OnPremServer. They just need to have the right url path that will match the route configuration.

[jsandv]

In our scenario we would have many internal networks and one public proxy.
Agent is installed on each on-prem location, the agent is registered in the public proxy.
When the agent starts up it makes the websocket connection at that route which identifies that agent, the agent needs to know "who he is". Now when incomming request to specific route, the public proxy can lookup the correct agent to pass the http-request.

"WebSocketHttpReverseProxyOptions": {
	"Clusters": [
	  {
		"Name": "agent01",
		"Routes": [
		  {
			"Path": "/SomeApp1",
			"Endpoint": "http://192.168.1.1:5005"
		  },
		  {
			"Path": "/SomeApp2",
			"Endpoint": "http://192.168.1.2:5005"
		  }
		]
	  },
	  {
		"Name": "agent02",
		"Routes": [
		  {
			"Path": "/someApp3",
			"Endpoint": "http://192.168.1.1:5005"
		  }
		]
	  }
	]
}

Also the system must support websockets, if the local app is a blazorserverapp, then public proxy will have to accept a websocket request from the browser, it will ask the agent to create a websocket connection to the local app, the agent will also make a websocket request to the public app, when the public app get this connection it will map it to the browser socket and proxy the connection.

[Danielku15]

I would also love to see a feature where a market reverse proxy (like YARP) allows LAN applications to connect outbound to a reverse proxy to get served through it. Currently most reverse proxies require that the proxy can call down to the application which is hard to achieve in LAN<->DMZ separations (firewall needs to be opened) and LAN <-> Cloud separations.

Here some posts where I tried to initiate similar discussions in the past:
Azure/azure-relay#60
dotnet/aspnetcore#6981
ThreeMammals/Ocelot#1271

We have an in-house developed solution where one ASP.net core application hosts a custom reverse proxy. LAN applications can be configured with a custom IServer implementation which connects to this proxy through websockets and register the application configured endpoints. These LAN connections are put to a connection pool and once a client comes along we pick a connection from the pool to process the request.

This is similar to Azure relay but with a home-brew protocol (using a combination of property bags with OWIN keys for HTTP support, and on-demand WebSocket upgrade).

If YARP would support such usecases out of the box we could drop our custom solution.

[davidfowl]

Once I free up, I'll spike this. I have it mapped out in my head but this isn't the highest priority right now.

[davidfowl]

OK I hacked together a demo https://github.com/davidfowl/YarpTunnelDemo

• Shows how to implement this using our current extensibility
• The endpoint to register connections needs to be secured (see the comments)
• I hardcode a single backend, this could be expanded to support arbitrary clusters by having the backend provide the cluster id
• I wrote it in 2 hours but it seems to work well enough 😄 (I look forward to your pull requests)

[Misiu]

I have a similar scenario and currently, I do that using RabbitMQ and NSQ.
One can call a REST endpoint or the main app (the only one that is publically available), the request is serialized, and sent to a message queue. The agent that is installed in the client infrastructure (behind a firewall) reads the messages, queries local API or DB, and places the response in another message queue.
The main app waits (with a timeout) for the response message and returns a response or a GUID (so the client can ask for a reply).

This works quite well, but a YARP version would be better and probably easier to maintain.

@davidfowl can the same technique that you have shown in your demo project be used to proxy REST requests?
You have a public REST API that has all the security set up (authentication, authorization, rate limiting, etc) and an agent that is installed on-premise that is doing all the hard work (querying local API, doing file operations, calling SQL). The public app is responsible only for security, rate-limiting, passing the requests to the client, and returning the responses.

[davidfowl]

Yes there's no real magic at all. The project has been cleaned up and the diagram updated to explain how it works. The backend is the agent that has a custom Kestrel transport that uses an outbound connection instead of an inbound one. It supports HTTP/2 or websockets and uses those connections to proxy HTTP request from the front end. This plugs in at the connection layer so HTTP/1/2 and streaming protocols just work OOTB. These are handled by the SocketsHttpHandler and Kestrel.

If you use an MQ you'd need to decide what layer you want to extend, the HTTP protocol itself or the connection layer? Either way you can extend YARP to support that.

[Misiu]

@davidfowl the requirement for using MQ was quite simple: some requests can take a long time to complete. For example, generating a report or doing a complex SQL query. So instead of waiting for the response, I return a 202 response code with a GUID that allows asking later for the response.
Use case 1:

1. do a request for the data
2. it takes less than 3 seconds
3. you get the response

Use case 2:

1. do a request for the data that is taking a long time
2. it takes more than 3 seconds so you get 202 response code with a GUID
3. the request is still proceeded by the backend and the response is sent to MQ
4. you do a GET request to a specific endpoint passing the GUID
5. you get the actual response (from the MQ)

The 2.3 is tricky because I think that if I set a request timeout then the request will stop and I have no way to store the response. Am I correct on that?

[karelz]

Triage: We need to take @davidfowl prototype, list open design questions and write design doc.

[karelz]

Design doc in PR #1766

[sam9291]

I would be very interested in this feature, very nice! Do you think an option to use gRPC bidirectional streaming instead of web socket would be feasible?

[davidfowl]

Yes, it would be, but why does it matter?

[sam9291]

Having the flexibility to choose between the two would be a great feature to YARP I think, I'm also wondering if gRPC would achieve faster performance as the means of transport instead of using WebSocket. I see gRPC being promoted as "high performance" RPC option, however I'm not an expert on this, the difference might not be as big as I think.

Do you think the performance difference between WebSocket and gRPC with this approach would be negligible?

[Tratcher]

WebSockets and gRPC won't have significantly different performance characteristics in this streaming scenario.

[samsp-msft]

There would likely be negligible performance benefits to gRPC - the benefits normally talked about are with respect to its binary serialization of message contents rather than JSON.

As the tunnel is persistent and would be using an inner protocol for multiplexing - there would not be much benefit to using HTTP/2 as the tunnel transport - HTTP/1.1 is simpler and likely to be supported by more firewalls and other devices along the communications path.

[Misiu]

any updates on this?
version 2.0.0 was released (https://github.com/microsoft/reverse-proxy/releases/tag/v2.0.0) but sadly tunneling isn't mentioned anywhere.

[DomenPigeon]

Just wanted to say that I would love to see this feature in YARP :), if with SignalR hub it would be even better.

[FlorianGrimm]

I tried to do something in this direction
https://github.com/FlorianGrimm/reverse-proxy/tree/reverse-proxy-tunneling
It's not done - But Can I have a comment if this is the right direction?

[tjmoore]

Very interested in this as it appears to be exactly what I need, but unfortunately I need something pretty much now.

i.e. on-premise or hosted servers (customer hosted installs of our product), and a cloud service to allow customers to use an app with a common public endpoint using a customer identifier or sub-domain that routes HTTP REST requests to their instance. The customers are mostly unable or unwilling to expose ports, so need a tunnel or similar.

Been looking to develop something by hand but just stumbled on YARP and then this. I could still go with hand developed but concerned it's a bigger task than I think and there may be lots of potential issues I've not considered, having limited experience in this area.

Is this sample viable enough to build a product based on it, or do I really need to wait for something to be built into YARP? #1618 (comment) (and can't really wait, unless it's a week or two away). What's the benefit with built-in support over a sample that may already work?

Or likewise this #1618 (comment)

[DomenPigeon]

@tjmoore I don't know if this is good enough for you solution, for me it worked as I need the tunnels mostly internally: Dev tunnels, or at least for a starting point I think they could serve you well. To have something working while doing the proper solution.

[tjmoore]

I think "not for production workloads" would be an issue as it's a customer solution I'm working on which needs on-prem install by customer and a cloud hosted service their users can connect to.

[tjmoore]

Design doc in PR #1766

Is this still just a design, nothing functional available?

It seems to be a little different to the POC demo here https://github.com/davidfowl/YarpTunnelDemo but both design and the demo are 2 years old.

I'm struggling to understand the demo though, particularly how it does the registration from backend and where it's actually setting up a websocket tunnel.

[davidfowl]

particularly how it does the registration from backend and where it's actually setting up a websocket tunnel.

https://github.com/davidfowl/YarpTunnelDemo/blob/bc2a6ffbadd78ef7f49f9e0e9091c334313c8a6f/Backend/appsettings.json#L9-L11

The backend makes a request to YARP to say "register me as a backend".

The front end has to registration endpoints:

https://github.com/davidfowl/YarpTunnelDemo/blob/bc2a6ffbadd78ef7f49f9e0e9091c334313c8a6f/Frontend/Program.cs#L12-L18

[tjmoore]

I saw the tunnel url config but wondered if it's actually registering.

I've created an issue on the demo app for my questions/issues - davidfowl/YarpTunnelDemo#13 (comment)

[FlorianGrimm]

Based on https://github.com/davidfowl/YarpTunnelDemo I added the configuration for tunnels.
https://github.com/FlorianGrimm/reverse-proxy/tree/yarptunnel
The code allows to listen/provide many tunnels (n-m) and mixing tunnels and normal forwarders.
Unlike davidfowl's version you enable the transport/tunnel in code and specify the url/protocol in the ReverseProxy config.
In the cluster you can specify the transport -e.g. so the tunnel alpha will be used

    "Clusters": {
      "alpha": {
        "Transport": "TunnelHTTP2"

On the otherside you specify the tunnels - e.g. this server will try to connect to the tunnel alpha

  "ReverseProxy": {
    "Tunnels": {
      "tunnelalphaFE1": {
        "Url": "https://localhost:5001",
        "RemoteTunnelId": "alpha",
        "Transport": "TunnelHTTP2",       

@Tratcher (or somebody) Is this the right direction? or better stop here?

---

I have a problem with AOT. I tried to add the EnableRequestDelegateGenerator, but it didn't generate code.
this is the way to avoid the cascade of RequiresUnreferencedCode? right?
So I need help for that:
https://github.com/FlorianGrimm/reverse-proxy/blob/yarptunnel/src/ReverseProxy/Tunnel/TunnelHTTP2Route.cs#L45

---

I added configation for clientcertifactes.
I borrowed some code from kestrel, but I'll strucle here.
Can somebody give me some hints/help?

---

I added a demo that starts 6 server (in one process - for debugging) commicate with tunnels (frontend - backend) and normal forwarders (backend - api)
https://github.com/FlorianGrimm/reverse-proxy/tree/yarptunnel/samples/Tunnel/ReverseProxy.Tunnel.AllInOne.Sample

ReverseProxy.Tunnel.AllInOne.Sample do also some kind of benchmark (more likely simple messurements) - if you test it and the durations are in the hundreds ms - please read the comment in the progam.cs line:18

[stephenwelsh]

Using @davidfowl YarpTunnelDemo project as inspiration we have created a small standalone library UFX.Relay that leverages the YARP HttpForwarder and forwards requests over a single WebSocket connection. With UFX.Relay the client end does not require YARP as it has a custom Listener/Endpoint that pushes the tunnelled request into the on-prem Kestrel server so can do pretty much anything with the requests from there. In our use-case we will also have YARP on the client and will be used to forward to request to a local service/device, enabling a double reverse proxy.
In our scenario we are using Microsoft.Orleans to co-ordinate distributed dynamic routing to the far end target (i.e. a Cisco Phone on a customer network) so do not intent to tie in with the YARP cluster routing. However looking to ensure any interfaces are flexible enough that tying in with cluster routing should be possible.

The current UFX.Relay sample establishes a static tunnel and with a little bit more work could replace the likes of ngrok for test/dev

I can see @FlorianGrimm has made progress on the original demo i.e. adding authentication and tying in with YARP cluster routing however it looks like it still uses multiple web socket connections.

We discovered the great MultiplexingStream created by @AArnott and leveraged it to allow for a single Websocket to handle multiple request/connections to the client

An additional scenario we plan to cover is connection aggregation, i.e. an on-prem client receives requests to it locally and forwards to the server which can then handle/forward to a cloud service. In our use case we can have 10,000+ outbound web sockets from Cisco phones in a single location that are idle 99% of the time as the connection is used for out of band device management.
This should mostly be a matter of reversing the opening of the WebSocket so terminology like server/client will become confusing soon ;)

I could see a scenario where this could become a complimentary library i.e Yarp.ReverseProxy.Tunnel that could be an option to extend YARP to remote destinations (i.e. on-prem) over a secure out-bound connection from the on-prem network.

We are planning to close off the loose ends and get this to production quality over the next few months so would appreciate any feedback on this approach as we polish it off.

[FlorianGrimm]

@stephenwelsh In the last months I had the growing feeling I knew nothing about HTTP. Please forgive me if this are stupid questions. I'm just curious.
Is it general better to use a multiplexed connection than more Websockets/HTTP2 - Connections? Or is this because of the crazy amount of 10k connections?
I don't understand the use-case with your IP-phones. The Websocket starts in your LAN? You want to save costs within your switches/firewall? How do handle the different TLS certifcate? Does the phone not notice this?
Hope you also can share a little bit of you Orleans idea as a demo/sample.

[stephenwelsh]

Hi @FlorianGrimm ,
No worries, I have a few years of networking experience, its my coding skills that hold me back ;)

I'm just curious. Is it general better to use a multiplexed connection than more Websockets/HTTP2 - Connections? Or is this because of the crazy amount of 10k connections? I don't understand the use-case with your IP-phones.

Generally keeping the number of WebSocket connection low is best, I'm a bit of a purist so prefer to only use resources when absolutely necessary, the multiplexed connection is perfect at enabling that. Also once multiplexed it's always possible to open more connection for additional throughput. In our case there is very little traffic but a high connection count so keeping connection to device ratio as low is possible is our objective.

The Websocket starts in your LAN? You want to save costs within your switches/firewall?

Yes the WebSocket is initiated from the customer network with an outbound connection (this is how ngrok works too). This means the customer does not need to open up ports on their firewall, so as long as we follow best practice with securing the WebSocket and content it is very convenient and secure.

Our primary use case is allowing requests from our cloud server to on-premis devices (ngrok replacement)
Our secondary use case (connection aggregation) allows phones to connect to the cloud service over a single web socket instead of 1000's and can save costs/resources in two ways:

1. The agent runs on the customer network and consolidations 1000's of connections into 1 reducing port usage/logging on the customer systems.
2. We use Azure App Container & Azure Application Gateway, a single App Gateway unit supports 10,000 concurrent connections/WebSockets. We are designing our system to support 1M+ phones so that's 100+ Units, we have worked out that when at max scale (which is the most efficient cost) its about $0.15/connection/year. If we can host the agent on a 3 year dedicated Azure VM behind a load balanced and connect back to our Azure App Container instance this connection consolidation makes a huge cost saving, we estimate it can reduce the cost to $0.01/connection/year. This frees us up to focus on features rather than worrying about cost of infrastructure, assuming most of our existing customers move to the cloud 'eventually' ;)
   Literally got the connection aggregation working today, aim to update our repo tomorrow assuming all is good just need to update the Readme to cover the scenarios etc.

Both of these use cases will be over an out-bound WebSocket from the Customers network

How do handle the different TLS certifcate? Does the phone not notice this?

Our on-prem agent will host on HTTPS with ideally a customer provided certificate, so the phone will connect to that fine. Cisco Phones use a manufacture client certificate that allows it to be authenticated. In our case we use Certificate Forwarding so the phones cert is added to a header and passed along the tunnel when configured correctly will appear like a HTTPS connection (including the cert & auth) to cloud service.

Note: We use certificate authentication from the agent to our cloud gateway service for an existing SignalR management connection. So we can use the same certificate for the tunnel WebSocket, free authentication ;)

Hope you also can share a little bit of you Orleans idea as a demo/sample.

That will be tricky as there is a lot of specific logic to our requirements (i.e. AgentGrain <=> SignalR) and plan to create a TunnelGrain that will co-ordinate with the AgentGrain. I only mentioned it because we need the Tunnel routing to be very dynamic, but the samples need to be simple/static, if we can come up with a clean example will add it but no promises ;)

We did publish an UFX.Orleans.SignalRBackplane library a little while back that you might find interesting. If we had the spare time a Yarp/Tunnel Backplane over Orleans would be cool.

P.S. feel free to open an issue on the UFX.Relay repo to discuss further, don't want to hijack this issue ;)