Docker-ized Stalwart Email Server with Socks/Proxy Protocol Attempt #557

shoutmarble · 2025-02-17T00:46:57Z

shoutmarble
Feb 17, 2025

I'm going to try and run a Dockerized Stalwart-Mail server using g3proxy

And have SPF, DKIM, TXT DNS records all validate for the Dockerized Stalwart-Mail server with the VPS's HOST's IP.

I want use g3proxy to try to use the proxy-protocol to send the HOST's IP to the Stalwart Email server.

Internally, in the Stalwart-Mail server I will need set in a config file the IP of the g3proxy proxy-protocol server.
And, I will also need to supply g3proxy the IP of the Stalwart-Mail server of where to send the proxy-protocol traffic.

These are some of the g3proxy use cases I will likely need to cover, proxy protocol, socks5, and sni.

escaper_proxy_socks5
chain_proxy_protocol
simple_sni_proxy

I have momentarily below, only stubbed out the Stalwart-Mail server behind a g3proxy http-rpoxy

compose.yml

services:
    g3proxy:
        image: g3proxy/comp:g3
        build:
            context: ./g3
            dockerfile: g3proxy/docker/debian.Dockerfile
        ports:
            - "88:88"
            - "80:80"
            - "8080:8080"
            - "443:443"
            - "25:25"
            - "587:587"
            - "465:465"
            - "143:143"
            - "993:993"
            - "4190:4190"
            - "110:110"
            - "995:995"
        volumes:
            - .:/g3/
        command:
            -c /g3/my_g3proxy.yml

    stalwart-mail:
        image: stalwartlabs/mail-server:latest
        hostname: stalwart-mail
        container_name: stalwart-mail
        volumes:
            - ./opt:/opt/stalwart-mail

my_g3proxy.yml

log: stdout

resolver:
    - name: default
      type: c-ares

escaper:
    - name: default
      type: direct_fixed
      # no_ipv6: true
      # resolve_strategy: IPv4Only
      resolver: default

server:
  - name: g3proxy
    escaper: default
    type: http_rproxy
    listen:
      address: 0.0.0.0:80
    hosts:
      - exact_match: mail.landingdev.xyz
        upstream: stalwart-mail:8080

PORTS USED

http		localIp = ::	localPort = 8080	tls = false
https		localIp = ::	localPort = 443		tls = true
imap		localIp = ::	localPort = 143		tls = false
imaptls		localIp = ::	localPort = 993		tls = true
pop3		localIp = ::	localPort = 110		tls = false
pop3s		localIp = ::	localPort = 995		tls = true
sieve		localIp = ::	localPort = 4190	tls = false
smtp	 	localIp = ::	localPort = 25		tls = false
submission	localIp = ::	localPort = 587		tls = false
submissions	localIp = ::	localPort = 465		tls = true

zh-jq-b · 2025-02-17T06:14:00Z

zh-jq-b
Feb 17, 2025
Maintainer

You can use several tcp_stream servers to map each of the ports, see https://github.com/bytedance/g3/blob/master/g3proxy/examples/simple_tcp_stream/g3proxy.yaml.
But proxy_protocol is currently not supported when connect to the backend service.

4 replies

shoutmarble Feb 17, 2025
Author

I will try that out

shoutmarble Feb 26, 2025
Author

@zh-jq The way I've always got Stalwart-Mail working prior was with using Proxy-Protocol (HAproxy, TRaefik, etc)

And allowing the HAproxy/TRaefik to sending the Real-IP of the client.

I'm assuming I'm wanting g3proxy to be a proxy-protocol server to the backend servers. The way I've
used HAproxy and TRaefik. I do not believe TRaefik is a Proxy-Protocol server but can consume
the receiving of Proxy-Protocol tcp-stream from Stalwart to TRaefik (not optimal, but works)

I will try to use g3proxy like TRaefik and have Stalwart Mail send proxy-protocol to g3proxy, which may work?

The downstream Stalwart Mail application sends all the proxy-protocol traffic/tcp-stream.

If I'm trying to squeeze something that doesn't fit with g3proxy I may just opt to use some nginx server
with the proxy-protocol plugin.


                  +-----------------+
                  | INTERNET/DNS/NS |
                  +-----------------+     +-----------------------------------------------------+
                       X                  |HOSTINGER VPS                    +----------------+  |
                       XX                 |44.55.66.77               +------> STALWART       |  |
    +-------------+      XXX              |                          |      | 10.5.0.8       |  |
    |Client       |        XX             |   +-------------+  PROXY PROTOCOL PORTS 25,110   |  |
    |12.23.34.45  |      XXXX             |   |G3PROXY      |  12.23.34.45  | 465,586        |  |
    |             +----------------------->   |10.0.5.5     |        ^      +----------------+  |
    |             |      X                |   |             +--------+                          |
    +-------------+       XXX             |   |             |               +--------------+    |
                            XX            |   +-------------+               |OTHER/BLOG/CMS|    |
                          XXXX            |                                 |10.5.0.10     |    |
                         X                |                                 |PORTS 80,443  |    |
                         X                |                                 +--------------+    |
                         XX               +-----------------------------------------------------+
                          XXX

root@server:~/stalwart# tree certs
certs
├── landingdev.xyz.crt
├── landingdev.xyz.issuer.crt
├── landingdev.xyz.json
├── landingdev.xyz.key
└── landingdev.xyz.pem

1 directory, 5 files

I hope what I said wasn't some ramble; I'm not able to communicate this level of expertise in this area.

zh-jq Feb 27, 2025
Maintainer

It's easy to add proxy-protocol support to tcp-stream server when connect to backend service. But I'm wondering why do you want a proxy server in front of your mail server. The tcp-stream server in g3proxy is really simple, so I don't think it can fulfill your needs.
I'm considering add mail proxy support to g3tiles which will support more features but that won't happen any soon.

shoutmarble Feb 27, 2025
Author

#557 (comment)

The reason I think I need a layer-4 or tcp-stream is so that when TLS-Termination occurs the TLS connection appears to be from the Real IP of the Client and the actual dedicated IP of the VPS Host.

So that the SPF, DKIM, DMARC, etc records that are set in DNS match the TLS termination values. So, when one email server sends an email to another email server the handshakes over the ports (110, 25, 465, 587, etc) are verified with the SPF, DKIM, DMARC records to the receiving/sending email servers (in the TLS session)

The g3proxy ingests a tcp_stream and produce a proxy-protocol stream to the backend stalwart mail server?

(I'm not running TRaefik or HAproxy, nor do I want to run TRaefik or HAProxy, sorry for including those, that was confusing, I was only mentioning those as being a similar use-case)

I believe all that I have to do now to #557 (comment) is to add my ACME certificates to my my-g3proxy.yml and configure the proxy protocol trusted networks in my stalwart email.

root@server:~/stalwart# tree certs/
certs/
├── landingdev.xyz.crt
├── landingdev.xyz.issuer.crt
├── landingdev.xyz.json
├── landingdev.xyz.key
└── landingdev.xyz.pem

1 directory, 5 files

*EDIT: oops, I conflated proxy_pass as being the same thing as proxy_protocol

I think I need to use chain_proxy_protocol; I bet I could keep my tcp_stream and remove proxy_pass and put proxy_protocol: 2 in it's place and do the same in the resolvers section as it is in chain_proxy_protocol

I will try that out now...

chain_proxy_protocol

---
runtime:
  thread_number: 2

server:
  - name: http1
    escaper: proxy_pp1
    type: http_proxy
    listen:
      address: "[::]:3128"
  - name: http2
    escaper: proxy_pp2
    type: http_proxy
    listen:
      address: "[::]:3129"
  - name: http0
    escaper: default
    type: http_proxy
  - name: port_pp1
    type: plain_tcp_port
    server: http0
    proxy_protocol: 1
    listen: "[::1]:3131"
  - name: port_pp2
    type: plain_tcp_port
    server: http0
    proxy_protocol: 2
    listen: "[::1]:3132"

resolver:
  - name: default
    type: c-ares

escaper:
  - name: proxy_pp1
    type: proxy_http
    proxy_addr: "[::1]:3131"
    use_proxy_protocol: 1
  - name: proxy_pp2
    type: proxy_http
    proxy_addr: "[::1]:3132"
    use_proxy_protocol: 2
  - name: default
    type: direct_fixed
    resolver: default

*EDIT2:
I'm thinking about what your pointing at and digesting the docs.

shoutmarble · 2025-02-18T00:11:56Z

shoutmarble
Feb 18, 2025
Author

I think that is the correct way. I can send the TCP stream traffic that has my hosts IP (to the routed docker containers)
185.28.22.166 to the internal Dockerized Stalwart-Mail server to do all the TLS/DKIM/SPF operations with DNS and
ACME.

I have only stubbed out the tcp_stream below to my g3proxy config file to peek into the g3proxy stdout logfiles.

docker compose up

root@server:~/stalwart# docker compose up
[+] Running 3/3
 ✔ Network stalwart_default      Created                                                                          0.1s
 ✔ Container stalwart-mail       Recreated                                                                        0.1s
 ✔ Container stalwart-g3proxy-1  Recreated                                                                        0.1s
Attaching to g3proxy-1, stalwart-mail
g3proxy-1      | Feb 17 23:22:27.754 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61982, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:27.754145Z, task_event: finished, task_id: 0d5f8e4eed8611ef8ad42a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 142.253µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
g3proxy-1      | Feb 17 23:22:27.760 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61983, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:27.760804Z, task_event: finished, task_id: 0d609269ed8611ef8ad52a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 52.728µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
g3proxy-1      | Feb 17 23:22:27.962 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61984, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:27.962000Z, task_event: finished, task_id: 0d7f45a9ed8611ef8ad62a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 64.680µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
g3proxy-1      | Feb 17 23:22:28.897 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61986, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:28.897452Z, task_event: finished, task_id: 0e0e02c0ed8611ef8ad72a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 256.134µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
g3proxy-1      | Feb 17 23:22:28.897 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61985, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:28.897795Z, task_event: finished, task_id: 0e0e1025ed8611ef8ad82a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 37.910µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
g3proxy-1      | Feb 17 23:22:34.052 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61988, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:34.052223Z, task_event: finished, task_id: 112090f6ed8611ef8ad92a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 78.806µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
g3proxy-1      | Feb 17 23:22:34.052 INFO c_rd_bytes: 0, c_wr_bytes: 0, client_addr: 97.83.98.141:61987, daemon_name: , escaper: , log_type: Task, pid: 1, r_rd_bytes: 0, r_wr_bytes: 0, reason: ForbiddenByRule, server_addr: 172.18.0.2:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Connecting, start_at: 2025-02-17T23:22:34.052462Z, task_event: finished, task_id: 11209a50ed8611ef8ada2a18c3342b79, task_type: TcpConnect, tcp_connect_tries: 0, total_time: 40.555µs, upstream: 127.0.0.1:8080, forbidden by rule: method unavailable
Gracefully stopping... (press Ctrl+C again to force)

compose.yml

root@server:~/stalwart# cat compose.yml

# networks:
#     g3-net:
#         driver: bridge
#         ipam:
#             config:
#                 - subnet: 10.5.0.0/16
#                   gateway: 10.5.0.1

services:
    g3proxy:
        image: g3proxy/comp:g3
        build:
            context: ./g3
            dockerfile: g3proxy/docker/debian.Dockerfile
        ports:
            - "88:88"
            - "80:80"
            - "8080:8080"
            - "443:443"
            - "25:25"
            - "587:587"
            - "465:465"
            - "143:143"
            - "993:993"
            - "4190:4190"
            - "110:110"
            - "995:995"
        # networks:
        #     g3-net:
        #         ipv4_address: 10.5.0.5
        volumes:
            - .:/g3/
        command:
            -c /g3/my_g3proxy.yml

    stalwart-mail:
        image: stalwartlabs/mail-server:latest
        hostname: stalwart-mail
        container_name: stalwart-mail
        # networks:
        #     g3-net:
        #         ipv4_address: 10.5.0.7
        volumes:
            - ./opt:/opt/stalwart-mail

my_g3proxy.yml

log: stdout

runtime:
    thread_number: 2

controller:
    local:
        recv_timeout: 30
        send_timeout: 1

resolver:
    - name: default
      type: c-ares

escaper:
    - name: esc_stwt_8080
      type: direct_fixed
      bind_ip: "127.0.0.1"
      resolver: default
      egress_network_filter:
        default: forbid
        permit:
          - 127.0.0.1


server:
    - name: stwt_http_8080
      escaper: esc_stwt_http_8080
      type: tcp_stream
      listen:
        address: "0.0.0.0:8080"
      proxy_pass: "127.0.0.1:8080"

docker ps

root@server:~/stalwart# docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS          PORTS          NAMES
98c468ce5c2f   stalwartlabs/mail-server:latest   "/bin/sh /usr/local/…"   34 minutes ago   Up 27 minutes   25/tcp, 
110/tcp, 
143/tcp, 
443/tcp, 
465/tcp, 
587/tcp, 
993/tcp, 
995/tcp, 
4190/tcp, 
8080/tcp
stalwart-mail


1f2f45801125   g3proxy/comp:g3                   "/usr/bin/g3proxy -c…"   34 minutes ago   Up 27 minutes   0.0.0.0:25->25/tcp, 
:::25->25/tcp, 
0.0.0.0:80->80/tcp, :::80->80/tcp, 
0.0.0.0:88->88/tcp, :::88->88/tcp, 
0.0.0.0:110->110/tcp, :::110->110/tcp, 
0.0.0.0:143->143/tcp, :::143->143/tcp, 
0.0.0.0:443->443/tcp, :::443->443/tcp, 
0.0.0.0:465->465/tcp, :::465->465/tcp, 
0.0.0.0:587->587/tcp, :::587->587/tcp, 
0.0.0.0:993->993/tcp, :::993->993/tcp, 
0.0.0.0:995->995/tcp, :::995->995/tcp, 
0.0.0.0:4190->4190/tcp, :::4190->4190/tcp, 
0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   
stalwart-g3proxy-1

2 replies

zh-jq Feb 18, 2025
Maintainer

you need to change the escaper name in the server config to esc_stwt_8080

shoutmarble Feb 27, 2025
Author

#557 (comment) I fixed the above my-g3proxy.yml in the below comment.

However, I'm using proxy_pass. but, I believe I should be using proxy_protocol
I'm investigating...

shoutmarble · 2025-02-18T05:00:36Z

shoutmarble
Feb 18, 2025
Author

I have mocked in all the ports to my g3proxy from my Stalwart-Mail.

I've looked at the SMTP ports: 25, 465, and 587
And they they look correct. If the Stalwart Mail is uses the actual client IP and the host IP
for verifying the send/receive of emails to the DNS SPF, DKIM, TXT entries it should be all set.

compose.yml

networks:
    g3-net:
        driver: bridge
        ipam:
            config:
                - subnet: 10.5.0.0/16
                  gateway: 10.5.0.1

services:
    g3proxy:
        image: g3proxy/comp:g3
        build:
            context: ./g3
            dockerfile: g3proxy/docker/debian.Dockerfile
        ports:
            - "88:88"
            - "80:80"
            - "8080:8080"
            - "443:443"
            - "25:25"
            - "587:587"
            - "465:465"
            - "143:143"
            - "993:993"
            - "4190:4190"
            - "110:110"
            - "995:995"
        networks:
            g3-net:
                ipv4_address: 10.5.0.5
        volumes:
            - .:/g3/
        command:
            -c /g3/my_g3proxy.yml

    stalwart-mail:
        image: stalwartlabs/mail-server:latest
        hostname: stalwart-mail
        container_name: stalwart-mail
        networks:
            g3-net:
                ipv4_address: 10.5.0.8
        volumes:
            - ./opt:/opt/stalwart-mail

my_g3proxy.yml

log: stdout

runtime:
    thread_number: 2

controller:
    local:
        recv_timeout: 30
        send_timeout: 1

server:
    - name: stwt_http_8080
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:8080"
      proxy_pass: "10.5.0.8:8080"
    - name: stwt_https_443
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:443"
      proxy_pass: "10.5.0.8:443"
    - name: stwt_smtp_25
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:25"
      proxy_pass: "10.5.0.8:25"
    - name: stwt_submission_587
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:587"
      proxy_pass: "10.5.0.8:587"
    - name: stwt_submission_465
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:465"
      proxy_pass: "10.5.0.8:465"
    - name: stwt_imap_143
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:143"
      proxy_pass: "10.5.0.8:143"
    - name: stwt_imaptls_993
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:993"
      proxy_pass: "10.5.0.8:993"
    - name: stwt_sieve_4190
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:4190"
      proxy_pass: "10.5.0.8:4190"
    - name: stwt_pop3_110
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:110"
      proxy_pass: "10.5.0.8:110"
    - name: stwt_pop3s_995
      escaper: esc_stwt
      type: tcp_stream
      listen:
        address: "10.5.0.5:995"
      proxy_pass: "10.5.0.8:995"

escaper:
    - name: esc_stwt
      type: direct_fixed
      bind_ip: "10.5.0.5"
      resolver: default
      egress_network_filter:
        default: forbid
        permit:
          - 10.5.0.8

resolver:
    - name: default
      type: c-ares

docker compose up

root@server:~/stalwart# docker compose up
[+] Running 9/9
 ✔ stalwart-mail Pulled                                                                                     8.8s 
   ✔ 3815f79548aa Pull complete                                                                             5.7s 
   ✔ d05ac034a050 Pull complete                                                                             5.7s 
   ✔ 69a176fe4a1f Pull complete                                                                             6.8s 
   ✔ ff59faa70d57 Pull complete                                                                             7.4s 
   ✔ 13eed8633459 Pull complete                                                                             7.6s 
   ✔ 599733dffedd Pull complete                                                                             7.6s 
   ✔ 6cc142849f3d Pull complete                                                                             7.6s 
 ! g3proxy Warning          pull access denied for g3proxy/comp, repository does not ex...                  0.9s 
[+] Building 697.2s (16/16) FINISHED                                                              docker:default
 => [g3proxy internal] load build definition from debian.Dockerfile                                         0.0s
 => => transferring dockerfile: 658B                                                                        0.0s 
 => [g3proxy internal] load metadata for docker.io/library/debian:bookworm-slim                             0.8s 
 => [g3proxy internal] load metadata for docker.io/library/rust:bookworm                                    1.1s 
 => [g3proxy internal] load .dockerignore                                                                   0.0s
 => => transferring context: 92B                                                                            0.0s 
 => [g3proxy builder 1/5] FROM docker.io/library/rust:bookworm@sha256:738ae99a3d75623f41e6882566b4ef37e38  42.3s 
 => => resolve docker.io/library/rust:bookworm@sha256:738ae99a3d75623f41e6882566b4ef37e38a9840244a47efd4a0  0.0s 
 => => sha256:cdb394ef6c0ddd4d077351c8ccd2d55af621b279730caf26cf113c1e562b8e78 4.36kB / 4.36kB              0.0s 
 => => sha256:a492eee5e55976c7d3feecce4c564aaf6f14fb07fdc5019d06f4154eddc93fde 48.48MB / 48.48MB            2.8s 
 => => sha256:32b550be6cb62359a0f3a96bc0dc289f8b45d097eaad275887f163c6780b4108 24.06MB / 24.06MB            4.3s 
 => => sha256:738ae99a3d75623f41e6882566b4ef37e38a9840244a47efd4a0ca22e9628b88 7.75kB / 7.75kB              0.0s 
 => => sha256:479476fa1dec14dfa9ed2dbcaa94cda5ab945e125d45c2d153267cc0135f3b69 1.94kB / 1.94kB              0.0s 
 => => sha256:35af2a7690f2b43e7237d1fae8e3f2350dfb25f3249e9cf65121866f9c56c772 64.39MB / 64.39MB            7.2s 
 => => sha256:7576b00d9bb10cc967bb5bdeeb3d5fa078ac8800e112aa03ed15ec199662d4f7 211.33MB / 211.33MB         15.2s 
 => => extracting sha256:a492eee5e55976c7d3feecce4c564aaf6f14fb07fdc5019d06f4154eddc93fde                   5.7s 
 => => sha256:d702af6a2b4f14eeed68b29d7e6676c2ac3a68619de372120a67811b71922b59 194.94MB / 194.94MB         17.1s
 => => extracting sha256:32b550be6cb62359a0f3a96bc0dc289f8b45d097eaad275887f163c6780b4108                   1.6s 
 => => extracting sha256:35af2a7690f2b43e7237d1fae8e3f2350dfb25f3249e9cf65121866f9c56c772                   6.1s 
 => => extracting sha256:7576b00d9bb10cc967bb5bdeeb3d5fa078ac8800e112aa03ed15ec199662d4f7                  15.9s 
 => => extracting sha256:d702af6a2b4f14eeed68b29d7e6676c2ac3a68619de372120a67811b71922b59                   9.1s 
 => [g3proxy internal] load build context                                                                   0.4s
 => => transferring context: 8.99MB                                                                         0.4s 
 => [g3proxy stage-1 1/4] FROM docker.io/library/debian:bookworm-slim@sha256:40b107342c492725bc7aacbe93a49  7.1s
 => => resolve docker.io/library/debian:bookworm-slim@sha256:40b107342c492725bc7aacbe93a49945445191ae36418  0.0s 
 => => sha256:44bccdd61bf09a081b1db8c61cf49bfabf30ac7afcc970010137c0ab587b209c 1.02kB / 1.02kB              0.0s
 => => sha256:a916d01f3536e5997c7efefcb146c7b80d355f2a51d86b75ad1a339a0f9582ea 453B / 453B                  0.0s
 => => sha256:c29f5b76f736a8b555fd191c48d6581bb918bcd605a7cbcc76205dd6acff3260 28.21MB / 28.21MB            2.9s
 => => sha256:40b107342c492725bc7aacbe93a49945445191ae364184a6d24fedb28172f6f7 8.56kB / 8.56kB              0.0s
 => => extracting sha256:c29f5b76f736a8b555fd191c48d6581bb918bcd605a7cbcc76205dd6acff3260                   4.0s
 => [g3proxy stage-1 2/4] RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt  13.4s 
 => [g3proxy builder 2/5] WORKDIR /usr/src/g3                                                               1.2s 
 => [g3proxy builder 3/5] COPY . .                                                                          0.5s 
 => [g3proxy builder 4/5] RUN apt-get update && apt-get install -y libclang-dev cmake capnproto            16.4s 
 => [g3proxy builder 5/5] RUN cargo build --profile release-lto  --no-default-features --features vendor  635.1s 
 => [g3proxy stage-1 3/4] COPY --from=builder /usr/src/g3/target/release-lto/g3proxy /usr/bin/g3proxy       0.1s 
 => [g3proxy stage-1 4/4] COPY --from=builder /usr/src/g3/target/release-lto/g3proxy-ctl /usr/bin/g3proxy-  0.0s 
 => [g3proxy] exporting to image                                                                            0.1s 
 => => exporting layers                                                                                     0.1s 
 => => writing image sha256:527ce6969fc30a0d242a612f36fad8a0533ad474e2c5c84fe60e030db247d1e9                0.0s 
 => => naming to docker.io/g3proxy/comp:g3                                                                  0.0s 
 => [g3proxy] resolving provenance for metadata file                                                        0.0s 
[+] Running 4/4
 ✔ g3proxy                       Built                                                                      0.0s 
 ✔ Network stalwart_g3-net       Created                                                                    0.1s 
 ✔ Container stalwart-mail       Created                                                                    0.1s 
 ✔ Container stalwart-g3proxy-1  Created                                                                    0.1s 
Attaching to g3proxy-1, stalwart-mail
stalwart-mail  | ✅ Configuration file written to /opt/stalwart-mail/etc/config.toml
stalwart-mail  | 🔑 Your administrator account is 'xxxxxxxxxxx' with password 'xxxxxxxxxxx'.
g3proxy-1      | Feb 18 04:37:02.860 INFO c_rd_bytes: 87, c_wr_bytes: 2299, client_addr: 97.83.98.141:64324, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:49444, next_peer_addr: 10.5.0.8:8080, pid: 1, r_rd_bytes: 2299, r_wr_bytes: 87, ready_time: 331.863µs, reason: ClosedByClient, server_addr: 10.5.0.ame: stwt_http_8080, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:37:02.796443Z, task_event: finished, task_id: ffc67216edb111ef986ed3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 289.756µs, tcp_connect_tries: 1, total_time: 63.626ms, upstream: 10.5.0.8:8080, closed by client
g3proxy-1      | Feb 18 04:40:54.450 INFO c_rd_bytes: 3845, c_wr_bytes: 322592, client_addr: 97.83.98.1t, t, server_addr: 10.5.0.5:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:34:58.068074Z, task_event: finished, task_id: b56e682cedb111ef986cd3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 585.354µs, tcp_connect_tries: 1, total_time: 356.382s, upstream: 10.5.0.8:8080, closed by client
g3proxy-1      | Feb 18 04:40:54.450 INFO c_rd_bytes: 801, c_wr_bytes: 68202, client_addr: 97.83.98.141:64314, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:33678, next_peer_addr: 10.5.0.8:8080, pid: 1, r_rd_bytes: 68202, r_wr_bytes: 801, ready_time: 640.998µs, reason: ClosedByClient, server_addr: 10.5.0.5:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:34:58.057261Z, task_event: finished, task_id: b56cc1cbedb111ef986bd3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 588.740µs, tcp_connect_tries: 1, total_time: 356.393s, upstream: 10.5.0.8:8080, closed by client
g3proxy-1      | Feb 18 04:40:54.450 INFO c_rd_bytes: 2394, c_wr_bytes: 6868118, client_addr: 97.83.98.141:64315, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:33692, next_peer_addr: 10.5.0.8:8080, pid: 1, r_rd_bytes: 6868118, r_wr_bytes: 2394, ready_time: 452.218µs, reason: ClosedByClient, server_addr: 10.5.0.5:8080, server_name: stwt_http_8080, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:34:58.210285Z, task_event: finished, task_id: b5841b42edb111ef986dd3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 403.057µs, tcp_connect_tries: 1, total_time: 356.240s, upstream: 10.5.0.8:8080, closed by client
g3proxy-1      | Feb 18 04:48:23.122 INFO c_rd_bytes: 11, c_wr_bytes: 7, client_addr: 97.83.98.141:62858, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:57852, next_peer_addr: 10.5.0.8:465, pid: 1, r_rd_bytes: 7, r_wr_bytes: 11, ready_time: 518.961µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:465, server_name: stwt_submission_465, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:48:18.907789Z, task_event: finished, task_id: 92c4d989edb311ef9871d3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 477.715µs, tcp_connect_tries: 1, total_time: 4.215s, upstream: 10.5.0.8:465, closed by upstream
g3proxy-1      | Feb 18 04:48:35.315 INFO c_rd_bytes: 12, c_wr_bytes: 7, client_addr: 97.83.98.141:62856, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:41662, next_peer_addr: 10.5.0.8:465, pid: 1, r_rd_bytes: 7, r_wr_bytes: 12, ready_time: 262.666µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:465, server_name: stwt_submission_465, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:48:32.447769Z, task_event: finished, task_id: 9ad6e2faedb311ef9872d3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 222.332µs, tcp_connect_tries: 1, total_time: 2.867s, upstream: 10.5.0.8:465, closed by upstream
g3proxy-1      | Feb 18 04:48:59.525 INFO c_rd_bytes: 21, c_wr_bytes: 7, client_addr: 97.83.98.141:62902, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:49232, next_peer_addr: 10.5.0.8:465, pid: 1, r_rd_bytes: 7, r_wr_bytes: 21, ready_time: 353.986µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:465, server_name: stwt_submission_465, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:48:47.710957Z, task_event: finished, task_id: a3efdd4aedb311ef9873d3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 311.998µs, tcp_connect_tries: 1, total_time: 11.814s, upstream: 10.5.0.8:465, closed by upstream
g3proxy-1      | Feb 18 04:52:10.053 INFO c_rd_bytes: 75, c_wr_bytes: 481, client_addr: 97.83.98.141:62870, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:36064, next_peer_addr: 10.5.0.8:25, pid: 1, r_rd_bytes: 481, r_wr_bytes: 75, ready_time: 484.888µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:25, server_name: stwt_smtp_25, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:45:54.518534Z, task_event: finished, task_id: 3cb4cc3dedb311ef986fd3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 442.059µs, tcp_connect_tries: 1, total_time: 375.535s, upstream: 10.5.0.8:25, closed by upstream
g3proxy-1      | Feb 18 04:52:51.796 INFO c_rd_bytes: 13, c_wr_bytes: 334, client_addr: 97.83.98.141:62864, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:35858, next_peer_addr: 10.5.0.8:587, pid: 1, r_rd_bytes: 334, r_wr_bytes: 13, ready_time: 400.151µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:587, server_name: stwt_submission_587, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:47:41.194116Z, task_event: finished, task_id: 7c4a3328edb311ef9870d3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 363.332µs, tcp_connect_tries: 1, total_time: 310.602s, upstream: 10.5.0.8:587, closed by upstream
g3proxy-1      | Feb 18 04:57:31.349 INFO c_rd_bytes: 449, c_wr_bytes: 1056, client_addr: 97.83.98.141:62922, daemon_name: , escaper: esc_stwt, log_type: Task, next_bind_ip: 10.5.0.5, next_bound_addr: 10.5.0.5:34142, next_peer_addr: 10.5.0.8:465, pid: 1, r_rd_bytes: 1056, r_wr_bytes: 449, ready_time: 353.604µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:465, server_name: stwt_submission_465, server_type: TcpStream, stage: Relaying, start_at: 2025-02-18T04:52:16.023925Z, task_event: finished, task_id: 2019e697edb411ef9874d3e3dc139339, task_type: TcpConnect, tcp_connect_spend: 319.932µs, tcp_connect_tries: 1, total_time: 315.325s, upstream: 10.5.0.8:465, closed by upstream

openssl s_client -connect mail.landingdev.xyz:465 -crlf -ign_eof

XXXXX@DESKTOP-UU82O8K:/mnt/c/Users/Dad$ openssl s_client -connect mail.landingdev.xyz:465 -crlf -ign_eof  
CONNECTED(00000003)
depth=0 CN = rcgen self signed cert
verify error:num=18:self-signed certificate
verify return:1
depth=0 CN = rcgen self signed cert
verify return:1
---
Certificate chain
 0 s:CN = rcgen self signed cert
   i:CN = rcgen self signed cert
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA256
   v:NotBefore: Jan  1 00:00:00 1975 GMT; NotAfter: Jan  1 00:00:00 4096 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIBXTCCAQSgAwIBAgIUYBbYunEn+Ax74Rp1UmaSqWZ5gD0wCgYIKoZIzj0EAwIw
ITEfMB0GA1UEAwwWcmNnZW4gc2VsZiBzaWduZWQgY2VydDAgFw03NTAxMDEwMDAw
MDBaGA80MDk2MDEwMTAwMDAwMFowITEfMB0GA1UEAwwWcmNnZW4gc2VsZiBzaWdu
ZWQgY2VydDBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKfZ3qbY/JVqD3q7YxPN
RlTp4mv56Mdcp3oSIQdljaDCxgsx56jwTlQE123jcy/CC6V/Ob4cvaYz6l5MoFC8
rSOjGDAWMBQGA1UdEQQNMAuCCWxvY2FsaG9zdDAKBggqhkjOPQQDAgNHADBEAiAe
oGJGyiIUVxTj0Ig/ZK9/z0edTGGF2k15H4r9GlwLJQIgIy+1qEHinNmUq3PtFN+J
MPiXGxK91BVJD+FBeu+DRUE=
-----END CERTIFICATE-----
subject=CN = rcgen self signed cert
issuer=CN = rcgen self signed cert
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 662 bytes and written 401 bytes
Verification error: self-signed certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 18 (self-signed certificate)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 5F8D09AE95900D31EE45C6306EC7FD03E187BA0A8022E4550F1EB19B7DE54811
    Session-ID-ctx:
    Resumption PSK: 164F5D19C44A55A4971CD0F6F1286391782B24EC518C1E8D6566CA5F5D080C96053ADA7B941C7B1D07DF7941129DA965
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
    0000 - 8d cb 54 b5 d8 e8 a5 c5-3b 3b 19 58 d3 1b 0d a1   ..T.....;;.X....
    0010 - 80 86 af 89 a7 00 2c c5-6c 99 e3 36 59 de 40 27   ......,.l..6Y.@'

    Start Time: 1739854335
    Timeout   : 7200 (sec)
    Verify return code: 18 (self-signed certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 0CE827DE497EA4499C05FCB93CCBE7B4C6F938DBD3265E70589C523D4E7A5D4E
    Session-ID-ctx:
    Resumption PSK: 3ECAF88D72449C19A834F784B83F2401DAE231FBD7DA7C1FF1428BA2AD4B3BA3DCABBC19B2190BD75C1D363B7C0FD6AE
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
    0000 - 59 8e 34 8d b1 b3 f9 7e-60 3e 3d bb 47 01 ff 44   Y.4....~`>=.G..D
    0010 - e7 1d 01 72 0e bb ef ae-d3 26 06 b5 5b 78 4b 92   ...r.....&..[xK.

    Start Time: 1739854335
    Timeout   : 7200 (sec)
    Verify return code: 18 (self-signed certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
220 stalwart-mail Stalwart ESMTP at your service
HELO mail.landingdev.xyz
250 stalwart-mail you had me at HELO

0 replies

shoutmarble · 2025-02-27T21:42:18Z

shoutmarble
Feb 27, 2025
Author

I'm looking diagnostically at plain_tls_port being processed by tcp_stream to understand g3proxy server sections.

This example successfully pulls a TLS encrypted webpage from my VPS using port 443

curl -l -vvv https://who.landingdev.xyz:443

C:\Users\Dad\Documents>curl -l -vvv https://who.landingdev.xyz:443
* Host who.landingdev.xyz:443 was resolved.
* IPv6: (none)
* IPv4: 185.28.22.166
*   Trying 185.28.22.166:443...
* Connected to who.landingdev.xyz (185.28.22.166) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server did not agree on a protocol. Uses default.
* using HTTP/1.x
> GET / HTTP/1.1
> Host: who.landingdev.xyz
> User-Agent: curl/8.8.0
> Accept: */*
>
* Request completely sent off
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 200 OK
< Date: Thu, 27 Feb 2025 21:01:22 GMT
< Content-Length: 157
< Content-Type: text/plain; charset=utf-8
<
Hostname: who
IP: 127.0.0.1
IP: ::1
IP: 10.5.0.9
RemoteAddr: 10.5.0.5:41048
GET / HTTP/1.1
Host: who.landingdev.xyz
User-Agent: curl/8.8.0
Accept: */*

* Connection #0 to host who.landingdev.xyz left intact

C:\Users\Dad\Documents>

docker compose up

root@server:~/stalwart# docker compose up
[+] Running 2/2
 ✔ Container stalwart-g3proxy-1  Created                                   0.0s 
 ✔ Container who                 Created                                   0.0s 
Attaching to g3proxy-1, who
who        | 2025/02/27 20:59:55 Starting up on port 80
g3proxy-1  | Feb 27 21:01:09.176 INFO c_rd_bytes: 81, c_wr_bytes: 275, client_addr: 97.83.98.141:65349, daemon_name: , escaper: default, log_type: Task, next_bound_addr: 10.5.0.5:35574, next_peer_addr: 10.5.0.9:80, pid: 1, r_rd_bytes: 275, r_wr_bytes: 81, ready_time: 361.409µs, reason: ClosedByClient, server_addr: 10.5.0.5:443, server_name: tcp, server_type: TcpStream, stage: Relaying, start_at: 2025-02-27T21:01:09.091736Z, task_event: finished, task_id: f7d3d7f3f54d11ef9dabcabaad70cfcf, task_type: TcpConnect, tcp_connect_spend: 285.539µs, tcp_connect_tries: 1, total_time: 84.540ms, upstream: 10.5.0.9:80, closed by client
g3proxy-1  | Feb 27 21:01:22.333 INFO c_rd_bytes: 81, c_wr_bytes: 275, client_addr: 97.83.98.141:65357, daemon_name: , escaper: default, log_type: Task, next_bound_addr: 10.5.0.5:41048, next_peer_addr: 10.5.0.9:80, pid: 1, r_rd_bytes: 275, r_wr_bytes: 81, ready_time: 353.464µs, reason: ClosedByClient, server_addr: 10.5.0.5:443, server_name: tcp, server_type: TcpStream, stage: Relaying, start_at: 2025-02-27T21:01:22.263621Z, task_event: finished, task_id: ffadb6b7f54d11ef9daccabaad70cfcf, task_type: TcpConnect, tcp_connect_spend: 291.089µs, tcp_connect_tries: 1, total_time: 70.232ms, upstream: 10.5.0.9:80, closed by client
g3proxy-1  | Feb 27 21:06:04.593 INFO c_rd_bytes: 198, c_wr_bytes: 411, client_addr: 10.5.0.1:57128, daemon_name: , escaper: default, log_type: Task, next_bound_addr: 10.5.0.5:40294, next_peer_addr: 10.5.0.9:80, pid: 1, r_rd_bytes: 411, r_wr_bytes: 198, ready_time: 312.037µs, reason: ClosedByUpstream, server_addr: 10.5.0.5:443, server_name: tcp, server_type: TcpStream, stage: Relaying, start_at: 2025-02-27T21:06:01.593846Z, task_event: finished, task_id: a62c1fa2f54e11ef9dadcabaad70cfcf, task_type: TcpConnect, tcp_connect_spend: 244.723µs, tcp_connect_tries: 1, total_time: 3.000s, upstream: 10.5.0.9:80, closed by upstream

compose.yml

networks:
    g3-net:
        driver: bridge
        ipam:
            config:
                - subnet: 10.5.0.0/16
                  gateway: 10.5.0.1

services:
    g3proxy:
        image: g3proxy/comp:g3
        build:
            context: ./g3
            dockerfile: g3proxy/docker/debian.Dockerfile
        ports:
            - "88:88"
            - "80:80"
            - "8080:8080"
            - "443:443"
            - "25:25"
            - "587:587"
            - "465:465"
            - "143:143"
            - "993:993"
            - "4190:4190"
            - "110:110"
            - "995:995"
        networks:
            g3-net:
                ipv4_address: 10.5.0.5
        volumes:
            - .:/g3/
        command:
            -c /g3/my_g3proxy.yml


    who:
        image: traefik/whoami:latest
        hostname: who
        container_name: who
        networks:
            g3-net:
                ipv4_address: 10.5.0.9

my_g3proxy.yml

log: stdout

resolver:
    - name: default
      type: c-ares

escaper:
    - name: default
      type: direct_fixed
      resolve_strategy: IPv4Only
      resolver: default

server:
  - name: tcp
    escaper: default
    type: tcp_stream
    proxy_pass:
      - "10.5.0.9:80"

  - name: tls
    type: plain_tls_port
    listen:
      address: 10.5.0.5:443
    tls_server:
      cert_pairs:
        certificate: /g3/certs/landingdev.xyz.crt
        private_key: /g3/certs/landingdev.xyz.key
    server: tcp

FILES

root@server:~/stalwart# tree -L 2
.
├── certs
│   ├── landingdev.xyz.crt
│   ├── landingdev.xyz.issuer.crt
│   ├── landingdev.xyz.json
│   ├── landingdev.xyz.key
│   └── landingdev.xyz.pem
├── compose.yml
├── g3
│   │
│   ├── ...
│   ├── g3proxy
│   └── ...
├── my_g3proxy.yml
└── ...

docker ps

root@server:~/stalwart# docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Status}}"
CONTAINER ID   IMAGE                   STATUS
4ba541ced706   g3proxy/comp:g3         Up 26 minutes
3dec1db46bda   traefik/whoami:latest   Up 26 minutes

0 replies

shoutmarble · 2025-02-28T02:51:05Z

shoutmarble
Feb 28, 2025
Author

Is there a way to implement the proxy_protocol example around my who compose.yml service to inspect the I/O proxy_protocol from the who service.

The PIPELINING of servers and escapers is very elegant but I do not have the feel for it yet.

+--------------------------------------------------------------+     +-----------------------------------------+
|                                                              |     |                                         |
|                                                              |     |                                         |
|   networks:                                                  |     |     log: stdout                         |
|       g3-net:                                                |     |                                         |
|           driver: bridge                                     |     |     resolver:                           |
|           ipam:                                              |     |         - name: default                 |
|               config:                                        |     |           type: c-ares                  |
|                   - subnet: 10.5.0.0/16                      |     |                                         |
|                     gateway: 10.5.0.1                        |     |     escaper:                            |
|                                                              |     |         - name: default                 |
|   services:                                                  |     |           type: direct_fixed            |
|       g3proxy:                                               |     |           resol^e_strategy: IPv4Only    |
|           container_name: g3proxy                            |     |           resolver: default             |
|           image: g3proxy/comp:g3                             |     |                                         |
|           build:                                             |     |         - name: proxy_pp2               |
|               context: ./g3                                  |     |           type: proxy_http              |
|               dockerfile: g3proxy/docker/debian.Dockerfile   |     |           proxy_addr: "10.5.0.9:80"     |
|           ports:                                             |     |           use_proxy_protocol: 2         |
|               - "88:88"                                      |     |                                         |
|               | "80:80"                                      |     |     server:                             |
|               | "8080:8080"                                  |     |                                         |
|               | "443:443"                                    |     |       - name: http2                     |
|               | "25:25"                                      |     |         escaper: proxy_pp2              |
|               | "587:587"                                    |     |         type: http_proxy                |
|               | "465:465"                                    |     |         listen:                         |
|               | "143:143"                                    |     |           address: "10.5.0.5:8080"      |
|               | "993:993"                                    |     |                                         |
|               | "4190:4190"                                  |     |       - name: port_pp2                  |
|               | "110:110"                                    |     |         type: plain_tcp_port            |
|               - "995:995"                                    |     |         server: http0                   |
|           networks:                                          |     |         proxy_protocol: 2               |
|               g3-net:                                        |     |         listen: "10.5.0.9:80"           |
|                   ipv4_address: 10.5.0.5                     |     |                                         |
|           volumes:                                           |     |       - name: http0                     |
|               - .:/g3/                                       |     |         escaper: default                |
|           command:                                           |     |         type: http_proxy                |
|               -c /g3/my_g3proxy.yml                          |     |                                         |
|                                                              |     |                                         |
|       who:                                                   |     +-----------------------------------------+
|           image: traefik/whoami:latest                       |
|           hostname: who                                      |
|           container_name: who                                |
|           networks:                                          |
|               g3-net:                                        |
|                   ipv4_address: 10.5.0.9                     |
|                                                              |
|                                                              |
|                                                              |
+--------------------------------------------------------------+



 +--------------------------------------------+       +-------------------------------------------------------+
 |                                            |       |                                                       |
 |  CLIENT                                    |       |  VPS                                                  |
 |  IP: 11.22.33.44                           |       |  IP: 44.55.66.77                                      |
 |                                            |       |                                                       |
 |                                            |       |                                                       |
 |                                            |       |                                                       |
 |                                            |       |   +-------------------+         +------------------+  |
 |                                            |       |   | G3PROXY           |         | WHO              |  |
 |                                            |       |   | IP: 10.5.0.5      |         | IP: 10.5.0.9     |  |
 |                                            |       |   |                   | proxy   |                  |  |
 |                                            |       |   |    +------------+ | protocol| +--------------+ |  |
 |                                            |       |   |    | ESCAPER    +-------------+ ESCAPER      | |  |
 |                                            |       |   |    | 10.5.0.9:80| |         | | 10.5.0.5:8080| |  |
 |                                            |       |   |    +------------+ |         | +--------------+ |  |
 |                                            |       |   |                   |         |                  |  |
 |                                            |       |   |                   |         |                  |  |
 |                                            |       |   +-------------------+         +------------------+  |
 |                                            |       |                                                       |
 |                                            |       |                                                       |
 |                                            |       |                                                       |
 |                                            |       |                                                       |
 |                                            |       |                                                       |
 +--------------------------------------------+       +-------------------------------------------------------+

compose.yml

networks:
    g3-net:
        driver: bridge
        ipam:
            config:
                - subnet: 10.5.0.0/16
                  gateway: 10.5.0.1

services:
    g3proxy:
        container_name: g3proxy
        image: g3proxy/comp:g3
        build:
            context: ./g3
            dockerfile: g3proxy/docker/debian.Dockerfile
        ports:
            - "88:88"
            - "80:80"
            - "8080:8080"
            - "443:443"
            - "25:25"
            - "587:587"
            - "465:465"
            - "143:143"
            - "993:993"
            - "4190:4190"
            - "110:110"
            - "995:995"
        networks:
            g3-net:
                ipv4_address: 10.5.0.5
        volumes:
            - .:/g3/
        command:
            -c /g3/my_g3proxy.yml

    who:
        image: traefik/whoami:latest
        hostname: who
        container_name: who
        networks:
            g3-net:
                ipv4_address: 10.5.0.9

my_g3proxy.yml

log: stdout

resolver:
    - name: default
      type: c-ares

escaper:
    - name: default
      type: direct_fixed
      resolve_strategy: IPv4Only
      resolver: default

    - name: proxy_pp2
      type: proxy_http
      proxy_addr: "10.5.0.9:80"
      use_proxy_protocol: 2

server:

  - name: http2
    escaper: proxy_pp2
    type: http_proxy
    listen:
      address: "10.5.0.5:8080"

  - name: port_pp2
    type: plain_tcp_port
    server: http0
    proxy_protocol: 2
    listen: "10.5.0.9:80"

  - name: http0
    escaper: default
    type: http_proxy

5 replies

zh-jq-b Feb 28, 2025
Maintainer

#582 This PR adds support for proxy_protocol in direct_fixed escaper, which will support this usecase.

shoutmarble Feb 28, 2025
Author

I'm trying it out now, thanks!

root@server:~/stalwart/g3# git log --pretty=format:"%h - %s" -n 1
c6d11205 - g3proxy: add proxy_protocol support in direct_fixed escaper

shoutmarble Mar 3, 2025
Author

My diagram is a mess.

I have a the http2 g3proxy server listening to incoming traffic from the CLIENT, 10.5.0.5
And I have port_pp2 g3proxy server listening to the incoming traffic from WHO, 10.5.0.9

The traffic incoming to proxy_pp2 10.5.0.5:8080 is escaped by proxy_pp2 and ...served by a proxy listener port_pp2?
This expectantly appears backwards and doesn't work. (not for reasons that I completely understand
for the compelling reasons that this is above my head currently right now)

                                    ┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
                                    │                                                                                                          │
                                    │ VPS: 22.22.22.22                                                                                         │
                                    │                                                                                                          │
                                    │  ┌──────────────────────────────────────────────────────────────────────────┐                            │
                                    │  │                        ┌───────────────────────────────────┐             │                            │
┌──────────────────────────┐        │  │  G3PROXY 10.5.0.5      │   server:                         │             │                            │
│                          │        │  │                        │     - name: http2                 │             │                            │
│ CLIENT IP: 11.11.11.11   │        │  │                        │     ┌────────────────────┐        │             │                            │
│                          │        │  │                  ┌─────────────escaper: proxy_pp2 │        │             │                            │
│ ┌─────────────────┐   ┌────┐   ┌──────────────────┐     │     │     └────────────────────┘        │             │                            │
│ │ BROWSER:        │   │  │ │   │resolver:         │─────│─┐   │       type: http_proxy            │             │                            │
│ │ PORTS: 80/443   │   │  │ └────  - name: default │     │ │   │       listen:                     │             │                            │
│ │                 │   │  │ │   │    type: c-ares  │◄──┐ │ │   │       ┌──────────────────────────┐│             │                            │
│ │                 │   └────┘   └──────────────────┘   │ │ └──────────►│ address: "10.5.0.5:8080" ││             │                            │
│ │                 │      │        │  │                │ │     │       └──────────────────────────┘│             │                            │
│ └─────────────────┘      │        │  │                │ │     │                                   │             │                            │
│                          │        │  │                │ │     └───────────────────────────────────┘             │                            │
│                          │        │  │                │ │                                                       │                            │
│                          │        │  │                └─│───────────────────┐   ┌────────────────────────────┐  │                            │
│                          │        │  │                  │                   │   │ [SERVER]                   │  │                            │
└──────────────────────────┘        │  │ ┌────────────────│───────────────┐   └───│   - name: port_pp2         │  │                            │
                                    │  │ │ escaper:       ▼               │       │     type: plain_tcp_port   │  │                            │
                                    │  │ │   - name: proxy_pp2            │     ┌───    server: http0          │  │                            │
                                    │  │ │     type: proxy_http           │     │ │     proxy_protocol: 2      │  │                            │
                                    │  │ │     proxy_addr: "10.5.0.9:80"─────┐  │ │     listen: "10.5.0.9:80"◄───────────────────────────────┐ │
                                    │  │ │     use_proxy_protocol: 2      │  │  │ │                            │  │                          │ │
                                    │  │ │                                │  │  │ └────────────────────────────┘  │   ┌──────────────────┐   │ │
                                    │  │ └────────────────────────────────┘  │  │                                 │   │                  │   │ │
                                    │  │                                     │  │                                 │   │ WHO (SERVICE)    │   │ │
                                    │  │                                     └──│──────────────────────────────────────►IP: 10.5.0.9:80 ─────┘ │
                                    │  │                                        │                                 │   │                  │     │
                                    │  │  ┌──────────────────────────┐          │ ┌────────────────────────┐      │   │                  │     │
                                    │  │  │                          │          │ │                        │      │   │                  │     │
                                    │  │  │ [ESCAPER]                │          │ │  [SERVER]              │      │   └──────────────────┘     │
                                    │  │  │   - name: default  ◄───────────┐    └───►  - name: http0       │      │                            │
                                    │  │  │     type: direct_fixed   │     └─────────    escaper: default  │      │                            │
                                    │  │  │     resolver: default    │            │      type: http_proxy  │      │                            │
                                    │  │  │                          │            │                        │      │                            │
                                    │  │  └──────────────────────────┘            └────────────────────────┘      │                            │
                                    │  └──────────────────────────────────────────────────────────────────────────┘                            │
                                    │                                                                                                          │
                                    └──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

compose.yml

networks:
    g3-net:
        driver: bridge
        ipam:
            config:
                - subnet: 10.5.0.0/16
                  gateway: 10.5.0.1

services:
    g3proxy:
        container_name: g3proxy
        image: g3proxy/comp:g3
        build:
            context: ./g3
            dockerfile: g3proxy/docker/debian.Dockerfile
        ports:
            - "88:88"
            - "80:80"
            - "8080:8080"
            - "443:443"
        networks:
            g3-net:
                ipv4_address: 10.5.0.5
        volumes:
            - .:/g3/
        command:
            -c /g3/my_g3proxy.yml


    who:
        image: traefik/whoami:latest
        hostname: who
        container_name: who
        networks:
            g3-net:
                ipv4_address: 10.5.0.9

my_g3proxy.yml

server:
  - name: http2
    escaper: proxy_pp2
    type: http_proxy
    listen:
      address: "10.5.0.5:8080"

  - name: http0
    escaper: default
    type: http_proxy

  - name: port_pp2
    type: plain_tcp_port
    server: http0
    proxy_protocol: 2
    listen: "10.5.0.9:80"

escaper:
  - name: proxy_pp2
    type: proxy_http
    proxy_addr: "10.5.0.9:80"
    use_proxy_protocol: 2
  - name: default
    type: direct_fixed
    resolver: default

resolver:
  - name: default
    type: c-ares

docker compose up

root@server:~/stalwart/test# docker compose up
[+] Running 2/2
 ✔ Container g3proxy  Created                                         0.0s 
 ✔ Container who      Created                                         0.0s 
Attaching to g3proxy, who
who      | 2025/03/03 05:12:41 Starting up on port 80
g3proxy  | Error: failed to spawn all servers
g3proxy  |
g3proxy  | Caused by:
g3proxy  |     Cannot assign requested address (os error 99)
g3proxy exited with code 1
Gracefully stopping... (press Ctrl+C again to force)
[+] Stopping 2/2
 ✔ Container who      Stopped                                         0.1s 
 ✔ Container g3proxy  Stopped                                         0.0s 
root@server:~/stalwart/test#

zh-jq-b Mar 3, 2025
Maintainer

g3proxy can't start as the address "10.5.0.9:80" has been used by "WHO"?
And in this case I think it's more suitable to use http_rproxy instead of http_proxy?

zh-jq-b Mar 3, 2025
Maintainer

you can try something like:

server:
  - name: http0
    escaper: default-with-pp2
    type: http_rproxy
    listen:
      address: 0.0.0.0:80
    hosts:
      - exact_match: mail.landingdev.xyz
        upstream: stalwart-mail:8080

escaper:
  - name: default-with-pp2
    type: direct_fixed
    resolver: default
    use_proxy_protocol: 2
    # other params

resolver:
  - name: default
    type: c-ares

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker-ized Stalwart Email Server with Socks/Proxy Protocol Attempt #557

{{title}}

Replies: 5 comments 11 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Docker-ized Stalwart Email Server with Socks/Proxy Protocol Attempt #557

shoutmarble Feb 17, 2025

Replies: 5 comments · 11 replies

zh-jq-b Feb 17, 2025 Maintainer

shoutmarble Feb 17, 2025 Author

shoutmarble Feb 26, 2025 Author

zh-jq Feb 27, 2025 Maintainer

shoutmarble Feb 27, 2025 Author

shoutmarble Feb 18, 2025 Author

zh-jq Feb 18, 2025 Maintainer

shoutmarble Feb 27, 2025 Author

shoutmarble Feb 18, 2025 Author

shoutmarble Feb 27, 2025 Author

shoutmarble Feb 28, 2025 Author

zh-jq-b Feb 28, 2025 Maintainer

shoutmarble Feb 28, 2025 Author

shoutmarble Mar 3, 2025 Author

zh-jq-b Mar 3, 2025 Maintainer

zh-jq-b Mar 3, 2025 Maintainer

shoutmarble
Feb 17, 2025

Replies: 5 comments 11 replies

zh-jq-b
Feb 17, 2025
Maintainer

shoutmarble Feb 17, 2025
Author

shoutmarble Feb 26, 2025
Author

zh-jq Feb 27, 2025
Maintainer

shoutmarble Feb 27, 2025
Author

shoutmarble
Feb 18, 2025
Author

zh-jq Feb 18, 2025
Maintainer

shoutmarble Feb 27, 2025
Author

shoutmarble
Feb 18, 2025
Author

shoutmarble
Feb 27, 2025
Author

shoutmarble
Feb 28, 2025
Author

zh-jq-b Feb 28, 2025
Maintainer

shoutmarble Feb 28, 2025
Author

shoutmarble Mar 3, 2025
Author

zh-jq-b Mar 3, 2025
Maintainer

zh-jq-b Mar 3, 2025
Maintainer