Internal Server Error With Caddy and Cloudflare

[Menghini]

Hello,

I am attempting to use Zulip with Caddy and Cloudflare.

version: "2"
services:
  database:
    image: "zulip/zulip-postgresql:14"
    restart: unless-stopped
    environment:
      POSTGRES_DB: "zulip"
      POSTGRES_USER: "zulip"
      # Note that you need to do a manual `ALTER ROLE` query if you
      # change this on a system after booting the postgres container
      # the first time on a host.  Instructions are available in README.md.
      POSTGRES_PASSWORD: "REPLACE_WITH_SECURE_POSTGRES_PASSWORD"
    volumes:
      - "postgresql-14:/var/lib/postgresql/data:rw"
  memcached:
    image: "memcached:alpine"
    restart: unless-stopped
    command:
      - "sh"
      - "-euc"
      - |
        echo 'mech_list: plain' > "$$SASL_CONF_PATH"
        echo "zulip@$$HOSTNAME:$$MEMCACHED_PASSWORD" > "$$MEMCACHED_SASL_PWDB"
        echo "zulip@localhost:$$MEMCACHED_PASSWORD" >> "$$MEMCACHED_SASL_PWDB"
        exec memcached -S
    environment:
      SASL_CONF_PATH: "/home/memcache/memcached.conf"
      MEMCACHED_SASL_PWDB: "/home/memcache/memcached-sasl-db"
      MEMCACHED_PASSWORD: "REPLACE_WITH_SECURE_MEMCACHED_PASSWORD"
  rabbitmq:
    image: "rabbitmq:3.7.7"
    restart: unless-stopped
    environment:
      RABBITMQ_DEFAULT_USER: "zulip"
      RABBITMQ_DEFAULT_PASS: "REPLACE_WITH_SECURE_RABBITMQ_PASSWORD"
    volumes:
      - "rabbitmq:/var/lib/rabbitmq:rw"
  redis:
    image: "redis:alpine"
    restart: unless-stopped
    command:
      - "sh"
      - "-euc"
      - |
        echo "requirepass '$$REDIS_PASSWORD'" > /etc/redis.conf
        exec redis-server /etc/redis.conf
    environment:
      REDIS_PASSWORD: "REPLACE_WITH_SECURE_REDIS_PASSWORD"
    volumes:
      - "redis:/data:rw"
  zulip:
    image: "zulip/docker-zulip:8.4-0"
    restart: unless-stopped
    build:
      context: .
      args:
        # Change these if you want to build zulip from a different repo/branch
        ZULIP_GIT_URL: https://github.com/zulip/zulip.git
        ZULIP_GIT_REF: "8.4"
        # Set this up if you plan to use your own CA certificate bundle for building
        # CUSTOM_CA_CERTIFICATES:
    ports:
      - "8751:80"
      #- "443:443"
    environment:
      DB_HOST: "database"
      DB_HOST_PORT: "5432"
      DB_USER: "zulip"
      SSL_CERTIFICATE_GENERATION: "self-signed"
      SETTING_MEMCACHED_LOCATION: "memcached:11211"
      SETTING_RABBITMQ_HOST: "rabbitmq"
      SETTING_REDIS_HOST: "redis"
      SECRETS_email_password: "123456789"
      # These should match RABBITMQ_DEFAULT_PASS, POSTGRES_PASSWORD,
      # MEMCACHED_PASSWORD, and REDIS_PASSWORD above.
      SECRETS_rabbitmq_password: "REPLACE_WITH_SECURE_RABBITMQ_PASSWORD"
      SECRETS_postgres_password: "REPLACE_WITH_SECURE_POSTGRES_PASSWORD"
      SECRETS_memcached_password: "REPLACE_WITH_SECURE_MEMCACHED_PASSWORD"
      SECRETS_redis_password: "REPLACE_WITH_SECURE_REDIS_PASSWORD"
      SECRETS_secret_key: "REPLACE_WITH_SECURE_SECRET_KEY"
      SETTING_EXTERNAL_HOST: "zulip.example.org"
      SETTING_ZULIP_ADMINISTRATOR: "[email protected]"
      SETTING_EMAIL_HOST: "" # e.g. smtp.example.com
      SETTING_EMAIL_HOST_USER: "[email protected]"
      SETTING_EMAIL_PORT: "587"
      # It seems that the email server needs to use ssl or tls and can't be used without it
      SETTING_EMAIL_USE_SSL: "False"
      SETTING_EMAIL_USE_TLS: "True"
      ZULIP_AUTH_BACKENDS: "EmailAuthBackend"
      DISABLE_HTTPS: "True"
      # Uncomment this when configuring the mobile push notifications service
      # SETTING_PUSH_NOTIFICATION_BOUNCER_URL: 'https://push.zulipchat.com'
    volumes:
      - "zulip:/data:rw"
    ulimits:
      nofile:
        soft: 1000000
        hard: 1048576
volumes:
  zulip:
  postgresql-14:
  rabbitmq:
  redis:

This is my Caddyfile

zulip.example.org {
    reverse_proxy 192.168.1.101:8751
}

I have followed the steps from issue #313 and added DISABLE_HTTPS: "True" but still get this issue.

Running sudo docker-compose exec zulip bash and finding the error log with cat /var/log/zulip/errors.log I get this output:

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/bbd84a06bc651effedfe39cab337a1bf300cee02/zulip-py3-venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/srv/zulip-venv-cache/bbd84a06bc651effedfe39cab337a1bf300cee02/zulip-py3-venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = middleware_method(
  File "/home/zulip/deployments/2024-05-09-18-59-10/zerver/middleware.py", line 666, in process_view
    raise ProxyMisconfigurationError(proxy_state_header)
zerver.middleware.ProxyMisconfigurationError: Reverse proxy misconfiguration: No proxies configured in Zulip, but proxy headers detected from proxy at 192.168.208.1; see https://zulip.readthedocs.io/en/latest/production/deployment.html#putting-the-zulip-application-behind-a-reverse-proxy 

I have given up and now am asking for advice on this. To be clear: my Caddyfile is giving an SSL certificate and I believe so is Cloudflare. Any advice?

[Menghini]

Update: I believe I have found the solution.

I added this line to my docker-compose.yml file to my environment variables

LOADBALANCER_IPS: "192.168.0.0/16"

I then ran docker compose up -d and then

docker-compose exec -u zulip zulip \
    /home/zulip/deployments/current/manage.py generate_realm_creation_link

to get a link to make a server. It now works 100%, to my knowledge. Just passing this info onto someone that may need it!

[timabbott]

It seems like this is a common enough failure mode that we should up the documentation of LOADBALANCER_IPS.

[shawn-monadical]

@timabbott I think this trailing comma in that change is not valid, see here:

2de7686#diff-e45e45baeda1c1e73482975a664062aa56f20c03dd9d64a827aba57775bed0d3R99