Lots of TLS errors

[superff]

Hello

I am seeing lots if TLS error in opa logs,

does anyone have any ideas how to fix those?

2022/01/19 00:27:41 http: TLS handshake error from 172.16.65.122:26835: EOF
2022/01/19 00:27:21 http: TLS handshake error from 172.16.66.13:7628: EOF
2022/01/19 00:27:21 http: TLS handshake error from 172.16.65.227:41959: EOF
2022/01/19 00:27:21 http: TLS handshake error from 172.16.64.118:40032: EOF
2022/01/19 00:27:21 http: TLS handshake error from 172.16.65.248:36861: EOF
2022/01/19 00:27:22 http: TLS handshake error from 172.16.64.4:18399: EOF
2022/01/19 00:27:23 http: TLS handshake error from 172.16.64.55:53269: EOF
2022/01/19 00:27:23 http: TLS handshake error from 172.16.65.122:9829: EOF

we host opa in K8s,

and the liveness and readiness are using https

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: '#{APP_NAME}#'
spec:
  replicas: '#{K8S_REPLICAS}#'
  selector:
    matchLabels:
      app: '#{APP_NAME}#'
  template:
    metadata:
      labels:
        app: '#{APP_NAME}#'
        aadpodidbinding: '#{IDENTITY_NAME}#'
      annotations:
        autocert.step.sm/name: '#{APP_NAME}#'
    spec:
      containers:
        - image: '#{IMAGE_REGISTRY}#/#{APP_NAME}#:#{IMAGE_TAG}#'
          name: '#{APP_NAME}#'
          resources:
            limits:
              memory: '1024Mi'
              cpu: '500m'
          args:
            - 'run'
            - '--server'
            - '--ignore=.*'
            - '--config-file'
            - '/etc/opa/config.yaml'
            - '--addr'
            - ':8443'
            - '--log-level'
            - '#{LOG_LEVEL}#'
            - '--tls-ca-cert-file'
            - '/var/run/autocert.step.sm/root.crt'
            - '--tls-cert-file'
            - '/var/run/autocert.step.sm/site.crt'
            - '--tls-private-key-file'
            - '/var/run/autocert.step.sm/site.key'
            - '--tls-cert-refresh-period'
            - '5m'
          ports:
            - name: https
              containerPort: 8443
          env:
            - name: POLICY_URL
              value: 'https://#{STORAGE_ACCOUNT}#.blob.core.windows.net'
          volumeMounts:
            - mountPath: /etc/opa
              name: '#{APP_NAME}#-config'
              readOnly: true
          livenessProbe:
            httpGet:
              scheme: HTTPS
              port: 8443
            initialDelaySeconds: 5
            periodSeconds: 60
            timeoutSeconds: 1
          readinessProbe:
            httpGet:
              scheme: HTTPS
              port: 8443
              path: /health?bundle=true
            initialDelaySeconds: 5
            periodSeconds: 30
            timeoutSeconds: 1
      terminationGracePeriodSeconds: 30
      tolerations:
        - key: pool
          operator: Equal
          value: services
      volumes:
        - name: '#{APP_NAME}#-config'
          configMap:
            name: '#{APP_NAME}#'

[anderseknert]

Hi Fred. That seems very random, and it's difficult to say what might cause that without more information. Have you identified the clients calling OPA from those IP addresses?

Are these OPAs deployed in Kubernetes? TCP probes configured for liveness checks rather than HTTP, maybe?

[superff]

Thanks @anderseknert , yes, I am using k8s

the liveness and readiness are using http

       livenessProbe:
            httpGet:
              scheme: HTTP
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 60
            timeoutSeconds: 1
          readinessProbe:
            httpGet:
              scheme: HTTP
              port: 8080
              path: /health?bundle=true
            initialDelaySeconds: 5
            periodSeconds: 30
            timeoutSeconds: 1

[anderseknert]

Looks good to me. Have you identified which clients are calling OPA when this happens? I take it the OPA is exposed to any pod in the cluster? Is this OPA serving admission review requests, and the client is the Kubernetes API server?