In-cluster deployment looks for a kubeconfig

[rosh-cha]

Trying to deploy headlamp using the kubernetes-headlamp.yaml in to a k8s cluster.
Does the in cluster deployment suppose to look for the kubeconfig?

{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":305,"time":"2024-03-18T22:51:07Z","message":"Creating Headlamp handler"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":306,"time":"2024-03-18T22:51:07Z","message":"Kubeconfig path: "}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":307,"time":"2024-03-18T22:51:07Z","message":"Static plugin dir: /headlamp/static-plugins"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":308,"time":"2024-03-18T22:51:07Z","message":"Plugins dir: /headlamp/plugins"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":309,"time":"2024-03-18T22:51:07Z","message":"Dynamic clusters support: false"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":310,"time":"2024-03-18T22:51:07Z","message":"Helm support: false"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":311,"time":"2024-03-18T22:51:07Z","message":"Proxy URLs: []"}
{"level":"info","context":"main","clusterURL":"https://10.96.0.1:443","source":"/headlamp/backend/pkg/kubeconfig/kubeconfig.go","line":172,"time":"2024-03-18T22:51:07Z","message":"Proxy setup"}
*** Headlamp Server ***
  API Routers:
{"level":"error","source":"/headlamp/backend/cmd/headlamp.go","line":365,"error":"read /: is a directory","time":"2024-03-18T22:51:07Z","message":"loading kubeconfig"}
{"level":"error","source":"/headlamp/backend/cmd/headlamp.go","line":376,"error":"open /home/headlamp/.config/Headlamp/kubeconfigs/config: no such file or directory","time":"2024-03-18T22:51:07Z","message":"loading dynamic kubeconfig"}

[thomaspetit]

+1 saw the same error

[GraysonCAdams]

Same issue here...

Is in-cluster deployment just straight up broken?

How did you resolve this @rosh-cha or @thomaspetit ?

[thomaspetit]

No, haven't had the time to look into it as I've been quite busy fixing my bootstrap script for my local clusters. Might want to revisit this soon with the latest multi-cluster features that I read about (for in-cluster deployments).

[dabecow]

I'm facing the same issue, no luck with deploying according to the https://headlamp.dev/docs/latest/installation/in-cluster/ :(

[mpdunlop]

Same issue here, not sure how to get started...

[quinhn]

Same issue with Keycloak OIDC configuration

[gavin-ob]

Same issue, I really just want to bypass the OIDC authentication as I already have auth happening via a reverse proxy. Is there anyway to pass kubeconfig in via values.yaml? or if not does anyone know how I might remove the need for auth? Apologies if that makes no sense, I am far from an expert :) it may be that kubeconfig doesn't need passing and I just need to somehow disable the need for OIDC token

[gberche-orange]

FYI, we're currently loading a static kubeconfig file in cluster by mounting onto /home/headlamp/.config/Headlamp/kubeconfigs/config a secret holding a kubeconfig file (in our case, we craft it using external secret operator).

    volumeMounts:
      - name: "headlamp-kubeconfig"
        mountPath: "/home/headlamp/.config/Headlamp/kubeconfigs/"

    volumes:
      - name: "headlamp-kubeconfig"
        secret:
          secretName: "kubeconfig-vc-crossplane-consumer"
          mountPath: "/home/headlamp/.config/Headlamp/kubeconfigs/"

We still need to test support for multiple kubeconfig files, by passing the -kubeconfig argument or the KUBECONFIG env (through helm values.env) with : separated paths as documented into https://headlamp.dev/docs/latest/installation/desktop/#use-several-kube-config-files

/CC @Guilamb

[illume]

Ah, I didn't think anyone would load kube configs in-cluster. As far as I know, the errors are just misleading for most people. In the case when Headlamp is running in-cluster (and not as an app) then these should be warning logs and not error logs.

@gberche-orange I wonder if there's another way to support what you're doing there with the kube config file?

It's unclear to me if we should support kube config at all in cluster? Or if it should be enabled explicitly when used in-cluster (because most people won't use it and the logs would be misleading).

@yolossn @knrt10 any thoughts?

[yolossn]

I agree that many wont be using kubeconfigs in the cluster but removing it completely can affect the existing workflows for people so making it available based on a config can be the way to go.

[gberche-orange]

Ah, I didn't think anyone would load kube configs in-cluster.
I wonder if there's another way to support what you're doing there with the kube config file?

Our use case is to use headlamp to demo RBAC user experience from various actors (app developers, app operators, platform operators). Ability to load multiple distinct kubeconfig files is useful to be as close to real conditions (i.e. where each actor is given a kubeconfig to use of cluster)

[Guilamb]

FYI, we're currently loading a static kubeconfig file in cluster by mounting onto /home/headlamp/.config/Headlamp/kubeconfigs/config a secret holding a kubeconfig file (in our case, we craft it using external secret operator).

    volumeMounts:
      - name: "headlamp-kubeconfig"
        mountPath: "/home/headlamp/.config/Headlamp/kubeconfigs/"

    volumes:
      - name: "headlamp-kubeconfig"
        secret:
          secretName: "kubeconfig-vc-crossplane-consumer"
          mountPath: "/home/headlamp/.config/Headlamp/kubeconfigs/"

We still need to test support for multiple kubeconfig files, by passing the -kubeconfig argument or the KUBECONFIG env (through helm values.env) with : separated paths as documented into https://headlamp.dev/docs/latest/installation/desktop/#use-several-kube-config-files

/CC @Guilamb

So I've tried using the KUBECONFIG environment variable but without success. However, using the -kubeconfig flag, which I added in the Helm chart using extraArgs, worked with the : separator: -kubeconfig=home/headlamp/.config/Headlamp/kubeconfigs/kubeconfig1:home/headlamp/.config/Headlamp/kubeconfigs/kubeconfig2.

[meysam81]

I'm on the latest stable version of the headlamp (ghcr.io/headlamp-k8s/headlamp:v0.27.0) and I still see this error.

I fixed it by modifying the following values to the helm values.yml file:

initContainers:
  - command:
      - /bin/sh
      - "-c"
      - |
        kubectl config set-cluster main --server=https://kubernetes.default.svc --certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        kubectl config set-credentials main --token=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
        kubectl config set-context main --cluster=main --user=main
        kubectl config use-context main
    env:
      - name: KUBERNETES_SERVICE_HOST
        valueFrom:
          fieldRef:
            fieldPath: status.hostIP
      - name: KUBERNETES_SERVICE_PORT
        value: "6443"
      - name: KUBECONFIG
        value: /home/headlamp/.config/Headlamp/kubeconfigs/config
    image: bitnami/kubectl:1.32.0
    name: create-kubeconfig
    securityContext:
      capabilities:
        drop:
          - ALL
      privileged: false
      readOnlyRootFilesystem: true
      runAsGroup: 101
      runAsNonRoot: true
      runAsUser: 100
    volumeMounts:
      - mountPath: /home/headlamp/.config/Headlamp/kubeconfigs
        name: kubeconfig


volumeMounts:
  - mountPath: /home/headlamp/.config/Headlamp/kubeconfigs/config
    name: kubeconfig
    readOnly: true
    subPath: config

volumes:
  - name: kubeconfig
    emptyDir: {}

[patsevanton]

values.yml works. but for some reason it doesn't ask for a token. immediately grants full rights

When is a new release expected in which access to the kubernetes cluster will be out of the box during the helm installation?