cannot handle multiple packages that have the same cmd: in provide

[smoser]

chainguard-dev/melange#1188 added code to melange that allowed symlinks to provide 'cmd:'

The justification for that is if a package provides /usr/bin/gcloud that is a symlink to /usr/lib/somewhere/gcloud, then the end result is that the package does provide cmd:gcloud and the user should be able to apk search cmd:gcloud to find it.

There was fallout of that change and it had to be reverted under chainguard-dev/melange#1200

The errors that showed up were (from a slack thread)

Two of them have errors during the apk test disqualified because ... already provides cmd:

We do ultimately

1. want to have symlinks able to provide 'cmd:' (for the 'gcloud' suggestion above and also for dash-binsh package, which provides /bin/sh)
2. need to have apk and go-apk have the same behavior.

[xnox]

@jonjohnsonjr :

(i.e. one can install multiple packages by name, with both of them having same cmd:foo provides, but go-apk says no, as it resolves and pins all provides)

i don't think this is true, it's that go-apk gives up sooner on finding a solution than apk

i think we need to create a quick demo of two things called "demo-pybin-3.12" and "demo-pybin-3.11" and "demo-pybin" with the manually specified relevant provides, and otherwise empty packages. to demonstrate how apk is happy to add first two into "world" despite conflicting/matching cmd:pybin provides, and go-apk does not.

In addition to that, a similar demo but with non-empty files, and replaces-priority is the next step after that.

[xnox]

package:
  name: xnox-demo
  version: 1
  epoch: 0
  description: Simple go application that prints 'hello world' in a loop when built and invoked.
  copyright:
    - license: Apache-2.0

subpackages:
  - name: xnox-demo-3.12  
    dependencies:
      provider-priority: 312
      provides:
        - cmd:xnox-demo
  - name: xnox-demo-3.11
    dependencies:
      provider-priority: 311
      provides:
        - cmd:xnox-demo
  - name: xnox-demo-3.10
    dependencies:
      provider-priority: 310
      provides:
        - cmd:xnox-demo

Given the above, apk can install it:

# apk add xnox-demo-3.12 xnox-demo-3.11 xnox-demo-3.10
(1/3) Installing xnox-demo-3.10 (1-r0)
(2/3) Installing xnox-demo-3.11 (1-r0)
(3/3) Installing xnox-demo-3.12 (1-r0)
OK: 14 MiB in 17 packages

But go-apk cannot:

Given this sample yaml

package:
  name: xnox-demo2
  version: 1
  epoch: 0
  description: Simple go application that prints 'hello world' in a loop when built and invoked.
  copyright:
    - license: Apache-2.0

environment:
  contents:
    packages:
      - build-base      
      - busybox
      - xnox-demo-3.12
      - xnox-demo-3.11
      - xnox-demo-3.10

pipeline:
  - runs: ls

It fails like so

2024/05/14 18:54:44 INFO melange is building:
2024/05/14 18:54:44 INFO   configuration file: ./xnox-demo2.yaml
2024/05/14 18:54:44 INFO   workspace dir: /tmp/melange-workspace-2995631733
2024/05/14 18:54:44 INFO evaluating pipelines for package requirements
2024/05/14 18:54:44 INFO populating workspace /tmp/melange-workspace-2995631733 from ./xnox-demo2
2024/05/14 18:54:44 INFO building workspace in '/tmp/melange-guest-3037386847' with apko
2024/05/14 18:54:44 INFO image configuration:
2024/05/14 18:54:44 INFO   contents:
2024/05/14 18:54:44 INFO     repositories: []
2024/05/14 18:54:44 INFO     keyring:      []
2024/05/14 18:54:44 INFO     packages:     [build-base busybox xnox-demo-3.10 xnox-demo-3.11 xnox-demo-3.12]
2024/05/14 18:54:44 INFO   accounts:
2024/05/14 18:54:44 INFO     runas:  
2024/05/14 18:54:44 INFO     users:
2024/05/14 18:54:44 INFO       - uid=1000(build) gid=1000
2024/05/14 18:54:44 INFO     groups:
2024/05/14 18:54:44 INFO       - gid=1000(build) members=[build]
2024/05/14 18:54:45 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/05/14 18:54:45 INFO   workspace dir: /tmp/melange-workspace-2995631733
2024/05/14 18:54:45 INFO   guest dir: /tmp/melange-guest-3037386847
2024/05/14 18:54:45 INFO error during command execution: failed to build package: unable to build guest: unable to generate image: installing apk packages: error getting package dependencies: solving "xnox-demo-3.11" constraint: solving "xnox-demo-3.11" constraint:   xnox-demo-3.11-1-r0.apk disqualified because xnox-demo-3.10-1-r0.apk already provides cmd:xnox-demo
make[1]: *** [Makefile:150: packages/x86_64/xnox-demo2-1-r0.apk] Error 1

I believe this is because apk only adds explicit "xnox-demo-3.*" in /etc/apk/world, whereas go-apk adds the provides as exclusive "owners" / top-level in "installed" sort of.

[smoser]

this recreates with just one yaml file due to the test havin the same issue.

just run make package/xnox-demo test/xnox-demo

package:
  name: xnox-demo
  version: 1
  epoch: 0
  description: xnoxs demo
  copyright:
    - license: Apache-2.0

subpackages:
  - name: xnox-demo-3.12  
    dependencies:
      provider-priority: 312
      provides:
        - cmd:xnox-demo
  - name: xnox-demo-3.11
    dependencies:
      provider-priority: 311
      provides:
        - cmd:xnox-demo
  - name: xnox-demo-3.10
    dependencies:
      provider-priority: 310
      provides:
        - cmd:xnox-demo

test:
  environment:
    contents:
      packages:
        - xnox-demo-3.12
        - xnox-demo-3.11
        - xnox-demo-3.10
  pipeline:
    - runs: |
        echo hi world

[xnox]

🥺 could maybe @jonjohnsonjr @pnasrat please fix this 🥺

All other package managers always worked like this just fine, and i'm sure this is safe.

[xnox]

note image xnox-demo-enterprise as follows:

package:
  name: xnox-demo-enterprise
  version: 1
  epoch: 1
  description: Simple go application that prints 'hello world' in a loop when built and invoked.
  copyright:
    - license: Apache-2.0

subpackages:
  - name: xnox-demo-3.9
    dependencies:
      provider-priority: 309
      provides:
        - cmd:xnox-demo

Note it is different full version. and yet it is still installable by apk:

# apk add xnox-demo-3.12 xnox-demo-3.9
fetch https://packages.wolfi.dev/os/x86_64/APKINDEX.tar.gz
(1/2) Installing xnox-demo-3.12 (1-r0)
(2/2) Installing xnox-demo-3.9 (1-r1)
OK: 14 MiB in 16 packages

# cat /etc/apk/world 
apk-tools=2.14.1-r1
busybox=1.36.1-r8
ca-certificates-bundle=20240315-r1
glibc=2.39-r3
glibc-locale-posix=2.39-r3
ld-linux=2.39-r3
libcrypt1=2.39-r3
libcrypto3=3.3.0-r6
libssl3=3.3.0-r6
libxcrypt=4.4.36-r6
wolfi-base=1-r5
wolfi-baselayout=20230201-r8
wolfi-keys=1-r7
xnox-demo-3.12
xnox-demo-3.9
zlib=1.3.1-r1

[xnox]

package:
  name: xnox-demo
  version: 1
  epoch: 0
  description: xnoxs demo
  copyright:
    - license: Apache-2.0

environment:
  contents:
    packages:
      - busybox
      
pipeline:
  - runs: |
      echo hello
      
subpackages:
  - name: xnox-demo-3.12
    dependencies:
      provider-priority: 312
      replaces-priority: 312
      replaces:
        - xnox-demo-3.12
        - xnox-demo-3.11
        - xnox-demo-3.10
    pipeline:
      - runs: |
          echo "#!/bin/true 312" > xnox-demo
          chmod +x xnox-demo
          mkdir -p ${{targets.contextdir}}/usr/bin/
          cp xnox-demo ${{targets.contextdir}}/usr/bin/xnox-demo
          cp xnox-demo ${{targets.contextdir}}/usr/bin/xnox-demo-3.12

  - name: xnox-demo-3.11
    dependencies:
      provider-priority: 311
      replaces-priority: 311
      replaces:
        - xnox-demo-3.12
        - xnox-demo-3.11
        - xnox-demo-3.10
    pipeline:
      - runs: |
          echo "#!/bin/true 311" > xnox-demo
          chmod +x xnox-demo
          mkdir -p ${{targets.contextdir}}/usr/bin/
          cp xnox-demo ${{targets.contextdir}}/usr/bin/xnox-demo
          cp xnox-demo ${{targets.contextdir}}/usr/bin/xnox-demo-3.11

  - name: xnox-demo-3.10
    dependencies:
      provider-priority: 310
      replaces-priority: 310
      replaces:
        - xnox-demo-3.12
        - xnox-demo-3.11
        - xnox-demo-3.10
    pipeline:
      - runs: |
          echo "#!/bin/true 310" > xnox-demo
          chmod +x xnox-demo
          mkdir -p ${{targets.contextdir}}/usr/bin/
          cp xnox-demo ${{targets.contextdir}}/usr/bin/xnox-demo
          cp xnox-demo ${{targets.contextdir}}/usr/bin/xnox-demo-3.10

test:
  environment:
    contents:
      packages:
        - xnox-demo-3.12
        - xnox-demo-3.11
        - xnox-demo-3.10
  pipeline:
    - runs: |
        echo hi world

so this doesn't work with neither APK or GO-APK.

so we shouldn't do it.