Chocolatey 2.0 creates shims for virtual/meta-pacakge

[davidkna]

Checklist

• I have verified this is the correct repository for opening this issue.
• I have verified no other issues exist related to my problem.
• I have verified this is not an issue for a specific package.
• I have verified this issue is not security related.

What You Are Seeing?

Chocolatey creates for a meta-package that does not contain any executables.

What is Expected?

No shim is created.

How Did You Get This To Happen?

1. Run: choco install starship
2. Observe that chocolatey creates a shim C:\ProgramData\chocolatey\lib\starship\lib\starship\tools\starship.exe, despite the package being empty.
3. Invoke-Expression (&starship init powershell) no longer works properly: C:\ProgramData\chocolatey\lib\starship\tools\starship.exe: The term 'C:\ProgramData\chocolatey\lib\starship\tools\starship.exe' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

System Details

• Operating System: 10.0.22621.0
• Windows PowerShell version: 7.3.4
• Chocolatey CLI Version: 2.0.0
• Chocolatey Licensed Extension version:
• Chocolatey License type: Foss
• Terminal/Emulator: Microsoft Terminal

Installed Packages

Chocolatey v2.0.0
chocolatey 2.0.0
starship 1.15.0
starship.install 1.15.0
3 packages installed.

Output Log

I'm not on a Windows machine right now, I can add this if necessary.

Additional Context

starship/starship#5227

[pauby]

Before we can look at this please go back to the issue section above and complete all of the sections. Please don't miss any.

Please add the details above and not in a new comment.

Thanks

[davidkna]

Updated, but I'm not posting the log because this time it didn't create the shim this time. The last time I tried it, like in the other issue, I got a message that a shim was created.

[pauby]

I can't reproduce this issue.

Notre that the starship package installs starship.install. The shim for that is created correctly. There is no shim created for the starship package (as there is no EXE to create it from).

There is no $env:chocolateyInstall\lib\starship\lib folder created.

Invoke-Expression (&starship init powershell) works correctly.

[pakoito]

I can repro consistently on my machine. I don't deny it may be some misconfiguration or bad state locally, I just want to fix it and get starship back.

Logs for 1.4: starship/starship#5227 (comment)

Logs for 2.0: starship/starship#5227 (comment)

I can confirm starship.portable isn't affected. Logs: starship/starship#5227 (comment)

[cobrabr]

Uninstalling both starship and starship.install then reinstalling starship worked for me. If it makes any difference, I'm using Chocolatey 2.1.0-alpha-20230608.

[pauby]

@cobrabr just to be clear, you got this issue when upgrading from a previous version?

Uninstalling and reinstalling resolved the issue?

And you were using the same version of Chocolatey CLI during the upgrade, uninstall and install again?

[cobrabr]

@pauby Yes to all 3 questions. :)

[cobrabr]

@pauby A bit more info on this: I tried on another machine, and to make it actually work, these are the steps I took:

1. Uninstall starship and starship.install
2. Install starship.portable
3. Uninstall starship.portable
4. Install starship

I closed and reopened my console after every step, buy don't know if it's really necessary.

[pauby]

@cobrabr Thank you for that!

It looks like it may be this issue that is causing the problem.

[gep13]

@davidkna @pakoito @cobrabr I wanted to provide some additional feedback on this issue, as I am writing up some follow up issues to address what I believe the underlying problem is here, and I wanted to capture it all in one place.

I believe that the problem that is being seen here is two-fold. Part of it comes down to a change that was made in the starship package between versions 1.11.0 and 1.12.0, and the other part comes down to a change that was made in the most recent major version of Chocolatey.

Let me try to explain in the form of a set of steps that I have taken to re-produce the problem, and some screenshots to go along with it.

First up, let's install starship with Chocolatey CLI 1.4.0:

With that done, I can immediately run the starship executable, since a shim has been created in the c:/programdata/chocolatey/bin folder, which is pointed at the actual starship.exe which is located in the package folder at c:/programdata/chocolatey/lib/starship/tools/starship.exe.

Immediately running starship.exe:

Confirmation of the starship.exe version:

Confirmation of the shim pointing at the starship.exe:

Confirmation of the starship package created in the lib folder:

Confirmation of the starship.exe in the tools folder:

Next up, if I upgrade the starship package using Chocolatey CLI 1.4.0, you will see that an additional package, starship.install was installed, due to a change in how the packaging for this application was done. This change was made in version 1.12.0, but I have upgraded to the latest version, which is currently 1.15.0:

As you will see, this upgrades successfully, and there are now two packages in the lib folder:

At this point, Chocolatey CLI didn't touch the shim which is located in Chocolatey's bin folder, nor did it touch the starship.exe which was placed into the starship package folder in Chocolatey's lib folder. As a result, I can still run the shim, and it "works", however it is pointing that wrong version of starship:

The starship shim is still using 1.11.0 (since this file was never removed from disk) and the physically installed version of starship is using version 1.15.0. It looks as though the starship installer is adding an entry into the path for this actual physical location of the starship executable, however, the existing Chocolatey shim "wins", since it is higher in the list:

Now, if I follow the same process using the latest version of the Chocolatey CLI, i.e. install the earlier version of the starship package, upgrade to the latest, I will see that things don't follow the same pattern.

Rather than leaving the starship.exe in the tools folder of the starship package, it is removed:

However, the shim is still in place, since Chocolatey CLI didn't clean this up. As a result, when you attempted to run starship from the command line, you will see this error:

Since the shim still thinks that the starship.exe is in that location.

This change came about in the latest version of Chocolatey CLI, since we are attempting to do a better job at cleaning up the files that are left on disk during an upgrade. As you can see from this issue, there is still some work to be done here, and part of the problem is set to be addressed in this issue:

#3206

As linked above by @pauby. However, this issue will be limited in scope to only remove the orphaned shims during an upgrade and uninstall operation. This doesn't help when the shim has already been orphaned. The intention here is to create a new command, such as choco verify that will be responsible for testing shims in the bin folder, to verify that the target still exists, and if it doesn't, giving the option to remove the shim.

So, while the upgrade to the latest starship package works using the 1.4.0 and earlier version of Chocolatey CLI, the shim itself was still pointing at the older starship version, and therefore wasn't actually working correctly.

Direct installation of the starship.portable package will work correctly, regardless of Chocolatey CLI version, since the was the original transition from the starship to the starship.install dependent package where the problem first got introduced.

I realise that there are a lot of words here, but I hope that I have explained what is going on. Please let me know if you have any questions about the above.

[gep13]

To further follow up with this one, I went looking for existing issues regarding the creation of a new Chocolatey CLI command for verifying the current state of a system, and I found this issue:

#1757

I believe that this one would be extended to include the verification (check) of existing shims, to ensure that the target of the shim is still in place.

[pakoito]

Thank you for the in-depth explanation @gep13! Is there any way to manually remove the shim while choco verify is in development?

[TheCakeIsNaOH]

Is there any way to manually remove the shim while choco verify is in development?

Delete the shim .exe from $env:ChocolateyInstall\bin. So if Chocolatey CLI is installed to the default location that would be C:\programdata\chocolatey\bin\starship.exe

[TheCakeIsNaOH]

I think there are a number of things that could be improved with regard to shim management, here are a couple of issues that are all partially interrelated:
#1670
#3
#1108
#1515

I have some notes somewhere on my thoughts for this, but I was waiting on #2003 to be merged first before looking into it again.

[gep13]

@pakoito said...
Thank you for the in-depth explanation @gep13! Is there any way to manually remove the shim while choco verify is in development?

You are very welcome! As @TheCakeIsNaOH mentioned, in the absence of the choco verify command, if you find a shim that has been orphaned (you can see where a shim is pointed at by using <shim-name>.exe --shimgen-help) you can simply delete that shim from within the bin folder.

[cobrabr]

Thanks all for the extra info, it's much appreciated! Looking forward to having choco verify available to clean up my local Chocolatey environment. 😁

@pakoito, I'm pretty sure bcurran3's choco-cleaner cleans up orphaned shims (along with various other Chocolatey residuals), if you're looking for an automated solution for now.

[pauby]

@pakoito @cobrabr

I'm pretty sure bcurran3's choco-cleaner cleans up orphaned shims (along with various other Chocolatey residuals), if you're looking for an automated solution for now.

We do not support or recommend that you use that package, as it removes or modifies things that should not be changed. As a result, you may end up in a situation with a broken Chocolatey CLI installation.

[cobrabr]

@pauby Interesting, good to know. I've been using it forever, never had any issues with it, but will keep that in mind. Thanks for the heads-up!