Third-party package managers like pip from [email protected]

[xyzzy-plugh-plover]

Output of brew config

n/a

Output of brew doctor

n/a

Description of issue

Would the following proposal be a useful way for brew to handle third-party package managers that are included in a formula/cask (like pip from [email protected])?

Instead of requiring formulae to be manually created for external packages (like cffi for the cffi Python package from pip), allow any package from any supported third-party package manager to be interacted with in brew via adapters to the package manager (the adapters would be supplied by the formula for the package manager).

The packages would be referenced in brew via a generalized formula naming convention like the following (exact syntax could be modified):

<formula>~<package>
example:
[email protected]~cffi

No manual formula would be required for [email protected]~cffi, which would replace the existing cffi formula. Adapter code for [email protected] would forward various brew subcommands to pip3.12 for any formula whose name begins with [email protected]~ (like brew info [email protected]~cffi getting info from pip3.12 show cffi, or brew list getting info from pip3.12 list).

If an external package requires additional processing by brew that must be manually specified, then a formula could be manually made for it (e.g., [email protected]~cffi.rb), but the default would be handling external packages without manual formulae.

If brew needs to perform any additional actions besides just forwarding calls to pip, then, to allow other code to call pip* from the command line as per usual (instead of requiring brew to be called to manage Python packages), wrappers for all pip executables/scripts would be provided by the [email protected] formula to be put on the PATH. The wrappers would perform all processing necessary for Python packages to be properly integrated with brew (e.g., if pip uninstall warns about uninstalling packages that other packages depend on, pip3.12 uninstall cffi could warn if a brew formula depends on cffi, even if no pip packages depend on cffi).

The same could be done for other formulae or casks that either are, or include, an external package manager.

[SMillerDev]

So this proposal is a bit odd to me for two reasons:

1. Things don't just pip install anymore with Python 3.12 since we adopted the system install PEP from that version onwards.
2. We're removing formulae that package Python packages with a few exceptions that need special handling, so very few formulae would be replaced with this method.
3. We already have brew create --Python which should make creating formulae pretty easy.

I'm not sure what the benefit of this system would be. I think for those reasons it might be easier to start this off as a custom command and if it proves popular you can try and add it to brew

[xyzzy-plugh-plover]

pip from the [email protected] formula wouldn't let me install any packages using it directly without using a switch that sounded like it could allow inconsistencies or other problems to occur.

Is there any way for users to use pip as per usual from the [email protected] formula that will ensure that pip & brew don't develop any inconsistencies or other problems?

If not, it doesn't make sense to me for users to have to deal with virtual environments, multiple Python installations, creating brew formulae, etc. to install Python packages. They should be able to use pip as per usual, not even knowing it was installed via brew.

[SMillerDev]

If you want to install tools with pip, use pipx which will manage the virtual environment. If you want to install libraries, the concencus in the Python community is to use virtual environments. And Homebrew implemented that.

You can search discussions for PEP, I'm sure it'll show up.

[xyzzy-plugh-plover]

Does brew now use pipx to install formulae like mercurial each in its own respective virtual environment, or does brew still install to the standard pip global/user environment instead?

If not using pipx, why not?

[p-linnane]

All of this has been discussed at length in https://github.com/orgs/Homebrew/discussions/3404.

[carlocab]

Most Python formulae install into their own virtual environment, but not using pipx (because we don't need it). You can check which ones with grep virtualenv.

You also don't need to use pipx if you don't want to, but it's one way to make installing packages into their own virtual env simpler.