pip3 install --user generates a spurious "externally managed" error

[jkugler]

brew gist-logs <formula> link OR brew config AND brew doctor output

$ brew config
HOMEBREW_VERSION: 4.3.21
ORIGIN: https://github.com/Homebrew/brew
HEAD: c5d09b411776547bdbc8e0435deaf690ca632413
Last commit: 10 hours ago
Core tap JSON: 12 Sep 18:56 UTC
Core cask tap JSON: 12 Sep 18:57 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 10
Homebrew Ruby: 3.3.4 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.4_1/bin/ruby
CPU: 10-core 64-bit arm_firestorm_icestorm
Clang: 15.0.0 build 1500
Git: 2.46.0 => /opt/homebrew/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 14.6.1-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: 15.4
Rosetta 2: false

and

$ brew doctor
Your system is ready to brew.

Verification

• My brew doctor output says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.
• I searched for recent similar issues at https://github.com/Homebrew/homebrew-core/issues?q=is%3Aissue and found no duplicates.

What were you trying to do (and why)?

I am trying to install a Python package in the user directory. According Python, user site is under my home directory:

$ python3
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> site.USER_SITE
'/Users/tek30584/Library/Python/3.12/lib/python/site-packages'

Also:

$ pip3 --version
pip 24.2 from /opt/homebrew/lib/python3.12/site-packages/pip (python 3.12)

What happened (include all command output)?

$ pip3 install --user uv
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.

    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:

    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz

    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with

    brew install pipx

    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.

    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.

    Read more about this behavior here: <https://peps.python.org/pep-0668/>

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.

What did you expect to happen?

I expected the package to install under my home directory.

Oddly enough, the error message says:

    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.

    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation

1. The USER_SITE directory are not system packages; it is under my home directory
2. I did pass --user to use the user site directory, but it still generated the error.

It said "If you believe this is a mistake, please contact your Python installation or OS distribution provider." So I'm opening an issue. :)

Step-by-step reproduction instructions (by running brew commands)

brew install [email protected]
pip3 install --user uv

FYI: This only shows up with 3.12. Using pip3.10 and pip3.11 seems to do the right thing.

$ python3.10
Python 3.10.15 (main, Sep  7 2024, 00:20:06) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> site.USER_SITE
'/Users/tek30584/Library/Python/3.10/lib/python/site-packages'

and

$ python3.11
imPython 3.11.10 (main, Sep  7 2024, 01:03:31) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> site.USER_SITE
'/Users/tek30584/Library/Python/3.11/lib/python/site-packages'

[jkugler]

FYI: pip3 install --user --break-system-packages uv does the right thing (puts it in the user directory, without the error).

[carlocab]

Related: https://github.com/orgs/Homebrew/discussions/3404

[carlocab]

Quick summary of that: we think this behaviour is really on upstream (Python/pip). Though they might point this back at us if you complain about this behaviour to them.