Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcc-13.3-darwin-r0 tag build error on macOS14 ARM #20

Open
cjones051073 opened this issue Jul 1, 2024 · 78 comments
Open

gcc-13.3-darwin-r0 tag build error on macOS14 ARM #20

cjones051073 opened this issue Jul 1, 2024 · 78 comments

Comments

@cjones051073
Copy link

Using latest gcc gcc-13.3-darwin-r0 tag

In file included from /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_gcc13/libgcc13/work/gcc-13.3.0/libiberty/floatformat.c:29:
:info:build /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/math.h:54:5: error: #error "Unsupported value of __FLT_EVAL_METHOD__."
:info:build    54 | #   error "Unsupported value of __FLT_EVAL_METHOD__."
:info:build       |     ^~~~~
:info:build make[3]: *** [floatformat.o] Error 1
:info:build make[3]: *** Waiting for unfinished jobs....

Full configure and build log attached
main.log.gz

cjones051073 referenced this issue in macports/macports-ports Jul 1, 2024
@iains
Copy link
Owner

iains commented Jul 1, 2024

:debug:main Starting logging for libgcc13 @13.2.0_4+stdlib_flag

^^^ This looks like it's related to 13.2 not 13.3?

the 13.3 branch includes:

commit 87e6cc0103369f8891c3c3a516f4d93187c2c12b
Author: Francois-Xavier Coudert <[email protected]>
Date:   2023-12-11 14:56:04 +0000

    fixincludes: Update darwin_flt_eval_method for macOS 14

@cjones051073
Copy link
Author

No it's for 13.3 ignore that first part of the log

@cjones051073
Copy link
Author

cjones051073 commented Jul 1, 2024

apologies, that last log had some crude from an older 13.2 build at the start, but the bulk of it is indeed for 13.3. Here is a clean up version
main.log.gz

@cjones051073
Copy link
Author

cjones051073 commented Jul 1, 2024

note, the way the macports builds work is they start from the stock upstream release tarballs, but then apply a patch file generated from the tag in your repo here,

:debug:patch system:  cd "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_gcc13/libgcc13/work/gcc-13.3.0" && /usr/bin/patch -p0 < '/Users/chris/Projects/MacPorts/ports/lang/gcc13/files/patch-darwin-gcc-13.3.0.diff'

so the end result is what is built is exactly equivalent to your gcc-13.3-darwin-r0 tag

@cjones051073
Copy link
Author

final small log file clean up....

main.log.gz

@cjones051073
Copy link
Author

I should add gcc 13.2 builds just fine using the exact same procedure, applying a patch file that takes the build to your gcc-darwin-13.2-r0 tag. The issue above is new for the 13.3 release (and FWIW, I see exactly the same when attempting to build gcc 14.1 using your branch there).

@iains
Copy link
Owner

iains commented Jul 1, 2024

note, the way the macports builds work is they start from the stock upstream release tarballs,

Do you know which tarball is being used in this case?
( I can then check if it should include the patch I referenced ).

but then apply a patch file generated from the tag in your repo here,

:debug:patch system:  cd "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_gcc13/libgcc13/work/gcc-13.3.0" && /usr/bin/patch -p0 < '/Users/chris/Projects/MacPorts/ports/lang/gcc13/files/patch-darwin-gcc-13.3.0.diff'

so the end result is what is built is exactly equivalent to your gcc-13.3-darwin-r0 tag

In principle, at least.

I do build the release sources on as many systems as I have available - but, I suspect, that for 13.3 I only had x86_64 macOS14).

@iains
Copy link
Owner

iains commented Jul 1, 2024

I should add gcc 13.2 builds just fine using the exact same procedure, applying a patch file that takes the build to your gcc-darwin-13.2-r0 tag. The issue above is new for the 13.3 release (and FWIW, I see exactly the same when attempting to build gcc 14.1 using your branch there).

This is surprising, let me see if I can repeat it - at the moment my test hardware is building/testing a rebase from the weekend, so it will not be immediate

@cjones051073
Copy link
Author

the tar ball is from the stock gcc release sites, e.g.

--->  Attempting to fetch gcc-13.3.0.tar.xz from ftp://ftp.lip6.fr/pub/gnu/gcc/gcc-13.3.0

but thats just the start, the p[atch file that is applied makes what is built identical to your tags here.

@iains
Copy link
Owner

iains commented Jul 1, 2024

final small log file clean up....

main.log.gz

one difference - I have not tried to build with Xcode 15.4 CLT .. are you able to test with 15.3?

@cjones051073
Copy link
Author

I'm not really that keen to downgrade TBH...

@iains
Copy link
Owner

iains commented Jul 1, 2024

I'm not really that keen to downgrade TBH...

I do not see an Xcode 15.4 CLT .. so perhaps there's no difference in those tools; which perhaps means there's a change in the SDK.. I know FX filed a Feedback about the __FLT_EVAL_METHOD__.

@fxcoudert any ideas here?

@iains
Copy link
Owner

iains commented Jul 1, 2024

This patch is before the 13.3.0 tag on releases/gcc-13 - so I would have expected it to be in the release tarballs too. Perhaps there's yet another change in this in the Xcode 15.4 SDKs.. I not have those, since there's no XC15.4 CLT download.

commit 87e6cc0103369f8891c3c3a516f4d93187c2c12b
Author: Francois-Xavier Coudert <[email protected]>
Date:   2023-12-11 14:56:04 +0000

    fixincludes: Update darwin_flt_eval_method for macOS 14
    
    On macOS 14, a guard in <math.h> changed:
    
    -- MacOSX13.3.sdk/usr/include/math.h 2023-04-19 01:54:44
    +++ MacOSX14.0.sdk/usr/include/math.h 2023-08-01 08:42:43
    @@ -22,0 +23 @@
    +
    @@ -43 +44 @@
    -#if __FLT_EVAL_METHOD__ == 0
    +#if __FLT_EVAL_METHOD__ == 0 || __FLT_EVAL_METHOD__ == -1
    @@ -49 +50 @@
    -#elif __FLT_EVAL_METHOD__ == 2 || __FLT_EVAL_METHOD__ == -1
    +#elif __FLT_EVAL_METHOD__ == 2
    
    Therefore the darwin_flt_eval_method fixincludes fix doesn't match any
    longer, leading to a large number of testsuite failures like
    
    /private/var/gcc/regression/master/14-gcc/build/gcc/include-fixed/math.h:69:5:
    error: #error "Unsupported value of __FLT_EVAL_METHOD__."
    
    where __FLT_EVAL_METHOD__ = 16.
    
    This patch adjusts the fix to allow for both forms.
    
    Tested with make check in fixincludes on x86_64-apple-darwin23.0.0 and
    verifying that <math.h> has indeed been fixed as expected.
    
    (backport of 93f803d53b5ccaabded9d7b4512b54da81c1c616)

@cjones051073
Copy link
Author

cjones051073 commented Jul 1, 2024

The source we build starts as the official 13.3.0 tarball

:notice:fetch --->  Attempting to fetch gcc-13.3.0.tar.xz from ftp://ftp.lip6.fr/pub/gnu/gcc/gcc-13.3.0

which we then patch using a patch file generated from your repo tag

git diff --no-prefix releases/gcc-13.3.0 gcc-13.3-darwin-r0 | cat > patch-darwin-gcc-13.3.0.diff

which gets applied as

:debug:patch system:  cd "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_gcc13/libgcc13/work/gcc-13.3.0" && /usr/bin/patch -p0 < '/Users/chris/Projects/MacPorts/ports/lang/gcc13/files/patch-darwin-gcc-13.3.0.diff'
:info:patch patching file Makefile.def
:info:patch patching file Makefile.in
:info:patch patching file README.md
:info:patch patching file 'c++tools/Makefile.in'
<snip>

so what we end up with is precisely whatever you have in your gcc-13-3-darwin-r0 tag. So if its in that, we are using it.

@cjones051073
Copy link
Author

for reference the patch file we apply to bring the stock release to your tagged version

patch-darwin-gcc-13.3.0.diff.gz

@cjones051073
Copy link
Author

cjones051073 commented Jul 1, 2024

FWIW I just forcibly removed my CLT installation

sudo rm -rf /Library/Developer/CommandLineTools

made sure my Xcode 15.4 was update to date, and then reinstalled the CLT with

xcode-select --install

with that I tried building gcc 13.3 and get the same error.

I am not sure what you mean by 'there's no XC15.4 CLT download' ? This procedure works fine for me.

@iains
Copy link
Owner

iains commented Jul 1, 2024

FWIW I just forcibly removed my CLT installation

sudo rm -rf /Library/Developer/CommandLineTools

made sure my Xcode 15.4 was update to date, and then reinstalled the CLT with

xcode-select --install

with that I tried building gcc 13.3 and get the same error.

I just checked the release tarball - and FX's patch is in it ..

I am not sure what you mean by 'there's no XC15.4 CLT download' ? This procedure works fine for me.

There is no separate Xcode 15.4 CLT download on Apple's Developer website; I do not, and will not, have multiple Xcode versions installed on all my test boxes it becomes unmanageable - CLT is hard enough.

@cjones051073
Copy link
Author

You don't have Xcode installed ? Only the CLT ?

@cjones051073
Copy link
Author

cjones051073 commented Jul 1, 2024

with that I tried building gcc 13.3 and get the same error.

I just checked the release tarball - and FX's patch is in it ..

Then I guess, for me with Xcode 15.4 (and whatever CLT running xcode-select --install then provides) that patch is no longer working.

@cjones051073
Copy link
Author

cjones051073 commented Jul 1, 2024

So I ran a test build, going back to your previous tag, gcc-13.2-darwin-r0. That builds just fine, on the same machine with Xcode 15.4 + CLT on which gcc-13.3-darwin-r0 fails to build. So I am not sure the issue is really the Xcode version here, and more some change between these two tags.

Here is the OK 13.2 build log for reference

main-13.2.0.log.gz

@iains
Copy link
Owner

iains commented Jul 1, 2024

So I ran a test build, going back to your previous tag, gcc-13.2-darwin-r0. That builds just fine, on the same machine with Xcode 15.4 + CLT on which gcc-13.3-darwin-r0 fails to build. So I am not sure the issue is really the Xcode version here, and more some change between these two tags.

two things have changed; (1) the branch has got some improvements (2) it seems that the SDK has changed, in some way.

Almost certainly updates that relate to the improved coverage of floating point types have triggered this; but it is also the case that the branch does build fine with earlier Apple SDKs using the fixincludes that are part of the current branch - so that you might need to figure out what changes are needed to those to cover the SDK version you are using.

@cjones051073
Copy link
Author

So I ran a test build, going back to your previous tag, gcc-13.2-darwin-r0. That builds just fine, on the same machine with Xcode 15.4 + CLT on which gcc-13.3-darwin-r0 fails to build. So I am not sure the issue is really the Xcode version here, and more some change between these two tags.

two things have changed; (1) the branch has got some improvements (2) it seems that the SDK has changed, in some way.

Almost certainly updates that relate to the improved coverage of floating point types have triggered this; but it is also the case that the branch does build fine with earlier Apple SDKs using the fixincludes that are part of the current branch - so that you might need to figure out what changes are needed to those to cover the SDK version you are using.

Here is the math.h header from the error

:info:build /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/math.h:54:5: error: #error "Unsupported value of __FLT_EVAL_METHOD__."
:info:build    54 | #   error "Unsupported value of __FLT_EVAL_METHOD__."
:info:build       |     ^~~~~

from the SDK I am using (15.4) . can you compare it to what you have with 15.3 to see what has changed ?
math.h.gz

@cjones051073
Copy link
Author

b.t.w., and please bare with me here as I know next to nothing about how GCC's 'fixincludes' work, but the error I see is

:info:build /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/math.h:54:5: error: #error "Unsupported value of __FLT_EVAL_METHOD__."
:info:build    54 | #   error "Unsupported value of __FLT_EVAL_METHOD__."
:info:build       |     ^~~~~

whereas in the patch you mention above it mentions errors like

/private/var/gcc/regression/master/14-gcc/build/gcc/include-fixed/math.h:69:5:
    error: #error "Unsupported value of __FLT_EVAL_METHOD__."

Could it be for some reason in the builds I have the fixincludes are not being used at all, for some reason, hence the error message referring directly to /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/math.h instead of a temporary path like /private/var/gcc/regression/master/14-gcc/build/gcc/include-fixed/math.h ?

@cjones051073
Copy link
Author

b.t.w. The CLT I have installed is still 15.3

DEBUG: Xcode 15.4, CLT 15.3.0.0.1.1708646388

So likely the same as what you have anyway.

@iains
Copy link
Owner

iains commented Jul 2, 2024

b.t.w. The CLT I have installed is still 15.3

DEBUG: Xcode 15.4, CLT 15.3.0.0.1.1708646388

So likely the same as what you have anyway.

In this case, what probably matters is the SDK in use.

@iains
Copy link
Owner

iains commented Jul 2, 2024

for the record I just confirmed a build of GCC-13.3-darwin-r1 on macOS14.5:

$ ./gcc/xgcc -v
Using built-in specs.
COLLECT_GCC=./gcc/xgcc
Target: aarch64-apple-darwin23
Configured with: /src-local/gcc-git-13/configure --prefix=/opt/iains/aarch64-apple-darwin23/gcc-13-3Dr1 --build=aarch64-apple-darwin23 --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk --disable-libstdcxx-pch --enable-languages=all CC=aarch64-apple-darwin23-gcc CXX=aarch64-apple-darwin23-g++
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.3.0 (GCC) 

(I'm using GCC to bootstrap, because of building Ada and D too).

@iains
Copy link
Owner

iains commented Jul 2, 2024

from the SDK I am using (15.4) . can you compare it to what you have with 15.3 to see what has changed ? math.h.gz

Now I am very confused - that file is the same as the one in the SDK I am using - and which [as per the comment above] built and tested fine from my branch. I'm now doing a build with clang as the bootstrap compiler (so no Ada or D).

@cjones051073
Copy link
Author

Could it also be related to the configure options we use, which are perhaps not the usual ones (you can find an example of what we use in the logs) ?

@fxcoudert
Copy link
Contributor

for reference the patch file we apply to bring the stock release to your tagged version
patch-darwin-gcc-13.3.0.diff.gz

Normally the fix for that header is in GCC 13.3 upstream (https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=87e6cc0103369f8891c3c3a516f4d93187c2c12b).

Can you confirm what is the value of __FLT_EVAL_METHOD__ when the error occurs?

@barracuda156
Copy link

OK so removing --includedir=/opt/local/include/gcc has indeed fixed the build issue.

I need to let the full builds complete and then assess what impact removing our use of this has on the way we package the GCC versions we distribution within MacPorts, as our packaging there is a bit specialised in a few ways. If though GCC is no longer using this option in the same way as it was when we first starting using it, which seems the case the more I look at it, it might well be just removing it at this point is fine. But I need to check a few things first.

@cjones051073 We probably need to verify it works the same when bootstrap it done with gcc instead of clang. The breakage can be clang-specific.

@barracuda156
Copy link

Okay, bootstrapping with gcc on 14.5 did not go far, failing right at configure:

configure:4526: /opt/local/libexec/gcc10-bootstrap/bin/gcc -arch arm64 -o conftest -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk conftest.c  >&5
configure:4530: $? = 0
configure:4552: result: 
configure:4574: checking whether we are cross compiling
configure:4582: /opt/local/libexec/gcc10-bootstrap/bin/gcc -arch arm64 -o conftest -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk conftest.c  >&5
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/_stdio.h:68,
                 from /opt/local/libexec/gcc10-bootstrap/lib/gcc/aarch64-apple-darwin23/10.5.0/include-fixed/stdio.h:78,
                 from conftest.c:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/sys/cdefs.h:554:30: error: missing ')' after "__has_attribute"
  554 | #if __has_cpp_attribute(clang::unsafe_buffer_usage)
      |                              ^~
/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/sys/cdefs.h:554:32: error: missing binary operator before token "unsafe_buffer_usage"
  554 | #if __has_cpp_attribute(clang::unsafe_buffer_usage)
      |                                ^~~~~~~~~~~~~~~~~~~
configure:4586: $? = 1

Notice it passes -I/opt/local/include.

@barracuda156
Copy link

gcc 15 when building from MacPorts set-up also passes MacPorts include path across the build, so yeah, I guess this is wrong and we should remove it:

:info:build /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/./prev-gcc/xg++ -B/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/./prev-gcc/ -B/opt/local/powerpc-apple-darwin10/bin/ -nostdinc++ -B/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/prev-powerpc-apple-darwin10/libstdc++-v3/src/.libs -B/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/prev-powerpc-apple-darwin10/libstdc++-v3/libsupc++/.libs  -isystem /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/prev-powerpc-apple-darwin10/libstdc++-v3/include/powerpc-apple-darwin10  -isystem /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/prev-powerpc-apple-darwin10/libstdc++-v3/include  -isystem /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/gcc-15-20240630/libstdc++-v3/libsupc++ -L/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/prev-powerpc-apple-darwin10/libstdc++-v3/src/.libs -L/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/build/prev-powerpc-apple-darwin10/libstdc++-v3/libsupc++/.libs -c   -g -O2 -fno-checking -DIN_GCC    -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Wconditionally-supported -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror   -DHAVE_CONFIG_H  -DGENERATOR_FILE -I. -Ibuild -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/gcc-15-20240630/gcc -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/gcc-15-20240630/gcc/build -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/gcc-15-20240630/gcc/../include  -I/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/gcc-15-20240630/gcc/../libcpp/include -I/opt/local/include \
:info:build 		-o build/gentarget-def.o /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_lang_gcc-powerpc/libgcc-powerpc/work/gcc-15-20240630/gcc/gentarget-def.cc

(Just some random line from the log to illustrate the point.)

@iains
Copy link
Owner

iains commented Jul 2, 2024

Okay, bootstrapping with gcc on 14.5 did not go far, failing right at configure:

configure:4526: /opt/local/libexec/gcc10-bootstrap/bin/gcc -arch arm64 -o conftest -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk conftest.c  >&5
configure:4530: $? = 0
configure:4552: result: 
configure:4574: checking whether we are cross compiling
configure:4582: /opt/local/libexec/gcc10-bootstrap/bin/gcc -arch arm64 -o conftest -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk conftest.c  >&5
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/_stdio.h:68,
                 from /opt/local/libexec/gcc10-bootstrap/lib/gcc/aarch64-apple-darwin23/10.5.0/include-fixed/stdio.h:78,
                 from conftest.c:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/sys/cdefs.h:554:30: error: missing ')' after "__has_attribute"
  554 | #if __has_cpp_attribute(clang::unsafe_buffer_usage)
      |                              ^~
/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/include/sys/cdefs.h:554:32: error: missing binary operator before token "unsafe_buffer_usage"
  554 | #if __has_cpp_attribute(clang::unsafe_buffer_usage)
      |                                ^~~~~~~~~~~~~~~~~~~
configure:4586: $? = 1

We've fixed this, and I've back ported to a local 10.5 + lots of Darwin extras .. I plan to publish that but (as usual) subject to time constraints - in short, the problem above is different.

@cjones051073
Copy link
Author

@barracuda156 boot strapping from gcc is not something we do on our primary platforms, its only an issue for people like yourself trying to keep ancient OSes going, so whilst you are welcome to test things there its not a concern in general. I also do not expect @iains to expend any energy on why it works for you on OSX10.6...

In any case these sorts of issues we should discuss elsewhere, on macports channels.

From what I can see dropping the use of the option seems OK so far (and likely its anyway no longer doing what it did when we started using it a good few years back) so I am likely to push an update to 13.3 soon and we can then follow up any residual issues elsewhere.

@iains
Copy link
Owner

iains commented Jul 2, 2024

OK so removing --includedir=/opt/local/include/gcc has indeed fixed the build issue.

--includedir=DIR C header files [PREFIX/include]

So, indeed, setting this somewhere unexpected is going to mess up the search process - what is somewhat amazing is that it has worked before.

Sadly, configure options get added for obscure reasons and then get brought forward to times long after they are needed or correct ... it would not do any harm to audit what remains for correctness.

@cjones051073
Copy link
Author

cjones051073 commented Jul 2, 2024

The way I read the description from the configure help you post above this option controls where the header files are written, not where a set of system headers are read from. So I do think what this option does in practise does not really match the help there, I must say.

@cjones051073
Copy link
Author

We set a number of options to control where things are written, see

https://github.com/macports/macports-ports/blob/4e65f6cc89de2d2f65dea66eaaa9e470752b277a/lang/gcc13/Portfile#L94

Unless I am mistaken the lib, info and man equivalents work the way we understood, they control where output is written no ? So if so I think the lack of consistency with what the includedir option does is a bit confusing.

@iains
Copy link
Owner

iains commented Jul 2, 2024

We set a number of options to control where things are written, see

https://github.com/macports/macports-ports/blob/4e65f6cc89de2d2f65dea66eaaa9e470752b277a/lang/gcc13/Portfile#L94

Unless I am mistaken the lib, info and man equivalents work the way we understood, they control where output is written no ? So if so I think the lack of consistency with what the includedir option does is a bit confusing.

I think, actually, they all work in similar ways - that is control the expected layout of the toolchain in the directory structure.

  • In the case of lib and include that could matter, since we use inputs from there as well as outputs.

    • for lib we are probably "saved" by the Darwin linkers ignoring that and falling back to /usr/lib
    • we do care about include tho and therefore would need some extra way to locate the other (/usr/include) path - which I'm not sure is available right now.
  • info and man are (from the PoV of the compiler build) outputs - so all that matters in those cases is that the tools that use their content know the special paths.

(BTW, just for info) The defaulting of linkers to search /usr/lib (or /lib on Linux) can cause its own set of problems with things being found that the user did not really intend. One day, perhaps this will be made explicit and then we would find the same issue with missing /usr/lib.

@barracuda156
Copy link

barracuda156 commented Jul 2, 2024

@barracuda156 boot strapping from gcc is not something we do on our primary platforms, its only an issue for people like yourself trying to keep ancient OSes going, so whilst you are welcome to test things there its not a concern in general.

@cjones051073 Well, in general bootstrapping with gcc is a concern, of course, in may not be a concern in MacPorts.
Also, it can be done in MacPorts, and I have used gcc10 to boostrap gcc13 on Sonoma earlier.

I also do not expect @iains to expend any energy on why it works for you on OSX10.6...

We have very few issues which are specific to 10.6, and @iains is well aware of those. With gcc it is perhaps 1 issue.
But above log is from Sonoma, if you haven’t noticed.

From what I can see dropping the use of the option seems OK so far (and likely its anyway no longer doing what it did when we started using it a good few years back) so I am likely to push an update to 13.3 soon and we can then follow up any residual issues elsewhere.

I kinda begin wondering how was it working fine so far – it should have broken things.

@cjones051073
Copy link
Author

cjones051073 commented Jul 3, 2024

So removing the use of --includedir to control where we want the installed headers to go seems to have one side effect, which is the gccjit headers end up in the wrong place. They end up in

/opt/local/include/libgccjit++.h
/opt/local/include/libgccjit.h

instead of where we want them which is

/opt/local/include/gcc13/libgccjit++.h
/opt/local/include/gcc13/libgccjit.h

Luckily, we can still control where the C++ headers end up with --with-gxx-include-dir= so this is the only difference I can see in terms of what we finally install. The only solution I see is to manually move them during our post-destroot phase, which is a pain but doable.

@iains I can only agree that this dual use of the option --includedir to define where output gets written, but also where some input headers are read in from is rather inconvenient and I would say not really clear from the configure --help docs that its what happens. It would be good to have a way to define these independently.

@barracuda156
Copy link

@cjones051073 Please ping me when to test your branch.

@iains
Copy link
Owner

iains commented Jul 3, 2024

So removing the use of --includedir to control where we want the installed headers to go seems to have one side effect, which is the gcchit headers end up in the wrong place. They end up in

/opt/local/include/libgccjit++.h
/opt/local/include/libgccjit.h

instead of where we want them which is

/opt/local/include/gcc13/libgccjit++.h
/opt/local/include/gcc13/libgccjit.h

Luckily, we can still control where the C++ headers end up with --with-gxx-include-dir= so this is the only difference I can see in terms of what we finally install. The only solution I see is to manually move them during our post-destroot phase, which is a pain but doable.

This is because you want them installed per gcc version?

@iains I can only agree that this dual use of the option --includedir to define where output gets written, but also where some input headers are read in from is rather inconvenient and I would say not really clear from the configure --help docs that its what happens. It would be good to have a way to define these independently.

I suppose that there's an expectation in some cases that the toolchain is part of the distribution - and therefore integrated into it; this is not the case with GCC on macOS (since 10.7 anyway).

Perhaps there is a way - but I have never used or tested it - so you'd need to experiment:

  --includedir=DIR        C header files [PREFIX/include]
  --oldincludedir=DIR     C header files for non-gcc [/usr/include]

IFF (and this I do not know) oldincludedir overrides includedir in the fixincludes machinery - perhaps it would work to make --includedir=where-you-want-it and --oldincludedir=/usr/include
(I am doubtful tho - because the default would seem to DTRT and clearly that did not work).

We could always make some darwin-specific configuration changes - but the general layout is quite deep in the libtool stuff, so we'd be carrying the patch forever (I'd expect).

@cjones051073
Copy link
Author

So removing the use of --includedir to control where we want the installed headers to go seems to have one side effect, which is the gcchit headers end up in the wrong place. They end up in

/opt/local/include/libgccjit++.h
/opt/local/include/libgccjit.h

instead of where we want them which is

/opt/local/include/gcc13/libgccjit++.h
/opt/local/include/gcc13/libgccjit.h

Luckily, we can still control where the C++ headers end up with --with-gxx-include-dir= so this is the only difference I can see in terms of what we finally install. The only solution I see is to manually move them during our post-destroot phase, which is a pain but doable.

This is because you want them installed per gcc version?

Correct. We offer multiple gcc versions in MacPorts and require them to be installable side by side, which implies each has to stick to its own versioned prefix.

@cjones051073
Copy link
Author

--oldincludedir

Yeah, I was just looking at that oldincludedir flag and was going to ask what its intended use case was.

We could try using it, but what still confuses me a bit is the configuration is already being told the SDK to use via --with-sysroot="/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk", so why is this not being used to define where to look for the 'system' headers, for the fixincludes stuff etc. ?

@iains
Copy link
Owner

iains commented Jul 3, 2024

--oldincludedir

Yeah, I was just looking at that oldincludedir flag and was going to ask what its intended use case was.

Actually, I did not (yet) read the install page to see if there's a larger description - will do that later.

We could try using it, but what still confuses me a bit is the configuration is already being told the SDK to use via --with-sysroot="/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk", so why is this not being used to define where to look for the 'system' headers, for the fixincludes stuff etc. ?

The "sysroot" is the base of a tree which represents what's installed on a system. So all the used paths are relative to that - and for systems (like Linux and Darwin before 10.11) that can have their headers and libraries installed in / it's not needed (or, equivalently, you can consider the sysroot to be /)

So it makes no statement about how the installation is laid out - only tells the compiler where to start looking.

cjones051073 added a commit to macports/macports-ports that referenced this issue Jul 3, 2024
Also introduces some small tweaks to the configuration to address issues in

iains/gcc-13-branch#20

most specifcally, removing the use of the --includedir configuration
parameter, as this does more than it was understood to do and in fact
also controls *where* gcc expects to find C headers, not just where it
writes its output too, which was the understanding.
@barracuda156
Copy link

@cjones051073 @iains It is still broken on Sonoma.

:info:build checking for unistd.h... 1 warning generated.
:info:build if [ x"" != x ]; then \
:info:build 	  /usr/bin/clang -arch arm64 -c -DHAVE_CONFIG_H -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I. -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/../include  -W -Wall -Wwrite-strings -Wc++-compat -Wstrict-prototypes -Wshadow=local -pedantic  -D_GNU_SOURCE  -fno-common  /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/fibheap.c -o noasan/fibheap.o; \
:info:build 	else true; fi
:info:build /usr/bin/clang -arch arm64 -c -DHAVE_CONFIG_H -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I. -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/../include  -W -Wall -Wwrite-strings -Wc++-compat -Wstrict-prototypes -Wshadow=local -pedantic  -D_GNU_SOURCE  /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/fibheap.c -o fibheap.o
:info:build yes
:info:build checking minix/config.h usability... warning: unknown warning option '-Wshadow=local' [-Wunknown-warning-option]
:info:build yes
:info:build checking for CFLocaleCopyPreferredLanguages... 1 warning generated.
:info:build if [ x"-fno-common" != x ]; then \
:info:build 	  /usr/bin/clang -arch arm64 -c -DHAVE_CONFIG_H -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I. -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/../include  -W -Wall -Wwrite-strings -Wc++-compat -Wstrict-prototypes -Wshadow=local -pedantic  -D_GNU_SOURCE  -fno-common /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/filedescriptor.c -o pic/filedescriptor.o; \
:info:build 	else true; fi
:info:build no
:info:build checking minix/config.h presence... warning: unknown warning option '-Wshadow=local' [-Wunknown-warning-option]
:info:build /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/gcc-14.1.0/libiberty/filedescriptor.c:45:10: error: call to undeclared function 'dup2'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
:info:build   return dup2 (fd, fd) < 0;
:info:build          ^
:info:build 1 warning and 1 error generated.
:info:build make[3]: *** [filedescriptor.o] Error 1
:info:build make[3]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_gcc14/libgcc14/work/build/build-arm64-apple-darwin23/libiberty'
:info:build make[2]: *** [all-build-libiberty] Error 2
:info:build make[2]: *** Waiting for unfinished jobs....

Same error as before. Also -I/opt/local/include is passed still.
libgcc14_sonoma.log

@cjones051073
Copy link
Author

Macports buildbots have no problems, so whatever this is its unique to your setup.

@barracuda156
Copy link

Macports buildbots have no problems, so whatever this is its unique to your setup.

Well, perhaps something in MacPorts prefix break it, but the point is that nothing changed in this regard, the flag is passed as before.

Here for gcc13:
gcc13.3.0_sonoma.log

@cjones051073
Copy link
Author

Please compare yourself to the logs from the buildbot

https://ports.macports.org/port/gcc13/builds/

@barracuda156
Copy link

barracuda156 commented Jul 4, 2024

Please compare yourself to the logs from the buildbot

https://ports.macports.org/port/gcc13/builds/

The -I/opt/local/include flag is passed there: https://build.macports.org/builders/ports-14_arm64-builder/builds/35884/steps/install-port/logs/stdio

I can see that the build succeeds, but that only means that it is unpredictable, and something installed breaks it. (Perhaps only on Sonoma or only on arm64, since I have no issues on 10.6.)

I will deactivate all ports and try building from scratch on 14.5.

@barracuda156
Copy link

@iains @cjones051073 Well, it still fails. Perhaps buildbots use an old Xcode? Or different tarballs give different results?

I tried deactivating everything and building libgcc14, and it errs out like before.

svacchanda@43-200 ~ % port installed | grep active
  cctools @949.0.1_3+xcode (active)
  db48 @4.8.30_5 (active)
  gdbm @1.24_0 (active)
  gettext-runtime @0.22.5_0 (active)
  gmp @6.3.0_0 (active)
  isl @0.24_1 (active)
  ld64 @3_6+ld64_xcode (active)
  ld64-xcode @2_6 (active)
  libiconv @1.17_0 (active)
  libmpc @1.3.1_0 (active)
  lz4 @1.9.4_0 (active)
  mpfr @4.2.1_0 (active)
  ncurses @6.5_0 (active)
  perl5.34 @5.34.3_1 (active)
  readline @8.2.001_0 (active)
  texinfo @7.1_0 (active)
  xz @5.4.7_0 (active)
  zlib @1.3.1_0 (active)
  zstd @1.5.6_0 (active)

I have also removed all custom ports (even though nothing relevant was there anyway).

libgcc14_sonoma_clean.log

@barracuda156
Copy link

@cjones051073 So nothing conflicting in MacPorts prefix can cause the breakage, that is ruled out.
What should I try? Build every dependency from source?

@cjones051073
Copy link
Author

Please stop posting about this here, as I doubt its related to this issue, and we should not spam @iains on something that might be a macports packaging issue. Open a ticket about it in the macports trac system and we can discuss there. It if turns out we think its a gcc issue you can open a new issue for that in due course.

@barracuda156
Copy link

Okay, please ignore, on a fifth attempt it worked. I have no idea what is the difference (isl version inconsequential, if anyone would notice that in logs).

@iains
Copy link
Owner

iains commented Jul 4, 2024

This is because you want them installed per gcc version?

Correct. We offer multiple gcc versions in MacPorts and require them to be installable side by side, which implies each has to stick to its own versioned prefix.

For the record, there is a facility --enable-version-specific-runtime-libs that is intended to allow multiple versions of the runtimes to co-exist. Other than that library versioning is supposed to ensure that library ABIs are compatible.

For parallel instals of different GCC versions, we should expect that to 'just work' - i.e. that files that are specific to one version of the compiler are local to that compiler's installation (e.g. fixed includes, or the libgcc runtimes). If that is not happening then we need to file bug(s) to fix it. NOTE; this is also a common practice on Linux (multiple GCC versions installed) so if it was failing disastrously we'd expect to have heard :)

It's possible that you are defeating some/all of the provisions by being specific about paths - the configure philosophy is to assume that the person configuring knows the exact consequences of what they are changing .. otherwise its goal is to pick the most suitable set of options for the platform.

Probably, since we have had some active maintenance now for several years and have moved quite a bit forward in supporting some of the macOS-specific requirements, it might be worth starting from "what do we want to achieve"? and making sure we only specify the minimum to do that.

@barracuda156
Copy link

i.e. that files that are specific to one version of the compiler are local to that compiler's installation (e.g. fixed includes, or the libgcc runtimes).

MacPorts uses a single libgcc runtime for all gcc versions above 6, which is provided [normally] by the latest release of gcc.

@iains
Copy link
Owner

iains commented Jul 4, 2024

i.e. that files that are specific to one version of the compiler are local to that compiler's installation (e.g. fixed includes, or the libgcc runtimes).

MacPorts uses a single libgcc runtime for all gcc versions above 6, which is provided [normally] by the latest release of gcc.

Which is fine, providing the user cannot down-grade it from the latest .. (barring some issues we have at the moment with macOS15 etc having dropped some symbols.. but let's not worry about that here, yet).

edit: NOTE I think that this would only apply to the shared library since static libraries and other CRTs are found local to the compiler - and that should be the case, since the set of CRTs can change.

@iains
Copy link
Owner

iains commented Sep 23, 2024

I had a quick run through the long discussion here - but was unable to determine if it's "fixed" or we figured out why there was an issue - is anyone still having problems?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants