metaprojects: grouping many projects together as a single meson unit

[dcbaker]

What Constitutes a metaproject?

Xorg

• A set of core libraries and applications developed tightly, but in separate repos
• A larger set of helper libraries and applications in separate repos, more loosely developed than the core libraries

LLVM

• A set of libraries and applications developed tightly, and released with a common version

GNOME

• A set of diverse applications and libraries built on separate release schedules
• released periodically as part of an SDK

Questions

Why not use a regular meson.build with subprojects?

Meson (for good reason) has very strict layout requirements for a single projects with subprojects, they must be placed in a sub directory of the source root. This might be fine for a hypothetical metaproject for Xorg, as one really could just slap git wraps in subprojects/ and go. For LLVM this wouldn't work, since the project layout has the individual projects in the project root, ie /llvm, /clang, etc.

Additionally, like a meson.options file, we likely want to place limitations on which language features are allowed in a metaproject, likely just control flow related actions, options, and subproject declarations.

What about introspection data on a metaproject?

Great question!

What features should we allow in a metaproject?

Also a great question!

what about subprojects from within the members of a metaproject

We likely need the same kind of sub-project promotion that happens in a regular meson.build. Additionally, we may want to allow metaprojects to define their own third party subprojects separately from the first party subprojects (ie, Xorg might want to require Mesa, which is technically not part of Xorg, but which the Xorg server uses in some configurations.)

[dontlaugh]

I'd like to see a Meson-ish take on monorepo builds, as well. Something that takes a few good ideas from Buck2 and Bazel, but simpler.

One way to attack this - at least initially - is a completely new command line that only deals with a metaproject layer.

So a hypothetical meta-meson command that speaks something like buck2/bazel target syntax

meta-meson compile //llvm/...   # build everything discovered "under" llvm

This hypothetical command would only be allowed to deal with metadata that meson exports, plus any new cross-project metadata that new files and syntax support.

Really, the idea is that composing a correct (if not necessarily optimal) meta project build should be possible with a tool that doesn't make modifications to meson's core. If you get that part right, then core can be adapted to support dependencies from other parts of the monorepo, etc.

[eli-schwartz]

There are two distinctive kinds of structures I'm thinking of: The first is Xorg, which is a group of tightly related projects living in separate repos. The second is LLVM, which is a group of related projects living in a mono repo, but which are distinct projects. In both cases it can be advantageous to have a single meson target to setup and initialize which can build the whole ecosystem.

I've talked about this topic a lot on IRC, good call on opening a tracking ticket.

My understanding is that much of the current design of subprojects has been guided by the use case of GNOME, which is neither of these cases, but is instead:

• a bunch of projects with separate release cycles, but part of the same wide ecosystem, for which it is desirable to build the lot together and install as a sort of SDK

It's basically the same overall goal as JHBuild: https://wiki.gnome.org/Projects/Jhbuild/Introduction

This clashes quite heavily with the interest of subprojects as dependency fallbacks, but for the same general reasons as xorg does. The solution (metaprojects) is roughly the same, probably.