Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record the count of modules compiled and the count of total modules #17

Open
parsonsmatt opened this issue Jan 30, 2024 · 2 comments · May be fixed by #21
Open

Record the count of modules compiled and the count of total modules #17

parsonsmatt opened this issue Jan 30, 2024 · 2 comments · May be fixed by #21

Comments

@parsonsmatt
Copy link
Contributor

A common build log will include this prefix text:

[356 / 1000] Foo.Bar.Baz 

The second number is the set of modules GHC knows that may need building, and the first is the placement of the current module in the linearized module graph.

If this plugin could report how many modules are built in a given run, and how many modules are to be built total, then we'd be able to know how many modules were skipped. This would be useful information for understanding how incremental builds or caching is working, as well as how entangled the module graph is (ie "changing these imports resulted in a X difference in how many modules are rebuilt").

@parsonsmatt
Copy link
Contributor Author

Our hs-opentelemetry library does not support metrics per se, but we can record an attribute on a span, which should work just as well.

@parsonsmatt
Copy link
Contributor Author

I talked to Ian-Woo Kim about this, so we have an implementation strategy:

  1. We can get the ModuleGraph which contains a list of modules to compile: https://github.com/ghc/ghc/blob/master/compiler/GHC/Unit/Module/Graph.hs#L155
  2. The ModuleGraph is in the HscEnv: https://github.com/ghc/ghc/blob/master/compiler/GHC/Driver/Env/Types.hs#L68
  3. ... but the ModuleGraph is not populated at driver initialization, so we'll want to record that at the start of the Hsc phase. We probably want to ensure that we only add the attribute once, otherwise it'll be expensive. Potentially, we can start with an MVar () that is full, and each thread can tryTakeMVar. The first one to get a Just () can do addAttribute to the parent span.

OK, here's I think another issue. The parent span from the environment is a FrozenSpan, so we can't attach to that. We'll need to attach to a different span, one that is created live, or we need a way to signal to an external process that we want to add an attribute to a span.

To keep track of how many modules are compiled, we'll need to have an IORef that is incremented with every Hsc phase. Part of plugin cleanup should read that IORef and attach an attribute.

@parsonsmatt parsonsmatt linked a pull request Jan 31, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant