[libc++] Unify the benchmarks with the test suite (#101399)

Instead of building the benchmarks separately via CMake and running them
separately from the test suite, this patch merges the benchmarks into
the test suite and handles both uniformly.

As a result:
- It is now possible to run individual benchmarks like we run tests
  (e.g. using libcxx-lit), which is a huge quality-of-life improvement.

- The benchmarks will be run under exactly the same configuration as
  the rest of the tests, which is a nice simplification. This does
  mean that one has to be careful to enable the desired optimization
  flags when running benchmarks, but that is easy with e.g.
  `libcxx-lit <...> --param optimization=speed`.

- Benchmarks can use the same annotations as the rest of the test
  suite, such as `// UNSUPPORTED` & friends.

When running the tests via `check-cxx`, we only compile the benchmarks
because running them would be too time consuming. This introduces a bit
of complexity in the testing setup, and instead it would be better to
allow passing a --dry-run flag to GoogleBenchmark executables, which is
the topic of https://github.com/google/benchmark/issues/1827.

I am not really satisfied with the layering violation of adding the
%{benchmark_flags} substitution to cmake-bridge, however I believe
this can be improved in the future.

1 files changed, 0 insertions, 3 deletions

diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index 15d2957ec1ac..2184ddd49537 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -159,9 +159,6 @@ jobs:
           'generic-no-rtti',
           'generic-optimized-speed',
           'generic-static',
-          # TODO Find a better place for the benchmark and bootstrapping builds to live. They're either very expensive
-          # or don't provide much value since the benchmark run results are too noise on the bots.
-          'benchmarks',
           'bootstrapping-build'
         ]
         machine: [ 'libcxx-runners-set' ]