Reapply "[NFC][DebugInfo][DWARF] Create new low-level dwarf library (#…
(#145959)
This reapplies cbf781f0bdf2f680abbe784faedeefd6f84c246e, with fixes for
the shared-library build and the unconventional sanitizer-runtime build.
Original Description:
This is the culmination of a series of changes described in [1].
Although somewhat large by line count, it is almost entirely mechanical,
creating a new library in DebugInfo/DWARF/LowLevel. This new library has
very minimal dependencies, allowing it to be used from more places than
the normal DebugInfo/DWARF library--in particular from MC.
1.
https://discourse.llvm.org/t/rfc-debuginfo-dwarf-refactor-into-to-lower-and-higher-level-libraries/86665/2
…145081)" This reverts commit cbf781f0bdf2f680abbe784faedeefd6f84c246e. Breaks a couple of buildbots.
This is the culmination of a series of changes described in [1].
Although somewhat large by line count, it is almost entirely mechanical,
creating a new library in DebugInfo/DWARF/LowLevel. This new library has
very minimal dependencies, allowing it to be used from more places than
the normal DebugInfo/DWARF library--in particular from MC.
I am happy to put it in another location, or to structure it differently
if that makes sense. Some have suggested in BinaryFormat, but it is not
a great fit there. But if that makes more sense to the reviewers, I can
do that.
Another possibility would be to use pass-through headers to allow
clients who don't care to depend only on DebugInfo/DWARF. This would be
a much less invasive change, and perhaps easier for clients. But also a
system that hides details.
Either way, I'm open.
1.
https://discourse.llvm.org/t/rfc-debuginfo-dwarf-refactor-into-to-lower-and-higher-level-libraries/86665/2
We saw occasional segfaults while processing some binaries. The reason probably is that we may clear the DIE while we are reading it's data from another thread which happens due to cross-unit references. --------- Co-authored-by: Arslan Khabutdinov <akhabutdinov@fb.com>
Verification crashed here https://github.com/llvm/llvm-project/blob/main/llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp#L519 The reason being that during verification to extract inline_info we recreate compile unit dies. Assert fails because we previously cleaned up just the DIEs but some other fields remained initialized. Co-authored-by: Arslan Khabutdinov <akhabutdinov@fb.com>
Same as https://github.com/llvm/llvm-project/pull/139907/ except there is now a special dovoidwork helper function. Previous approach with assert(f();return success;) failed tests for release builds, so I created a separate helper. Open to suggestions how to solve this more elegantly. Co-authored-by: Arslan Khabutdinov <akhabutdinov@fb.com>
Reverts llvm/llvm-project#139907 as per discussion in https://github.com/llvm/llvm-project/issues/140545 due to tests becoming flaky
For large binaries gsymutil ends up using too much memory. This diff adds DIE tree cleanup per compile unit to reduce memory usage. P. S. Not sure about formatting. Maybe it hasn't been run in a while, or I have misconfigured something. `$ git clang-format HEAD~1 clang-format did not modify any files $ clang-format --version clang-format version 21.0.0git (git@github.com:peremyach/llvm-project.git 8d945c8357e1bd9872a34f92620d4916bfd27482) ` Co-authored-by: Arslan Khabutdinov <akhabutdinov@fb.com>
Reverts llvm/llvm-project#91023 Build break found in clang-ppc64le-linux-multistage build no. 583.
llvm-gsymutil eats a lot of RAM. On some large binaries, it causes OOM's on smaller hardware, consuming well over 64GB of RAM. This change frees line tables once we're done with them, and frees DWARFUnits's DIE's when we finish processing each DU, though they may get reconstituted if there are references from other DU's during processing. Once the conversion is complete, all DIE's are freed. The reduction in peak memory usage from these changes showed between 7-12% in my tests. The double-checked locking around the creation & freeing of the data structures was tested on a 166 core system. I validated that it trivially malfunctioned without the locks (and with stupid reordering of the locks) and worked reliably with them. --------- Co-authored-by: Kevin Frei <freik@meta.com>