Note: This is a resubmission of #106791. I had to revert this a year ago for a failing test that I could not understand. I have time now to try and get this in again. Summary: This improves the performance of ObjectFileMacho::ParseSymtab by removing eager and expensive work in favor of doing it later in a less-expensive fashion. Experiment: My goal was to understand LLDB's startup time. First, I produced a Debug build of LLDB (no dSYM) and a Release+NoAsserts build of LLDB. The Release build debugged the Debug build as it debugged a small C++ program. I found that ObjectFileMachO::ParseSymtab accounted for somewhere between 1.2 and 1.3 seconds consistently. After applying this change, I consistently measured a reduction of approximately 100ms, putting the time closer to 1.1s and 1.2s on average. Background: ObjectFileMachO::ParseSymtab will incrementally create symbols by parsing nlist entries from the symtab section of a MachO binary. As it does this, it eagerly tries to determine the size of symbols (e.g. how long a function is) using LC_FUNCTION_STARTS data (or eh_frame if LC_FUNCTION_STARTS is unavailable). Concretely, this is done by performing a binary search on the function starts array and calculating the distance to the next function or the end of the section (whichever is smaller). However, this work is unnecessary for 2 reasons: 1. If you have debug symbol entries (i.e. STABs), the size of a function is usually stored right after the function's entry. Performing this work right before parsing the next entry is unnecessary work. 2. Calculating symbol sizes for symbols of size 0 is already performed in `Symtab::InitAddressIndexes` after all the symbols are added to the Symtab. It also does this more efficiently by walking over a list of symbols sorted by address, so the work to calculate the size per symbol is constant instead of O(log n).
This PR adds support for parsing the WebAssembly symbol table. The symbol table is encoded in the "names" section and contains names and indexes into other sections. For now we only support parsing function (code) symbols. The result is that you can set breakpoints by symbol name, while previously breakpoints by name required debug info (DWARF). This is also necessary for Swift, which checks for the presence of `swift_release` as a heuristic to determine if there's a static Swift stdlib.
unbreak gcc CI bots.
This patch fixes: lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp:415:7: error: label at end of compound statement is a C++23 extension [-Werror,-Wc++23-extensions] lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp:536:7: error: label at end of compound statement is a C++23 extension [-Werror,-Wc++23-extensions] lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp:672:7: error: label at end of compound statement is a C++23 extension [-Werror,-Wc++23-extensions]
While fixing bugs in the x86_64 LC_THREAD parser in ObjectFileMachO, I noticed that the other LC_THREAD parsers are all less clear than they should be. To recap, a Mach-O LC_THREAD load command has a byte size for the entire payload. Within the payload, there will be one or more register sets provided. A register set starts with a UInt32 "flavor", the type of register set defined in the system headers, and a UInt32 "count", the number of UInt32 words of memory for this register set. After one register set, there may be additional sets. A parser can skip an unknown register set flavor by using the count field to get to the next register set. When the total byte size of the LC_THREAD load command has been parsed, it is completed. This patch fixes the riscv/arm/arm64 LC_THREAD parsers to use the total byte size as the exit condition, and to skip past unrecognized register sets, instead of stopping parsing. Instead of fixing the i386 corefile support, I removed it. The last macOS that supported 32-bit Intel code was macOS 10.14 in 2018. I also removed i386 KDP support, 32-bit intel kernel debugging hasn't been supported for even longer than that. It would be preferable to do these things separately, but I couldn't bring myself to update the i386 LC_THREAD parser, and it required very few changes to remove this support entirely.
reading, and one bug in the new RegisterContextUnifiedCore class. The PR I landed a few days ago to allow Mach-O corefiles to augment their registers with additional per-thread registers in metadata exposed a few bugs in the x86_64 corefile reader when running under different CI environments. It also showed a bug in my RegisterContextUnifiedCore class where I wasn't properly handling lookups of unknown registers (e.g. the LLDB_GENERIC_RA when debugging an intel target). The Mach-O x86_64 corefile support would say that it had fpu & exc registers available in every corefile, regardless of whether they were actually present. It would only read the bytes for the first register flavor in the LC_THREAD, the GPRs, but it read them incorrectly, so sometimes you got more register context than you'd expect. The LC_THREAD register context specifies a flavor and the number of uint32_t words; the ObjectFileMachO method would read that number of uint64_t's, exceeding the GPR register space, but it was followed by FPU and then EXC register space so it didn't crash. If you had a corefile with GPR and EXC register bytes, it would be written into the GPR and then FPU register areas, with zeroes filling out the rest of the context.
The "process metadata" LC_NOTE allows for thread IDs to be specified in a Mach-O corefile. This extends the JSON recognzied in that LC_NOTE to allow for additional registers to be supplied on a per-thread basis. The registers included in a Mach-O corefile LC_THREAD load command can only be one of the register flavors that the kernel (xnu) defines in <mach/arm/thread_status.h> for arm64 -- the general purpose registers, floating point registers, exception registers. JTAG style corefile producers may have access to many additional registers beyond these that EL0 programs typically use, for instance TCR_EL1 on AArch64, and people developing low level code need access to these registers. This patch defines a format for including these registers for any thread. The JSON in "process metadata" is a dictionary that must have a `threads` key. The value is an array of entries, one per LC_THREAD in the Mach-O corefile. The number of entries must match the LC_THREADs so they can be correctly associated. Each thread's dictionary must have two keys, `sets`, and `registers`. `sets` is an array of register set names. If a register set name matches one from the LC_THREAD core registers, any registers that are defined will be added to that register set. e.g. metadata can add a register to the "General Purpose Registers" set that lldb shows users. `registers` is an array of dictionaries, one per register. Each register must have the keys `name`, `value`, `bitsize`, and `set`. It may provide additional keys like `alt-name`, that `DynamicRegisterInfo::SetRegisterInfo` recognizes. This `sets` + `registers` formatting is the same that is used by the `target.process.python-os-plugin-path` script interface uses, both are parsed by `DynamicRegisterInfo`. The one addition is that in this LC_NOTE metadata, each register must also have a `value` field, with the value provided in big-endian base 10, as usual with JSON. In RegisterContextUnifiedCore, I combine the register sets & registers from the LC_THREAD for a specific thread, and the metadata sets & registers for that thread from the LC_NOTE. Even if no LC_NOTE is present, this class ingests the LC_THREAD register contexts and reformats it to its internal stores before returning itself as the RegisterContex, instead of shortcutting and returning the core's native RegisterContext. I could have gone either way with that, but in the end I decided if the code is correct, we should live on it always. I added a test where we process save-core to create a userland corefile, then use a utility "add-lcnote" to strip the existing "process metadata" LC_NOTE that lldb put in it, and adds a new one from a JSON string. rdar://74358787 --------- Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
A unification of the DWARF section names, https://github.com/llvm/llvm-project/pull/141344 broke dwarf5 debugging with Mach-O files. The str_offset and str_offset.dwo names are different in Mach-O from other object files.
Different object file formats support DWARF sections (COFF, ELF, MachO, PE/COFF, WASM). COFF and PE/COFF only matched a subset. This caused some GCC executables produced on MinGW to have issue later on when debugging. One example is that `.debug_rnglists` was not matched, which caused range-extraction to fail when printing a backtrace. This unifies the parsing of section names in `ObjectFile::GetDWARFSectionTypeFromName`, so all file formats can use the same naming convention. Since the prefixes are different, `GetDWARFSectionTypeFromName` only matches the suffixes (i.e. `.debug_` needs to be stripped before). I added two tests to ensure the sections are correctly identified on Windows executables.
This reverts commit d4d2f069dec4fb8b13447f52752d4ecd08d976d6. Temporarily reverting until we can find a way to get the correct ObjectFile set in Module's Triples without adding "-macho" to the triple string for each Module. This is breaking TestUniversal.py on the x86_64 macOS CI bots.