The "process metadata" LC_NOTE allows for thread IDs to be specified in a Mach-O corefile. This extends the JSON recognzied in that LC_NOTE to allow for additional registers to be supplied on a per-thread basis. The registers included in a Mach-O corefile LC_THREAD load command can only be one of the register flavors that the kernel (xnu) defines in <mach/arm/thread_status.h> for arm64 -- the general purpose registers, floating point registers, exception registers. JTAG style corefile producers may have access to many additional registers beyond these that EL0 programs typically use, for instance TCR_EL1 on AArch64, and people developing low level code need access to these registers. This patch defines a format for including these registers for any thread. The JSON in "process metadata" is a dictionary that must have a `threads` key. The value is an array of entries, one per LC_THREAD in the Mach-O corefile. The number of entries must match the LC_THREADs so they can be correctly associated. Each thread's dictionary must have two keys, `sets`, and `registers`. `sets` is an array of register set names. If a register set name matches one from the LC_THREAD core registers, any registers that are defined will be added to that register set. e.g. metadata can add a register to the "General Purpose Registers" set that lldb shows users. `registers` is an array of dictionaries, one per register. Each register must have the keys `name`, `value`, `bitsize`, and `set`. It may provide additional keys like `alt-name`, that `DynamicRegisterInfo::SetRegisterInfo` recognizes. This `sets` + `registers` formatting is the same that is used by the `target.process.python-os-plugin-path` script interface uses, both are parsed by `DynamicRegisterInfo`. The one addition is that in this LC_NOTE metadata, each register must also have a `value` field, with the value provided in big-endian base 10, as usual with JSON. In RegisterContextUnifiedCore, I combine the register sets & registers from the LC_THREAD for a specific thread, and the metadata sets & registers for that thread from the LC_NOTE. Even if no LC_NOTE is present, this class ingests the LC_THREAD register contexts and reformats it to its internal stores before returning itself as the RegisterContex, instead of shortcutting and returning the core's native RegisterContext. I could have gone either way with that, but in the end I decided if the code is correct, we should live on it always. I added a test where we process save-core to create a userland corefile, then use a utility "add-lcnote" to strip the existing "process metadata" LC_NOTE that lldb put in it, and adds a new one from a JSON string. rdar://74358787 --------- Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
We have a binary image on Darwin that has no code, only metadata. It has a large symbol table with many external symbol names that will not be needed in the debugger. And it is possible to not have this binary on the debugger system - so lldb must read all of the symbol names out of memory, one at a time, which can be quite slow. We're adding a section __TEXT,__lldb_no_nlist, to this binary to indicate that lldb should not read the nlist symbols for it when we are reading out of memory. If lldb is run with an on-disk version of the binary, we will load the symbol table as we normally would, there's no benefit to handling this binary differently. I added a test where I create a dylib with this specially named section, launch the process. The main binary deletes the dylib from the disk so lldb is forced to read it out of memory. lldb attaches to the binary, confirms that the dylib is present in the process and is a memory Module. If the binary is not present, or lldb found the on-disk copy because it hasn't been deleted yet, we delete the target, flush the Debugger's module cache, sleep and retry, up to ten times. I create the specially named section by compiling an assembly file that puts a byte in the section which makes for a bit of a messy Makefile (the pre-canned actions to build a dylib don't quite handle this case) but I don't think it's much of a problem. This is a purely skipUnlessDarwin test case. Relanding this change with a restructured Makefiles for the test case that should pass on the CI bots. rdar://146167816
This reverts commit 397696bb3d26c1298bf265e4907b0b6416ad59c9. This breaks the macOS CI bots, I need to use $LDFLAGS in the $LD invocation when building the dylib to get the dylibs to build on the CI bots. But I've added "-lno-nlists -lhas-nlists" to the LDFLAGS for the main binary in the same directory, so using LDFLAGS will result in a compile error for the dylibs. I'll need to build the dylibs in a subdir with a different Makefile, will reland with that change in a bit.
We have a binary image on Darwin that has no code, only metadata. It has a large symbol table with many external symbol names that will not be needed in the debugger. And it is possible to not have this binary on the debugger system - so lldb must read all of the symbol names out of memory, one at a time, which can be quite slow. We're adding a section __TEXT,__lldb_no_nlist, to this binary to indicate that lldb should not read the nlist symbols for it when we are reading out of memory. If lldb is run with an on-disk version of the binary, we will load the symbol table as we normally would, there's no benefit to handling this binary differently. I added a test where I create a dylib with this specially named section, launch the process. The main binary deletes the dylib from the disk so lldb is forced to read it out of memory. lldb attaches to the binary, confirms that the dylib is present in the process and is a memory Module. If the binary is not present, or lldb found the on-disk copy because it hasn't been deleted yet, we delete the target, flush the Debugger's module cache, sleep and retry, up to ten times. I create the specially named section by compiling an assembly file that puts a byte in the section which makes for a bit of a messy Makefile (the pre-canned actions to build a dylib don't quite handle this case) but I don't think it's much of a problem. This is a purely skipUnlessDarwin test case. rdar://146167816
The Mach-O load commands have an LC_SYMTAB / struct symtab_command which represents the offset of the symbol table (nlist records) and string table for this binary. In a mach-o binary on disk, these are file offsets. If a mach-o binary is loaded in memory with all segments consecutive, the `symoff` and `stroff` are the offsets from the TEXT segment (aka the mach-o header) virtual address to the virtual address of the start of these tables. However, if a Mach-O binary is a part of the shared cache, then the segments will be separated -- they will have different slide values. And it is possible for the LINKEDIT segment to be greater than 4GB away from the TEXT segment in the virtual address space, so these 32-bit offsets cannot express the offset from TEXT segment to these tables. Create separate uint64_t variables to track the offset to the symbol table and string table, instead of reusing the 32-bit ones in the symtab_command structure. rdar://140432279
Recently in #107731 this change was revereted due to excess memory size in `TestSkinnyCore`. This was due to a bug where a range's end was being passed as size. Creating massive memory ranges. Additionally, and requiring additional review, I added more unit tests and more verbose logic to the merging of save core memory regions. @jasonmolenda as an FYI.
Reverts llvm/llvm-project#107159 as this is still causing `TestSkinnyCorefile.py` to time out. https://ci.swift.org/view/all/job/llvm.org/view/LLDB/job/as-lldb-cmake/11099/ https://ci.swift.org/view/all/job/llvm.org/view/LLDB/job/lldb-cmake/5544/
Reapplies #106293, testing identified issue in the merging code. I used this opportunity to strip CoreFileMemoryRanges to it's own file and then add unit tests on it's behavior.
Reverts #105442. Due to `TestSkinnyCoreFailing` and root causing of the failure will likely take longer than EOD.
This patch adds the option to specify specific memory ranges to be included in a given core file. The current implementation lets user specified ranges either be in addition to a certain save style, or independent of them via the newly added custom enum. To achieve being inclusive of save style, I've moved from a std::vector of ranges to a RangeDataVector, and to join overlapping ranges to prevent duplication of memory ranges in the core file. As a non function bonus, when SBSavecore was initially created, the header was included in the lldb-private interfaces, and I've fixed that and moved it the forward declare as an oversight. CC @bulbazord in case we need to include that into swift.