| Age | Commit message (Collapse) | Author |
|---|
|
Another attempt at resolving the deadlock issue @GeorgeHuyubo discovered
(his previous
[attempt](https://github.com/llvm/llvm-project/pull/160225)).
This change can be summarized as the following:
* Plumb through a boolean flag to force no preload in
`GetOrCreateModules` all the way through to `LoadModuleAtAddress`.
* Parallelize `Module::PreloadSymbols` separately from
`Target::GetOrCreateModule` and its caller `LoadModuleAtAddress` (this
is what avoids the deadlock).
These changes roughly maintain the performance characteristics of the
previous implementation of parallel module loading. Testing on targets
with between 5000 and 14000 modules, I saw similar numbers as before,
often more than 10% faster in the new implementation across multiple
trials for these massive targets. I think it's because we have less lock
contention with this approach.
# The deadlock
See [bt.txt](https://github.com/user-attachments/files/22524471/bt.txt)
for a sample backtrace of LLDB when the deadlock occurs.
As @GeorgeHuyubo explains in his
[PR](https://github.com/llvm/llvm-project/pull/160225), the deadlock
occurs from an ABBA deadlock that happens when a thread context-switches
out of `Module::PreloadSymbols`, goes into `Target::GetOrCreateModule`
for another module, possibly entering this block:
```
if (!module_sp) {
// The platform is responsible for finding and caching an appropriate
// module in the shared module cache.
if (m_platform_sp) {
error = m_platform_sp->GetSharedModule(
module_spec, m_process_sp.get(), module_sp, &search_paths,
&old_modules, &did_create_module);
} else {
error = Status::FromErrorString("no platform is currently set");
}
}
```
`Module::PreloadSymbols` holds a module-level mutex, and then
`GetSharedModule` *attempts* to hold the mutex of the global shared
`ModuleList`. So, this thread holds the module mutex, and waits on the
global shared `ModuleList` mutex.
A competing thread may execute `Target::GetOrCreateModule`, enter the
same block as above, grabbing the global shared `ModuleList` mutex.
Then, in `ModuleList::GetSharedModule`, we eventually call
`ModuleList::FindModules` which eventually waits for the `Module` mutex
held by the first thread (via `Module::GetUUID`). Thus, we deadlock.
## Reproducing the deadlock
It might be worth noting that I've never been able to observe this
deadlock issue during live debugging (e.g. launching or attaching to
processes), however we were able to consistently reproduce this issue
with coredumps when using the following settings:
```
(lldb) settings set target.parallel-module-load true
(lldb) settings set target.preload-symbols true
(lldb) settings set symbols.load-on-demand false
(lldb) target create --core /some/core/file/here
# deadlock happens
```
## How this change avoids this deadlock
This change avoids concurrent executions of `Module::PreloadSymbols`
with `Target::GetOrCreateModule` by waiting until after the
`Target::GetOrCreateModule` executions to run `Module::PreloadSymbols`
in parallel. This avoids the ordering of holding a Module lock *then*
the ModuleList lock, as `Target::GetOrCreateModule` executions maintain
the ordering of the shared ModuleList lock first (from what I've read
and tested).
## Why not read-write lock?
Some feedback in https://github.com/llvm/llvm-project/pull/160225 was to
modify mutexes used in these components with read-write locks. This
might be a good idea overall, but I don't think it would *easily*
resolve this specific deadlock. `Module::PreloadSymbols` would probably
need a write lock to Module, so even if we had a read lock in
`Module::GetUUID` we would still contend. Maybe the `ModuleList` lock
could be a read lock that converts to a write lock if it chooses to
update the module, but it seems likely that some thread would try to
update the shared module list and then the write lock would contend
again.
Perhaps with deeper architectural changes, we could fix this issue?
# Other attempts
One downside of this approach (and the former approach of parallel
module loading) is that each DYLD would need to implement this pattern
themselves. With @clayborg's help, I looked at a few other approaches:
* In `Target::GetOrCreateModule`, backgrounding the
`Module::PreloadSymbols` call by adding it directly to the thread pool
via `Debugger::GetThreadPool().async()`. This required adding a lock to
`Module::SetLoadAddress` (probably should be one there already) since
`ObjectFileELF::SetLoadAddress` is not thread-safe (updates sections).
Unfortunately, during execution, this causes the preload symbols to run
synchronously with `Target::GetOrCreateModule`, preventing us from truly
parallelizing the execution.
* In `Module::PreloadSymbols`, backgrounding the `symtab` and `sym_file`
`PreloadSymbols` calls individually, but similar issues as the above.
* Passing a callback function like
https://github.com/swiftlang/llvm-project/pull/10746 instead of the
boolean I use in this change. It's functionally the same change IMO,
with some design tradeoffs:
* Pro: the caller doesn't need to explicitly call
`Module::PreloadSymbols` itself, and can instead call whatever function
is passed into the callback.
* Con: the caller needs to delay the execution of the callback such that
it occurs after the `GetOrCreateModule` logic, otherwise we run into the
same issue. I thought this would be trickier for the caller, requiring
some kinda condition variable or otherwise storing the calls to execute
afterwards.
# Test Plan:
```
ninja check-lldb
```
---------
Co-authored-by: Tom Yang <toyang@fb.com>
|
|
Main executables were bypassing the locate module callback that shared
libraries use, preventing custom symbol file location logic from working
consistently.
This PR fix this by
* Adding target context to ModuleSpec
* Leveraging that context to use target search path and platform's
locate module callback in ModuleList::GetSharedModule
This ensures both main executables and shared libraries get the same
callback treatment for symbol file resolution.
---------
Co-authored-by: George Hu <hyubo@meta.com>
Co-authored-by: George Hu <georgehuyubo@gmail.com>
|
|
The current display is missing a space, for example:
```
no target │ Locating binary: 24906A83-0182-361B-8B4A-90A249B04FD7at 0x0000000c0d108000
```
Co-authored-by: Dominic Chen <daming_chen@apple.com>
|
|
July 14 2024 I landed a change to update progress reporting when
loading kernel/firmware binaries
https://github.com/llvm/llvm-project/pull/98845
In DynamicLoader::LoadBinaryWithUUIDAndAddress I removed code that
was setting the ModuleSpec to the provided name, if the name provided
is that of a file on disk. With this code missing, if a filepath
name is passed in, this code will fail to find that binary on the local
disk. There's nothing in the PR / intention that would lead to this
change, it was unintentional.
|
|
Identical PR to: https://github.com/llvm/llvm-project/pull/134563
Previous PR was approved and landed but broke the build due to bad
merge.
Manually resolve the merge conflict and try to land again.
Co-authored-by: George Hu <georgehuyubo@gmail.com>
|
|
This reverts commit 070a4ae2f9bcf6967a7147ed2972f409eaa7d3a6.
Multiple buildbot failures have been reported:
https://github.com/llvm/llvm-project/pull/134563
The build fails with:
lldb/source/Target/Statistics.cpp:75:39: error: use of undeclared
identifier 'num_symbols_loaded'
|
|
In statistics, add locate time for each module in statistics, we time
the PluginManager::LocateExecutableSymbolFile and
PluginManager::LocateExecutableObjectFile call, save the duration for
the succeeded symbol locator plugin in the Module class as a map.
New key being added:
Module level: "_symbolLocatorTime_"
Summary level: "_totalSymbolLocatorTime_"
which would be a map of symbol_locator_plugin_name to time.
Sample statistic dump output after this change:
```
Command: statistics dump
===============Output===============
{
"commands": {
"command container add": 1,
"command script add": 51,
"command script import": 59,
"statistics dump": 1,
"target create": 1,
"type summary add": 36,
"type synthetic add": 21
},
"memory": {
"strings": {
"bytesTotal": 2801664,
"bytesUnused": 1704256,
"bytesUsed": 1097408
}
},
"modules": [
{
"debugInfoByteSize": 244927,
"debugInfoEnabled": true,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0.61934599999999995,
"debugInfoParseTime": 0.00033100000000000002,
"identifier": 94868115726768,
"path": "/home/hyubo/.cache/llvm-debuginfod/client/llvmcache-720c9a0f5ba8b460a1524a25597226f0fa551f71-c4crasher",
"symbolLocatorTime": {
"debuginfod": 1.4547020000000002
},
"symbolTableIndexTime": 0.0035370000000000002,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.0055040000000000002,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--linux",
"uuid": "720C9A0F-5BA8-B460-A152-4A25597226F0-FA551F71"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868111142256,
"path": "[vdso](0x00007ffd41c59000)",
"symbolLocatorTime": {
"debuginfod": 0.80597700000000005
},
"symbolTableIndexTime": 2.8e-05,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.00037300000000000001,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--linux",
"uuid": "4D1F38BD-BD34-DFB3-C9A5-B49A2A912219-AC713763"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868116162096,
"path": "/usr/local/fbcode/platform010/lib/libc.so.6",
"symbolLocatorTime": {
"debuginfod": 0.286356
},
"symbolTableIndexTime": 0.0091780000000000004,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.081841999999999998,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--linux",
"uuid": "ACE9DF01-8872-3A35-6D14-3C92527EF739-4BE32C75"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868111990032,
"path": "/usr/local/fbcode/platform010/lib/libm.so.6",
"symbolLocatorTime": {
"debuginfod": 0.63356699999999999
},
"symbolTableIndexTime": 0.0023960000000000001,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.021706,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--linux",
"uuid": "85932B54-0DE7-4FC1-23B0-FB09AD1A5A61-8E1098B7"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868116399488,
"path": "/usr/local/fbcode/platform010/lib/libmvec.so.1",
"symbolLocatorTime": {
"debuginfod": 0.66705099999999995
},
"symbolTableIndexTime": 0.00053700000000000004,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.0034429999999999999,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--linux",
"uuid": "4537CA22-8E6E-495B-7A03-FC2CEDCAD71C-8A7B2067"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868111778368,
"path": "/usr/local/fbcode/platform010/lib/libatomic.so.1",
"symbolLocatorTime": {
"debuginfod": 0.83268299999999995
},
"symbolTableIndexTime": 0.000243,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.0026640000000000001,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--",
"uuid": "F90F9111-BBD1-C2A9-972A-34EB75ABE62E-3BDED9CF"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868116403264,
"path": "/usr/local/fbcode/platform010/lib/libgcc_s.so.1",
"symbolLocatorTime": {
"debuginfod": 0.58871499999999999
},
"symbolTableIndexTime": 0.000292,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.0033,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--",
"uuid": "1C715A92-40BE-BE95-E148-1B401749BAB8-15D09F9D"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868113225328,
"path": "/usr/local/fbcode/platform010/lib/libstdc++.so.6",
"symbolLocatorTime": {
"debuginfod": 0.76993400000000001
},
"symbolTableIndexTime": 0.042455,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.081374000000000002,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--",
"uuid": "C9DAEE53-F607-981B-6BED-F2042833BFC7-71A1E66C"
},
{
"debugInfoByteSize": 0,
"debugInfoEnabled": false,
"debugInfoHadIncompleteTypes": false,
"debugInfoHadVariableErrors": false,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 94868113600960,
"path": "/usr/local/fbcode/platform010/lib/ld.so",
"symbolLocatorTime": {
"debuginfod": 0.48382199999999997
},
"symbolTableIndexTime": 0.00054799999999999998,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.0057219999999999997,
"symbolTableSavedToCache": false,
"symbolTableStripped": false,
"triple": "x86_64--linux",
"uuid": "CCA86CF4-E4FF-42C8-7056-2F7D8B83AEE0-530B4095"
}
],
"targets": [
{
"breakpoints": [],
"dyldPluginName": "dump-modulelist-dyld",
"expressionEvaluation": {
"failures": 0,
"successes": 0
},
"frameVariable": {
"failures": 0,
"successes": 0
},
"mismatchCoredumpModuleCount": 0,
"moduleIdentifiers": [
94868115726768,
94868111142256,
94868116162096,
94868111990032,
94868116399488,
94868111778368,
94868116403264,
94868113225328,
94868113600960
],
"signals": [
{
"SIGABRT": 1
}
],
"sourceMapDeduceCount": 0,
"sourceRealpathAttemptCount": 0,
"sourceRealpathCompatibleCount": 0,
"stopCount": 1,
"summaryProviderStatistics": [],
"targetCreateTime": 4.1999999999999998e-05,
"totalBreakpointResolveTime": 0,
"totalSharedLibraryEventHitCount": 0
}
],
"totalDebugInfoByteSize": 244927,
"totalDebugInfoEnabled": 1,
"totalDebugInfoIndexLoadedFromCache": 0,
"totalDebugInfoIndexSavedToCache": 0,
"totalDebugInfoIndexTime": 0.61934599999999995,
"totalDebugInfoParseTime": 0.00033100000000000002,
"totalModuleCount": 9,
"totalModuleCountHasDebugInfo": 1,
"totalModuleCountWithIncompleteTypes": 0,
"totalModuleCountWithVariableErrors": 0,
"totalSymbolLocatorTime": {
"debuginfod": 6.5228070000000002
},
"totalSymbolTableIndexTime": 0.059214000000000003,
"totalSymbolTableParseTime": 0.205928,
"totalSymbolTableStripped": 0,
"totalSymbolTablesLoadedFromCache": 0,
"totalSymbolTablesSavedToCache": 0,
"transcript": [
{
"command": "symsrv",
"commandArguments": "",
"commandName": "symsrv",
"durationInSeconds": 0.00069099999999999999,
"error": "",
"output": "",
"timestampInEpochSeconds": 1744934015
},
{
"command": "settings set symbols.enable-external-lookup true",
"commandArguments": "symbols.enable-external-lookup true",
"commandName": "settings set",
"durationInSeconds": 4.1999999999999998e-05,
"error": "",
"output": "",
"timestampInEpochSeconds": 1744934015
},
{
"command": "settings insert-before plugin.symbol-locator.debuginfod.server-urls 0 https://our.intern.facebook.com/intern/debuginfod",
"commandArguments": "plugin.symbol-locator.debuginfod.server-urls 0 https://our.intern.facebook.com/intern/debuginfod",
"commandName": "settings insert-before",
"durationInSeconds": 5.5999999999999999e-05,
"error": "",
"output": "",
"timestampInEpochSeconds": 1744934015
},
{
"command": "target create --core \"/var/tmp/cores/c4crasher.crasher.1576894\"",
"commandArguments": "--core \"/var/tmp/cores/c4crasher.crasher.1576894\"",
"commandName": "target create",
"durationInSeconds": 6.7297630000000002,
"error": "",
"output": "Core file '/var/tmp/cores/c4crasher.crasher.1576894' (x86_64) was loaded.\n",
"timestampInEpochSeconds": 1744934017
},
{
"command": "fbpaste statistics dump",
"commandArguments": "statistics dump",
"commandName": "fbpaste",
"timestampInEpochSeconds": 1744934055
},
{
"command": "statistics dump",
"commandArguments": "",
"commandName": "statistics dump",
"timestampInEpochSeconds": 1744934055
}
]
}
```
Co-authored-by: George Hu <georgehuyubo@gmail.com>
|
|
Make StreamAsynchronousIO an unique_ptr instead of a shared_ptr. I tried
passing the class by value, but the llvm::raw_ostream forwarder stored
in the Stream parent class isn't movable and I don't think it's worth
changing that. Additionally, there's a few places that expect a
StreamSP, which are easily created from a StreamUP.
|
|
Remove Debugger::GetOutputStream and Debugger::GetErrorStream in
preparation for replacing both with a new variant that needs to be
locked and hence can't be handed out like we do right now.
The patch replaces most uses with GetAsyncOutputStream and
GetAsyncErrorStream respectively. There methods return new StreamSP
objects that automatically get flushed on destruction.
See #126630 for more details.
|
|
ELF core debugging fix #117070 broke TestLoadUnload.py tests due to
GetModuleSpec call, ProcessGDBRemote fetches modules from remote. Revise
the original PR, renamed FindBuildId to FindModuleUUID.
|
|
DynamicLoader does not use ProcessElfCore NT_FILE entries to get
UUID. Use GetModuleSpec to get UUID from Process.
|
|
This patch adds the support to `Process.cpp` to automatically save off
TLS sections, either via loading the memory region for the module, or
via reading `fs_base` via generic register. Then when Minidumps are
loaded, we now specify we want the dynamic loader to be the `POSIXDYLD`
so we can leverage the same TLS accessor code as `ProcessELFCore`. Being
able to access TLS Data is an important step for LLDB generated
minidumps to have feature parity with ELF Core dumps.
|
|
When doing firmware/kernel debugging, it is frequent that binaries and
debug info need to be retrieved / downloaded, and the lack of progress
reports made for a poor experience, with lldb seemingly hung while
downloading things over the network. This PR adds progress reports to
the critical sites for these use cases.
|
|
This completes the conversion of LocateSymbolFile into a SymbolLocator
plugin. The only remaining function is DownloadSymbolFileAsync which
doesn't really fit into the plugin model, and therefore moves into the
SymbolLocator class, while still relying on the plugins to do the
underlying work.
|
|
This builds on top of the work started in c3a302d to convert
LocateSymbolFile to a SymbolLocator plugin. This commit moves
DownloadObjectAndSymbolFile.
|
|
This builds on top of the work started in c3a302d to convert
LocateSymbolFile to a SymbolLocator plugin. This commit moves
LocateExecutableSymbolFile.
|
|
This commit contains the initial scaffolding to convert the
functionality currently implemented in LocateSymbolFile to a plugin
architecture. The plugin approach allows us to easily add new ways to
find symbols and fixes some issues with the current implementation.
For instance, currently we (ab)use the host OS to include support for
querying the DebugSymbols framework on macOS. The plugin approach
retains all the benefits (including the ability to compile this out on
other platforms) while maintaining a higher level of separation with the
platform independent code.
To limit the scope of this patch, I've only converted a single function:
LocateExecutableObjectFile. Future commits will convert the remaining
LocateSymbolFile functions and eventually remove LocateSymbolFile. To
make reviewing easier, that will done as follow-ups.
|
|
The DebugSymbols DBGShellsCommand, which can find the symbols
for binaries, has a mechanism to return error messages when
it cannot find a symbol file. Those errors were not printed
to the user for several corefile use case scenarios; this
patch fixes that.
Also add dyld/process logging for the LC_NOTE metadata parsers
in ObjectFileMachO, to help in seeing what lldb is basing its
searches on.
Differential Revision: https://reviews.llvm.org/D157160
|
|
DynamicLoader::LoadBinaryWithUUIDAndAddress can create a Module based
on the binary image in memory, which in some cases contains symbol
names and can be genuinely useful. If we don't have a filename, it
creates a name in the form `memory-image-0x...` with the header address.
In practice, this is most useful with Darwin userland corefiles
where the binary was stored in the corefile in whole, and we can't
find a binary with the matching UUID. Using the binary out of
the corefile memory in this case works well.
But in other cases, akin to firmware debugging, we merely end up
with an oddly named binary image and no symbols.
Add a flag to control whether we will create these memory images
and add them to the Target or not; only set it to true when working
with a userland Mach-O image with the "all image infos" LC_NOTE for
a userland corefile.
Differential Revision: https://reviews.llvm.org/D157167
|
|
In ProcessMachCore::LoadBinariesViaMetadata(), if we did
load some binaries via metadata in the core file, don't
then search for a userland dyld in the corefile / kernel
and throw away that binary list. Also fix a little bug
with correctly recognizing corefiles using a `main bin spec`
LC_NOTE that explicitly declare that this is a userland
corefile.
LocateSymbolFileMacOSX.cpp's Symbols::DownloadObjectAndSymbolFile
clarify the comments on how the force_lookup and how the
dbgshell_command local both have the same effect.
In PlatformDarwinKernel::LoadPlatformBinaryAndSetup, don't
log a message unless we actually found a kernel fileset.
Reorganize ObjectFileMachO::LoadCoreFileImages so that it delegates
binary searching to DynamicLoader::LoadBinaryWithUUIDAndAddress and
doesn't duplicate those searches. For searches that fail, we would
perform them multiple times in both methods. When we have the
mach-o segment vmaddrs for a binary, don't let LoadBinaryWithUUIDAndAddress
load the binary first at its mach-o header address in the Target;
we'll load the segments at the correct addresses individually later
in this method.
DynamicLoaderDarwin::ImageInfo::PutToLog fix a LLDB_LOG logging
formatter.
In DynamicLoader::LoadBinaryWithUUIDAndAddress, instead of using
Target::GetOrCreateModule as a way to find a binary already registered
in lldb's global module cache (and implicitly add it to the Target
image list), use ModuleList::GetSharedModule() which only searches
the global module cache, don't add it to the Target. We may not
want to add an unstripped binary to the Target.
Add a call to Symbols::DownloadObjectAndSymbolFile() even if
"force_symbol_search" isn't set -- this will turn into a
DebugSymbols call / Spotlight search on a macOS system, which
we want.
Only set the Module's LoadAddress if the caller asked us to do that.
Differential Revision: https://reviews.llvm.org/D150928
rdar://109186357
|
|
Add support for recognizing a platform binary in the ObjectFileMachO
method that parses the "load binary" LC_NOTEs in a corefile.
A bit of reorganization to ProcessMachCore::DoLoadCore to separate
all of the unrelated things being done in that method into their own
separate methods, as well as small fixes to improve the handling of
a corefile with multiple kernel images in the corefile.
Differential Revision: https://reviews.llvm.org/D133680
rdar://98754861
|
|
I went over the output of the following mess of a command:
(ulimit -m 2000000; ulimit -v 2000000; git ls-files -z | parallel
--xargs -0 cat | aspell list --mode=none --ignore-case | grep -E
'^[A-Za-z][a-z]*$' | sort | uniq -c | sort -n | grep -vE '.{25}' |
aspell pipe -W3 | grep : | cut -d' ' -f2 | less)
and proceeded to spend a few days looking at it to find probable typos
and fixed a few hundred of them in all of the llvm project (note, the
ones I found are not anywhere near all of them, but it seems like a
good start).
Differential revision: https://reviews.llvm.org/D131122
|
|
Complete support of the binary-addresses key in the qProcessInfo packet
in ProcessGDBRemote, for detecting if one of the binaries needs to be
handled by a Platform plugin, and can be used to set the Process'
DynamicLoader plugin and the Target's Platform plugin.
Implement this method in PlatformDarwinKernel to recognize a kernel
fileset at that address, find the actual kernel address in the
fileset, set DynamicLoaderDarwinKernel and PlatformDarwinKernel
in the Process/Target; register the kernel address with the dynamic
loader so it will be loaded later during attach.
This patch only addresses the live debug scenario with a gdb remote
serial protocol connection. I'll handle corefiles in a subsequent
patch that builds on this.
Differential Revision: https://reviews.llvm.org/D133534
rdar://98754861
|
|
Add support to Mach-O corefiles and to live gdb remote serial protocol
connections for the corefile/remote stub to provide a list of load
addresses of binaries that should be found & loaded by lldb, and nothing
else. lldb will try to parse the binary out of memory, and if it can
find a UUID, try to find a binary & its debug information based on the
UUID, falling back to using the memory image if it must.
A bit of code unification from three parts of lldb that were loading
individual binaries already, so there is a shared method in
DynamicLoader to handle all of the variations they were doing.
Re-landing this with a uuid_is_null() implementation added to
Utility/UuidCompatibility.h for non-Darwin systems.
Differential Revision: https://reviews.llvm.org/D130813
rdar://94249937
rdar://94249384
|
|
This reverts commit d8879fba8825b9799166ba0ea552d4027bfb8ad1.
Debian bot failure; I included <uuid/uuid.h> to get uuid_is_null() but
don't get it there. Will memcmp or whatever & recommit.
|
|
Add support to Mach-O corefiles and to live gdb remote serial protocol
connections for the corefile/remote stub to provide a list of load
addresses of binaries that should be found & loaded by lldb, and nothing
else. lldb will try to parse the binary out of memory, and if it can
find a UUID, try to find a binary & its debug information based on the
UUID, falling back to using the memory image if it must.
A bit of code unification from three parts of lldb that were loading
individual binaries already, so there is a shared method in
DynamicLoader to handle all of the variations they were doing.
Differential Revision: https://reviews.llvm.org/D130813
rdar://94249937
rdar://94249384
|
|
Places calling LoadModuleAtAddress() already call ModulesDidLoad()
after a loop calling LoadModuleAtAddress(), so it's not necessary
to call it from there, and the batched ModulesDidLoad() may be
more efficient than this place calling it one after one.
This also makes the ModuleLoadedNotifys test pass on Linux now that
the duplicates no longer bring down the average of modules notified
per call.
Differential Revision: https://reviews.llvm.org/D123128
|
|
When opening core files (and also in some other situations) we could end
up with two vdso modules. This could happen because the vdso module is
very special, and over the years, we have accumulated various ways to
load it.
In D10800, we added one mechanism for loading it, which took the form of
a generic load-from-memory capability. Unfortunately loading an elf file
from memory is not possible (because the loader never loads the entire
file), and our attempts to do so were causing crashes. So, in D34352, we
partially reverted D10800 and implemented a custom mechanism specific to
the vdso.
Unfortunately, enough of D10800 remained such that, under the right
circumstances, it could end up loading a second (non-functional) copy of
the vdso module. This happened when the process plugin did not support
the extended MemoryRegionInfo query (added in D22219, to workaround a
different bug), which meant that the loader plugin was not able to
recognise that the linux-vdso.so.1 module (this is how the loader calls
it) is in fact the same as the [vdso] module (the name used in
/proc/$PID/maps) we loaded before. This typically happened in a core
file, as they don't store this kind of information.
This patch fixes the issue by completing the revert of D10800 -- the
memory loading code is removed completely. It also reduces the scope of
the hackaround introduced in D22219 -- it isn't completely sound and is
only relevant for fairly old (but still supported) versions of android.
I added the memory loading logic to the wasm dynamic loader, which has
since appeared and is relying on this feature (it even has a test). As
far as I can tell loading wasm modules from memory is possible and
reliable. MachO memory loading is not affected by this patch, as it uses
a completely different code path.
Since the scenarios/patches I described came without test cases, I have
created two new gdb-client tests cases for them. They're not
particularly readable, but right now, this is the best way we can
simulate the behavior (bugs) of a particular dynamic linker.
Differential Revision: https://reviews.llvm.org/D122660
|
|
|
|
The C headers are deprecated so as requested in D102845, this is replacing them
all with their (not deprecated) C++ equivalent.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D103084
|
|
This is a polymorphic class, copying it is a bad idea.
This was not a problem because most classes inheriting from it were
deleting their copy operations themselves. However, this enables us to
delete those explicit deletions, and ensure noone forgets to add them in
the future.
|
|
StringRef conversions
StringRef will call strlen on the C string which is inefficient (as ConstString already
knows the string lenght and so does StringRef). This patch replaces all those calls
with GetStringRef() which doesn't recompute the length.
|
|
Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).
This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).
Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere
Reviewed By: JDevlieghere
Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73258
|
|
A lot of comments in LLDB are surrounded by an ASCII line to delimit the
begging and end of the comment.
Its use is not really consistent across the code base, sometimes the
lines are longer, sometimes they are shorter and sometimes they are
omitted. Furthermore, it looks kind of weird with the 80 column limit,
where the comment actually extends past the line, but not by much.
Furthermore, when /// is used for Doxygen comments, it looks
particularly odd. And when // is used, it incorrectly gives the
impression that it's actually a Doxygen comment.
I assume these lines were added to improve distinguishing between
comments and code. However, given that todays editors and IDEs do a
great job at highlighting comments, I think it's worth to drop this for
the sake of consistency. The alternative is fixing all the
inconsistencies, which would create a lot more churn.
Differential revision: https://reviews.llvm.org/D60508
llvm-svn: 358135
|
|
Add a flag to control whether the ModulesDidLoad notification is
called when a module is added. If the notifications are disabled,
the caller must call ModulesDidLoad after adding all the new modules,
but postponing this notification until they're all batched up can
allow for better efficiency than notifying one-by-one.
Change the name of the ModuleList notifier functions that a subclass
can implement to start with 'Notify' to make it clear what they are.
Add a NotifyModulesRemoved.
Add header documentation for the changed/updated methods.
Added defaulted-value 'notify' argument to ModuleList Append,
AppendIfNeeded, and Remove because callers working with a local
ModuleList don't have an obvious idea of what notify means in this
context. When the ModuleList is a part of the Target class, the
notify behavior matters.
DynamicLoaderDarwin has been updated so that libraries being
added/removed are correctly batched up before notifications are
sent. Added the TestModuleLoadedNotifys.py test to run on
Darwin to test this.
<rdar://problem/48293064>
Differential Revision: https://reviews.llvm.org/D60172
llvm-svn: 357955
|
|
The `ap` suffix is a remnant of lldb's former use of auto pointers,
before they got deprecated. Although all their uses were replaced by
unique pointers, some variables still carried the suffix.
In r353795 I removed another auto_ptr remnant, namely redundant calls to
::get for unique_pointers. Jim justly noted that this is a good
opportunity to clean up the variable names as well.
I went over all the changes to ensure my find-and-replace didn't have
any undesired side-effects. I hope I didn't miss any, but if you end up
at this commit doing a git blame on a weirdly named variable, please
know that the change was unintentional.
llvm-svn: 353912
|
|
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
|
|
This patch removes the comments following the header includes. They were
added after running IWYU over the LLDB codebase. However they add little
value, are often outdates and burdensome to maintain.
Differential revision: https://reviews.llvm.org/D54385
llvm-svn: 346625
|
|
This patch removes the logic for resolving paths out of FileSpec and
updates call sites to rely on the FileSystem class instead.
Differential revision: https://reviews.llvm.org/D53915
llvm-svn: 345890
|
|
This patch removes the Exists method from FileSpec and updates its uses
with calls to the FileSystem.
Differential revision: https://reviews.llvm.org/D53845
llvm-svn: 345854
|
|
This is an NFC commit to refactor the "load dependent files" parameter
from a boolean to an enum value. We want to be able to specify a
default, in which case we decide whether or not to load the dependent
files based on whether the target is an executable or not (i.e. a
dylib).
This is a dependency for D51934.
Differential revision: https://reviews.llvm.org/D51859
llvm-svn: 342633
|
|
This is intended as a clean up after the big clang-format commit
(r280751), which unfortunately resulted in many of the comment
paragraphs in LLDB being very hard to read.
FYI, the script I used was:
import textwrap
import commands
import os
import sys
import re
tmp = "%s.tmp"%sys.argv[1]
out = open(tmp, "w+")
with open(sys.argv[1], "r") as f:
header = ""
text = ""
comment = re.compile(r'^( *//) ([^ ].*)$')
special = re.compile(r'^((([A-Z]+[: ])|([0-9]+ )).*)|(.*;)$')
for line in f:
match = comment.match(line)
if match and not special.match(match.group(2)):
# skip intentionally short comments.
if not text and len(match.group(2)) < 40:
out.write(line)
continue
if text:
text += " " + match.group(2)
else:
header = match.group(1)
text = match.group(2)
continue
if text:
filled = textwrap.wrap(text, width=(78-len(header)),
break_long_words=False)
for l in filled:
out.write(header+" "+l+'\n')
text = ""
out.write(line)
os.rename(tmp, sys.argv[1])
Differential Revision: https://reviews.llvm.org/D46144
llvm-svn: 331197
|
|
This renames the LLDB error class to Status, as discussed
on the lldb-dev mailing list.
A change of this magnitude cannot easily be done without
find and replace, but that has potential to catch unwanted
occurrences of common strings such as "Error". Every effort
was made to find all the obvious things such as the word "Error"
appearing in a string, etc, but it's possible there are still
some lingering occurences left around. Hopefully nothing too
serious.
llvm-svn: 302872
|
|
This adjusts header file includes for headers and source files
in Core. In doing so, one dependency cycle is eliminated
because all the includes from Core to that project were dead
includes anyway. In places where some files in other projects
were only compiling due to a transitive include from another
header, fixups have been made so that those files also include
the header they need. Tested on Windows and Linux, and plan
to address failures on OSX and FreeBSD after watching the
bots.
llvm-svn: 299714
|
|
corresponding module.
Reviewers: labath, clayborg
Subscribers: jaydeep, bhushan, lldb-commits, slthakur
Differential Revision: https://reviews.llvm.org/D30454
llvm-svn: 299196
|
|
<rdar://problem/30735021>
llvm-svn: 296504
|
|
*** to conform to clang-format’s LLVM style. This kind of mass change has
*** two obvious implications:
Firstly, merging this particular commit into a downstream fork may be a huge
effort. Alternatively, it may be worth merging all changes up to this commit,
performing the same reformatting operation locally, and then discarding the
merge for this particular commit. The commands used to accomplish this
reformatting were as follows (with current working directory as the root of
the repository):
find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} +
find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ;
The version of clang-format used was 3.9.0, and autopep8 was 1.2.4.
Secondly, “blame” style tools will generally point to this commit instead of
a meaningful prior commit. There are alternatives available that will attempt
to look through this change and find the appropriate prior commit. YMMV.
llvm-svn: 280751
|
|
"Incorrect" file name seen on Android whene the main executable is
called "app_process32" (or 64) but the linker specifies the package
name (e.g. com.android.calculator2). Additionally it can be present
in case of some linker bugs.
This CL adds logic to try to fetch the correct file name from the proc
file system based on the base address sepcified by the linker in case
we are failed to load the module by name.
Differential revision: http://reviews.llvm.org/D22219
llvm-svn: 276411
|
|
case Process::GetFileLoadAddress fails to set it to a real
value. (fixing "conditional use of garbage value" clang warning)
llvm-svn: 275731
|
|
other minor fixes.
llvm-svn: 262570
|
