llvm-project.git/lldb/source/Core/DynamicLoader.cpp, branch main Unnamed repository; edit this file 'description' to name the repository. [lldb] fix parallel module loading deadlock for Linux DYLD (#166480) 2025-11-14T23:58:43+00:00 Tom Yang zhenyutyang@gmail.com 2025-11-14T23:58:43+00:00 66d5f6a60550a123638bbdf91ec8cff76cb29c5a Another attempt at resolving the deadlock issue @GeorgeHuyubo discovered (his previous [attempt](https://github.com/llvm/llvm-project/pull/160225)). This change can be summarized as the following: * Plumb through a boolean flag to force no preload in `GetOrCreateModules` all the way through to `LoadModuleAtAddress`. * Parallelize `Module::PreloadSymbols` separately from `Target::GetOrCreateModule` and its caller `LoadModuleAtAddress` (this is what avoids the deadlock). These changes roughly maintain the performance characteristics of the previous implementation of parallel module loading. Testing on targets with between 5000 and 14000 modules, I saw similar numbers as before, often more than 10% faster in the new implementation across multiple trials for these massive targets. I think it's because we have less lock contention with this approach. # The deadlock See [bt.txt](https://github.com/user-attachments/files/22524471/bt.txt) for a sample backtrace of LLDB when the deadlock occurs. As @GeorgeHuyubo explains in his [PR](https://github.com/llvm/llvm-project/pull/160225), the deadlock occurs from an ABBA deadlock that happens when a thread context-switches out of `Module::PreloadSymbols`, goes into `Target::GetOrCreateModule` for another module, possibly entering this block: ``` if (!module_sp) { // The platform is responsible for finding and caching an appropriate // module in the shared module cache. if (m_platform_sp) { error = m_platform_sp->GetSharedModule( module_spec, m_process_sp.get(), module_sp, &search_paths, &old_modules, &did_create_module); } else { error = Status::FromErrorString("no platform is currently set"); } } ``` `Module::PreloadSymbols` holds a module-level mutex, and then `GetSharedModule` *attempts* to hold the mutex of the global shared `ModuleList`. So, this thread holds the module mutex, and waits on the global shared `ModuleList` mutex. A competing thread may execute `Target::GetOrCreateModule`, enter the same block as above, grabbing the global shared `ModuleList` mutex. Then, in `ModuleList::GetSharedModule`, we eventually call `ModuleList::FindModules` which eventually waits for the `Module` mutex held by the first thread (via `Module::GetUUID`). Thus, we deadlock. ## Reproducing the deadlock It might be worth noting that I've never been able to observe this deadlock issue during live debugging (e.g. launching or attaching to processes), however we were able to consistently reproduce this issue with coredumps when using the following settings: ``` (lldb) settings set target.parallel-module-load true (lldb) settings set target.preload-symbols true (lldb) settings set symbols.load-on-demand false (lldb) target create --core /some/core/file/here # deadlock happens ``` ## How this change avoids this deadlock This change avoids concurrent executions of `Module::PreloadSymbols` with `Target::GetOrCreateModule` by waiting until after the `Target::GetOrCreateModule` executions to run `Module::PreloadSymbols` in parallel. This avoids the ordering of holding a Module lock *then* the ModuleList lock, as `Target::GetOrCreateModule` executions maintain the ordering of the shared ModuleList lock first (from what I've read and tested). ## Why not read-write lock? Some feedback in https://github.com/llvm/llvm-project/pull/160225 was to modify mutexes used in these components with read-write locks. This might be a good idea overall, but I don't think it would *easily* resolve this specific deadlock. `Module::PreloadSymbols` would probably need a write lock to Module, so even if we had a read lock in `Module::GetUUID` we would still contend. Maybe the `ModuleList` lock could be a read lock that converts to a write lock if it chooses to update the module, but it seems likely that some thread would try to update the shared module list and then the write lock would contend again. Perhaps with deeper architectural changes, we could fix this issue? # Other attempts One downside of this approach (and the former approach of parallel module loading) is that each DYLD would need to implement this pattern themselves. With @clayborg's help, I looked at a few other approaches: * In `Target::GetOrCreateModule`, backgrounding the `Module::PreloadSymbols` call by adding it directly to the thread pool via `Debugger::GetThreadPool().async()`. This required adding a lock to `Module::SetLoadAddress` (probably should be one there already) since `ObjectFileELF::SetLoadAddress` is not thread-safe (updates sections). Unfortunately, during execution, this causes the preload symbols to run synchronously with `Target::GetOrCreateModule`, preventing us from truly parallelizing the execution. * In `Module::PreloadSymbols`, backgrounding the `symtab` and `sym_file` `PreloadSymbols` calls individually, but similar issues as the above. * Passing a callback function like https://github.com/swiftlang/llvm-project/pull/10746 instead of the boolean I use in this change. It's functionally the same change IMO, with some design tradeoffs: * Pro: the caller doesn't need to explicitly call `Module::PreloadSymbols` itself, and can instead call whatever function is passed into the callback. * Con: the caller needs to delay the execution of the callback such that it occurs after the `GetOrCreateModule` logic, otherwise we run into the same issue. I thought this would be trickier for the caller, requiring some kinda condition variable or otherwise storing the calls to execute afterwards. # Test Plan: ``` ninja check-lldb ``` --------- Co-authored-by: Tom Yang <toyang@fb.com> 
Another attempt at resolving the deadlock issue @GeorgeHuyubo discovered
(his previous
[attempt](https://github.com/llvm/llvm-project/pull/160225)).

This change can be summarized as the following:
* Plumb through a boolean flag to force no preload in
`GetOrCreateModules` all the way through to `LoadModuleAtAddress`.
* Parallelize `Module::PreloadSymbols` separately from
`Target::GetOrCreateModule` and its caller `LoadModuleAtAddress` (this
is what avoids the deadlock).

These changes roughly maintain the performance characteristics of the
previous implementation of parallel module loading. Testing on targets
with between 5000 and 14000 modules, I saw similar numbers as before,
often more than 10% faster in the new implementation across multiple
trials for these massive targets. I think it's because we have less lock
contention with this approach.

# The deadlock

See [bt.txt](https://github.com/user-attachments/files/22524471/bt.txt)
for a sample backtrace of LLDB when the deadlock occurs.

As @GeorgeHuyubo explains in his
[PR](https://github.com/llvm/llvm-project/pull/160225), the deadlock
occurs from an ABBA deadlock that happens when a thread context-switches
out of `Module::PreloadSymbols`, goes into `Target::GetOrCreateModule`
for another module, possibly entering this block:
```
      if (!module_sp) {
        // The platform is responsible for finding and caching an appropriate
        // module in the shared module cache.
        if (m_platform_sp) {
          error = m_platform_sp->GetSharedModule(
              module_spec, m_process_sp.get(), module_sp, &search_paths,
              &old_modules, &did_create_module);
        } else {
          error = Status::FromErrorString("no platform is currently set");
        }
      }
```
`Module::PreloadSymbols` holds a module-level mutex, and then
`GetSharedModule` *attempts* to hold the mutex of the global shared
`ModuleList`. So, this thread holds the module mutex, and waits on the
global shared `ModuleList` mutex.

A competing thread may execute `Target::GetOrCreateModule`, enter the
same block as above, grabbing the global shared `ModuleList` mutex.
Then, in `ModuleList::GetSharedModule`, we eventually call
`ModuleList::FindModules` which eventually waits for the `Module` mutex
held by the first thread (via `Module::GetUUID`). Thus, we deadlock.

## Reproducing the deadlock

It might be worth noting that I've never been able to observe this
deadlock issue during live debugging (e.g. launching or attaching to
processes), however we were able to consistently reproduce this issue
with coredumps when using the following settings:
```
(lldb) settings set target.parallel-module-load true
(lldb) settings set target.preload-symbols true
(lldb) settings set symbols.load-on-demand false
(lldb) target create --core /some/core/file/here
# deadlock happens
```

## How this change avoids this deadlock

This change avoids concurrent executions of `Module::PreloadSymbols`
with `Target::GetOrCreateModule` by waiting until after the
`Target::GetOrCreateModule` executions to run `Module::PreloadSymbols`
in parallel. This avoids the ordering of holding a Module lock *then*
the ModuleList lock, as `Target::GetOrCreateModule` executions maintain
the ordering of the shared ModuleList lock first (from what I've read
and tested).

## Why not read-write lock?

Some feedback in https://github.com/llvm/llvm-project/pull/160225 was to
modify mutexes used in these components with read-write locks. This
might be a good idea overall, but I don't think it would *easily*
resolve this specific deadlock. `Module::PreloadSymbols` would probably
need a write lock to Module, so even if we had a read lock in
`Module::GetUUID` we would still contend. Maybe the `ModuleList` lock
could be a read lock that converts to a write lock if it chooses to
update the module, but it seems likely that some thread would try to
update the shared module list and then the write lock would contend
again.

Perhaps with deeper architectural changes, we could fix this issue?

# Other attempts

One downside of this approach (and the former approach of parallel
module loading) is that each DYLD would need to implement this pattern
themselves. With @clayborg's help, I looked at a few other approaches:
* In `Target::GetOrCreateModule`, backgrounding the
`Module::PreloadSymbols` call by adding it directly to the thread pool
via `Debugger::GetThreadPool().async()`. This required adding a lock to
`Module::SetLoadAddress` (probably should be one there already) since
`ObjectFileELF::SetLoadAddress` is not thread-safe (updates sections).
Unfortunately, during execution, this causes the preload symbols to run
synchronously with `Target::GetOrCreateModule`, preventing us from truly
parallelizing the execution.
* In `Module::PreloadSymbols`, backgrounding the `symtab` and `sym_file`
`PreloadSymbols` calls individually, but similar issues as the above.
* Passing a callback function like
https://github.com/swiftlang/llvm-project/pull/10746 instead of the
boolean I use in this change. It's functionally the same change IMO,
with some design tradeoffs:
* Pro: the caller doesn't need to explicitly call
`Module::PreloadSymbols` itself, and can instead call whatever function
is passed into the callback.
* Con: the caller needs to delay the execution of the callback such that
it occurs after the `GetOrCreateModule` logic, otherwise we run into the
same issue. I thought this would be trickier for the caller, requiring
some kinda condition variable or otherwise storing the calls to execute
afterwards.

# Test Plan:
```
ninja check-lldb
```

---------

Co-authored-by: Tom Yang <toyang@fb.com>
 [lldb] Enable locate module callback for all module loading (#160199) 2025-11-06T20:48:21+00:00 GeorgeHuyubo 113479859+GeorgeHuyubo@users.noreply.github.com 2025-11-06T20:48:21+00:00 fce58897ce82de84c8d794609132eb547b2b4871 Main executables were bypassing the locate module callback that shared libraries use, preventing custom symbol file location logic from working consistently. This PR fix this by * Adding target context to ModuleSpec * Leveraging that context to use target search path and platform's locate module callback in ModuleList::GetSharedModule This ensures both main executables and shared libraries get the same callback treatment for symbol file resolution. --------- Co-authored-by: George Hu <hyubo@meta.com> Co-authored-by: George Hu <georgehuyubo@gmail.com> 
Main executables were bypassing the locate module callback that shared 
libraries use, preventing custom symbol file location logic from working
consistently. 

This PR fix this by
*   Adding target context to ModuleSpec
* Leveraging that context to use target search path and platform's
locate module callback in ModuleList::GetSharedModule

This ensures both main executables and shared libraries get the same 
callback treatment for symbol file resolution.

---------

Co-authored-by: George Hu <hyubo@meta.com>
Co-authored-by: George Hu <georgehuyubo@gmail.com>
 [lldb] Fix incorrect print of UUID and load address (#152560) 2025-08-08T04:31:48+00:00 Dominic Chen 1108560+ddcc@users.noreply.github.com 2025-08-08T04:31:48+00:00 92ac1ac9046d785f5f0c68e2d9f74b05c4db5d9c The current display is missing a space, for example: ``` no target │ Locating binary: 24906A83-0182-361B-8B4A-90A249B04FD7at 0x0000000c0d108000 ``` Co-authored-by: Dominic Chen <daming_chen@apple.com> 
The current display is missing a space, for example:
```
no target │ Locating binary: 24906A83-0182-361B-8B4A-90A249B04FD7at 0x0000000c0d108000
```

Co-authored-by: Dominic Chen <daming_chen@apple.com>
 [lldb] Re-insert code to search for a binary by filepath if provided 2025-06-18T00:41:31+00:00 Jason Molenda jmolenda@apple.com 2025-06-18T00:34:09+00:00 4e090b6e84e33e2a442e3951253ca570f8f842f8 July 14 2024 I landed a change to update progress reporting when loading kernel/firmware binaries https://github.com/llvm/llvm-project/pull/98845 In DynamicLoader::LoadBinaryWithUUIDAndAddress I removed code that was setting the ModuleSpec to the provided name, if the name provided is that of a file on disk. With this code missing, if a filepath name is passed in, this code will fail to find that binary on the local disk. There's nothing in the PR / intention that would lead to this change, it was unintentional. 
July 14 2024 I landed a change to update progress reporting when
loading kernel/firmware binaries
https://github.com/llvm/llvm-project/pull/98845
In DynamicLoader::LoadBinaryWithUUIDAndAddress I removed code that
was setting the ModuleSpec to the provided name, if the name provided
is that of a file on disk.  With this code missing, if a filepath
name is passed in, this code will fail to find that binary on the local
disk.  There's nothing in the PR / intention that would lead to this
change, it was unintentional.
Add symbol locator time for each module in statistics (#137379) 2025-04-25T18:48:31+00:00 GeorgeHuyubo 113479859+GeorgeHuyubo@users.noreply.github.com 2025-04-25T18:48:31+00:00 77f8335a07e65f88a2c2925f175f48c458911cee Identical PR to: https://github.com/llvm/llvm-project/pull/134563 Previous PR was approved and landed but broke the build due to bad merge. Manually resolve the merge conflict and try to land again. Co-authored-by: George Hu <georgehuyubo@gmail.com> 
Identical PR to: https://github.com/llvm/llvm-project/pull/134563
Previous PR was approved and landed but broke the build due to bad
merge.
Manually resolve the merge conflict and try to land again.

Co-authored-by: George Hu <georgehuyubo@gmail.com>
 Revert "Add symbol locator time for each module in statistics (#134563)" 2025-04-25T18:01:19+00:00 Kazu Hirata kazu@google.com 2025-04-25T18:01:19+00:00 bc716a755a9bd0e5f0b5de0da0ffcc77374690db This reverts commit 070a4ae2f9bcf6967a7147ed2972f409eaa7d3a6. Multiple buildbot failures have been reported: https://github.com/llvm/llvm-project/pull/134563 The build fails with: lldb/source/Target/Statistics.cpp:75:39: error: use of undeclared identifier 'num_symbols_loaded' 
This reverts commit 070a4ae2f9bcf6967a7147ed2972f409eaa7d3a6.

Multiple buildbot failures have been reported:
https://github.com/llvm/llvm-project/pull/134563

The build fails with:

  lldb/source/Target/Statistics.cpp:75:39: error: use of undeclared
  identifier 'num_symbols_loaded'
Add symbol locator time for each module in statistics (#134563) 2025-04-25T17:43:17+00:00 GeorgeHuyubo 113479859+GeorgeHuyubo@users.noreply.github.com 2025-04-25T17:43:17+00:00 070a4ae2f9bcf6967a7147ed2972f409eaa7d3a6 In statistics, add locate time for each module in statistics, we time the PluginManager::LocateExecutableSymbolFile and PluginManager::LocateExecutableObjectFile call, save the duration for the succeeded symbol locator plugin in the Module class as a map. New key being added: Module level: "_symbolLocatorTime_" Summary level: "_totalSymbolLocatorTime_" which would be a map of symbol_locator_plugin_name to time. Sample statistic dump output after this change: ``` Command: statistics dump ===============Output=============== { "commands": { "command container add": 1, "command script add": 51, "command script import": 59, "statistics dump": 1, "target create": 1, "type summary add": 36, "type synthetic add": 21 }, "memory": { "strings": { "bytesTotal": 2801664, "bytesUnused": 1704256, "bytesUsed": 1097408 } }, "modules": [ { "debugInfoByteSize": 244927, "debugInfoEnabled": true, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0.61934599999999995, "debugInfoParseTime": 0.00033100000000000002, "identifier": 94868115726768, "path": "/home/hyubo/.cache/llvm-debuginfod/client/llvmcache-720c9a0f5ba8b460a1524a25597226f0fa551f71-c4crasher", "symbolLocatorTime": { "debuginfod": 1.4547020000000002 }, "symbolTableIndexTime": 0.0035370000000000002, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.0055040000000000002, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--linux", "uuid": "720C9A0F-5BA8-B460-A152-4A25597226F0-FA551F71" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868111142256, "path": "[vdso](0x00007ffd41c59000)", "symbolLocatorTime": { "debuginfod": 0.80597700000000005 }, "symbolTableIndexTime": 2.8e-05, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.00037300000000000001, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--linux", "uuid": "4D1F38BD-BD34-DFB3-C9A5-B49A2A912219-AC713763" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868116162096, "path": "/usr/local/fbcode/platform010/lib/libc.so.6", "symbolLocatorTime": { "debuginfod": 0.286356 }, "symbolTableIndexTime": 0.0091780000000000004, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.081841999999999998, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--linux", "uuid": "ACE9DF01-8872-3A35-6D14-3C92527EF739-4BE32C75" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868111990032, "path": "/usr/local/fbcode/platform010/lib/libm.so.6", "symbolLocatorTime": { "debuginfod": 0.63356699999999999 }, "symbolTableIndexTime": 0.0023960000000000001, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.021706, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--linux", "uuid": "85932B54-0DE7-4FC1-23B0-FB09AD1A5A61-8E1098B7" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868116399488, "path": "/usr/local/fbcode/platform010/lib/libmvec.so.1", "symbolLocatorTime": { "debuginfod": 0.66705099999999995 }, "symbolTableIndexTime": 0.00053700000000000004, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.0034429999999999999, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--linux", "uuid": "4537CA22-8E6E-495B-7A03-FC2CEDCAD71C-8A7B2067" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868111778368, "path": "/usr/local/fbcode/platform010/lib/libatomic.so.1", "symbolLocatorTime": { "debuginfod": 0.83268299999999995 }, "symbolTableIndexTime": 0.000243, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.0026640000000000001, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--", "uuid": "F90F9111-BBD1-C2A9-972A-34EB75ABE62E-3BDED9CF" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868116403264, "path": "/usr/local/fbcode/platform010/lib/libgcc_s.so.1", "symbolLocatorTime": { "debuginfod": 0.58871499999999999 }, "symbolTableIndexTime": 0.000292, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.0033, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--", "uuid": "1C715A92-40BE-BE95-E148-1B401749BAB8-15D09F9D" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868113225328, "path": "/usr/local/fbcode/platform010/lib/libstdc++.so.6", "symbolLocatorTime": { "debuginfod": 0.76993400000000001 }, "symbolTableIndexTime": 0.042455, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.081374000000000002, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--", "uuid": "C9DAEE53-F607-981B-6BED-F2042833BFC7-71A1E66C" }, { "debugInfoByteSize": 0, "debugInfoEnabled": false, "debugInfoHadIncompleteTypes": false, "debugInfoHadVariableErrors": false, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 94868113600960, "path": "/usr/local/fbcode/platform010/lib/ld.so", "symbolLocatorTime": { "debuginfod": 0.48382199999999997 }, "symbolTableIndexTime": 0.00054799999999999998, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.0057219999999999997, "symbolTableSavedToCache": false, "symbolTableStripped": false, "triple": "x86_64--linux", "uuid": "CCA86CF4-E4FF-42C8-7056-2F7D8B83AEE0-530B4095" } ], "targets": [ { "breakpoints": [], "dyldPluginName": "dump-modulelist-dyld", "expressionEvaluation": { "failures": 0, "successes": 0 }, "frameVariable": { "failures": 0, "successes": 0 }, "mismatchCoredumpModuleCount": 0, "moduleIdentifiers": [ 94868115726768, 94868111142256, 94868116162096, 94868111990032, 94868116399488, 94868111778368, 94868116403264, 94868113225328, 94868113600960 ], "signals": [ { "SIGABRT": 1 } ], "sourceMapDeduceCount": 0, "sourceRealpathAttemptCount": 0, "sourceRealpathCompatibleCount": 0, "stopCount": 1, "summaryProviderStatistics": [], "targetCreateTime": 4.1999999999999998e-05, "totalBreakpointResolveTime": 0, "totalSharedLibraryEventHitCount": 0 } ], "totalDebugInfoByteSize": 244927, "totalDebugInfoEnabled": 1, "totalDebugInfoIndexLoadedFromCache": 0, "totalDebugInfoIndexSavedToCache": 0, "totalDebugInfoIndexTime": 0.61934599999999995, "totalDebugInfoParseTime": 0.00033100000000000002, "totalModuleCount": 9, "totalModuleCountHasDebugInfo": 1, "totalModuleCountWithIncompleteTypes": 0, "totalModuleCountWithVariableErrors": 0, "totalSymbolLocatorTime": { "debuginfod": 6.5228070000000002 }, "totalSymbolTableIndexTime": 0.059214000000000003, "totalSymbolTableParseTime": 0.205928, "totalSymbolTableStripped": 0, "totalSymbolTablesLoadedFromCache": 0, "totalSymbolTablesSavedToCache": 0, "transcript": [ { "command": "symsrv", "commandArguments": "", "commandName": "symsrv", "durationInSeconds": 0.00069099999999999999, "error": "", "output": "", "timestampInEpochSeconds": 1744934015 }, { "command": "settings set symbols.enable-external-lookup true", "commandArguments": "symbols.enable-external-lookup true", "commandName": "settings set", "durationInSeconds": 4.1999999999999998e-05, "error": "", "output": "", "timestampInEpochSeconds": 1744934015 }, { "command": "settings insert-before plugin.symbol-locator.debuginfod.server-urls 0 https://our.intern.facebook.com/intern/debuginfod", "commandArguments": "plugin.symbol-locator.debuginfod.server-urls 0 https://our.intern.facebook.com/intern/debuginfod", "commandName": "settings insert-before", "durationInSeconds": 5.5999999999999999e-05, "error": "", "output": "", "timestampInEpochSeconds": 1744934015 }, { "command": "target create --core \"/var/tmp/cores/c4crasher.crasher.1576894\"", "commandArguments": "--core \"/var/tmp/cores/c4crasher.crasher.1576894\"", "commandName": "target create", "durationInSeconds": 6.7297630000000002, "error": "", "output": "Core file '/var/tmp/cores/c4crasher.crasher.1576894' (x86_64) was loaded.\n", "timestampInEpochSeconds": 1744934017 }, { "command": "fbpaste statistics dump", "commandArguments": "statistics dump", "commandName": "fbpaste", "timestampInEpochSeconds": 1744934055 }, { "command": "statistics dump", "commandArguments": "", "commandName": "statistics dump", "timestampInEpochSeconds": 1744934055 } ] } ``` Co-authored-by: George Hu <georgehuyubo@gmail.com> 
In statistics, add locate time for each module in statistics, we time
the PluginManager::LocateExecutableSymbolFile and
PluginManager::LocateExecutableObjectFile call, save the duration for
the succeeded symbol locator plugin in the Module class as a map.
New key being added:
Module level: "_symbolLocatorTime_"
Summary level: "_totalSymbolLocatorTime_"
which would be a map of symbol_locator_plugin_name to time.

Sample statistic dump output after this change:

```
Command: statistics dump
===============Output===============
{
  "commands": {
    "command container add": 1,
    "command script add": 51,
    "command script import": 59,
    "statistics dump": 1,
    "target create": 1,
    "type summary add": 36,
    "type synthetic add": 21
  },
  "memory": {
    "strings": {
      "bytesTotal": 2801664,
      "bytesUnused": 1704256,
      "bytesUsed": 1097408
    }
  },
  "modules": [
    {
      "debugInfoByteSize": 244927,
      "debugInfoEnabled": true,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0.61934599999999995,
      "debugInfoParseTime": 0.00033100000000000002,
      "identifier": 94868115726768,
      "path": "/home/hyubo/.cache/llvm-debuginfod/client/llvmcache-720c9a0f5ba8b460a1524a25597226f0fa551f71-c4crasher",
      "symbolLocatorTime": {
        "debuginfod": 1.4547020000000002
      },
      "symbolTableIndexTime": 0.0035370000000000002,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.0055040000000000002,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--linux",
      "uuid": "720C9A0F-5BA8-B460-A152-4A25597226F0-FA551F71"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868111142256,
      "path": "[vdso](0x00007ffd41c59000)",
      "symbolLocatorTime": {
        "debuginfod": 0.80597700000000005
      },
      "symbolTableIndexTime": 2.8e-05,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.00037300000000000001,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--linux",
      "uuid": "4D1F38BD-BD34-DFB3-C9A5-B49A2A912219-AC713763"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868116162096,
      "path": "/usr/local/fbcode/platform010/lib/libc.so.6",
      "symbolLocatorTime": {
        "debuginfod": 0.286356
      },
      "symbolTableIndexTime": 0.0091780000000000004,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.081841999999999998,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--linux",
      "uuid": "ACE9DF01-8872-3A35-6D14-3C92527EF739-4BE32C75"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868111990032,
      "path": "/usr/local/fbcode/platform010/lib/libm.so.6",
      "symbolLocatorTime": {
        "debuginfod": 0.63356699999999999
      },
      "symbolTableIndexTime": 0.0023960000000000001,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.021706,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--linux",
      "uuid": "85932B54-0DE7-4FC1-23B0-FB09AD1A5A61-8E1098B7"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868116399488,
      "path": "/usr/local/fbcode/platform010/lib/libmvec.so.1",
      "symbolLocatorTime": {
        "debuginfod": 0.66705099999999995
      },
      "symbolTableIndexTime": 0.00053700000000000004,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.0034429999999999999,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--linux",
      "uuid": "4537CA22-8E6E-495B-7A03-FC2CEDCAD71C-8A7B2067"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868111778368,
      "path": "/usr/local/fbcode/platform010/lib/libatomic.so.1",
      "symbolLocatorTime": {
        "debuginfod": 0.83268299999999995
      },
      "symbolTableIndexTime": 0.000243,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.0026640000000000001,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--",
      "uuid": "F90F9111-BBD1-C2A9-972A-34EB75ABE62E-3BDED9CF"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868116403264,
      "path": "/usr/local/fbcode/platform010/lib/libgcc_s.so.1",
      "symbolLocatorTime": {
        "debuginfod": 0.58871499999999999
      },
      "symbolTableIndexTime": 0.000292,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.0033,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--",
      "uuid": "1C715A92-40BE-BE95-E148-1B401749BAB8-15D09F9D"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868113225328,
      "path": "/usr/local/fbcode/platform010/lib/libstdc++.so.6",
      "symbolLocatorTime": {
        "debuginfod": 0.76993400000000001
      },
      "symbolTableIndexTime": 0.042455,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.081374000000000002,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--",
      "uuid": "C9DAEE53-F607-981B-6BED-F2042833BFC7-71A1E66C"
    },
    {
      "debugInfoByteSize": 0,
      "debugInfoEnabled": false,
      "debugInfoHadIncompleteTypes": false,
      "debugInfoHadVariableErrors": false,
      "debugInfoIndexLoadedFromCache": false,
      "debugInfoIndexSavedToCache": false,
      "debugInfoIndexTime": 0,
      "debugInfoParseTime": 0,
      "identifier": 94868113600960,
      "path": "/usr/local/fbcode/platform010/lib/ld.so",
      "symbolLocatorTime": {
        "debuginfod": 0.48382199999999997
      },
      "symbolTableIndexTime": 0.00054799999999999998,
      "symbolTableLoadedFromCache": false,
      "symbolTableParseTime": 0.0057219999999999997,
      "symbolTableSavedToCache": false,
      "symbolTableStripped": false,
      "triple": "x86_64--linux",
      "uuid": "CCA86CF4-E4FF-42C8-7056-2F7D8B83AEE0-530B4095"
    }
  ],
  "targets": [
    {
      "breakpoints": [],
      "dyldPluginName": "dump-modulelist-dyld",
      "expressionEvaluation": {
        "failures": 0,
        "successes": 0
      },
      "frameVariable": {
        "failures": 0,
        "successes": 0
      },
      "mismatchCoredumpModuleCount": 0,
      "moduleIdentifiers": [
        94868115726768,
        94868111142256,
        94868116162096,
        94868111990032,
        94868116399488,
        94868111778368,
        94868116403264,
        94868113225328,
        94868113600960
      ],
      "signals": [
        {
          "SIGABRT": 1
        }
      ],
      "sourceMapDeduceCount": 0,
      "sourceRealpathAttemptCount": 0,
      "sourceRealpathCompatibleCount": 0,
      "stopCount": 1,
      "summaryProviderStatistics": [],
      "targetCreateTime": 4.1999999999999998e-05,
      "totalBreakpointResolveTime": 0,
      "totalSharedLibraryEventHitCount": 0
    }
  ],
  "totalDebugInfoByteSize": 244927,
  "totalDebugInfoEnabled": 1,
  "totalDebugInfoIndexLoadedFromCache": 0,
  "totalDebugInfoIndexSavedToCache": 0,
  "totalDebugInfoIndexTime": 0.61934599999999995,
  "totalDebugInfoParseTime": 0.00033100000000000002,
  "totalModuleCount": 9,
  "totalModuleCountHasDebugInfo": 1,
  "totalModuleCountWithIncompleteTypes": 0,
  "totalModuleCountWithVariableErrors": 0,
  "totalSymbolLocatorTime": {
    "debuginfod": 6.5228070000000002
  },
  "totalSymbolTableIndexTime": 0.059214000000000003,
  "totalSymbolTableParseTime": 0.205928,
  "totalSymbolTableStripped": 0,
  "totalSymbolTablesLoadedFromCache": 0,
  "totalSymbolTablesSavedToCache": 0,
  "transcript": [
    {
      "command": "symsrv",
      "commandArguments": "",
      "commandName": "symsrv",
      "durationInSeconds": 0.00069099999999999999,
      "error": "",
      "output": "",
      "timestampInEpochSeconds": 1744934015
    },
    {
      "command": "settings set symbols.enable-external-lookup true",
      "commandArguments": "symbols.enable-external-lookup true",
      "commandName": "settings set",
      "durationInSeconds": 4.1999999999999998e-05,
      "error": "",
      "output": "",
      "timestampInEpochSeconds": 1744934015
    },
    {
      "command": "settings insert-before plugin.symbol-locator.debuginfod.server-urls 0 https://our.intern.facebook.com/intern/debuginfod",
      "commandArguments": "plugin.symbol-locator.debuginfod.server-urls 0 https://our.intern.facebook.com/intern/debuginfod",
      "commandName": "settings insert-before",
      "durationInSeconds": 5.5999999999999999e-05,
      "error": "",
      "output": "",
      "timestampInEpochSeconds": 1744934015
    },
    {
      "command": "target create --core \"/var/tmp/cores/c4crasher.crasher.1576894\"",
      "commandArguments": "--core \"/var/tmp/cores/c4crasher.crasher.1576894\"",
      "commandName": "target create",
      "durationInSeconds": 6.7297630000000002,
      "error": "",
      "output": "Core file '/var/tmp/cores/c4crasher.crasher.1576894' (x86_64) was loaded.\n",
      "timestampInEpochSeconds": 1744934017
    },
    {
      "command": "fbpaste statistics dump",
      "commandArguments": "statistics dump",
      "commandName": "fbpaste",
      "timestampInEpochSeconds": 1744934055
    },
    {
      "command": "statistics dump",
      "commandArguments": "",
      "commandName": "statistics dump",
      "timestampInEpochSeconds": 1744934055
    }
  ]
}

```

Co-authored-by: George Hu <georgehuyubo@gmail.com>
 [lldb] Store StreamAsynchronousIO in a unique_ptr (NFC) (#127961) 2025-02-20T19:13:46+00:00 Jonas Devlieghere jonas@devlieghere.com 2025-02-20T19:13:46+00:00 78d82d3ae7ac99833e1b9c0b529c256f90b6c6cc Make StreamAsynchronousIO an unique_ptr instead of a shared_ptr. I tried passing the class by value, but the llvm::raw_ostream forwarder stored in the Stream parent class isn't movable and I don't think it's worth changing that. Additionally, there's a few places that expect a StreamSP, which are easily created from a StreamUP. 
Make StreamAsynchronousIO an unique_ptr instead of a shared_ptr. I tried
passing the class by value, but the llvm::raw_ostream forwarder stored
in the Stream parent class isn't movable and I don't think it's worth
changing that. Additionally, there's a few places that expect a
StreamSP, which are easily created from a StreamUP.
 [lldb] Remove Debugger::Get{Output,Error}Stream (NFC) (#126821) 2025-02-12T16:29:06+00:00 Jonas Devlieghere jonas@devlieghere.com 2025-02-12T16:29:06+00:00 eff3c343b08cfc46016708b3182ac062d45b3e21 Remove Debugger::GetOutputStream and Debugger::GetErrorStream in preparation for replacing both with a new variant that needs to be locked and hence can't be handed out like we do right now. The patch replaces most uses with GetAsyncOutputStream and GetAsyncErrorStream respectively. There methods return new StreamSP objects that automatically get flushed on destruction. See #126630 for more details. 
Remove Debugger::GetOutputStream and Debugger::GetErrorStream in
preparation for replacing both with a new variant that needs to be
locked and hence can't be handed out like we do right now.

The patch replaces most uses with GetAsyncOutputStream and
GetAsyncErrorStream respectively. There methods return new StreamSP
objects that automatically get flushed on destruction.

See #126630 for more details.
 [lldb] Fix TestLoadUnload.py (#117416) 2024-11-24T19:04:47+00:00 Kazuki Sakamoto sakamoto@splhack.org 2024-11-24T19:04:47+00:00 c2ffb42893eb543f64169e32851e00352feaca69 ELF core debugging fix #117070 broke TestLoadUnload.py tests due to GetModuleSpec call, ProcessGDBRemote fetches modules from remote. Revise the original PR, renamed FindBuildId to FindModuleUUID. 
ELF core debugging fix #117070 broke TestLoadUnload.py tests due to
GetModuleSpec call, ProcessGDBRemote fetches modules from remote. Revise
the original PR, renamed FindBuildId to FindModuleUUID.