Change the default from 256 to 24. The argument is that 256 is too large to be scanned by eye, and too large to print in a terminal which can be only 40-50 lines in height. When all children must be shown, `frame variable` and `expression` both support the `-A` (`--show-all-children`) flag. rdar://145327522
This reverts commit daa4061d61216456baa83ab404e096200e327fb4.
Original PR https://github.com/llvm/llvm-project/pull/129092.
I have restricted the test to X86 Windows because it turns out the only
reason that `expr x.get()` would change m_memory_id is that on x86 we
have to write the return address to the stack in ABIWindows_X86_64::PrepareTrivialCall:
```
// Save return address onto the stack
if (!process_sp->WritePointerToMemory(sp, return_addr, error))
return false;
```
This is not required on AArch64 so m_memory_id was not changed:
```
(lldb) expr x.get()
(int) $0 = 0
(lldb) process status -d
Process 15316 stopped
* thread #1, stop reason = Exception 0x80000003 encountered at address 0x7ff764a31034
frame #0: 0x00007ff764a31038 TestProcessModificationIdOnExpr.cpp.tmp`main at TestProcessModificationIdOnExpr.cpp:35
32 __builtin_debugtrap();
33 __builtin_debugtrap();
34 return 0;
-> 35 }
36
37 // CHECK-LABEL: process status -d
38 // CHECK: m_stop_id: 2
ProcessModID:
m_stop_id: 3
m_last_natural_stop_id: 0
m_resume_id: 0
m_memory_id: 0
```
Really we should find a better way to force a memory write here, but
I can't think of one right now.
And a follow up warning fix. This reverts commit 6aa963f780d63d4c8fa80de97dd79c932bc35f4e and 2bff80f25d51e24d3c552e033a2863dd36ef648b. This is failing on Windows on Arm: https://lab.llvm.org/buildbot/#/builders/141/builds/8375 Seems to produce the line the test wants but not in the right place. Reverting while I investigate.
This change adds a setting `target.process.track-memory-cache-changes`. Disabling this setting prevents invalidating and updating values in `ValueObject::UpdateValueIfNeeded` when only "internal" debugger memory is updated. Writing to "internal" debugger memory happens when, for instance, expressions are evaluated by visualizers (pretty printers). One of the examples when cache invalidation has a particularly heavy impact is visualizations of some collections: in some collections getting collection size is an expensive operation (it requires traversal of the collection). At the same time evaluating user expression with side effects (visible to target, not only to debugger) will still bump memory ID because: - If expression is evaluated via interpreter: it will cause write to "non-internal" memory - If expression is JIT-compiled: then to call the function LLDB will write to "non-internal" stack memory The downside of disabled `target.process.track-memory-cache-changes` setting is that convenience variables won't reevaluate synthetic children automatically. --------- Co-authored-by: Mikhail Zakharov <mikhail.zakharov@jetbrains.com>
This patch improves LLDB launch time on Linux machines for **preload
scenarios**, particularly for executables with a lot of shared library
dependencies (or modules). Specifically:
* Launching a binary with `target.preload-symbols = true`
* Attaching to a process with `target.preload-symbols = true`.
It's completely controlled by a new flag added in the first commit
`plugin.dynamic-loader.posix-dyld.parallel-module-load`, which *defaults
to false*. This was inspired by similar work on Darwin #110646.
Some rough numbers to showcase perf improvement, run on a very beefy
machine:
* Executable with ~5600 modules: baseline 45s, improvement 15s
* Executable with ~3800 modules: baseline 25s, improvement 10s
* Executable with ~6650 modules: baseline 67s, improvement 20s
* Executable with ~12500 modules: baseline 185s, improvement 85s
* Executable with ~14700 modules: baseline 235s, improvement 120s
A lot of targets we deal with have a *ton* of modules, and unfortunately
we're unable to convince other folks to reduce the number of modules, so
performance improvements like this can be very impactful for user
experience.
This patch achieves the performance improvement by parallelizing
`DynamicLoaderPOSIXDYLD::RefreshModules` for the launch scenario, and
`DynamicLoaderPOSIXDYLD::LoadAllCurrentModules` for the attach scenario.
The commits have some context on their specific changes as well --
hopefully this helps the review.
# More context on implementation
We discovered the bottlenecks by via `perf record -g -p <lldb's pid>` on
a Linux machine. With an executable known to have 1000s of shared
library dependencies, I ran
```
(lldb) b main
(lldb) r
# taking a while
```
and showed the resulting perf trace (snippet shown)
```
Samples: 85K of event 'cycles:P', Event count (approx.): 54615855812
Children Self Command Shared Object Symbol
- 93.54% 0.00% intern-state libc.so.6 [.] clone3
clone3
start_thread
lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(void*) r
std::_Function_handler<void* (), lldb_private::Process::StartPrivateStateThread(bool)::$_0>::_M_invoke(std::_Any_data const&)
lldb_private::Process::RunPrivateStateThread(bool) n
- lldb_private::Process::HandlePrivateEvent(std::shared_ptr<lldb_private::Event>&)
- 93.54% lldb_private::Process::ShouldBroadcastEvent(lldb_private::Event*)
- 93.54% lldb_private::ThreadList::ShouldStop(lldb_private::Event*)
- lldb_private::Thread::ShouldStop(lldb_private::Event*) *
- 93.53% lldb_private::StopInfoBreakpoint::ShouldStopSynchronous(lldb_private::Event*) t
- 93.52% lldb_private::BreakpointSite::ShouldStop(lldb_private::StoppointCallbackContext*) i
lldb_private::BreakpointLocationCollection::ShouldStop(lldb_private::StoppointCallbackContext*) k
lldb_private::BreakpointLocation::ShouldStop(lldb_private::StoppointCallbackContext*) b
lldb_private::BreakpointOptions::InvokeCallback(lldb_private::StoppointCallbackContext*, unsigned long, unsigned long) i
DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit(void*, lldb_private::StoppointCallbackContext*, unsigned long, unsigned lo
- DynamicLoaderPOSIXDYLD::RefreshModules() O
- 93.42% DynamicLoaderPOSIXDYLD::RefreshModules()::$_0::operator()(DYLDRendezvous::SOEntry const&) const u
- 93.40% DynamicLoaderPOSIXDYLD::LoadModuleAtAddress(lldb_private::FileSpec const&, unsigned long, unsigned long, bools
- lldb_private::DynamicLoader::LoadModuleAtAddress(lldb_private::FileSpec const&, unsigned long, unsigned long, boos
- 83.90% lldb_private::DynamicLoader::FindModuleViaTarget(lldb_private::FileSpec const&) o
- 83.01% lldb_private::Target::GetOrCreateModule(lldb_private::ModuleSpec const&, bool, lldb_private::Status*
- 77.89% lldb_private::Module::PreloadSymbols()
- 44.06% lldb_private::Symtab::PreloadSymbols()
- 43.66% lldb_private::Symtab::InitNameIndexes()
...
```
We saw that majority of time was spent in `RefreshModules`, with the
main culprit within it `LoadModuleAtAddress` which eventually calls
`PreloadSymbols`.
At first, `DynamicLoaderPOSIXDYLD::LoadModuleAtAddress` appears fairly
independent -- most of it deals with different files and then getting or
creating Modules from these files. The portions that aren't independent
seem to deal with ModuleLists, which appear concurrency safe. There were
members of `DynamicLoaderPOSIXDYLD` I had to synchronize though: namely
`m_loaded_modules` which `DynamicLoaderPOSIXDYLD` maintains to map its
loaded modules to their link addresses. Without synchronizing this, I
ran into SEGFAULTS and other issues when running `check-lldb`. I also
locked the assignment and comparison of `m_interpreter_module`, which
may be unnecessary.
# Alternate implementations
When creating this patch, another implementation I considered was
directly background-ing the call to `Module::PreloadSymbol` in
`Target::GetOrCreateModule`. It would have the added benefit of working
across platforms generically, and appeared to be concurrency safe. It
was done via `Debugger::GetThreadPool().async` directly. However, there
were a ton of concurrency issues, so I abandoned that approach for now.
# Testing
With the feature active, I tested via `ninja check-lldb` on both Debug
and Release builds several times (~5 or 6 altogether?), and didn't spot
additional failing or flaky tests.
I also tested manually on several different binaries, some with around
14000 modules, but just basic operations: launching, reaching main,
setting breakpoint, stepping, showing some backtraces.
I've also tested with the flag off just to make sure things behave
properly synchronously.
Currently, an LLDB target option controls whether plugins report all threads. However, it seems natural for this knowledge could come from the plugin itself. To support this, this commits adds a virtual method to the plugin base class, making the Python OS query the target option to preserve existing behavior.
Add the framework code for hooking up and calling the Data Inspection Language (DIL) implementation, as an alternate implementation for the 'frame variable' command. For now, this is an opt-in option, via a target setting 'target.experimental.use-DIL'. See https://discourse.llvm.org/t/rfc-data-inspection-language/69893 for more information about this project. This PR does not actually call any of the DIL code; instead the piece that will eventually call the DIL code (StackFrame::DILEvaluateVariableExpression) calls back into the original 'frame variable' implementation.
Add the ability to override the disassembly CPU and CPU features through a target setting (`target.disassembly-cpu` and `target.disassembly-features`) and a `disassemble` command option (`--cpu` and `--features`). This is especially relevant for architectures like RISC-V which relies heavily on CPU extensions. The majority of this patch is plumbing the options through. I recommend looking at DisassemblerLLVMC and the test for the observable change in behavior.
This retries the PR 113521 skipping a test in a remote environment.
Reverts llvm/llvm-project#113521 due to build bot failures mentioned in the original PR.