| Age | Commit message (Collapse) | Author |
|---|
|
https://github.com/codespell-project/codespell
```bash
codespell bolt --skip="*.yaml,Maintainers.txt" --write-changes \
--ignore-words-list=acount,alledges,ans,archtype,defin,iself,mis,mmaped,othere,outweight,vas
```
|
|
The BOLT runtime currently does not close the FD pointing to
/proc/self/map_files. This does not actually hurt anything but was at
least a bit of a red herring for me when looking through strace on a
BOLT instrumented binary.
|
|
`__bolt_instr_data_dump()` will find instrumented binary name by
iterating through entries under directory `/proc/self/map_files`,
and then open the binary and memory map it onto heap in order
to locate `.bolt.instr.tables` section to read the descriptions.
If binary name is already known and/or binary is already opened
as memory mapped, we can pass binary name and/or memory
buffer directly to `__bolt_instr_data_dump()` to save some work.
|
|
(#148543)
|
|
This patch adds code generation for RISCV64 instrumentation.The work
involved includes the following three points:
a) Implements support for instrumenting direct function call and jump
on RISC-V which relies on , Atomic instructions
(used to increment counters) are only available on RISC-V when the A
extension is used.
b) Implements support for instrumenting direct function inderect call
by implementing the createInstrumentedIndCallHandlerEntryBB and
createInstrumentedIndCallHandlerExitBB interfaces. In this process, we
need to accurately record the target address and IndCallID to ensure
the correct recording of the indirect call counters.
c)Implemented the RISCV64 Bolt runtime library, implemented some system
call interfaces through embedded assembly. Get the difference between
runtime addrress of .text section andstatic address in section header
table, which in turn can be used to search for indirect call
description.
However, the community code currently has problems with relocation in
some scenarios, but this has nothing to do with instrumentation. We
may continue to submit patches to fix the related bugs.
|
|
Addressing #81441
|
|
Closes https://github.com/llvm/llvm-project/issues/63097
Before merging please make sure the change to
bolt/include/bolt/Passes/StokeInfo.h is correct.
bolt/include/bolt/Passes/StokeInfo.h
```diff
// This Pass solves the two major problems to use the Stoke program without
- // proting its code:
+ // probing its code:
```
I'm still not happy about the awkward wording in this comment.
bolt/include/bolt/Passes/FixRelaxationPass.h
```
$ ed -s bolt/include/bolt/Passes/FixRelaxationPass.h <<<'9,12p'
// This file declares the FixRelaxations class, which locates instructions with
// wrong targets and fixes them. Such problems usually occures when linker
// relaxes (changes) instructions, but doesn't fix relocations types properly
// for them.
$
```
bolt/docs/doxygen.cfg.in
bolt/include/bolt/Core/BinaryContext.h
bolt/include/bolt/Core/BinaryFunction.h
bolt/include/bolt/Core/BinarySection.h
bolt/include/bolt/Core/DebugData.h
bolt/include/bolt/Core/DynoStats.h
bolt/include/bolt/Core/Exceptions.h
bolt/include/bolt/Core/MCPlusBuilder.h
bolt/include/bolt/Core/Relocation.h
bolt/include/bolt/Passes/FixRelaxationPass.h
bolt/include/bolt/Passes/InstrumentationSummary.h
bolt/include/bolt/Passes/ReorderAlgorithm.h
bolt/include/bolt/Passes/StackReachingUses.h
bolt/include/bolt/Passes/StokeInfo.h
bolt/include/bolt/Passes/TailDuplication.h
bolt/include/bolt/Profile/DataAggregator.h
bolt/include/bolt/Profile/DataReader.h
bolt/lib/Core/BinaryContext.cpp
bolt/lib/Core/BinarySection.cpp
bolt/lib/Core/DebugData.cpp
bolt/lib/Core/DynoStats.cpp
bolt/lib/Core/Relocation.cpp
bolt/lib/Passes/Instrumentation.cpp
bolt/lib/Passes/JTFootprintReduction.cpp
bolt/lib/Passes/ReorderData.cpp
bolt/lib/Passes/RetpolineInsertion.cpp
bolt/lib/Passes/ShrinkWrapping.cpp
bolt/lib/Passes/TailDuplication.cpp
bolt/lib/Rewrite/BoltDiff.cpp
bolt/lib/Rewrite/DWARFRewriter.cpp
bolt/lib/Rewrite/RewriteInstance.cpp
bolt/lib/Utils/CommandLineOpts.cpp
bolt/runtime/instr.cpp
bolt/test/AArch64/got-ld64-relaxation.test
bolt/test/AArch64/unmarked-data.test
bolt/test/X86/Inputs/dwarf5-cu-no-debug-addr-helper.s
bolt/test/X86/Inputs/linenumber.cpp
bolt/test/X86/double-jump.test
bolt/test/X86/dwarf5-call-pc-function-null-check.test
bolt/test/X86/dwarf5-split-dwarf4-monolithic.test
bolt/test/X86/dynrelocs.s
bolt/test/X86/fallthrough-to-noop.test
bolt/test/X86/tail-duplication-cache.s
bolt/test/runtime/X86/instrumentation-ind-calls.s
|
|
This commit adds support for AArch64 in instrumentation runtime library,
including AArch64 system calls.
Also this commit divides syscalls into target-specific files.
Reviewed By: rafauler, yota9
Differential Revision: https://reviews.llvm.org/D151942
|
|
Because indirect call tables use static addresses for call sites, but pc
values recorded by runtime may be subject to ASLR in PIE, we couldn't
find indirect call descriptions by their runtime address in PIE. It
resulted in [unknown] entries in profile for all indirect calls. We need
to substract base address of .text from runtime addresses to get the
corresponding static addresses. Here we create a getter for base address
of .text and substract it's return value from recorded PC values. It
converts them to static addresses, which then may be used to find the
corresponding indirect call descriptions.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D154121
|
|
When a binary is instrumented with --instrumentation-sleep-time and
instrumentation-wait-forks options and lauched, the profile is
periodically written until all the forks die. The problem is that we
cannot wait for the whole process tree, and we have no way to tell when
it's safe to read the profile. Hovewer, if we keep profile open
throughout the life of the process tree, we can use fuser to determine
when writing is finished.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D154436
|
|
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D154419
|
|
default"
The issue was caused by the absence of placement new definition. It
worked for clang and thus passed Phabricator checks, but broke when
compiled with GCC on buildbot.
Full problem description: https://reviews.llvm.org/D153771#4468239
Original patch description:
In absence of instrumentation-file-append-pid option,
global allocator uses shared pages for allocation. However, since it is a
global variable, it gets COW'd after fork if instrumentation-sleep-time
is used, or if a process forks by itself. This means it handles the same
pages to every process which causes hash table corruption. Thus, if we
want shared pages, we need to put the allocator itself in a shared page,
which we do in this commit in __bolt_instr_setup.
I also added a couple of assertions to sanity-check the hash table.
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D153771
|
|
This reverts commit c15e9b6814e53bccb0194268a826c1213a84b395.
The issue appears unrelated as the crash happened in the BOLTed binary, not
instrumented binary.
|
|
In a very rare case that mmap call fails, we'll at least get a message
instead of segfault.
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D154056
|
|
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D154056
|
|
This reverts commit f0b45fba4b64ab0b5d6c50d978e02f0d12d4d070.
The stack broke https://lab.llvm.org/buildbot/#/builders/252.
|
|
This reverts commit 8f7c53ef81c17ae9d773818181d04ef1c3890912.
The stack broke https://lab.llvm.org/buildbot/#/builders/252.
|
|
This reverts commit 02c3724d43840339fdc91d21747e96b5f7405bb0.
This change breaks instrumented Clang:
https://lab.llvm.org/buildbot/#/builders/252/builds/2700
|
|
default"
This reverts commit ad4e0770ca7ebbc4dd6635b17421819b2393aa33.
Breaks BOLT upstream testing:
https://lab.llvm.org/buildbot/#/builders/244/builds/13736
|
|
Since there are no other means to debug the instrumentation library
other than using stdout, having a function to print hash table entries
is very useful.
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D153771
|
|
In absence of instrumentation-file-append-pid option,
global allocator uses shared pages for allocation. However, since it is a
global variable, it gets COW'd after fork if instrumentation-sleep-time
is used, or if a process forks by itself. This means it handles the same
pages to every process which causes hash table corruption. Thus, if we
want shared pages, we need to put the allocator itself in a shared page,
which we do in this commit in __bolt_instr_setup.
I also added a couple of assertions to sanity-check the hash table.
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D153771
|
|
The point of append-pid option is to record separate profiles for
separate forks, which is impossible when counters are the same for
every process. It leads to a sum of all profiles in every file, plus
GlobalWriteProfileMutex located in a shared memory prevents some
processes from dumping their data at all.
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D153771
|
|
In a very rare case that mmap call fails, we'll at least get a message
instead of segfault.
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D154056
|
|
Reviewed By: rafauler, Amir
Differential Revision: https://reviews.llvm.org/D154056
|
|
writing resulting profile
The BOLT instrumentation runtime uses a bump allocator that has a fixed amount of maximum memory. In some cases, this memory limit is not large enough (https://github.com/llvm/llvm-project/issues/59174). We are hitting this limit when instrumenting the Rust compiler with BOLT.
This change increases the memory of the bump allocator used for writing the resulting BOLT profile.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D151891
|
|
`__bolt_instr_data_dump()` does not lock the hash tables when iterating
over them, so the iteration can happen concurrently with a modification
done in another thread, when the table is in an inconsistent state. This
also has been observed in practice, when it caused a segmentation fault.
We fix this by locking hash tables during iteration. This is done by taking
the lock in `forEachElement()`.
The only other site of iteration, `resetCounters()`, has been correctly
locking the table even before this patch. This patch removes its `Lock`
because the lock is now taken in the inner `forEachElement()`.
Reviewed By: maksfb, yota9
Differential Revision: https://reviews.llvm.org/D129089
|
|
Compiler can generate calls to some functions implicitly, even under
constraints of freestanding environment. Make sure these functions are
available in our runtime objects.
Fixes test failures on some systems after https://reviews.llvm.org/D128960.
Reviewed By: yota9
Differential Revision: https://reviews.llvm.org/D129168
|
|
Summary:
Refactor remaining bolt sources to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html).
(cherry picked from FBD33345885)
|
|
Summary: Fix comments at the start of source files.
(cherry picked from FBD33274597)
|
|
Summary: In some cases __open() is returning 0 for me. The open
syscall will return a negative number (-1) on error so 0 should be
valid.
This happens when one compiles a BOLT instrumented executable of
pyston (python implementation) and afterwards pip installs a package
which needs to be compiled and sets CFLAGS=-pipe. I could not reduce
it down to a small testcase but I guess it one can trigger when
manually closing std{in, out, err}. Everything seems to work
normally when disabling the assert for 0 in getBinaryPath() - I
decided to also modify the second case in readDescriptions() even
though I did not run into that one yet.
(cherry picked from FBD32409548)
|
|
Summary:
Increase the hard limit from 256 to 4096.
This fixes the 'Assertion failed: failed to open binary path' error I'm seeing.
(cherry picked from FBD31911946)
|
|
Summary:
This patch temporarily disables instrumentation and higufy build not for
x86 platforms to be able to build llvm-bolt tool on aarch64.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD31738306)
|
|
Summary:
Sync the file with storage device on data dump to stabilize
instrumentation testing
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD31738021)
|
|
Summary:
This commit introduces TryLock usage for SimpleHashTable getter to
avoid deadlock and relax syscalls usage which causes significant
overhead in runtime.
The old behavior left under -conservative-instrumentation option passed
to instrumentation library.
Also, this commit includes a corresponding test case: instrumentation of
executable which performs indirect calls from common code and signal
handler.
Note: in case if TryLock was failed to acquire the lock - this indirect
call will not be accounted in the resulting profile.
Vasily Leonenko,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30821949)
|
|
Summary:
To avoid RELATIVE relocations avoid using of GOT table
by using hidden visibility for all symbols in library.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30092712)
|
|
Summary:
The trampolines are no loger pointers to the functions. For
propper name resolving by bolt use extern "C" for all external symbols
in instr.cpp
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30092698)
|
|
Summary:
This commit introduces -instrumentation-binpath argument used
to point instuqmented binary in runtime in case if /proc/self/map_files
path is not accessible due to access restriction issues.
Vasily Leonenko
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30092681)
|
|
Summary:
This commit fixes runtime instrumentation handlers for PIE
binaries case.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30092522)
|
|
/proc/self/map_files
Summary:
This commit adds support for opening libs based on links
/proc/self/map_files. For this we're getting current virtual address
and searching the lib in the directory with such address range. After
that, we're getting full path to the binary by using readlink
function. Direct read from link in /proc/self/map_files entries is not
possible because of lack of permissions.
Elvina Yakubova,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30092422)
|
|
Summary:
This commit implements new method for _start & _fini functions hooking
which allows to use relative jumps for future PIE & .so library support.
Instead of using absolute address of _start & _fini functions known on
linking stage - we'll use dynamically created trampoline functions and
use corresponding symbols in instrumentation runtime library.
As we would like to use instrumentation for dynamically loaded binaries
(with PIE & .so), thus we need to compile instrumentation library with
"-fPIC" flag to support relative address resolution for functions and
data.
For shared libraries we need to handle initialization of instrumentation
library case by using DT_INIT section entry point.
Also this commit adds detection if the binary is executable or shared
library based on existence of PT_INTERP header. In case of shared
library we save information about real library init function address
for further usage for instrumentation library init trampoline function
creation and also update DT_INIT to point instrumentation library init
function.
Functions called from init/fini functions should be called with forced
stack alignment to avoid issues with instructions which relies on it.
E.g. optimized string operations.
Vasily Leonenko,
Advanced Software Technology Lab, Huawei
(cherry picked from FBD30092316)
|
|
Summary:
Expanded auto types across BOLT semi-automatically with the aid
of clangd LSP
(cherry picked from FBD33289309)
|
|
Summary:
This PR introduces 2 new instrumentation options:
1. instrumentation-no-counters-clear: Discussed at https://github.com/facebookincubator/BOLT/issues/121
2. instrumentation-wait-forks: Since the instrumentation counters are mapped as MAP_SHARED it will be nice to add ability to wait until all forks of the parent process will die using tracking of process group.
The last patch is just emitBinary code refactor.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
Pull Request resolved: https://github.com/facebookincubator/BOLT/pull/125
GitHub Author: Vladislav Khmelevskyi <Vladislav.Khmelevskyi@huawei.com>
(cherry picked from FBD26919011)
|
|
Summary: As titled.
(cherry picked from FBD28112137)
|
|
Summary: Add support for dumping profile on MacOS.
(cherry picked from FBD25751363)
|
|
Summary: Add support for dumping counters on MacOS
(cherry picked from FBD25750516)
|
|
Summary:
Use size_t for the first parameter of operator new.
https://en.cppreference.com/w/cpp/memory/new/operator_new
(cherry picked from FBD25750921)
|
|
Summary:
This diff is a preparation for dumping the profile generated by BOLT's instrumenation on MachO.
1/ Function "bolt_instr_fini" is placed into the predefined section "__fini"
2/ In the instrumentation pass we create a symbol "bolt_instr_fini" and
replace the last global destructor with it.
This is a temporary solution, in the future we need to register bolt_instr_fini in addition to the existing destructors without dropping the last one.
(cherry picked from FBD25071864)
|
|
Summary: Link the instrumentation runtime on OSX.
(cherry picked from FBD24390019)
|
|
Summary: Add first bits to cross-compile the runtime for OSX.
(cherry picked from FBD24330977)
|
|
Summary:
Right now, the SAVE_ALL sequence executed upon entry of both
of our runtime libs (hugify and instrumentation) will cause the stack to
not be aligned at a 16B boundary because it saves 15 8-byte regs. Change
the code sequence to adjust for that. The compiler may generate code
that assumes the stack is aligned by using movaps instructions, which
will crash.
(cherry picked from FBD22744307)
|
