<feed xmlns='http://www.w3.org/2005/Atom'>
<title>llvm-project.git/lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp, branch users/nico/python-2</title>
<subtitle>Unnamed repository; edit this file 'description' to name the repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/'/>
<entry>
<title>JSON: llvm::Optional =&gt; std::optional</title>
<updated>2022-12-16T07:56:52+00:00</updated>
<author>
<name>Fangrui Song</name>
<email>i@maskray.me</email>
</author>
<published>2022-12-16T07:56:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=1da3a795fcf61a2c931d9320738db7d5c0444ce2'/>
<id>1da3a795fcf61a2c931d9320738db7d5c0444ce2</id>
<content type='text'>
Many files are from language servers.

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Many files are from language servers.

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intel pt] Add a cgroup filter</title>
<updated>2022-07-13T19:26:11+00:00</updated>
<author>
<name>Gaurav Gaur</name>
<email>gaur@fb.com</email>
</author>
<published>2022-06-29T20:09:12+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=d30fd5c3a17b5b8301e3413ae9b398ca7ee1f69c'/>
<id>d30fd5c3a17b5b8301e3413ae9b398ca7ee1f69c</id>
<content type='text'>
It turns out that cgroup filtering is relatively trivial and works
really nicely. Thid diffs adds automatic cgroup filtering when in
per-cpu mode, unless a new --disable-cgroup-filtering flag is passed in
the start command. At least on Meta machines, all processes are spawned
inside a cgroup by default, which comes super handy, because per cpu
tracing is now much more precise.

A manual test gave me this result

- Without filtering:
    Total number of trace items: 36083
    Total number of continuous executions found: 229
    Number of continuous executions for this thread: 2
    Total number of PSB blocks found: 98
    Number of PSB blocks for this thread 2
    Total number of unattributed PSB blocks found: 38

- With filtering:
    Total number of trace items: 87756
    Total number of continuous executions found: 123
    Number of continuous executions for this thread: 2
    Total number of PSB blocks found: 10
    Number of PSB blocks for this thread 3
    Total number of unattributed PSB blocks found: 2

Filtering gives us great results. The number of instructions collected
more than double (probalby because we have less noise in the trace), and
we have much less unattributed PSBs blocks and unrelated PSBs in
general. The ones that are unrelated probably belong to other processes
in the same cgroup.

Differential Revision: https://reviews.llvm.org/D129257
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It turns out that cgroup filtering is relatively trivial and works
really nicely. Thid diffs adds automatic cgroup filtering when in
per-cpu mode, unless a new --disable-cgroup-filtering flag is passed in
the start command. At least on Meta machines, all processes are spawned
inside a cgroup by default, which comes super handy, because per cpu
tracing is now much more precise.

A manual test gave me this result

- Without filtering:
    Total number of trace items: 36083
    Total number of continuous executions found: 229
    Number of continuous executions for this thread: 2
    Total number of PSB blocks found: 98
    Number of PSB blocks for this thread 2
    Total number of unattributed PSB blocks found: 38

- With filtering:
    Total number of trace items: 87756
    Total number of continuous executions found: 123
    Number of continuous executions for this thread: 2
    Total number of PSB blocks found: 10
    Number of PSB blocks for this thread 3
    Total number of unattributed PSB blocks found: 2

Filtering gives us great results. The number of instructions collected
more than double (probalby because we have less noise in the trace), and
we have much less unattributed PSBs blocks and unrelated PSBs in
general. The ones that are unrelated probably belong to other processes
in the same cgroup.

Differential Revision: https://reviews.llvm.org/D129257
</pre>
</div>
</content>
</entry>
<entry>
<title>[lldb] Use value_or instead of getValueOr (NFC)</title>
<updated>2022-06-19T16:12:01+00:00</updated>
<author>
<name>Kazu Hirata</name>
<email>kazu@google.com</email>
</author>
<published>2022-06-19T16:12:01+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=aa88161b378ecb49388eefc28abe2926a229bcfc'/>
<id>aa88161b378ecb49388eefc28abe2926a229bcfc</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [21] - Support long numbers in JSON</title>
<updated>2022-06-16T18:42:22+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-06-15T02:05:30+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=9f45f23d860251edceed6128110855c51af4f39f'/>
<id>9f45f23d860251edceed6128110855c51af4f39f</id>
<content type='text'>
llvm's JSON parser supports 64 bit integers, but other tools like the
ones written in JS don't support numbers that big, so we need to
represent these possibly big numbers as a string. This diff uses that to
represent addresses and tsc zero. The former is printed in hex for and
the latter in decimal string form. The schema was updated mentioning
that.

Besides that, I fixed some remaining issues and now all test pass. Before I wasn't running all tests because for some reason my computer reverted perf_paranoid to 1.

Differential Revision: https://reviews.llvm.org/D127819
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
llvm's JSON parser supports 64 bit integers, but other tools like the
ones written in JS don't support numbers that big, so we need to
represent these possibly big numbers as a string. This diff uses that to
represent addresses and tsc zero. The former is printed in hex for and
the latter in decimal string form. The schema was updated mentioning
that.

Besides that, I fixed some remaining issues and now all test pass. Before I wasn't running all tests because for some reason my computer reverted perf_paranoid to 1.

Differential Revision: https://reviews.llvm.org/D127819
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [20] - Rename some fields in the schema</title>
<updated>2022-06-16T18:42:22+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-06-14T22:53:59+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=6a5355e8a159bd9c058d4fbba2a87e0465fe0dc7'/>
<id>6a5355e8a159bd9c058d4fbba2a87e0465fe0dc7</id>
<content type='text'>
As discusses offline with @jj10305, we are updating some naming used throughout the code, specially in the json schema

- traceBuffer -&gt; iptTrace
- core -&gt; cpu

Differential Revision: https://reviews.llvm.org/D127817
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
As discusses offline with @jj10305, we are updating some naming used throughout the code, specially in the json schema

- traceBuffer -&gt; iptTrace
- core -&gt; cpu

Differential Revision: https://reviews.llvm.org/D127817
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [14] - Decode per cpu</title>
<updated>2022-06-16T18:23:01+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-05-24T19:16:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=a19fcc2bec81989f90700cfc63b89a0dfd330197'/>
<id>a19fcc2bec81989f90700cfc63b89a0dfd330197</id>
<content type='text'>
This is the final functional patch to support intel pt decoding per cpu.
It works by doing the following:

- First, all context switches are split by tid and sorted in order. This produces a list of continuous executes per thread per core.
- Then, all intel pt subtraces are split by PSB boundaries and assigned to individual thread continuous executions on the same core by doing simple TSC-based comparisons.
- With this, we have, per thread, a sorted list of continuous executions each one with a list of intel pt subtraces. Up to this point, this is really fast because no instructions were actually decoded.
- Then, each thread can be decoded by traversing their continuous executions and intel pt subtraces. An advantage of having these continuous executions is that we can identify if a continuous exexecution doesn't have intel pt data, and thus has a gap in it. We can later to more sofisticated comparisons to identify if within a continuous execution there are gaps.

I'm adding a test as well.

Differential Revision: https://reviews.llvm.org/D126394
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is the final functional patch to support intel pt decoding per cpu.
It works by doing the following:

- First, all context switches are split by tid and sorted in order. This produces a list of continuous executes per thread per core.
- Then, all intel pt subtraces are split by PSB boundaries and assigned to individual thread continuous executions on the same core by doing simple TSC-based comparisons.
- With this, we have, per thread, a sorted list of continuous executions each one with a list of intel pt subtraces. Up to this point, this is really fast because no instructions were actually decoded.
- Then, each thread can be decoded by traversing their continuous executions and intel pt subtraces. An advantage of having these continuous executions is that we can identify if a continuous exexecution doesn't have intel pt data, and thus has a gap in it. We can later to more sofisticated comparisons to identify if within a continuous execution there are gaps.

I'm adding a test as well.

Differential Revision: https://reviews.llvm.org/D126394
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [13] - Add context switch decoding</title>
<updated>2022-06-16T18:23:01+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-05-19T23:39:20+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=1a3f996972b175a6f7e4db1eabb8310e2e93f634'/>
<id>1a3f996972b175a6f7e4db1eabb8310e2e93f634</id>
<content type='text'>
- Add the logic that parses all cpu context switch traces and produces blocks of continuous executions, which will be later used to assign intel pt subtraces to threads and to identify gaps. This logic can also identify if the context switch trace is malformed.
- The continuous executions blocks are able to indicate when there were some contention issues when producing the context switch trace. See the inline comments for more information.
- Update the 'dump info' command to show information and stats related to the multicore decoding flow, including timing about context switch decoding.
- Add the logic to conver nanoseconds to TSCs.
- Fix a bug when returning the context switches. Now they data returned makes sense and even empty traces can be returned from lldb-server.
- Finish the necessary bits for loading and saving a multi-core trace bundle from disk.
- Change some size_t to uint64_t for compatibility with 32 bit systems.

Tested by saving a trace session of a program that sleeps 100 times, it was able to produce the following 'dump info' text:

```
(lldb) trace load /tmp/trace3/trace.json                                                                   (lldb) thread trace dump info                                                                              Trace technology: intel-pt

thread #1: tid = 4192415
  Total number of instructions: 1

  Memory usage:
    Total approximate memory usage (excluding raw trace): 2.51 KiB
    Average memory usage per instruction (excluding raw trace): 2573.00 bytes

  Timing for this thread:

  Timing for global tasks:
    Context switch trace decoding: 0.00s

  Events:
    Number of instructions with events: 0
    Number of individual events: 0

  Multi-core decoding:
    Total number of continuous executions found: 2499
    Number of continuous executions for this thread: 102

  Errors:
    Number of TSC decoding errors: 0
```

Differential Revision: https://reviews.llvm.org/D126267
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Add the logic that parses all cpu context switch traces and produces blocks of continuous executions, which will be later used to assign intel pt subtraces to threads and to identify gaps. This logic can also identify if the context switch trace is malformed.
- The continuous executions blocks are able to indicate when there were some contention issues when producing the context switch trace. See the inline comments for more information.
- Update the 'dump info' command to show information and stats related to the multicore decoding flow, including timing about context switch decoding.
- Add the logic to conver nanoseconds to TSCs.
- Fix a bug when returning the context switches. Now they data returned makes sense and even empty traces can be returned from lldb-server.
- Finish the necessary bits for loading and saving a multi-core trace bundle from disk.
- Change some size_t to uint64_t for compatibility with 32 bit systems.

Tested by saving a trace session of a program that sleeps 100 times, it was able to produce the following 'dump info' text:

```
(lldb) trace load /tmp/trace3/trace.json                                                                   (lldb) thread trace dump info                                                                              Trace technology: intel-pt

thread #1: tid = 4192415
  Total number of instructions: 1

  Memory usage:
    Total approximate memory usage (excluding raw trace): 2.51 KiB
    Average memory usage per instruction (excluding raw trace): 2573.00 bytes

  Timing for this thread:

  Timing for global tasks:
    Context switch trace decoding: 0.00s

  Events:
    Number of instructions with events: 0
    Number of individual events: 0

  Multi-core decoding:
    Total number of continuous executions found: 2499
    Number of continuous executions for this thread: 102

  Errors:
    Number of TSC decoding errors: 0
```

Differential Revision: https://reviews.llvm.org/D126267
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [12] - Support multi-core trace load and save</title>
<updated>2022-06-15T20:28:36+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-05-19T04:36:34+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=fc5ef57c7df5805c828de2e3e334c2cf74648e58'/>
<id>fc5ef57c7df5805c828de2e3e334c2cf74648e58</id>
<content type='text'>
:q!
This diff is massive, but it's because it connects the client with lldb-server
and also ensures that the postmortem case works.

- Flatten the postmortem trace schema. The reason is that the schema has become quite complex due to the new multicore case, which defeats the original purpose of having a schema that could work for every trace plug-in. At this point, it's better that each trace plug-in defines it's own full schema. This means that the only common field is "type".
-- Because of this new approach, I merged the "common" trace load and saving functionalities into the IntelPT one. This simplified the code quite a bit. If we eventually implement another trace plug-in, we can see then what we could reuse.
-- The new schema, which is flattened, has now better comments and is parsed better. A change I did was to disallow hex addresses, because they are a bit error prone. I'm asking now to print the address in decimal.
-- Renamed "intel" to "GenuineIntel" in the schema because that's what you see in /proc/cpuinfo.
- Implemented reading the context switch trace data buffer. I had to do
some refactors to do that cleanly.
-- A major change that I did here was to simplify the perf_event circular buffer reading logic. It was too complex. Maybe the original Intel author had something different in mind.
- Implemented all the necessary bits to read trace.json files with per-core data.
- Implemented all the necessary bits to save to disk per-core trace session.
- Added a test that ensures that parsing and saving to disk works.

Differential Revision: https://reviews.llvm.org/D126015
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
:q!
This diff is massive, but it's because it connects the client with lldb-server
and also ensures that the postmortem case works.

- Flatten the postmortem trace schema. The reason is that the schema has become quite complex due to the new multicore case, which defeats the original purpose of having a schema that could work for every trace plug-in. At this point, it's better that each trace plug-in defines it's own full schema. This means that the only common field is "type".
-- Because of this new approach, I merged the "common" trace load and saving functionalities into the IntelPT one. This simplified the code quite a bit. If we eventually implement another trace plug-in, we can see then what we could reuse.
-- The new schema, which is flattened, has now better comments and is parsed better. A change I did was to disallow hex addresses, because they are a bit error prone. I'm asking now to print the address in decimal.
-- Renamed "intel" to "GenuineIntel" in the schema because that's what you see in /proc/cpuinfo.
- Implemented reading the context switch trace data buffer. I had to do
some refactors to do that cleanly.
-- A major change that I did here was to simplify the perf_event circular buffer reading logic. It was too complex. Maybe the original Intel author had something different in mind.
- Implemented all the necessary bits to read trace.json files with per-core data.
- Implemented all the necessary bits to save to disk per-core trace session.
- Added a test that ensures that parsing and saving to disk works.

Differential Revision: https://reviews.llvm.org/D126015
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [10] - Return warnings and tsc information from lldb-server.</title>
<updated>2022-06-15T19:08:00+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-05-18T21:32:35+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=1f2d49a8e7636a11cc4660a779dce81770943bf5'/>
<id>1f2d49a8e7636a11cc4660a779dce81770943bf5</id>
<content type='text'>
- Add a warnings field in the jLLDBGetState response, for warnings to be delivered to the client for troubleshooting. This removes the need to silently log lldb-server's llvm::Errors and not expose them easily to the user
- Simplify the tscPerfZeroConversion struct and schema. It used to extend a base abstract class, but I'm doubting that we'll ever add other conversion mechanisms because all modern kernels support perf zero. It is also the one who is supposed to work with the timestamps produced by the context switch trace, so expecting it is imperative.
- Force tsc collection for cpu tracing.
- Add a test checking that tscPerfZeroConversion is returned by the GetState request
- Add a pre-check for cpu tracing that makes sure that perf zero values are available.

Differential Revision: https://reviews.llvm.org/D125932
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Add a warnings field in the jLLDBGetState response, for warnings to be delivered to the client for troubleshooting. This removes the need to silently log lldb-server's llvm::Errors and not expose them easily to the user
- Simplify the tscPerfZeroConversion struct and schema. It used to extend a base abstract class, but I'm doubting that we'll ever add other conversion mechanisms because all modern kernels support perf zero. It is also the one who is supposed to work with the timestamps produced by the context switch trace, so expecting it is imperative.
- Force tsc collection for cpu tracing.
- Add a test checking that tscPerfZeroConversion is returned by the GetState request
- Add a pre-check for cpu tracing that makes sure that perf zero values are available.

Differential Revision: https://reviews.llvm.org/D125932
</pre>
</div>
</content>
</entry>
<entry>
<title>[trace][intelpt] Support system-wide tracing [9] - Collect and return context switch traces</title>
<updated>2022-06-15T19:07:59+00:00</updated>
<author>
<name>Walter Erquinigo</name>
<email>wallace@fb.com</email>
</author>
<published>2022-05-18T15:01:01+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=a758205951d3a1eb60bdb65146ab23e8567f30a7'/>
<id>a758205951d3a1eb60bdb65146ab23e8567f30a7</id>
<content type='text'>
- Add collection of context switches per cpu grouped with the per-cpu intel pt traces.
- Move the state handling from the interl pt trace class to the PerfEvent one.
- Add support for stopping and enabling perf event groups.
- Return context switch entries as part of the jLLDBTraceGetState response.
- Move the triggers of whenever the process stopped or resumed. Now the will-resume notification is in a better location, which will ensure that we'll capture the instructions that will be executed.
- Remove IntelPTSingleBufferTraceUP. The unique pointer was useless.
- Add unit tests

Differential Revision: https://reviews.llvm.org/D125897
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Add collection of context switches per cpu grouped with the per-cpu intel pt traces.
- Move the state handling from the interl pt trace class to the PerfEvent one.
- Add support for stopping and enabling perf event groups.
- Return context switch entries as part of the jLLDBTraceGetState response.
- Move the triggers of whenever the process stopped or resumed. Now the will-resume notification is in a better location, which will ensure that we'll capture the instructions that will be executed.
- Remove IntelPTSingleBufferTraceUP. The unique pointer was useless.
- Add unit tests

Differential Revision: https://reviews.llvm.org/D125897
</pre>
</div>
</content>
</entry>
</feed>
