<feed xmlns='http://www.w3.org/2005/Atom'>
<title>llvm-project.git/llvm/docs/CommandGuide/lit.rst, branch users/nico/python-2</title>
<subtitle>Unnamed repository; edit this file 'description' to name the repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/'/>
<entry>
<title>[lit] show retry attempts (#142413)</title>
<updated>2025-06-06T08:24:26+00:00</updated>
<author>
<name>Konrad Kleine</name>
<email>kkleine@redhat.com</email>
</author>
<published>2025-06-06T08:24:26+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=69183149189095966b9e7740d3b66126b754fbd1'/>
<id>69183149189095966b9e7740d3b66126b754fbd1</id>
<content type='text'>
If a test took more than one attempt to complete, show the number of attempts and the maximum allowed attempts as `2 of 4 attempts` inside the `&lt;progress info&gt;` (see [TEST RUN OUTPUT FORMAT](https://llvm.org/docs/CommandGuide/lit.html#test-run-output-format)).

NOTE: Additionally this is a fixup for #141851 where the tests were not quite right. `max-retries-per-test/allow-retries-test_retry_attempts/test.py` was added but never used there. Now we're calling it. To correlate better between the test output and the test script I've used higher numbers of max allowed retries.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If a test took more than one attempt to complete, show the number of attempts and the maximum allowed attempts as `2 of 4 attempts` inside the `&lt;progress info&gt;` (see [TEST RUN OUTPUT FORMAT](https://llvm.org/docs/CommandGuide/lit.html#test-run-output-format)).

NOTE: Additionally this is a fixup for #141851 where the tests were not quite right. `max-retries-per-test/allow-retries-test_retry_attempts/test.py` was added but never used there. Now we're calling it. To correlate better between the test output and the test script I've used higher numbers of max allowed retries.</pre>
</div>
</content>
</entry>
<entry>
<title>[docs] don't use "=" in lit options with arguments (#142340)</title>
<updated>2025-06-04T10:36:12+00:00</updated>
<author>
<name>Konrad Kleine</name>
<email>kkleine@redhat.com</email>
</author>
<published>2025-06-04T10:36:12+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=107d8c792f5df7166615629a1c7bb84425def143'/>
<id>107d8c792f5df7166615629a1c7bb84425def143</id>
<content type='text'>
This is a fixup for #141851 and removes `=` from all
options with additional arguments.

Before 14 out of 22 options with arguments used "=" and 7 didn't.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is a fixup for #141851 and removes `=` from all
options with additional arguments.

Before 14 out of 22 options with arguments used "=" and 7 didn't.</pre>
</div>
</content>
</entry>
<entry>
<title>[lit] add --max-retries-per-test execution option (#141851)</title>
<updated>2025-05-31T01:46:12+00:00</updated>
<author>
<name>Konrad Kleine</name>
<email>kkleine@redhat.com</email>
</author>
<published>2025-05-31T01:46:12+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=84fd907aa7de2c35721ab6d1fd0d64d7c9345747'/>
<id>84fd907aa7de2c35721ab6d1fd0d64d7c9345747</id>
<content type='text'>
When packaging LLVM we've seen arbitrary tests fail.
It happened sporadically and most of the times the test
do work if they are run a second time on the next day.

The tests themselves were always different and we didn't
know ahead of time which ones we wanted to re-run.
That's we filter-out a lot of `libomp` and `libarcher` tests [1].

This change allows us to set
`LIT_OPTS="--max-retries-per-test=12"`
when running any "check-XXX" build target. Then any lit test
will at most be re-run 12 times, unless there's an `ALLOW_RETRIES:`
in one of the test scripts that's specifying a different value
than `12`. `12` is just an example here, any positive integer
will work.

Please note, that this only adds the possibility to re-run
lit tests. It does not actually do it until the caller specifies
`--max-retries-per-test=&lt;POSITIVE_INT&gt;` either on a call to `lit` or
in `LIT_OPTS`.

Also note, that one can still use `ALLOW_RETRIES:` in test scripts
and it will always rule over `--max-retries-per-test`. When
`--max-retries-per-test` is set too low, but the
`config.test_retry_attempts`
is high enough, it works as well.

Any option in the list below overrules its predecessor:

* `--max-retries-per-test`
* `config.test_retry_attempts`
* `ALLOW_RETRIES` keyword

From the above options to re-run tests, `--max-retries-per-test` is the
only one that doesn't require a change in the test scripts or the test
config.

[1]:
https://src.fedoraproject.org/rpms/llvm/blob/rawhide/f/llvm.spec#_2326

Downstream PR to make use of the `--max-retries-per-test` option:
https://src.fedoraproject.org/rpms/llvm/pull-request/434
Downstream ticket: https://issues.redhat.com/browse/LLVM-145</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When packaging LLVM we've seen arbitrary tests fail.
It happened sporadically and most of the times the test
do work if they are run a second time on the next day.

The tests themselves were always different and we didn't
know ahead of time which ones we wanted to re-run.
That's we filter-out a lot of `libomp` and `libarcher` tests [1].

This change allows us to set
`LIT_OPTS="--max-retries-per-test=12"`
when running any "check-XXX" build target. Then any lit test
will at most be re-run 12 times, unless there's an `ALLOW_RETRIES:`
in one of the test scripts that's specifying a different value
than `12`. `12` is just an example here, any positive integer
will work.

Please note, that this only adds the possibility to re-run
lit tests. It does not actually do it until the caller specifies
`--max-retries-per-test=&lt;POSITIVE_INT&gt;` either on a call to `lit` or
in `LIT_OPTS`.

Also note, that one can still use `ALLOW_RETRIES:` in test scripts
and it will always rule over `--max-retries-per-test`. When
`--max-retries-per-test` is set too low, but the
`config.test_retry_attempts`
is high enough, it works as well.

Any option in the list below overrules its predecessor:

* `--max-retries-per-test`
* `config.test_retry_attempts`
* `ALLOW_RETRIES` keyword

From the above options to re-run tests, `--max-retries-per-test` is the
only one that doesn't require a change in the test scripts or the test
config.

[1]:
https://src.fedoraproject.org/rpms/llvm/blob/rawhide/f/llvm.spec#_2326

Downstream PR to make use of the `--max-retries-per-test` option:
https://src.fedoraproject.org/rpms/llvm/pull-request/434
Downstream ticket: https://issues.redhat.com/browse/LLVM-145</pre>
</div>
</content>
</entry>
<entry>
<title>Clarify `lit`'s definition of failure and conditions when it exits with exit code 1 (#136190)</title>
<updated>2025-04-28T06:06:42+00:00</updated>
<author>
<name>i-ky</name>
<email>gl.ivanovsky@gmail.com</email>
</author>
<published>2025-04-28T06:06:42+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=6f6af49938bfd0b1649f25064fd9619552f82b23'/>
<id>6f6af49938bfd0b1649f25064fd9619552f82b23</id>
<content type='text'>
Documentation of when `lit` exits with code 1 is out of date.

It is no longer just "FAIL or XPASS", there are more failure types:

https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/utils/lit/lit/Test.py#L51-L55

Exit code can also be affected by `--ignore-fail` option:

https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/utils/lit/lit/main.py#L154-L162

This PR extracts a clear definition of "failure" from the description of
`--report-failures-only` option:

https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/docs/CommandGuide/lit.rst?plain=1#L194-L196
...puts it into "Test Status Results" section and references it when
describing exit codes and `--ignore-fail` option.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Documentation of when `lit` exits with code 1 is out of date.

It is no longer just "FAIL or XPASS", there are more failure types:

https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/utils/lit/lit/Test.py#L51-L55

Exit code can also be affected by `--ignore-fail` option:

https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/utils/lit/lit/main.py#L154-L162

This PR extracts a clear definition of "failure" from the description of
`--report-failures-only` option:

https://github.com/llvm/llvm-project/blob/b30100b87f24847afd6407b4939a184ebcf16ef9/llvm/docs/CommandGuide/lit.rst?plain=1#L194-L196
...puts it into "Test Status Results" section and references it when
describing exit codes and `--ignore-fail` option.</pre>
</div>
</content>
</entry>
<entry>
<title>Reapply "lit: Move RUN at line comment after the command."</title>
<updated>2025-03-24T19:01:49+00:00</updated>
<author>
<name>Peter Collingbourne</name>
<email>peter@pcc.me.uk</email>
</author>
<published>2025-03-24T18:54:29+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=11044de3dcc1dd7a47c8c3f98ecf77c00978b570'/>
<id>11044de3dcc1dd7a47c8c3f98ecf77c00978b570</id>
<content type='text'>
This reverts commit 57e89c97c2c1b4e41f07a90c2f4d36649696e619.

Updated lit tests.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit 57e89c97c2c1b4e41f07a90c2f4d36649696e619.

Updated lit tests.
</pre>
</div>
</content>
</entry>
<entry>
<title>Revert "lit: Move RUN at line comment after the command."</title>
<updated>2025-03-24T18:47:56+00:00</updated>
<author>
<name>Peter Collingbourne</name>
<email>peter@pcc.me.uk</email>
</author>
<published>2025-03-24T18:47:31+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=57e89c97c2c1b4e41f07a90c2f4d36649696e619'/>
<id>57e89c97c2c1b4e41f07a90c2f4d36649696e619</id>
<content type='text'>
This reverts commit 8d3dc1ed5656a3e69e4195d58684a7f4bf0ff5cc.

Test needs to be updated.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit 8d3dc1ed5656a3e69e4195d58684a7f4bf0ff5cc.

Test needs to be updated.
</pre>
</div>
</content>
</entry>
<entry>
<title>lit: Move RUN at line comment after the command.</title>
<updated>2025-03-24T18:29:57+00:00</updated>
<author>
<name>Peter Collingbourne</name>
<email>pcc@google.com</email>
</author>
<published>2025-03-24T18:29:57+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=8d3dc1ed5656a3e69e4195d58684a7f4bf0ff5cc'/>
<id>8d3dc1ed5656a3e69e4195d58684a7f4bf0ff5cc</id>
<content type='text'>
When a developer copy/pastes a failing command line into their
shell to rerun it, they have to manually delete the "RUN: at line
N:" prefix. To make life easier for such developers, let's make it
possible to copy/paste a command without needing to modify it while
still showing the line number in the output by moving the line number
to a comment at the end of the command line.

Reviewers: jroelofs, MaskRay

Reviewed By: jroelofs, MaskRay

Pull Request: https://github.com/llvm/llvm-project/pull/132485
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When a developer copy/pastes a failing command line into their
shell to rerun it, they have to manually delete the "RUN: at line
N:" prefix. To make life easier for such developers, let's make it
possible to copy/paste a command without needing to modify it while
still showing the line number in the output by moving the line number
to a comment at the end of the command line.

Reviewers: jroelofs, MaskRay

Reviewed By: jroelofs, MaskRay

Pull Request: https://github.com/llvm/llvm-project/pull/132485
</pre>
</div>
</content>
</entry>
<entry>
<title>[lit] Add --report-failures-only option for lit test reports (#115439)</title>
<updated>2024-11-13T08:30:33+00:00</updated>
<author>
<name>Rakshit Patel</name>
<email>rakshit.patel@sony.com</email>
</author>
<published>2024-11-13T08:30:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=c63e83f49575c024cf89fce9bc95d64988f3177b'/>
<id>c63e83f49575c024cf89fce9bc95d64988f3177b</id>
<content type='text'>
- Add option (--report-failures-only) to generate a reduced report for
lit tests that only includes failing tests
- This is a continuation of proposed patches by @gregbedwell here:
    - https://reviews.llvm.org/D143516
    - https://reviews.llvm.org/D143519

---------

Co-authored-by: Greg Bedwell &lt;greg.bedwell@sony.com&gt;
Co-authored-by: James Henderson &lt;James.Henderson@sony.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Add option (--report-failures-only) to generate a reduced report for
lit tests that only includes failing tests
- This is a continuation of proposed patches by @gregbedwell here:
    - https://reviews.llvm.org/D143516
    - https://reviews.llvm.org/D143519

---------

Co-authored-by: Greg Bedwell &lt;greg.bedwell@sony.com&gt;
Co-authored-by: James Henderson &lt;James.Henderson@sony.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>[llvm][docs] Update list of llvm-lit options</title>
<updated>2024-10-29T10:35:27+00:00</updated>
<author>
<name>David Spickett</name>
<email>david.spickett@linaro.org</email>
</author>
<published>2024-10-29T10:33:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=a8398bd81770a2801ec083fd2cd8a19140fe92a9'/>
<id>a8398bd81770a2801ec083fd2cd8a19140fe92a9</id>
<content type='text'>
Fixes #62899

In this commit I have updated the list of options
to include any missing options and re-rordered
some of them to match the order in lit's --help.

Where there was a larger description in this document
I've used that instead of the --help description.

This *does not* include --use-unique-output-file-name
as this was only added recently and we are still
debating whether it will be kept.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fixes #62899

In this commit I have updated the list of options
to include any missing options and re-rordered
some of them to match the order in lit's --help.

Where there was a larger description in this document
I've used that instead of the --help description.

This *does not* include --use-unique-output-file-name
as this was only added recently and we are still
debating whether it will be kept.
</pre>
</div>
</content>
</entry>
<entry>
<title>[LIT] Rename substitution `%basename_s` to `%{s:basename}` (#111062)</title>
<updated>2024-10-04T01:18:10+00:00</updated>
<author>
<name>Rahul Joshi</name>
<email>rjoshi@nvidia.com</email>
</author>
<published>2024-10-04T01:18:10+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=876f661dbeb42a76767edfb1f36214baacb27fd4'/>
<id>876f661dbeb42a76767edfb1f36214baacb27fd4</id>
<content type='text'>
Also added `%{t:stem}` as an alias for `%basename_t` and modified unit
test to test these new substitutions.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Also added `%{t:stem}` as an alias for `%basename_t` and modified unit
test to test these new substitutions.</pre>
</div>
</content>
</entry>
</feed>
