summaryrefslogtreecommitdiff
path: root/llvm/lib/Support/Compression.cpp
AgeCommit message (Collapse)Author
2024-11-25[Support] Check zstd decompress result before msan unpoison (#117276)Wu Yingcong
We should check the zstd decompress result before doing the msan unpoison. If the res is abnormal, then it would be a huge number, which will cause undesired msan unpoison behavior and will run for a long time.
2024-03-09[HIP] add --offload-compression-level= option (#83605)Yaxun (Sam) Liu
Added --offload-compression-level= option to clang and -compression-level= option to clang-offload-bundler for controlling compression level. Added support of long distance matching (LDM) for llvm::zstd which is off by default. Enable it for clang-offload-bundler by default since it improves compression rate in general. Change default compression level to 3 for zstd for clang-offload-bundler since it works well for bundle entry size from 1KB to 32MB, which should cover most of the clang-offload-bundler usage. Users can still specify compression level by -compression-level= option if necessary.
2022-09-19[Object] Add zstd decompression support to DecompressorFangrui Song
llvm::object::Decompressor is used by many DWARF consumers like llvm-dwarfdump, llvm-dwp, llvm-symbolizer. Add tests to them. The lldb test can be left to D133530. Reviewed By: dblaikie Differential Revision: https://reviews.llvm.org/D134116
2022-09-17[Support] Rename llvm::compression::{zlib,zstd}::uncompress to more ↵Fangrui Song
appropriate decompress This improves consistency with other places (e.g. llvm::compression::decompress, llvm::object::Decompressor::decompress, llvm-objcopy). Note: when zstd::uncompress was added, we noticed that the API `ZSTD_decompress` is fine while the zlib API `uncompress` is a misnomer.
2022-09-08[Support] Add llvm::compression::{getReasonIfUnsupported,compress,decompress}Fangrui Song
as high-level API on top of `llvm::compression::{zlib,zstd}::*`: * getReasonIfUnsupported: return nullptr if the specified format is supported, or (if unsupported) a string like `LLVM was not built with LLVM_ENABLE_ZLIB ...` * compress: dispatch to zlib::uncompress or zstd::uncompress * decompress: dispatch to zlib::uncompress or zstd::uncompress Move `llvm::DebugCompressionType` from MC to Support to avoid Support->MC cyclic dependency. There are 40+ uses in llvm-project. Add another enum class `llvm::compression::Format` to represent supported compression formats, which may be a superset of ELF compression formats. See D130458 (llvm-objcopy --{,de}compress-debug-sections for zstd) for a use case. Link: https://discourse.llvm.org/t/rfc-zstandard-as-a-second-compression-method-to-llvm/63399 ("[RFC] Zstandard as a second compression method to LLVM") --- Note: this patch alone will cause -Wswitch to llvm/lib/ObjCopy/ELF/ELFObject.cpp Reviewed By: ckissane, dblaikie Differential Revision: https://reviews.llvm.org/D130506
2022-09-08Revert "[Support] Add ↵Nikita Popov
llvm::compression::{getReasonIfUnsupported,compress,decompress}" This reverts commit 19dc3cff0f771bb8933136ef68e782553e920d04. This reverts commit 5b19a1f8e88da9ec92b995bfee90043795c2c252. This reverts commit 9397648ac8ad192f7e6e6a8e6894c27bf7e024e9. This reverts commit 10842b44759f987777b08e7714ef77da2526473a. Breaks the GCC build, as reported here: https://reviews.llvm.org/D130506#3776415
2022-09-07[Support] Add llvm::compression::{getReasonIfUnsupported,compress,decompress}Fangrui Song
as high-level API on top of `llvm::compression::{zlib,zstd}::*`: * getReasonIfUnsupported: return nullptr if the specified format is supported, or (if unsupported) a string like `LLVM was not built with LLVM_ENABLE_ZLIB ...` * compress: dispatch to zlib::uncompress or zstd::uncompress * decompress: dispatch to zlib::uncompress or zstd::uncompress Move `llvm::DebugCompressionType` from MC to Support to avoid Support->MC cyclic dependency. There are 40+ uses in llvm-project. Add another enum class `llvm::compression::Format` to represent supported compression formats, which may be a superset of ELF compression formats. See D130458 (llvm-objcopy --{,de}compress-debug-sections for zstd) for a use case. Link: https://discourse.llvm.org/t/rfc-zstandard-as-a-second-compression-method-to-llvm/63399 ("[RFC] Zstandard as a second compression method to LLVM") Differential Revision: https://reviews.llvm.org/D130506
2022-07-19[llvm] add zstd to `llvm::compression` namespaceCole Kissane
- add zstd to `llvm::compression` namespace - add a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB` - add tests for zstd to `llvm/unittests/Support/CompressionTest.cpp` - debian users should install libzstd when using `LLVM_ENABLE_ZSTD=FORCE_ON` from source due to this bug https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1941956 Reviewed By: leonardchan, MaskRay Differential Revision: https://reviews.llvm.org/D128465
2022-07-14Revert "[llvm] add zstd to llvm::compression namespace"Amara Emerson
This reverts commit d449c600767284486615f3b79601ced15a00af61. Breaks macOS builds with this: llvm/lib/Support/Compression.cpp:24:10: fatal error: 'zstd.h' file not found
2022-07-13[llvm] add zstd to llvm::compression namespaceCole Kissane
- add `FindZSTD.cmake` - add zstd to `llvm::compression` namespace - add a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB` - add tests for zstd to `llvm/unittests/Support/CompressionTest.cpp` Reviewed By: leonardchan, MaskRay Differential Revision: https://reviews.llvm.org/D128465
2022-07-13Revert "[llvm] add zstd to `llvm::compression` namespace"Cole Kissane
This reverts commit cef07169ec9f46fd25291a3218cf12bef324ea0c.
2022-07-13[llvm] add zstd to `llvm::compression` namespaceCole Kissane
- add `FindZSTD.cmake` - add zstd to `llvm::compression` namespace - add a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB` - add tests for zstd to `llvm/unittests/Support/CompressionTest.cpp` Reviewed By: leonardchan, MaskRay Differential Revision: https://reviews.llvm.org/D128465
2022-07-13[Support] Fix LLVM_ENABLE_ZLIB==0 buildsFangrui Song
2022-07-13[Support] Change compression::zlib::{compress,uncompress} to use uint8_t *Fangrui Song
It's more natural to use uint8_t * (std::byte needs C++17 and llvm has too much uint8_t *) and most callers use uint8_t * instead of char *. The functions are recently moved into `llvm::compression::zlib::`, so downstream projects need to make adaption anyway.
2022-07-13[llvm] fix zlib buffer truncate edge cases and fix nits in testsCole Kissane
- add check before truncating (un)compressed data buffer if the buffer is already a perfect length, to avoid triggering truncate assertion in edge case. - explictly coerce LLVM_ENABLE_ZLIB to a 0 or 1 value in OFF case, to match current ON, FORCE_ON behavior. - fix code style nits in zlib tests Reviewed By: phosek Differential Revision: https://reviews.llvm.org/D129698
2022-07-08[llvm] Remove unused and redundant crc32 funcction from ↵Cole Kissane
llvm::compression::zlib namespace * Remove crc32 from zlib compression namespace, people should use the `llvm::crc32` instead. Reviewed By: MaskRay, leonardchan Differential Revision: https://reviews.llvm.org/D128754
2022-07-08[NFC] Refactor llvm::zlib namespaceCole Kissane
* Refactor compression namespaces across the project, making way for a possible introduction of alternatives to zlib compression. Changes are as follows: * Relocate the `llvm::zlib` namespace to `llvm::compression::zlib`. Reviewed By: MaskRay, leonardchan, phosek Differential Revision: https://reviews.llvm.org/D128953
2022-03-14[Support] Change zlib::compress to return voidFangrui Song
With a sufficiently large output buffer, the only failure is Z_MEM_ERROR. Check it and call the noreturn report_bad_alloc_error if applicable. resize_for_overwrite may call report_bad_alloc_error as well. Now that there is no other error type, we can replace the return type with void and simplify call sites. Reviewed By: ikudrin Differential Revision: https://reviews.llvm.org/D121512
2021-12-08Support: Avoid using SmallVector::set_size() in zlibDuncan P. N. Exon Smith
Stop using `SmallVector::set_size()` in zlib. Replace pairs of `reserve()` / `set_size()` with `resize_for_overwrite()` and `truncate()`. Differential Revision: https://reviews.llvm.org/D115391
2020-08-11[CMake] Simplify CMake handling for zlibPetr Hosek
Rather than handling zlib handling manually, use find_package from CMake to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is set to YES, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This is a reland of abb0075 with all followup changes and fixes that should address issues that were reported in PR44780. Differential Revision: https://reviews.llvm.org/D79219
2020-08-08Revert "[CMake] Simplify CMake handling for zlib"Petr Hosek
This reverts commit ccbc1485b55ff4acd21bcfafbf7aec4ed0fd818d which is still failing on the Windows MLIR bots.
2020-08-08[CMake] Simplify CMake handling for zlibPetr Hosek
Rather than handling zlib handling manually, use find_package from CMake to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is set to YES, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This is a reland of abb0075 with all followup changes and fixes that should address issues that were reported in PR44780. Differential Revision: https://reviews.llvm.org/D79219
2020-08-07Revert "[CMake] Simplify CMake handling for zlib"Christian Kühnel
This reverts commit 1adc494bce44f6004994deed61b30d4b71fe1d05. This patch broke the Windows compilation on buildbot and pre-merge testing: http://lab.llvm.org:8011/builders/mlir-windows/builds/5945 https://buildkite.com/llvm-project/llvm-master-build/builds/780
2020-08-05[CMake] Simplify CMake handling for zlibPetr Hosek
Rather than handling zlib handling manually, use find_package from CMake to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is set to YES, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This is a reland of abb0075 with all followup changes and fixes that should address issues that were reported in PR44780. Differential Revision: https://reviews.llvm.org/D79219
2020-08-05Revert "[CMake] Simplify CMake handling for zlib"Hans Wennborg
This quietly disabled use of zlib on Windows even when building with -DLLVM_ENABLE_ZLIB=FORCE_ON. > Rather than handling zlib handling manually, use find_package from CMake > to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, > HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is > set to YES, which requires the distributor to explicitly select whether > zlib is enabled or not. This simplifies the CMake handling and usage in > the rest of the tooling. > > This is a reland of abb0075 with all followup changes and fixes that > should address issues that were reported in PR44780. > > Differential Revision: https://reviews.llvm.org/D79219 This reverts commit 10b1b4a231a485f1711d576e6131f6755e008abe and follow-ups 64d99cc6abed78c00a2a7863b02ce54911a5264f and f9fec0447e12da9e8cf4b628f6d45f4941e7d182.
2020-07-23[CMake] Simplify CMake handling for zlibPetr Hosek
Rather than handling zlib handling manually, use find_package from CMake to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is set to YES, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This is a reland of abb0075 with all followup changes and fixes that should address issues that were reported in PR44780. Differential Revision: https://reviews.llvm.org/D79219
2020-07-23Revert "[CMake] Simplify CMake handling for zlib"Petr Hosek
This reverts commit 1d09ecf36175f7910ffedd6d497c07b5c74c22fb since it breaks sanitizer bots.
2020-07-23[CMake] Simplify CMake handling for zlibPetr Hosek
Rather than handling zlib handling manually, use find_package from CMake to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is set to YES, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This is a reland of abb0075 with all followup changes and fixes that should address issues that were reported in PR44780. Differential Revision: https://reviews.llvm.org/D79219
2020-07-14Revert "[CMake] Simplify CMake handling for zlib"Petr Hosek
This reverts commit 8c1a79dc12f3cc600e16153961cd8cc50ba2c33b because it fails when zlib isn't installed.
2020-07-14[CMake] Simplify CMake handling for zlibPetr Hosek
Rather than handling zlib handling manually, use find_package from CMake to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB, HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is set to YES, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This is a reland of abb0075 with all followup changes and fixes that should address issues that were reported in PR44780. Differential Revision: https://reviews.llvm.org/D79219
2020-05-03[Support] Don't initialize buffer allocated by zlib::uncompressBenjamin Kramer
This is a somewhat annoying API, but not without precedend in this low level API.
2020-03-03Revert abb00753 "build: reduce CMake handling for zlib" (PR44780)Hans Wennborg
and follow-ups: a2ca1c2d "build: disable zlib by default on Windows" 2181bf40 "[CMake] Link against ZLIB::ZLIB" 1079c68a "Attempt to fix ZLIB CMake logic on Windows" This changed the output of llvm-config --system-libs, and more importantly it broke stand-alone builds. Instead of piling on more fix attempts, let's revert this to reduce the risk of more breakages.
2020-01-02build: reduce CMake handling for zlibSaleem Abdulrasool
Rather than handling zlib handling manually, use `find_package` from CMake to find zlib properly. Use this to normalize the `LLVM_ENABLE_ZLIB`, `HAVE_ZLIB`, `HAVE_ZLIB_H`. Furthermore, require zlib if `LLVM_ENABLE_ZLIB` is set to `YES`, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling. This restores 68a235d07f9e7049c7eb0c8091f37e385327ac28, e6c7ed6d2164a0659fd9f6ee44f1375d301e3cad. The problem with the windows bot is a need for clearing the cache.
2020-01-02Revert "build: reduce CMake handling for zlib"James Henderson
This reverts commit 68a235d07f9e7049c7eb0c8091f37e385327ac28. This commit broke the clang-x64-windows-msvc build bot and a follow-up commit did not fix it. Reverting to fix the bot.
2020-01-01build: reduce CMake handling for zlibSaleem Abdulrasool
Rather than handling zlib handling manually, use `find_package` from CMake to find zlib properly. Use this to normalize the `LLVM_ENABLE_ZLIB`, `HAVE_ZLIB`, `HAVE_ZLIB_H`. Furthermore, require zlib if `LLVM_ENABLE_ZLIB` is set to `YES`, which requires the distributor to explicitly select whether zlib is enabled or not. This simplifies the CMake handling and usage in the rest of the tooling.
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-08-04Fix buildbot breakage.Rui Ueyama
llvm-svn: 338940
2018-08-04Use the same constants as zlib to represent compression level.Rui Ueyama
This change allows users pass compression level that was not listed in the enum. Also, I think using different values than zlib's compression levels was just confusing. Differential Revision: https://reviews.llvm.org/D50196 llvm-svn: 338939
2018-08-03[Support] Don't initialize compressed buffer allocated by zlib::compressFangrui Song
resize() (zeroing) makes every allocated page resident. The actual size of the compressed buffer is usually much smaller. Making every page resident is wasteful. When linking a test binary with ~1.9GiB uncompressed debug info with LLD, this optimization decreases max RSS by ~1.5GiB. Differential Revision: https://reviews.llvm.org/50223 llvm-svn: 338913
2017-01-17Recommit r292214 "[Support/Compression] - Change zlib API to return Error ↵George Rimar
instead of custom status" No any changes, will follow up with D28807 commit containing APLi change for clang to fix build issues happened. Original commit message: [Support/Compression] - Change zlib API to return Error instead of custom status. Previously API returned custom enum values. Patch changes it to return Error with string description. That should help users to report errors in universal way. Differential revision: https://reviews.llvm.org/D28684 llvm-svn: 292226
2017-01-17Revert r292214 "[Support/Compression] - Change zlib API to return Error ↵George Rimar
instead of custom status." It broked clang: http://lab.llvm.org:8080/green//job/clang-stage1-cmake-RA-incremental_build/34218/consoleFull#46141505449ba4694-19c4-4d7e-bec5-911270d8a58c llvm-svn: 292217
2017-01-17[Support/Compression] - Change zlib API to return Error instead of custom ↵George Rimar
status. Previously API returned custom enum values. Patch changes it to return Error with string description. That should help users to report errors in universal way. Differential revision: https://reviews.llvm.org/D28684 llvm-svn: 292214
2016-09-12Define a dummy zlib::uncompress when zlib is not available.Rafael Espindola
Should fix link errors in some bots when it is used. llvm-svn: 281208
2016-09-09Add a lower level zlib::uncompress.Rafael Espindola
SmallVectors are convenient, but they don't cover every use case. In particular, they are fairly large (3 pointers + one element) and there is no way to take ownership of the buffer to put it somewhere else. This patch then adds a lower lever interface that works with any buffer. llvm-svn: 281082
2015-03-23Purge unused includes throughout libSupport.Benjamin Kramer
NFC. llvm-svn: 232976
2014-11-25[msan] Annotate zlib functions for MemorySanitizer.Evgeniy Stepanov
Mark destination buffer in zlib::compress and zlib::decompress as fully initialized. When building LLVM with system zlib and MemorySanitizer instrumentation, MSan does not observe memory writes in zlib code and erroneously considers zlib output buffers as uninitialized, resulting in false use-of-uninitialized memory reports. This change helps MSan understand the state of that memory and prevents such reports. llvm-svn: 222763
2014-04-05Simplify compression API by compressing into a SmallVector rather than a ↵David Blaikie
MemoryBuffer This is the other half of r205676. llvm-svn: 205677
2014-04-05Simplify compression API by decompressing into a SmallVector rather than a ↵David Blaikie
MemoryBuffer This avoids an extra copy during decompression and avoids the use of MemoryBuffer which is a weirdly esoteric device that includes unrelated concepts like "file name" (its rather generic name is a bit misleading). Similar refactoring of zlib::compress coming up. llvm-svn: 205676
2014-03-06Replace OwningPtr<T> with std::unique_ptr<T>.Ahmed Charles
This compiles with no changes to clang/lld/lldb with MSVC and includes overloads to various functions which are used by those projects and llvm which have OwningPtr's as parameters. This should allow out of tree projects some time to move. There are also no changes to libs/Target, which should help out of tree targets have time to move, if necessary. llvm-svn: 203083
2013-08-14Expose CRC-32 implementation from zlibAlexey Samsonov
llvm-svn: 188380
generated by cgit v1.2.3 (git 2.25.1) at 2026-10-24 21:33:53 +0000