This AutoConvert.h header frequently gets mislabeled as an unused include because it is guarded by MVS internally and every usage is also guarded. This refactors the change to remove this guard and instead make these functions a noop on other non-z/OS platforms.
This fixes the following error when reading source and header files on z/OS: error: source file is not valid UTF-8
Identified with misc-include-cleaner.
This patch renames the functions in AutoConvert.h/cpp to have a less generic name because they are z/OS specific.
This amends dceaa0f4491ebe30c0b0f1bc7fa5ec365b60ced6 because ASAN caught an issue where the allocation and deallocation were not properly paired: https://lab.llvm.org/buildbot/#/builders/239/builds/7001 Use malloc and free throughout this file to ensure that all kinds of memory buffers use the proper pairing.
When allocating a memory buffer, we use a non-throwing new so that we can explicitly handle memory buffers that are too large to fit into memory. However, when exceptions are disabled, LLVM installs a custom new handler (https://github.com/llvm/llvm-project/blob/90109d444839683b09f0aafdc50b749cb4b3203b/llvm/lib/Support/InitLLVM.cpp#L61) that explicitly crashes when we run out of memory (https://github.com/llvm/llvm-project/blob/de14b749fee41d4ded711e771e43043ae3100cb3/llvm/lib/Support/ErrorHandling.cpp#L188) and that means this particular out-of-memory situation cannot be gracefully handled. This was discovered while working on #embed (https://github.com/llvm/llvm-project/pull/68620) on Windows and resulted in a crash rather than the preprocessor issuing a diagnostic as expected. This patch switches away from the non-throwing new to a call to malloc (and free), which will return a null pointer without calling a custom new handler. It is the only instance in Clang or LLVM that I could find which used a non-throwing new, so I did not think we would need anything more involved than this change. Testing this would be highly platform dependent and so it does not come with test coverage. And because it doesn't change behavior that users are likely to be able to observe, it does not come with a release note.
Empty StringRefs are usually identified by their length being zero, and sometimes they'll have Data==nullptr (e.g. default constructed, or derived from an operation like split/copy and result turned out to be empty). If such StringRef objects are passed to llvm::MemoryBuffer::getMemBufferCopy, it'll result in UB as neither src nor dst can be null, even if size is zero. This patch prevents that UB by not issuing a copy whenever StringRef is empty. Differential Revision: https://reviews.llvm.org/D144706
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
This avoids repeated calls to strlen while we already know its value. When preprocessing sqlite3.c, this gives a surprising 2% speedup. Full benchmark available here: https://llvm-compile-time-tracker.com/compare.php?from=d14c2d408dccd8c6defa7d151e9a96be3cac8cc3&to=04f0641c1cbdcd0bdbd11cd910ca6091420bf52e&stat=instructions:u Recommit 1824432174b3166b40bce59477beb5821170748e, with restored '\0' at the end of buffer name. Differential Revision: https://reviews.llvm.org/D138555
This reverts commit 1824432174b3166b40bce59477beb5821170748e. The change triggers an assertion when building clang on macOS: FAILED: tools/clang/include/clang/Tooling/Syntax/Nodes.inc cd /Users/florianhahn/projects/llvm-project/builds/release-with-assertions/ccache-stage1 && /Users/florianhahn/projects/llvm-project/builds/release-with-assertions/ccache-stage1/bin/clang-tblgen -gen-clang-syntax-node-list -I /Users/florianhahn/projects/llvm-project/clang/include/clang/Tooling/Syntax -I/Users/florianhahn/projects/llvm-project/clang/include -I/Users/florianhahn/projects/llvm-project/builds/release-with-assertions/ccache-stage1/tools/clang/include -I/Users/florianhahn/projects/llvm-project/builds/release-with-assertions/ccache-stage1/include -I/Users/florianhahn/projects/llvm-project/llvm/include /Users/florianhahn/projects/llvm-project/clang/include/clang/Tooling/Syntax/Nodes.td --write-if-changed -o tools/clang/include/clang/Tooling/Syntax/Nodes.inc -d tools/clang/include/clang/Tooling/Syntax/Nodes.inc.d Assertion failed: ((!RequiresNullTerminator || BufEnd[0] == 0) && "Buffer is not null terminated!"), function init, file MemoryBuffer.cpp, line 52. Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 clang-tblgen 0x000000010466b68c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 clang-tblgen 0x000000010466a808 llvm::sys::RunSignalHandlers() + 64 2 clang-tblgen 0x000000010466bd60 SignalHandler(int) + 344 3 libsystem_platform.dylib 0x00000001877a92a4 _sigtramp + 56 4 libsystem_pthread.dylib 0x000000018777acec pthread_kill + 288 5 libsystem_c.dylib 0x00000001876b42c8 abort + 180 6 libsystem_c.dylib 0x00000001876b3620 err + 0 7 clang-tblgen 0x00000001046ce150 llvm::ErrorOr<std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>> getOpenFileImpl<llvm::MemoryBuffer>(int, llvm::Twine const&, unsigned long long, unsigned long long, long long, bool, bool, llvm::Optional<llvm::Align>) (.cold.1) + 0 8 clang-tblgen 0x000000010464a548 llvm::WritableMemoryBuffer::getNewUninitMemBuffer(unsigned long, llvm::Twine const&, llvm::Optional<llvm::Align>) + 496 9 clang-tblgen 0x000000010464a7a0 llvm::ErrorOr<std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>> getOpenFileImpl<llvm::MemoryBuffer>(int, llvm::Twine const&, unsigned long long, unsigned long long, long long, bool, bool, llvm::Optional<llvm::Align>) + 528 10 clang-tblgen 0x000000010464a254 llvm::ErrorOr<std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>> getFileAux<llvm::MemoryBuffer>(llvm::Twine const&, unsigned long long, unsigned long long, bool, bool, bool, llvm::Optional<llvm::Align>) + 176 11 clang-tblgen 0x000000010466fbbc llvm::TableGenMain(char const*, bool (*)(llvm::raw_ostream&, llvm::RecordKeeper&)) + 1428 12 clang-tblgen 0x0000000104626114 main + 156 13 dyld 0x0000000187453e50 start + 2544