summaryrefslogtreecommitdiff
path: root/llvm/lib/Analysis/TrainingLogger.cpp
AgeCommit message (Collapse)Author
2023-11-16[MLGO] Remove -tfutils-use-simplelogger flag (#72492)Aiden Grossman
This flag was redundant and the value was not used anywhere, so it should be removed.
2023-02-01[mlgo] Allow logging the spec for the "advice", if neededMircea Trofin
This is for the interactive model runner, so it can confirm the tensor spec of the advice with its host.
2023-01-20[mlgo] Stream the training dataMircea Trofin
This leverages the new logging format in that we don't need to buffer the training data, we can just write it out. Differential Revision: https://reviews.llvm.org/D142168
2023-01-17[nfc][mlgo] Remove abstraction layers for training loggerMircea Trofin
This follows from D141720 Differential Revision: https://reviews.llvm.org/D141967
2023-01-17Fix OSX build break introduced by D141720Mircea Trofin
2023-01-17[Analysis] Fix a warningKazu Hirata
This patch fixes: llvm/include/llvm/Analysis/Utils/TrainingLogger.h:94:14: error: private field 'IncludeReward' is not used [-Werror,-Wunused-private-field]
2023-01-17[mlgo] Remove the protobuf dependencyMircea Trofin
The dependency was due to the log format. This change switches to the previously-introduced (D139370) "dependency-free" logger instead of the protobuf-based one. A subsequent change will clean out the unnecessary abstraction left behind. This change drops the logger unittest, we have sufficient test coverage via lit tests, and a unit test would require adding, unnecesarily, a log reader (the reader is expected to be python, for the ML side, and there is a reader for that under Analysis/models, used for tests). Differential Revision: https://reviews.llvm.org/D141720
2022-12-12[mlgo] Use LLVM_HAVE_TFLITE instead of LLVM_HAVE_TF_API in C++ code (NFC)Kazu Hirata
We use LLVM_HAVE_TFLITE as the key to enable the mlgo work these days, and LLVM_HAVE_TF_API is defined whenever LLVM_HAVE_TF_API is defined. I'm posting this patch because it's purely mechanical. I'll post a follow-up patch to remove LLVM_HAVE_TF_API in non-C++ files, and that will not be as mechanical as this one. Differential Revision: https://reviews.llvm.org/D139863
2022-12-06Reapply "[mlgo] Dependency-free training mode logger"Mircea Trofin
This reverts commit 8abe7b11f74bea63d3134c144137b72146da0c7b. Added the missing cast which was causing a build problem on certain compilers.
2022-12-06Revert "[mlgo] Dependency-free training mode logger"Florian Hahn
This reverts commit c5ff6f72342e0a4b0ba2ec9f603bedca86721e80. This breaks building on macOS: FAILED: lib/Analysis/CMakeFiles/LLVMAnalysis.dir/TensorSpec.cpp.o /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBUILD_EXAMPLES -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/clang-build/lib/Analysis -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/lib/Analysis -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/clang-build/include -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -mmacosx-version-min=10.14 -fno-exceptions -fno-rtti -UNDEBUG -std=c++17 -MD -MT lib/Analysis/CMakeFiles/LLVMAnalysis.dir/TensorSpec.cpp.o -MF lib/Analysis/CMakeFiles/LLVMAnalysis.dir/TensorSpec.cpp.o.d -o lib/Analysis/CMakeFiles/LLVMAnalysis.dir/TensorSpec.cpp.o -c /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/lib/Analysis/TensorSpec.cpp In file included from /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/lib/Analysis/TensorSpec.cpp:16: In file included from /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include/llvm/Analysis/TensorSpec.h:16: /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include/llvm/Support/JSON.h:354:29: error: non-constant-expression cannot be narrowed from type 'unsigned long' to 'int64_t' (aka 'long long') in initializer list [-Wc++11-narrowing] create<int64_t>(int64_t{I}); ^ /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/lib/Analysis/TensorSpec.cpp:55:18: note: in instantiation of function template specialization 'llvm::json::Value::Value<unsigned long, void, void, void>' requested here OS.value(D); ^ /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include/llvm/Support/JSON.h:354:29: note: insert an explicit cast to silence this issue create<int64_t>(int64_t{I}); ^ static_cast<int64_t>( ) 1 error generated. https://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-incremental/33120/consoleFull#-145995569149ba4694-19c4-4d7e-bec5-911270d8a58c
2022-12-06[mlgo] Dependency-free training mode loggerMircea Trofin
This is the next step in dropping the dependency on protobuf. The simple logger produces an output consisting of lines of json strings. Tensor values - which should constitute the bulk of the data - are serialized as raw byte buffers. This allows for light-weight reading of the values. The next step is to switch the training logic to the new logging format, following which the protobuf-based logger will be dropped, together with the training dependency on protobuf. Subsequent changes will also stop buffering and stream, instead - the buffering model is just as a convenient point-in-time. Differential Revision: https://reviews.llvm.org/D139370
2022-12-01[mlgo][nfc] Virtualize Logger implementationMircea Trofin
This is in preparation for dropping the dependency on protobuf. This first step allows us to subsequently introduce the non-protobuf implementation behind a flag. After that we can update the training side to ingest the new format, after which we can drop the protobuf implementation and de-virtualize everything. Differential Revision: https://reviews.llvm.org/D139062
2022-11-30[mlgo][nfc] Make `LoggedFeatureSpec` an implementation detailMircea Trofin
It's an artifact very specific to using TFAgents during training, so it belongs with ModelUnderTrainingRunner. Differential Revision: https://reviews.llvm.org/D139031
2022-08-03[nfc][mlgo] Separate logger and training-mode model evaluatorMircea Trofin
This just shuffles implementations and declarations around. Now the logger and the TF C API-based model evaluator are separate. Differential Revision: https://reviews.llvm.org/D131116