Sometimes, there may be no matched anchors but the functions still match. e.g. if the function’s template typename changes, all the callsites that use the type are mismatched and the caller function that contains those callsite are mismatched. Introduce a check to match the functions if their demangled base names are the same.
We observed that the number of IR anchors is usually greater than the number of profile anchors, because IR anchors can be optimized away later and llvm-profgen might not generate profiles for cold callsites. This can cause missing function matches. I’m changing the similarity calculation to use only the profile anchor size. In another point of view, It also makes sense to reuse as many profile anchors as possible regardless of the new functions in the IR.
This is a split of https://github.com/llvm/llvm-project/pull/125756
Tested call-graph matching on some of Meta's large services, it works to reuse some renamed function profiles, no negative perf or significant build speed regression observed. Turned it on by default for CSSPGO mode.
When using -sample-profile-remove-probe, pseudo probe desc should also be removed and dwarf discriminator for call instruction should be restored.
If the gep is nusw (usually via inbounds) and the offset is non-negative, we can infer nuw. Proof: https://alive2.llvm.org/ce/z/ihztLy
With extbinary profile format, initial profile loading only reads profile based on current function names in the module. However, if a function is renamed, sample loader skips to load its original profile(which has a different name), we will miss this case. To address this, we load the top-level profile candidate explicitly for the matching. If a match is found later, the function profile will be further preserved for use by the sample loader.
We should undo this https://reviews.llvm.org/D102007 after undoing https://reviews.llvm.org/D104477 to use missing probe to represent unknown count, zero count and unknown count are different to profile inference. It only affects post-inline(linker) pipeline(with `--overwrite-existing-weights` on ) and flow-sensitive FDO.
Profile staleness could be due to function renaming. Given that sample profile loader relies on exact string matching, a trivial change in the function signature( such as `int foo()` --> `long foo()` ) can make the mangled name different, the function profile(including all nested children profile) becomes unavailable. This patch introduces stale profile call-graph level matching, targeting at identifying the trivial function renaming and reusing the old function profile. Some noteworthy details: 1. Extend the LCS based CFG level matching to identify new function. - Extend to match function and profile have different name instead of the exact function name matching. This leverages LCS, i.e during the finding of callsite anchor matching, when two function name are different, try matching the functions instead of return. - In LCS, the equal function check is replaced by `functionMatchesProfile`. - Only try matching functions that are new functions(neither appears on each side). This reduces the matching scope as we don't need to match the originally matched function. 2. Determine the matching by call-site anchor similarity check. - A new function `functionMatchesProfile(IRFunc, ProfFunc)` is used to check the renaming for the possible <IRFunc, ProfFunc> pair, use the LCS(diff) matching to compute the equal set and we define: `Similarity = |equalSet * 2| / (|A| + |B|)`. The profile name is marked as renamed if the similarity is above a threshold(`-func-profile-similarity-threshold`) 3. Process the matching in top-down function order - when a caller's is done matching, the new function names are saved for later use, using top-down order will maximize the reused results. - `ProfileNameToFuncMap` is used to save or cache the matching result. 4. Update the original profile at the end using `ProfileNameToFuncMap`. 5. Added a new switch --salvage-unused-profile to control this, default is false. Verified on one Meta's internal big service, confirmed 90%+ of the found renaming pair is good. (There could be incorrect renaming pair if the num of the anchor is small, but checked that those functions are simple cold function)
The algorithm added by PR #87375 can be potentially quadratic in the number of anchors. This is almost never a problem because normally functions have a reasonable number of function calls. However, in some rare cases of auto-generated code we observed very large functions that trigger quadratic behaviour here (resulting in >130GB of peak heap memory usage for clang). Let's add a knob for controlling the max number of callsites in a function above which stale profile matching won't be performed.