The following removes the confusion around num_mask_args that was
added to properly "guess" the number of mask elements in a AVX512
mask that's just represented as int.  The actual mistake lies in
the mixup of 'ncopies' which is used to track the number of
OMP SIMD calls to be emitted rather than the number of input
vectors.  So this reverts the earlier r16-5374-g5c2fdfc24e343c,
uses the proper 'ncopies' for loop mask record/query and adjusts
the guessing of the SIMD arg mask elements.

	PR tree-optimization/122762
	PR tree-optimization/122736
	PR tree-optimization/122790
	* cgraph.h (cgraph_simd_clone_arg::linear_step): Document
	use for SIMD_CLONE_ARG_TYPE_MASK.
	* omp-simd-clone.cc (simd_clone_adjust_argument_types):
	Record the number of mask arguments in linear_step if
	mask_mode is not VOIDmode.
	* tree-vect-stmts.cc (vectorizable_simd_clone_call):
	Remove num_mask_args computation, use a proper ncopies
	to query/register loop masks, use linear_step for the
	number of mask arguments when determining the number of
	mask elements in a mask argument.

	* gcc.dg/vect/vect-simd-clone-23.c: New testcase.

The problem here r16-4747-g529c25ed6e0a06 changed
symtab_type to toplevel_type and then added 2 types
at the begining but didn't update symtab_type_names.
This obvious patch changes symtab_type_names to toplevel_type_names,
adds a _MAX type to toplevel_type and a comment about keeping them in sync.
Also adds an static assert to make sure if someone adds a toplevel_type
that toplevel_type_names get updated too.

Pushed as obvious after a build and test.

	PR ipa/122512
gcc/ChangeLog:

	* cgraph.h (enum toplevel_type): Add TOPLEVEL_MAX. Add comment
	about keeping toplevel_type_names in sync.
	* symtab.cc (symtab_type_names): Rename to ...
	(toplevel_type_names): Add new types and add an assert about
	the size.
	(symtab_node::get_symtab_type_string): Use toplevel_type_names
	instead of symtab_type_names.
	(symtab_node::dump_base): Likewise.

Signed-off-by: Andrew Pinski <andrew.pinski@oss.qualcomm.com>

asm_node and symbol_node will now inherit from toplevel_node.
This is now useful for lto partitioning, in future it should be also
useful for toplevel extended assembly.

gcc/ChangeLog:

	* cgraph.h (enum symtab_type): Replace with toplevel_type.
	(enum toplevel_type): New.
	(struct toplevel_node): New.
	(struct asm_node): Inherits from toplevel_node.
	(test): New.
	(symbol_table::finalize_toplevel_asm): Use constructor.
	* coretypes.h (struct toplevel_node): New.
	(struct asm_node): New.
	* lto-streamer-in.cc (lto_input_toplevel_asms): Add
	lto_file_data.

This patch enables constant propagation to outlined OpenMP kernels.
It does so using a new function attribute called ' callback' (note the
space).

The attribute ' callback' captures the notion of a function calling one
of its arguments with some of its parameters as arguments.  An OpenMP
example of such function is GOMP_parallel.
We implement the attribute with new callgraph edges called callback
edges. They are imaginary edges pointing from the caller of the function
with the attribute (e.g. caller of GOMP_parallel) to the body function
itself (e.g. the outlined OpenMP body).  They share their call statement
with the edge from which they are derived (direct edge caller -> GOMP_parallel
in this case).  These edges allow passes such as ipa-cp to see the hidden
call site to the body function and optimize the function accordingly.

To illustrate on an example, the body GOMP_parallel looks something
like this:

void GOMP_parallel (void (*fn) (void *), void *data, /* ... */)
{
  /* ... */
  fn (data);
  /* ... */
}

If we extend it with the attribute ' callback(1, 2)', we express that the
function calls its first argument and passes it its second argument.
This is represented in the call graph in this manner:

             direct                         indirect
caller -----------------> GOMP_parallel ---------------> fn
  |
  ----------------------> fn
          callback

The direct edge is then the callback-carrying edge, all new edges
are the derived callback edges.
While constant propagation is the main focus of this patch, callback
edges can be useful for different passes (for example, they improve icf
for OpenMP kernels), as they allow for address redirection.
If the outlined body function gets optimized and cloned, from body_fn to
body_fn.optimized, the callback edge allows us to replace the
address in the arguments list:

GOMP_parallel (body_fn, &data_struct, /* ... */);

becomes

GOMP_parallel (body_fn.optimized, &data_struct, /* ... */);

This redirection is possible for any function with the attribute.

This callback attribute implementation is partially compatible with
clang's implementation. Its semantics, arguments and argument indexing style are
the same, but we represent an unknown argument position with 0
(precedent set by attributes such as 'format'), while clang uses -1 or '?'.
We use the index 1 for the 'this' pointer in member functions, clang
uses 0. We also allow for multiple callback attributes on the same function,
while clang only allows one.

The attribute is currently for GCC internal use only, thanks to the
space in its name.  Originally, it was supposed to be called
'callback' like its clang counterpart, but we cannot use this name, as
clang uses non-standard indexing style, leading to inconsistencies.  The
attribute will be introduced into the public API as 'gnu::callback_only'
in a future patch.

The attribute allows us to propagate constants into body functions of
OpenMP constructs. Currently, GCC won't propagate the value 'c' into the
OpenMP body in the following example:

int a[100];
void test(int c) {
#pragma omp parallel for
  for (int i = 0; i < c; i++) {
    if (!__builtin_constant_p(c)) {
      __builtin_abort();
    }
    a[i] = i;
  }
}
int main() {
  test(100);
  return a[5] - 5;
}

With this patch, the body function will get cloned and the constant 'c'
will get propagated.

Some functions may utilize the attribute's infrastructure without being
declared with it, for example GOMP_task.  These functions are special
cases and use the special case functions found in attr-callback.h.  Special
cases use the attribute under certain circumstances, for example
GOMP_task uses it when the copy function is not being used required.

gcc/ChangeLog:

	* Makefile.in: Add attr-callback.o to OBJS.
	* builtin-attrs.def (ATTR_CALLBACK): Callback attr identifier.
	(DEF_CALLBACK_ATTRIBUTE): Macro for callback attr creation.
	(GOMP): Attr for libgomp functions.
	(ATTR_CALLBACK_GOMP_LIST): ATTR_NOTHROW_LIST with GOMP callback
	attr added.
	* cgraph.cc (cgraph_add_edge_to_call_site_hash): Always hash the
	callback-carrying edge.
	(cgraph_node::get_edge): Always return the callback-carrying
	edge.
	(cgraph_edge::set_call_stmt): Add cascade for callback edges.
	(symbol_table::create_edge): Allow callback edges to share call
	stmts, initialize new flags.
	(cgraph_edge::make_callback): New method, derives a new callback
	edge.
	(cgraph_edge::get_callback_carrying_edge): New method.
	(cgraph_edge::first_callback_edge): Likewise.
	(cgraph_edge::next_callback_edge): Likewise.
	(cgraph_edge::purge_callback_edges): Likewise.
	(cgraph_edge::redirect_callee): When redirecting a callback
	edge, redirect its ref as well.
	(cgraph_edge::redirect_call_stmt_to_callee): Add callback edge
	redirection logic, set update_derived_edges to true hwne
	redirecting the carrying edge.
	(cgraph_node::remove_callers): Add cascade for callback edges.
	(cgraph_edge::dump_edge_flags): Print callback flags.
	(cgraph_node::verify_node): Add sanity checks for callback
	edges.
	* cgraph.h: Add new 1 bit flags and 16 bit callback_id to
	cgraph_edge class.
	* cgraphclones.cc (cgraph_edge::clone): Copy over callback data.
	* cif-code.def (CALLBACK_EDGE): Add CIF_CALLBACK_EDGE code.
	* ipa-cp.cc (purge_useless_callback_edges): New function,
	deletes callback edges when necessary.
	(ipcp_decision_stage): Call purge_useless_callback_edges.
	* ipa-fnsummary.cc (ipa_call_summary_t::duplicate): Add
	an exception for callback edges.
	(analyze_function_body): Copy over summary from carrying to
	callback edge.
	* ipa-inline-analysis.cc (do_estimate_growth_1): Skip callback
	edges when estimating growth.
	* ipa-inline-transform.cc (inline_transform): Add redirection
	cascade for callback edges.
	* ipa-param-manipulation.cc
	(drop_decl_attribute_if_params_changed_p): New function.
	(ipa_param_adjustments::build_new_function_type): Add
	args_modified out param.
	(ipa_param_adjustments::adjust_decl): Drop callback attrs when
	modifying args.
	* ipa-param-manipulation.h: Adjust decl of
	build_new_function_type.
	* ipa-prop.cc (ipa_duplicate_jump_function): Add decl.
	(init_callback_edge_summary): New function.
	(ipa_compute_jump_functions_for_edge): Add callback edge
	creation logic.
	* lto-cgraph.cc (lto_output_edge): Stream out callback data.
	(input_edge): Input callback data.
	* omp-builtins.def (BUILT_IN_GOMP_PARALLEL_LOOP_STATIC): Use new
	attr list.
	(BUILT_IN_GOMP_PARALLEL_LOOP_GUIDED): Likewise.
	(BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_DYNAMIC): Likewise.
	(BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_RUNTIME): Likewise.
	(BUILT_IN_GOMP_PARALLEL): Likewise.
	(BUILT_IN_GOMP_PARALLEL_SECTIONS): Likewise.
	(BUILT_IN_GOMP_TEAMS_REG): Likewise.
	* tree-core.h (ECF_CB_1_2): New constant for callback(1,2).
	* tree-inline.cc (copy_bb): Copy callback edges when copying the
	carrying edge.
	(redirect_all_calls): Redirect callback edges.
	* tree.cc (set_call_expr_flags): Create callback attr according
	to the ECF_CB flag.
	* attr-callback.cc: New file.
	* attr-callback.h: New file.

gcc/c-family/ChangeLog:

	* c-attribs.cc: Define callback attr.

gcc/fortran/ChangeLog:

	* f95-lang.cc (ATTR_CALLBACK_GOMP_LIST): New attr list
	corresponding to the list in builtin-attrs.def.

gcc/testsuite/ChangeLog:

	* gcc.dg/ipa/ipcp-cb-spec1.c: New test.
	* gcc.dg/ipa/ipcp-cb-spec2.c: New test.
	* gcc.dg/ipa/ipcp-cb1.c: New test.

Signed-off-by: Josef Melcr <jmelcr02@gmail.com>

Add support for a FMV set defined by a combination of target_clones and
target_version definitions.

Additionally, change is_function_default_version to consider a function
declaration annotated with target_clones containing default to be a
default version.

Lastly, add support for the case that a target_clone has all versions filtered
out and therefore the declaration should be removed. This is relevant as now
the default could be defined in a target_version, so a target_clones no longer
necessarily contains the default.

This takes advantage of refactoring done in previous patches changing how
target_clones are expanded and how conflicting decls are handled.

gcc/ChangeLog:

	* attribs.cc (is_function_default_version): Update to handle
	target_clones.
	* cgraph.h (FOR_EACH_FUNCTION_REMOVABLE): New macro.
	* multiple_target.cc (expand_target_clones): Update logic to delete
	empty target_clones and modify diagnostic.
	(ipa_target_clone): Update to use FOR_EACH_FUNCTION_REMOVABLE.

gcc/c-family/ChangeLog:

	* c-attribs.cc: Add support for target_version and target_clone mixing.

gcc/testsuite/ChangeLog:

	* g++.target/aarch64/mv-and-mvc1.C: New test.
	* g++.target/aarch64/mv-and-mvc2.C: New test.
	* g++.target/aarch64/mv-and-mvc3.C: New test.
	* g++.target/aarch64/mv-and-mvc4.C: New test.

This patch changes the semantics of target_version and target_clones attributes
to match the behavior described in the Arm C Language extension.

The changes to behavior are:

- The scope and signature of an FMV function set is now that of the default
  version.
- The FMV resolver is now created at the locations of the default version
  implementation. Previously this was at the first call to an FMV function.
- When a TU has a single annotated function version, it gets mangled.
  - This includes a lone annotated default version.

This only affects targets with TARRGET_HAS_FMV_TARGET_ATTRIBUTE set to false.
Currently that is aarch64 and riscv.

This is achieved by:

- Skipping the existing FMV dispatching code at C++ gimplification and instead
  making use of the target_clones dispatching code in multiple_targets.cc.
  (This fixes PR target/118313 for aarch64 and riscv).
- Splitting target_clones pass in two, an early and late pass, where the early
  pass handles cases where multiple declarations are used to define a version,
  and the late pass handling target semantics targets, and cases where a FMV
  set is defined by a single target_clones decl.
- Changing the logic in add_candidates and resolve_address of overloaded
  function to prevent resolution of any version except a default version.
  (thus making the default version determine scope and signature of the
  versioned function set).
- Adding logic for dispatching a lone annotated default version in
  multiple_targets.cc
  - As as annotated default version gets mangled an alias is created from the
    dispatched symbol to the default version as no ifunc resolution is required
    in this case. (ie. an alias from `_Z3foov` to `_Z3foov.default`)
- Adding logic to `symbol_table::remove_unreachable_nodes` and analyze_functions
  that a reference to the default function version also implies a possible
  reference to the other versions (so they shouldnt be deleted and do need to
  be analyzed).

gcc/ChangeLog:

	PR target/118313
	* cgraph.cc (delete_function_version): Made public static member of
	cgraph_node.
	* cgraph.h (delete_function_version): Ditto.
	* cgraphunit.cc (analyze_functions): Add logic for target version
	dependencies.
	* ipa.cc (symbol_table::remove_unreachable_nodes): Ditto.
	* multiple_target.cc (create_dispatcher_calls): Change to support
	target version semantics.
	(ipa_target_clone): Change to dispatch all function sets in
	target_version semantics, and to have early and late pass.
	(expand_target_clones): Add logic for cases of target_clones with no
	defaults.
	(is_simple_target_clones_case): New function.
	(class pass_target_clone): New parameter for early or late pass.
	* config/aarch64/aarch64.cc: (aarch64_get_function_versions_dispatcher):
	Refactor with the assumption that the DECL node will be default.
	* config/riscv/riscv.cc: (riscv_get_function_versions_dispatcher):
	Refactor with the assumption that the DECL node will be default.
	* passes.def: Split target_clones pass into early and late version.

gcc/cp/ChangeLog:

	PR target/118313
	* call.cc (add_candidates): Change to not resolve non-default versions
	in target_version semantics.
	* class.cc (resolve_address_of_overloaded_function): Ditto.
	* cp-gimplify.cc (cp_genericize_r): Change logic to not apply for
	target_version semantics.
	* decl.cc (maybe_mark_function_versioned): Remove static.
	* cp-tree.h (maybe_mark_function_versioned): New function.
	* decl2.cc (cplus_decl_attributes ): Change to mark and therefore
	mangle all target_version decls in target_version semantics.
	* typeck.cc (cp_build_function_call_vec): Add error for calling
	unresolvable non-default node in target_version semantics.

gcc/testsuite/ChangeLog:

	* g++.target/aarch64/mv-1.C: Change for target_version semantics.
	* g++.target/aarch64/mv-symbols2.C: Ditto.
	* g++.target/aarch64/mv-symbols3.C: Ditto.
	* g++.target/aarch64/mv-symbols4.C: Ditto.
	* g++.target/aarch64/mv-symbols5.C: Ditto.
	* g++.target/aarch64/mvc-symbols3.C: Ditto.
	* g++.target/riscv/mv-symbols2.C: Ditto.
	* g++.target/riscv/mv-symbols3.C: Ditto.
	* g++.target/riscv/mv-symbols4.C: Ditto.
	* g++.target/riscv/mv-symbols5.C: Ditto.
	* g++.target/riscv/mvc-symbols3.C: Ditto.
	* g++.target/aarch64/mv-symbols10.C: New test.
	* g++.target/aarch64/mv-symbols11.C: New test.
	* g++.target/aarch64/mv-symbols12.C: New test.
	* g++.target/aarch64/mv-symbols13.C: New test.
	* g++.target/aarch64/mv-symbols6.C: New test.
	* g++.target/aarch64/mv-symbols7.C: New test.
	* g++.target/aarch64/mv-symbols8.C: New test.
	* g++.target/aarch64/mv-symbols9.C: New test.

This patch is an overhaul of how FMV name mangling works. Previously
mangling logic was duplicated in several places across both target
specific and independent code. This patch changes this such that all
mangling is done in targetm.mangle_decl_assembler_name (including for the
dispatched symbol and dispatcher resolver).

Adds the assembler_name member to cgraph_function_version_info to store
the base assembler name of the function set, before FMV mangling.

This allows for the removing of previous hacks, such as where the default
mangled decl's assembler name was unmangled to then remangle all versions
and the resolver and dispatched symbol.

This introduces a change (shown in test changes) for the assembler name of the
dispatched symbol for a x86 versioned function set. Previously it used the
function name mangled twice. This was hard to reproduce without hacks I
wasn't comfortable with. Therefore, the mangling is changed to instead append
".ifunc" which matches clang's behavior.

This change also refactors expand_target_clone using
targetm.mangle_decl_assembler_name for mangling and get_clone_versions.
It is modified such that if the target_clone is in a FMV structure
the ordering is preserved once expanded. This is used later for ACLE semantics
and target_clone/target_version mixing.

gcc/ChangeLog:

	* attribs.cc (make_dispatcher_decl): Move duplicated cgraph logic into
	this function and change to use targetm.mangle_decl_assembler_name for
	mangling.
	* cgraph.cc (cgraph_node::insert_new_function_version): Record
	assembler_name.
	* cgraph.h (struct cgraph_function_version_info): Add assembler_name.
	(struct cgraph_node): Add dispatcher_resolver_function and
	is_target_clone.
	* config/aarch64/aarch64.cc (aarch64_parse_fmv_features): Change to
	support string_slice.
	(aarch64_process_target_version_attr): Ditto.
	(get_feature_mask_for_version): Ditto.
	(aarch64_mangle_decl_assembler_name): Add logic for mangling dispatched
	symbol and resolver.
	(get_suffixed_assembler_name): Removed.
	(make_resolver_func): Refactor to use
	aarch64_mangle_decl_assembler_name for mangling.
	(aarch64_generate_version_dispatcher_body): Remove remangling.
	(aarch64_get_function_versions_dispatcher): Refactor to remove
	duplicated cgraph logic.
	* config/i386/i386-features.cc
	(ix86_mangle_function_version_assembler_name): Refactor to use
	clone_identifier and to mangle default.
	(ix86_mangle_decl_assembler_name): Add logic for mangling dispatched
	symbol and resolver.
	(ix86_get_function_versions_dispatcher): Remove duplicated cgraph
	logic.
	(make_resolver_func): Refactor to use ix86_mangle_decl_assembler_name
	for mangling.
	* config/riscv/riscv.cc (riscv_mangle_decl_assembler_name): Add logic
	for FMV mangling.
	(get_suffixed_assembler_name): Removed.
	(make_resolver_func): Refactor to use riscv_mangle_decl_assembler_name
	for mangling.
	(riscv_generate_version_dispatcher_body): Remove unnecessary remangling.
	(riscv_get_function_versions_dispatcher): Remove duplicated cgraph
	logic.
	* config/rs6000/rs6000.cc (rs6000_mangle_decl_assembler_name): New
	function.
	(rs6000_get_function_versions_dispatcher): Remove duplicated cgraph
	logic.
	(make_resolver_func): Refactor to use rs6000_mangle_decl_assembler_name
	for mangling.
	(rs6000_mangle_function_version_assembler_name): New function.
	* multiple_target.cc (create_dispatcher_calls): Remove mangling code.
	(get_attr_str): Removed.
	(separate_attrs): Ditto.
	(is_valid_asm_symbol): Removed.
	(create_new_asm_name): Ditto.
	(expand_target_clones): Refactor to use
	targetm.mangle_decl_assembler_name for mangling and be more general.
	* tree.cc (get_target_clone_attr_len): Removed.
	* tree.h (get_target_clone_attr_len): Removed.

gcc/cp/ChangeLog:

	* decl.cc (maybe_mark_function_versioned): Change to insert function version
	and therefore record assembler name.

gcc/testsuite/ChangeLog:

	* g++.target/i386/mv-symbols1.C: Update x86 FMV mangling.
	* g++.target/i386/mv-symbols3.C: Ditto.
	* g++.target/i386/mv-symbols4.C: Ditto.
	* g++.target/i386/mv-symbols5.C: Ditto.

This is similar to clone_function_name and its siblings but takes an
identifier tree node rather than a function declaration.

This is to be used in conjunction with the identifier node stored in
cgraph_function_version_info::assembler_name to mangle FMV functions in
later patches.

gcc/ChangeLog:

	* cgraph.h (clone_identifier): New function.
	* cgraphclones.cc (clone_identifier): New function.
	(clone_function_name): Refactored to use clone_identifier.
	(is_valid_asm_symbol): New helper function.

This patch fixed devirtualization time benefit of ipa-cp which should be scaled
by edge frequency but it is not.  The cost model is still not correct for speculative
calls, since it does not take into account the fact that code size will shrink if
speculation is removed.
I also made cgraph_edge::make_direct to not ICE when there are multiple speculations
to same target. This can happen in combination of devirtualization and ICF in some
rare cases.

gcc/ChangeLog:

	* cgraph.cc (cgraph_edge::make_direct): Do not ICE when there are
	multiple speculations to comptaible targets
	(cgraph_edge::combined_sreal_frequency): New member function.
	* cgraph.h (cgraph_edge::combined_sreal_frequency): Declare.
	* ipa-cp.cc (devirtualization_time_bonus): Return sreal; consider
	profile.
	(estimate_local_effects): Likewise.

Renames record_function_versions to add_function_version, and make it
explicit that it is adding a single version to the function structure.

Additionally, change the insertion point to always maintain priority ordering
of the versions.

This allows for removing logic for moving the default to the first
position which was duplicated across target specific code and enables
easier reasoning about function sets.

gcc/ChangeLog:

	* cgraph.cc (cgraph_node::record_function_versions): Refactor and
	rename to...
	(cgraph_node::add_function_version): new function.
	* cgraph.h (cgraph_node::record_function_versions): Refactor and
	rename to...
	(cgraph_node::add_function_version): new function.
	* config/aarch64/aarch64.cc (aarch64_get_function_versions_dispatcher):
	Remove reordering.
	* config/i386/i386-features.cc (ix86_get_function_versions_dispatcher):
	Remove reordering.
	* config/riscv/riscv.cc (riscv_get_function_versions_dispatcher):
	Remove reordering.
	* config/rs6000/rs6000.cc (rs6000_get_function_versions_dispatcher):
	Remove reordering.

gcc/cp/ChangeLog:

	* decl.cc (maybe_version_functions): Change record_function_versions
	call to add_function_version.