The added function is currently '#if 0' but is planned to be used to enable
self mapping automatically. Prerequisite for auto self maps is still mapping
'declare target' variables (if any, in libgomp) or converting all
'declare target' variables to 'declare target link' in the compiler
(as required for 'omp requires self_maps').

include/ChangeLog:

	* hsa_ext_amd.h (enum hsa_amd_agent_info_s): Add
	HSA_AMD_AGENT_INFO_MEMORY_PROPERTIES.
	(enum): Add HSA_AMD_MEMORY_PROPERTY_AGENT_IS_APU.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (is_integrated_apu): New; currently '#if 0'.
	* plugin/plugin-nvptx.c (is_integrated_apu): Likewise.

Especially with unified-shared memory and especially with C++'s virtual
functions, it is not uncommon to have on the device a function pointer
that points to the host function - but has an associated device.
If the pointed-to function is (explicitly or implicitly) 'declare target'
with the 'indirect' clause, it is added to the lookup table.

Before this commit, the conversion of the lookup table into a lookup
hash table happened every time a device kernel was launched on the first
team - albeit if already converted, the function immediately returned.

Ignoring the overhead, there was also a race: If multiple teams were
launched, it could happen that another team of the same target region
already tried to use the lookup table which it was still being created.
Likewise when lauching a kernel with 'nowait' and directly afterward
another kernel, there could be a race of creating the table.

With this commit, the creating of the kernel has been moved to the
host-plugin's GOMP_OFFLOAD_load_image. The previous code stored a
pointer to the host/device pointer array, which makes it hard when
creating the hash table on the host (data is needed for finding the
slot) - but accessing it on the device (where the lookup has to work
as well). As the hash-table implementation (only) supports integral
value as payload (0 and 1 having special meaning), the solution was
to move to an uint128_t variable to store both the host and device
address.

As the host-side library is typically dynamically linked and the
device-side one statically, there is the problem of backward
compatibility. The current implementation permits both older
binaries and newer libgomp and newer binaries with older libgomp.
I could imagine us breaking the latter eventually, but for now
there is up and downward compatibility. (Obviously, the race is
only fixed if new + new is combined.)

Code wise, on the device exist GOMP_INDIRECT_ADDR_MAP which was
updated to point to the host/device-address array. Now additionally
GOMP_INDIRECT_ADDR_HMAP exists, which contains the hash-table map.

If the latter exists, libgomp only updates it and the former remains
a NULL pointer; it is also untouched if there are no indirect functions.
Being NULL therefore avoids the call to the device-side build_indirect_map.
The code also currently supports to have no hash and a linear walk. I think
that remained from testing; due to the backward-compat feature, it can
actually be turned of on either side.

libgomp/ChangeLog:

	PR libgomp/119857
	PR libgomp/114445
	* config/accel/target-indirect.c: Change to use uint128_t instead
	of a struct as data structure and add GOMP_INDIRECT_ADDR_HMAP as
	host-accessible variable.
	(struct indirect_map_t): Remove.
	(USE_HASHTAB_LOOKUP, INDIRECT_DEV_ADDR, INDIRECT_HOST_ADDR,
	SET_INDIRECT_HOST_ADDR, SET_INDIRECT_ADDRS): Define.
	(htab_free): Use __builtin_unreachable.
	(htab_hash, htab_eq, GOMP_target_map_indirect_ptr,
	build_indirect_map): Update for new representation and new
	pointer-to-hash variable.
	* config/gcn/team.c (gomp_gcn_enter_kernel): Only call
	build_indirect_map when GOMP_INDIRECT_ADDR_MAP.
	* config/nvptx/team.c (gomp_nvptx_main): Likewise.
	* libgomp-plugin.h (GOMP_INDIRECT_ADDR_HMAP): Define.
	* plugin/plugin-gcn.c: Conditionally include
	build-target-indirect-htab.h.
	(USE_HASHTAB_LOOKUP_FOR_INDIRECT): Define.
	(create_target_indirect_map): New prototype.
	(GOMP_OFFLOAD_load_image): Update to create the device's
	indirect-function hash table on the host.
	* plugin/plugin-nvptx.c: Conditionally include
	build-target-indirect-htab.h.
	(USE_HASHTAB_LOOKUP_FOR_INDIRECT): Define.
	(create_target_indirect_map): New prototype.
	(GOMP_OFFLOAD_load_image): Update to create the device's
	indirect-function hash table on the host.
	* plugin/build-target-indirect-htab.h: New file.

	PR libgomp/120444

include/ChangeLog:

	* cuda/cuda.h (cuMemsetD8, cuMemsetD8Async): Declare.

libgomp/ChangeLog:

	* libgomp-plugin.h (GOMP_OFFLOAD_memset): Declare.
	* libgomp.h (struct gomp_device_descr): Add memset_func.
	* libgomp.map (GOMP_6.0.1): Add omp_target_memset{,_async}.
	* libgomp.texi (Device Memory Routines): Document them.
	* omp.h.in (omp_target_memset, omp_target_memset_async): Declare.
	* omp_lib.f90.in (omp_target_memset, omp_target_memset_async):
	Add interfaces.
	* omp_lib.h.in (omp_target_memset, omp_target_memset_async): Likewise.
	* plugin/cuda-lib.def: Add cuMemsetD8.
	* plugin/plugin-gcn.c (struct hsa_runtime_fn_info): Add
	hsa_amd_memory_fill_fn.
	(init_hsa_runtime_functions): DLSYM_OPT_FN load it.
	(GOMP_OFFLOAD_memset): New.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_memset): New.
	* target.c (omp_target_memset_int, omp_target_memset,
	omp_target_memset_async_helper, omp_target_memset_async): New.
	(gomp_load_plugin_for_device): Add DLSYM (memset).
	* testsuite/libgomp.c-c++-common/omp_target_memset.c: New test.
	* testsuite/libgomp.c-c++-common/omp_target_memset-2.c: New test.
	* testsuite/libgomp.c-c++-common/omp_target_memset-3.c: New test.
	* testsuite/libgomp.fortran/omp_target_memset.f90: New test.
	* testsuite/libgomp.fortran/omp_target_memset-2.f90: New test.

libgomp/ChangeLog:

	PR libgomp/93226
	* libgomp-plugin.h (GOMP_OFFLOAD_openacc_async_dev2dev): New
	prototype.
	* libgomp.h (struct acc_dispatch_t): Add dev2dev_func.
	(gomp_copy_dev2dev): New prototype.
	* libgomp.map (OACC_2.6.1): New; add acc_memcpy_device{,_async}.
	* libgomp.texi (acc_memcpy_device): New.
	* oacc-mem.c (memcpy_tofrom_device): Change to take from/to
	device boolean; use memcpy not memmove; add early return if
	size == 0 or same device + same ptr.
	(acc_memcpy_to_device, acc_memcpy_to_device_async,
	acc_memcpy_from_device, acc_memcpy_from_device_async): Update.
	(acc_memcpy_device, acc_memcpy_device_async): New.
	* openacc.f90 (acc_memcpy_device, acc_memcpy_device_async):
	Add interface.
	* openacc_lib.h (acc_memcpy_device, acc_memcpy_device_async):
	Likewise.
	* openacc.h (acc_memcpy_device, acc_memcpy_device_async): Add
	prototype.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_openacc_async_host2dev):
	Update comment.
	(GOMP_OFFLOAD_openacc_async_dev2host): Update call.
	(GOMP_OFFLOAD_openacc_async_dev2dev): New.
	* plugin/plugin-nvptx.c (cuda_memcpy_dev_sanity_check): New.
	(GOMP_OFFLOAD_dev2dev): Call it.
	(GOMP_OFFLOAD_openacc_async_dev2dev): New.
	* target.c (gomp_copy_dev2dev): New.
	(gomp_load_plugin_for_device): Load dev2dev and async_dev2dev.
	* testsuite/libgomp.oacc-c-c++-common/acc_memcpy_device-1.c: New test.
	* testsuite/libgomp.oacc-fortran/acc_memcpy_device-1.f90: New test.

The interop directive operates on an opaque object that represents a
foreign runtime. This commit adds support for
this to the two offloading plugins.

For nvptx, it supports cuda, cuda_driver and hip; the latter is AMD's
version of CUDA which for Nvidia devices boils down to normal CUDA.
Thus, at the end for this limited use, cuda/cuda_driver/hip are all
the same - and for plugin-nvptx.c, the they differ only in terms of
what gets fr_id, fr_name and get_interop_type_desc return.

For gcn, it supports hip and hsa.

Regarding get-mapped-ptr-1.c: That's actually a fix for the
GOMP_interop commit r15-8654-g99e2906ae255fc that added
GOMP_DEVICE_DEFAULT_OMP_61 alias omp_default_device, which is
a conforming device number. But that test used -5 as check for a
non-conforming device number.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (_LIBGOMP_PLUGIN_INCLUDE): Define.
	(struct hsa_runtime_fn_info): Add two queue functions.
	(hipError_t, hipCtx_t, hipStream_s, hipStream_t): New types.
	(struct hip_runtime_fn_info): New.
	(hip_runtime_lib, hip_fns): New global vars.
	(init_environment_variables): Handle hip_runtime_lib.
	(init_hsa_runtime_functions): Load the two queue functions.
	(init_hip_runtime_functions, GOMP_OFFLOAD_interop,
	GOMP_OFFLOAD_get_interop_int, GOMP_OFFLOAD_get_interop_ptr,
	GOMP_OFFLOAD_get_interop_str,
	GOMP_OFFLOAD_get_interop_type_desc): New.
	* plugin/plugin-nvptx.c (_LIBGOMP_PLUGIN_INCLUDE): Define.
	(GOMP_OFFLOAD_interop, GOMP_OFFLOAD_get_interop_int,
	GOMP_OFFLOAD_get_interop_ptr, GOMP_OFFLOAD_get_interop_str,
	GOMP_OFFLOAD_get_interop_type_desc): New.
	* testsuite/libgomp.c/interop-fr-1.c: New test.
	* testsuite/libgomp.c-c++-common/get-mapped-ptr-1.c: Use -6
	not -5 as non-conforming device number.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (ELFABIVERSION_AMDGPU_HSA_V6,
	EF_AMDGPU_GENERIC_VERSION_V, EF_AMDGPU_GENERIC_VERSION_OFFSET,
	GET_GENERIC_VERSION): New #define.
	(elf_gcn_isa_is_generic): New.
	(isa_matches_agent): Accept all generic code objects on the first
	go; extend the diagnostic and handle runtime-failed case.
	(create_and_finalize_hsa_program): Call it also after loading
	the code failed, pass the status.

Follow up to r15-5392-g884637b6362391. As the name implies,
GOMP_OFFLOAD_openacc_async_construct is also externally called.
Hence, partially revert previous commit to permit unlocking handling
in oacc-async.c's lookup_goacc_asyncqueue by not failing fatally.

Hence, also the other (indirect) callers had to be updated:
GOMP_OFFLOAD_dev2dev fails now with 'false' and
GOMP_OFFLOAD_async_run fatally.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_dev2dev, GOMP_OFFLOAD_async_run):
	Handle omp_async_queue == NULL after call to maybe_init_omp_async.
	(GOMP_OFFLOAD_openacc_async_construct): Use error not fatal error,
	partially reverting r15-5392.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_openacc_async_construct): In
	case of an error, call GOMP_PLUGIN_fatal not ..._error; use NULL
	not false in return.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (isa_matches_agent): Mention the device number
	and ROCR_VISIBLE_DEVICES when reporting an ISA mismatch error.