summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2025-11-18AArch64: Optimise SVE scalar callbacksrelease/2.38/masterJoe Ramsay
Instead of using SVE instructions to marshall special results into the correct lane, just write the entire vector (and the predicate) to memory, then use cheaper scalar operations. Geomean speedup of 16% in special intervals on Neoverse with GCC 14. Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com> (cherry picked from commit 5b82fb18827e962af9f080fdf3c1a69802783f67)
2025-11-04x86: fix wmemset ifunc stray '!' (bug 33542)Jiamei Xie
The ifunc selector for wmemset had a stray '!' in the X86_ISA_CPU_FEATURES_ARCH_P(...) check: if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2) && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, AVX_Fast_Unaligned_Load, !)) This effectively negated the predicate and caused the AVX2/AVX512 paths to be skipped, making the dispatcher fall back to the SSE2 implementation even on CPUs where AVX2/AVX512 are available. The regression leads to noticeable throughput loss for wmemset. Remove the stray '!' so the AVX_Fast_Unaligned_Load capability is tested as intended and the correct AVX2/EVEX variants are selected. Impact: - On AVX2/AVX512-capable x86_64, wmemset no longer incorrectly falls back to SSE2; perf now shows __wmemset_evex/avx2 variants. Testing: - benchtests/bench-wmemset shows improved bandwidth across sizes. - perf confirm the selected symbol is no longer SSE2. Signed-off-by: xiejiamei <xiejiamei@hygon.com> Signed-off-by: Li jing <lijing@hygon.cn> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit 4d86b6cdd8132e0410347e07262239750f86dfb4)
2025-10-17x86: Skip XSAVE state size reset if ISA level requires XSAVEFlorian Weimer
If we have to use XSAVE or XSAVEC trampolines, do not adjust the size information they need. Technically, it is an operator error to try to run with -XSAVE,-XSAVEC on such builds, but this change here disables some unnecessary code with higher ISA levels and simplifies testing. Related to commit befe2d3c4dec8be2cdd01a47132e47bdb7020922 ("x86-64: Don't use SSE resolvers for ISA level 3 or above"). Reviewed-by: H.J. Lu <hjl.tools@gmail.com> (cherry picked from commit 59585ddaa2d44f22af04bb4b8bd4ad1e302c4c02)
2025-10-17x86-64: Simplify minimum ISA check ifdef conditional with ifSunil K Pandey
Replace minimum ISA check ifdef conditional with if. Since MINIMUM_X86_ISA_LEVEL and AVX_X86_ISA_LEVEL are compile time constants, compiler will perform constant folding optimization, getting same results. Reviewed-by: H.J. Lu <hjl.tools@gmail.com> (cherry picked from commit b6e3898194bbae78910bbe9cd086937014961e45)
2025-10-17x86-64: Don't use SSE resolvers for ISA level 3 or aboveH.J. Lu
When glibc is built with ISA level 3 or above enabled, SSE resolvers aren't available and glibc fails to build: ld: .../elf/librtld.os: in function `init_cpu_features': .../elf/../sysdeps/x86/cpu-features.c:1200:(.text+0x1445f): undefined reference to `_dl_runtime_resolve_fxsave' ld: .../elf/librtld.os: relocation R_X86_64_PC32 against undefined hidden symbol `_dl_runtime_resolve_fxsave' can not be used when making a shared object /usr/local/bin/ld: final link failed: bad value For ISA level 3 or above, don't use _dl_runtime_resolve_fxsave nor _dl_tlsdesc_dynamic_fxsave. This fixes BZ #31429. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit befe2d3c4dec8be2cdd01a47132e47bdb7020922)
2025-08-20i386: Add GLIBC_ABI_GNU_TLS version [BZ #33221]H.J. Lu
On i386, programs and shared libraries with __thread usage may fail silently at run-time against glibc without the TLS run-time fix for: https://sourceware.org/bugzilla/show_bug.cgi?id=32996 Add GLIBC_ABI_GNU_TLS version to indicate that glibc has the working GNU TLS run-time. Linker can add the GLIBC_ABI_GNU_TLS version to binaries which depend on the working TLS run-time so that such programs and shared libraries will fail to load and run at run-time against libc.so without the GLIBC_ABI_GNU_TLS version, instead of fail silently at random. This fixes BZ #33221. Signed-off-by: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Sam James <sam@gentoo.org> (cherry picked from commit ed1b7a5a489ab555a27fad9c101ebe2e1c1ba881)
2025-08-20i386: Also add GLIBC_ABI_GNU2_TLS version [BZ #33129]H.J. Lu
Since the GNU2 TLS run-time bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31372 affects both i386 and x86-64, also add GLIBC_ABI_GNU2_TLS version to i386 to indicate the working GNU2 TLS run-time. For x86-64, the additional GNU2 TLS run-time bug fix is needed for https://sourceware.org/bugzilla/show_bug.cgi?id=31501 Signed-off-by: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Sam James <sam@gentoo.org> (cherry picked from commit bd4628f3f18ac312408782eea450429c6f044860)
2025-08-20x86-64: Add GLIBC_ABI_GNU2_TLS version [BZ #33129]H.J. Lu
Programs and shared libraries compiled with -mtls-dialect=gnu2 may fail silently at run-time against glibc without the GNU2 TLS run-time fix for: https://sourceware.org/bugzilla/show_bug.cgi?id=31372 Add GLIBC_ABI_GNU2_TLS version to indicate that glibc has the working GNU2 TLS run-time. Linker can add the GLIBC_ABI_GNU2_TLS version to binaries which depend on the working GNU2 TLS run-time: https://sourceware.org/bugzilla/show_bug.cgi?id=33130 so that such programs and shared libraries will fail to load and run at run-time against libc.so without the GLIBC_ABI_GNU2_TLS version, instead of fail silently at random. This fixes BZ #33129. Signed-off-by: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Sam James <sam@gentoo.org> (cherry picked from commit 9df8fa397d515dc86ff5565f6c45625e672d539e)
2025-08-20i386: Update ___tls_get_addr to preserve vector registersH.J. Lu
Compiler generates the following instruction sequence for dynamic TLS access: leal tls_var@tlsgd(,%ebx,1), %eax call ___tls_get_addr@PLT CALL instruction is transparent to compiler which assumes all registers, except for EFLAGS, AX, CX, and DX, are unchanged after CALL. But ___tls_get_addr is a normal function which doesn't preserve any vector registers. 1. Rename the generic __tls_get_addr function to ___tls_get_addr_internal. 2. Change ___tls_get_addr to a wrapper function with implementations for FNSAVE, FXSAVE, XSAVE and XSAVEC to save and restore all vector registers. 3. dl-tlsdesc-dynamic.h has: _dl_tlsdesc_dynamic: /* Like all TLS resolvers, preserve call-clobbered registers. We need two scratch regs anyway. */ subl $32, %esp cfi_adjust_cfa_offset (32) It is wrong to use movl %ebx, -28(%esp) movl %esp, %ebx cfi_def_cfa_register(%ebx) ... mov %ebx, %esp cfi_def_cfa_register(%esp) movl -28(%esp), %ebx to preserve EBX on stack. Fix it with: movl %ebx, 28(%esp) movl %esp, %ebx cfi_def_cfa_register(%ebx) ... mov %ebx, %esp cfi_def_cfa_register(%esp) movl 28(%esp), %ebx 4. Update _dl_tlsdesc_dynamic to call ___tls_get_addr_internal directly. 5. Add have-test-mtls-traditional to compile tst-tls23-mod.c with traditional TLS variant to verify the fix. 6. Define DL_RUNTIME_RESOLVE_REALIGN_STACK in sysdeps/x86/sysdep.h. This fixes BZ #32996. Co-Authored-By: Adhemerval Zanella <adhemerval.zanella@linaro.org> Signed-off-by: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit 848f0e46f03f22404ed9a8aabf3fd5ce8809a1be)
2025-08-20x86: Optimize xstate size calculationSunil K Pandey
Scan xstate IDs up to the maximum supported xstate ID. Remove the separate AMX xstate calculation. Instead, exclude the AMX space from the start of TILECFG to the end of TILEDATA in xsave_state_size. Completed validation on SKL/SKX/SPR/SDE and compared xsave state size with "ld.so --list-diagnostics" option, no regression. Co-Authored-By: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com> (cherry picked from commit 70b648855185e967e54668b101d24704c3fb869d)
2025-08-20x86: Link tst-gnu2-tls2-x86-noxsave{,c,xsavec} with libpthreadFlorian Weimer
This fixes a test build failure on Hurd. Fixes commit 145097dff170507fe73190e8e41194f5b5f7e6bf ("x86: Use separate variable for TLSDESC XSAVE/XSAVEC state size (bug 32810)"). Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit c6e2895695118ab59c7b17feb0fcb75a53e3478c) (cherry picked from commit 837a36c371f18a3152d032e8060f4e5120c25e2b)
2025-08-20x86: Use separate variable for TLSDESC XSAVE/XSAVEC state size (bug 32810)Florian Weimer
Previously, the initialization code reused the xsave_state_full_size member of struct cpu_features for the TLSDESC state size. However, the tunable processing code assumes that this member has the original XSAVE (non-compact) state size, so that it can use its value if XSAVEC is disabled via tunable. This change uses a separate variable and not a struct member because the value is only needed in ld.so and the static libc, but not in libc.so. As a result, struct cpu_features layout does not change, helping a future backport of this change. Fixes commit 9b7091415af47082664717210ac49d51551456ab ("x86-64: Update _dl_tlsdesc_dynamic to preserve AMX registers"). Reviewed-by: H.J. Lu <hjl.tools@gmail.com> (cherry picked from commit 145097dff170507fe73190e8e41194f5b5f7e6bf)
2025-08-20Apply the Makefile sorting fixH.J. Lu
Apply the Makefile sorting fix generated by sort-makefile-lines.py. (cherry picked from commit ef7f4b1fef67430a8f3cfc77fa6aada2add851d7)
2025-08-19x86-64: Allocate state buffer space for RDI, RSI and RBXH.J. Lu
_dl_tlsdesc_dynamic preserves RDI, RSI and RBX before realigning stack. After realigning stack, it saves RCX, RDX, R8, R9, R10 and R11. Define TLSDESC_CALL_REGISTER_SAVE_AREA to allocate space for RDI, RSI and RBX to avoid clobbering saved RDI, RSI and RBX values on stack by xsave to STATE_SAVE_OFFSET(%rsp). +==================+<- stack frame start aligned at 8 or 16 bytes | |<- RDI saved in the red zone | |<- RSI saved in the red zone | |<- RBX saved in the red zone | |<- paddings for stack realignment of 64 bytes |------------------|<- xsave buffer end aligned at 64 bytes | |<- | |<- | |<- |------------------|<- xsave buffer start at STATE_SAVE_OFFSET(%rsp) | |<- 8-byte padding for 64-byte alignment | |<- 8-byte padding for 64-byte alignment | |<- R11 | |<- R10 | |<- R9 | |<- R8 | |<- RDX | |<- RCX +==================+<- RSP aligned at 64 bytes Define TLSDESC_CALL_REGISTER_SAVE_AREA, the total register save area size for all integer registers by adding 24 to STATE_SAVE_OFFSET since RDI, RSI and RBX are saved onto stack without adjusting stack pointer first, using the red-zone. This fixes BZ #31501. Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com> (cherry picked from commit 717ebfa85c8240d32d0d19d86a484c31c55c9617)
2025-08-19x86-64: Update _dl_tlsdesc_dynamic to preserve AMX registersH.J. Lu
_dl_tlsdesc_dynamic should also preserve AMX registers which are caller-saved. Add X86_XSTATE_TILECFG_ID and X86_XSTATE_TILEDATA_ID to x86-64 TLSDESC_CALL_STATE_SAVE_MASK. Compute the AMX state size and save it in xsave_state_full_size which is only used by _dl_tlsdesc_dynamic_xsave and _dl_tlsdesc_dynamic_xsavec. This fixes the AMX part of BZ #31372. Tested on AMX processor. AMX test is enabled only for compilers with the fix for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114098 GCC 14 and GCC 11/12/13 branches have the bug fix. Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com> (cherry picked from commit 9b7091415af47082664717210ac49d51551456ab)
2025-08-19x86: Update _dl_tlsdesc_dynamic to preserve caller-saved registersH.J. Lu
Compiler generates the following instruction sequence for GNU2 dynamic TLS access: leaq tls_var@TLSDESC(%rip), %rax call *tls_var@TLSCALL(%rax) or leal tls_var@TLSDESC(%ebx), %eax call *tls_var@TLSCALL(%eax) CALL instruction is transparent to compiler which assumes all registers, except for EFLAGS and RAX/EAX, are unchanged after CALL. When _dl_tlsdesc_dynamic is called, it calls __tls_get_addr on the slow path. __tls_get_addr is a normal function which doesn't preserve any caller-saved registers. _dl_tlsdesc_dynamic saved and restored integer caller-saved registers, but didn't preserve any other caller-saved registers. Add _dl_tlsdesc_dynamic IFUNC functions for FNSAVE, FXSAVE, XSAVE and XSAVEC to save and restore all caller-saved registers. This fixes BZ #31372. Add GLRO(dl_x86_64_runtime_resolve) with GLRO(dl_x86_tlsdesc_dynamic) to optimize elf_machine_runtime_setup. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit 0aac205a814a8511e98d02b91a8dc908f1c53cde)
2025-08-19x32/cet: Support shadow stack during startup for Linux 6.10H.J. Lu
Use RXX_LP in RTLD_START_ENABLE_X86_FEATURES. Support shadow stack during startup for Linux 6.10: commit 2883f01ec37dd8668e7222dfdb5980c86fdfe277 Author: H.J. Lu <hjl.tools@gmail.com> Date: Fri Mar 15 07:04:33 2024 -0700 x86/shstk: Enable shadow stacks for x32 1. Add shadow stack support to x32 signal. 2. Use the 64-bit map_shadow_stack syscall for x32. 3. Set up shadow stack for x32. Add the map_shadow_stack system call to <fixup-asm-unistd.h> and regenerate arch-syscall.h. Tested on Intel Tiger Lake with CET enabled x32. There are no regressions with CET enabled x86-64. There are no changes in CET enabled x86-64 _dl_start_user. Signed-off-by: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit 8344c1f5514b1b5b1c8c6e48f4b802653bd23b71)
2025-08-19x86-64: Remove sysdeps/x86_64/x32/dl-machine.hH.J. Lu
Remove sysdeps/x86_64/x32/dl-machine.h by folding x32 ARCH_LA_PLTENTER, ARCH_LA_PLTEXIT and RTLD_START into sysdeps/x86_64/dl-machine.h. There are no regressions on x86-64 nor x32. There are no changes in x86-64 _dl_start_user. On x32, _dl_start_user changes are <_dl_start_user>: mov %eax,%r12d + mov %esp,%r13d mov (%rsp),%edx mov %edx,%esi - mov %esp,%r13d and $0xfffffff0,%esp mov 0x0(%rip),%edi # <_dl_start_user+0x14> lea 0x8(%r13,%rdx,4),%ecx Signed-off-by: H.J. Lu <hjl.tools@gmail.com> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit 652c6cf26927352fc0e37e4e60c6fc98ddf6d3b4)
2025-08-19x86/cet: fix shadow stack test scriptsMichael Jeanson
Some shadow stack test scripts use the '==' operator with the 'test' command to validate exit codes resulting in the following error: sysdeps/x86_64/tst-shstk-legacy-1e.sh: 31: test: 139: unexpected operator The '==' operator is invalid for the 'test' command, use '-eq' like the previous call to 'test'. Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-by: H.J. Lu <hjl.tools@gmail.com> (cherry picked from commit 155bb9d036646138348fee0ac045de601811e0c5)
2025-08-19x86-64/cet: Make CET feature check specific to Linux/x86H.J. Lu
CET feature bits in TCB, which are Linux specific, are used to check if CET features are active. Move CET feature check to Linux/x86 directory. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit f2b65a44714e8fa13c7637cd9413169590795879)
2025-08-19i386: Remove CET support bitsH.J. Lu
1. Remove _dl_runtime_resolve_shstk and _dl_runtime_profile_shstk. 2. Move CET offsets from x86 cpu-features-offsets.sym to x86-64 features-offsets.sym. 3. Rename x86 cet-control.h to x86-64 feature-control.h since it is only for x86-64 and also used for PLT rewrite. 4. Add x86-64 ldsodefs.h to include feature-control.h. 5. Change TUNABLE_CALLBACK (set_plt_rewrite) to x86-64 only. 6. Move x86 dl-procruntime.c to x86-64. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit 874214db624a8e6c5d2dbe47419fab126f330d68)
2025-08-19x86-64/cet: Move check-cet.awk to x86_64H.J. Lu
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit 7d544dd049a2e3f1480b668f51b72dcc89e376ab)
2025-08-19x86: Move CET infrastructure to x86_64Adhemerval Zanella
The CET is only supported for x86_64 and there is no plan to add kernel support for i386. Move the Makefile rules and files from the generic x86 folder to x86_64 one. Checked on x86_64-linux-gnu and i686-linux-gnu. (cherry picked from commit b7fc4a07f206a640e6d807d72f5c1ee3ea7a25b6)
2025-08-19x86-64/cet: Move dl-cet.[ch] to x86_64 directoriesH.J. Lu
Since CET is only enabled for x86-64, move dl-cet.[ch] to x86_64 directories. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit a1bbee9fd17a84d4b550f8405d5e4d31ff24f87d)
2025-08-19i386: Fail if configured with --enable-cetAdhemerval Zanella
Since it is only supported for x86_64. Checked on i686-linux-gnu. (cherry picked from commit a0cfc48e8a67506e3f0b2d3ea5e04b45408b3683)
2025-08-19x86-64/cet: Check the restore token in longjmpH.J. Lu
setcontext and swapcontext put a restore token on the old shadow stack which is used to restore the target shadow stack when switching user contexts. When longjmp from a user context, the target shadow stack can be different from the current shadow stack and INCSSP can't be used to restore the shadow stack pointer to the target shadow stack. Update longjmp to search for a restore token. If found, use the token to restore the shadow stack pointer before using INCSSP to pop the shadow stack. Stop the token search and use INCSSP if the shadow stack entry value is the same as the current shadow stack pointer. It is a user error if there is a shadow stack switch without leaving a restore token on the old shadow stack. The only difference between __longjmp.S and __longjmp_chk.S is that __longjmp_chk.S has a check for invalid longjmp usages. Merge __longjmp.S and __longjmp_chk.S by adding the CHECK_INVALID_LONGJMP macro. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit 35694d3416b273ac19d67ffa49b7969f36684ae1)
2025-08-19i386: Ignore --enable-cetH.J. Lu
Since shadow stack is only supported for x86-64, ignore --enable-cet for i386. Always setting $(enable-cet) for i386 to "no" to support ifneq ($(enable-cet),no) in x86 Makefiles. We can't use ifeq ($(enable-cet),yes) since $(enable-cet) can be "yes", "no" or "permissive". Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit bbfb54930cdd85269504a34b362e77a3ac2a207a)
2025-08-19x86/cet: Add -fcf-protection=none before -fcf-protection=branchH.J. Lu
When shadow stack is enabled, some CET tests failed when compiled with GCC 14: FAIL: elf/tst-cet-legacy-4 FAIL: elf/tst-cet-legacy-5a FAIL: elf/tst-cet-legacy-6a which are caused by https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113039 These tests use -fcf-protection -fcf-protection=branch and assume that -fcf-protection=branch will override -fcf-protection. But this GCC 14 commit: https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=1c6231c05bdcca changed the -fcf-protection behavior such that -fcf-protection -fcf-protection=branch is treated the same as -fcf-protection Use -fcf-protection -fcf-protection=none -fcf-protection=branch as the workaround. This fixes BZ #31187. Tested with GCC 13 and GCC 14 on Intel Tiger Lake. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit b5dcccfb12385ee492eb074f6beb9ead56b5e5fd)
2025-08-19x86/cet: Run some CET tests with shadow stackH.J. Lu
When CET is disabled by default, run some CET tests with shadow stack enabled using $ export GLIBC_TUNABLES=glibc.cpu.hwcaps=SHSTK (cherry picked from commit cf9481724bcb86ad4a86cca7befed74bb9cc15eb)
2025-08-19x86/cet: Don't set CET active by defaultH.J. Lu
Not all CET enabled applications and libraries have been properly tested in CET enabled environments. Some CET enabled applications or libraries will crash or misbehave when CET is enabled. Don't set CET active by default so that all applications and libraries will run normally regardless of whether CET is active or not. Shadow stack can be enabled by $ export GLIBC_TUNABLES=glibc.cpu.hwcaps=SHSTK at run-time if shadow stack can be enabled by kernel. NB: This commit can be reverted if it is OK to enable CET by default for all applications and libraries. (cherry picked from commit 55d63e731253de82e96ed4ddca2e294076cd0bc5)
2025-08-19x86/cet: Check feature_1 in TCB for active IBT and SHSTKH.J. Lu
Initially, IBT and SHSTK are marked as active when CPU supports them and CET are enabled in glibc. They can be disabled early by tunables before relocation. Since after relocation, GLRO(dl_x86_cpu_features) becomes read-only, we can't update GLRO(dl_x86_cpu_features) to mark IBT and SHSTK as inactive. Instead, check the feature_1 field in TCB to decide if IBT and SHST are active. (cherry picked from commit d360dcc001cb12504cd3e8dbddee20df6bebb0f8)
2025-08-19x86/cet: Enable shadow stack during startupH.J. Lu
Previously, CET was enabled by kernel before passing control to user space and the startup code must disable CET if applications or shared libraries aren't CET enabled. Since the current kernel only supports shadow stack and won't enable shadow stack before passing control to user space, we need to enable shadow stack during startup if the application and all shared library are shadow stack enabled. There is no need to disable shadow stack at startup. Shadow stack can only be enabled in a function which will never return. Otherwise, shadow stack will underflow at the function return. 1. GL(dl_x86_feature_1) is set to the CET features which are supported by the processor and are not disabled by the tunable. Only non-zero features in GL(dl_x86_feature_1) should be enabled. After enabling shadow stack with ARCH_SHSTK_ENABLE, ARCH_SHSTK_STATUS is used to check if shadow stack is really enabled. 2. Use ARCH_SHSTK_ENABLE in RTLD_START in dynamic executable. It is safe since RTLD_START never returns. 3. Call arch_prctl (ARCH_SHSTK_ENABLE) from ARCH_SETUP_TLS in static executable. Since the start function using ARCH_SETUP_TLS never returns, it is safe to enable shadow stack in ARCH_SETUP_TLS. (cherry picked from commit 541641a3de8d89464151bd879552755e882c832e)
2025-08-19x86/cet: Sync with Linux kernel 6.6 shadow stack interfaceH.J. Lu
Sync with Linux kernel 6.6 shadow stack interface. Since only x86-64 is supported, i386 shadow stack codes are unchanged and CET shouldn't be enabled for i386. 1. When the shadow stack base in TCB is unset, the default shadow stack is in use. Use the current shadow stack pointer as the marker for the default shadow stack. It is used to identify if the current shadow stack is the same as the target shadow stack when switching ucontexts. If yes, INCSSP will be used to unwind shadow stack. Otherwise, shadow stack restore token will be used. 2. Allocate shadow stack with the map_shadow_stack syscall. Since there is no function to explicitly release ucontext, there is no place to release shadow stack allocated by map_shadow_stack in ucontext functions. Such shadow stacks will be leaked. 3. Rename arch_prctl CET commands to ARCH_SHSTK_XXX. 4. Rewrite the CET control functions with the current kernel shadow stack interface. Since CET is no longer enabled by kernel, a separate patch will enable shadow stack during startup. (cherry picked from commit edb5e0c8f915a798629717b5680a852c8bb3db25)
2025-08-19x86/cet: Don't disable CET if not single threadedH.J. Lu
In permissive mode, don't disable IBT nor SHSTK when dlopening a legacy shared library if not single threaded since IBT and SHSTK may be still enabled in other threads. Other threads with IBT or SHSTK enabled will crash when calling functions in the legacy shared library. Instead, an error will be issued. (cherry picked from commit 41560a9312ce0ec7203480eef8f865076bff9edb)
2025-08-19x86: Modularize sysdeps/x86/dl-cet.cH.J. Lu
Improve readability and make maintenance easier for dl-feature.c by modularizing sysdeps/x86/dl-cet.c: 1. Support processors with: a. Only IBT. Or b. Only SHSTK. Or c. Both IBT and SHSTK. 2. Lock CET features only if IBT or SHSTK are enabled and are not enabled permissively. (cherry picked from commit c04035809a393c0c6f1cc523df6b316b05fdb50f)
2025-08-19x86/cet: Update tst-cet-vfork-1H.J. Lu
Change tst-cet-vfork-1.c to verify that vfork child return triggers SIGSEGV due to shadow stack mismatch. (cherry picked from commit 1a23b39f9d2caeca72dc12adbbcb5d2d632d942a)
2025-08-19x86/cet: Check CPU_FEATURE_ACTIVE in permissive modeH.J. Lu
Verify that CPU_FEATURE_ACTIVE works properly in permissive mode. (cherry picked from commit 4d8a01d2b0963f7c7714ff53c313430599f0722f)
2025-08-19x86/cet: Check legacy shadow stack code in .init_array sectionH.J. Lu
Verify that legacy shadow stack code in .init_array section in application and shared library, which are marked as shadow stack enabled, will trigger segfault. (cherry picked from commit 28bd6f832d4c8ec9a223c153427c1ab6fd19a548)
2025-08-19x86/cet: Add tests for GLIBC_TUNABLES=glibc.cpu.hwcaps=-SHSTKH.J. Lu
Verify that GLIBC_TUNABLES=glibc.cpu.hwcaps=-SHSTK turns off shadow stack properly. (cherry picked from commit 9424ce80c2a08f4dfc06d5442b770ed5ec798c4b)
2025-08-19x86/cet: Check CPU_FEATURE_ACTIVE when CET is disabledH.J. Lu
Verify that CPU_FEATURE_ACTIVE (SHSTK) works properly when CET is disabled. (cherry picked from commit 71c0cc3357fe6d72f1dbef1c695e54b117d91b96)
2025-08-19x86/cet: Check legacy shadow stack applicationsH.J. Lu
Add tests to verify that legacy shadow stack applications run properly when shadow stack is enabled in Linux kernel. (cherry picked from commit f418fe6f973300c4c61461ed241928cba11017c2)
2025-08-19x86/cet: Don't assume that SHSTK implies IBTH.J. Lu
Since shadow stack (SHSTK) is enabled in the Linux kernel without enabling indirect branch tracking (IBT), don't assume that SHSTK implies IBT. Use "CPU_FEATURE_ACTIVE (IBT)" to check if IBT is active and "CPU_FEATURE_ACTIVE (SHSTK)" to check if SHSTK is active. (cherry picked from commit 442983319ba70de801fc856e8dd4748fba8f7f1b)
2025-08-19x86/cet: Check user_shstk in /proc/cpuinfoH.J. Lu
Linux kernel reports CPU shadow stack feature in /proc/cpuinfo as user_shstk, instead of shstk. (cherry picked from commit 0b850186fd3177311f10dcb938b668cc750fa3be)
2025-08-19Update syscall lists for Linux 6.7Joseph Myers
Linux 6.7 adds the futex_requeue, futex_wait and futex_wake syscalls, and enables map_shadow_stack for architectures previously missing it. Update syscall-names.list and regenerate the arch-syscall.h headers with build-many-glibcs.py update-syscalls. Tested with build-many-glibcs.py. (cherry picked from commit df11c05be91fda5ef490c76fd0d4a53821750116)
2025-08-19Update syscall lists for Linux 6.6Adhemerval Zanella
Linux 6.6 has one new syscall for all architectures, fchmodat2, and the map_shadow_stack on x86_64. (cherry picked from commit 582383b37d95b133c1ee6855ffaa2b1f5cb3d3b8)
2025-08-19Remove installed header rule on $(..)include/%.hH.J. Lu
On x86-64 machine with [hjl@gnu-cfl-3 x86-glibc]$ ls -l /usr/include/asm/prctl.h sysdeps/unix/sysv/linux/x86_64/include/asm/prctl.h -rw-r--r-- 1 hjl hjl 825 Jan 9 09:41 sysdeps/unix/sysv/linux/x86_64/include/asm/prctl.h -rw-r--r-- 1 root root 1170 Nov 27 16:00 /usr/include/asm/prctl.h [hjl@gnu-cfl-3 x86-glibc]$ glibc configured with --enable-cet build failed: make[2]: Entering directory '/export/gnu/import/git/gitlab/x86-glibc/iconv' ../Makerules:327: update target '/export/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/gnu/lib-names-64.h' due to: /export/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/gnu/lib-names-64.stmp : ../Makeconfig:1216: update target '/export/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/libc-modules.h' due to: /export/build/gnu/tools-build/glibc-cet-gitlab/build-x86_64-linux/libc-modules.stmp : ../Makerules:1126: update target '/usr/include/asm/prctl.h' due to: ../sysdeps/unix/sysv/linux/x86_64/64/../include/asm/prctl.h force-install /usr/bin/install -c -m 644 ../sysdeps/unix/sysv/linux/x86_64/64/../include/asm/prctl.h /usr/include/asm/prctl.h /usr/bin/install: cannot remove '/usr/include/asm/prctl.h': Permission denied make[2]: *** [../Makerules:1126: /usr/include/asm/prctl.h] Error 1 make[2]: Leaving directory '/export/gnu/import/git/gitlab/x86-glibc/iconv' make[1]: *** [Makefile:484: iconv/subdir_lib] Error 2 make[1]: Leaving directory '/export/gnu/import/git/gitlab/x86-glibc' make: *** [Makefile:9: all] Error 2 This is triggered by the rule in Makerules: $(inst_includedir)/%.h: $(..)include/%.h $(+force) $(do-install) Since no files under include/ should be installed, remove it from Makerules. Tested it on x86-64. There are no differences in the installed header files. (cherry picked from commit 1eae989cb7632760fd6f4008be73549da861b202)
2025-08-19debug: Fix tst-longjmp_chk3 build failure on HurdFlorian Weimer
Explicitly include <unistd.h> for _exit and getpid. (cherry picked from commit 4836a9af89f1b4d482e6c72ff67e36226d36434c)
2025-08-19debug: Wire up tst-longjmp_chk3Florian Weimer
The test was added in commit ac8cc9e300a002228eb7e660df3e7b333d9a7414 without all the required Makefile scaffolding. Tweak the test so that it actually builds (including with dynamic SIGSTKSZ). Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> (cherry picked from commit 4b7cfcc3fbfab55a1bbb32a2da69c048060739d6)
2025-08-19debug: Adapt fortify tests to libsupportAdhemerval Zanella
Checked on aarch64, armhf, x86_64, and i686. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org> (cherry picked from commit 9556acd249687ac562deb6309503165d66eb06fa)
2025-08-19x86-64: Save APX registers in ld.so trampolineH.J. Lu
Add APX registers to STATE_SAVE_MASK so that APX registers are saved in ld.so trampoline. This fixes BZ #31371. Also update STATE_SAVE_OFFSET and STATE_SAVE_MASK for i386 which will be used by i386 _dl_tlsdesc_dynamic. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com> (cherry picked from commit dfb05f8e704edac70db38c4c8ee700769d91a413)
generated by cgit v1.2.3 (git 2.25.1) at 2026-05-31 10:14:14 +0000