glibc.git/malloc, branch zack/no-nested-includes

Don’t include signal.h from sys/wait.h or sys/param.h.

2020-01-08T18:45:44+00:00

Besides the snarl of debugger/ucontext interfaces, these are the only
public headers that include signal.h.

sys/wait.h includes signal.h only for the definition of siginfo_t.
We already have a single-type header for that, so use it.  siginfo_t
contains a field whose type is uid_t, but sys/wait.h is not specified
to define uid_t, so, as is already done for pid_t, the conformance
test is modified to expect that field to have type __uid_t instead.

It is not clear what subset of the definitions from signal.h are
actually expected by historical users of sys/param.h; I’ve chosen to
take the comment at face value and cut it down to bits/signum.h, which
supplies _NSIG and all of the SIG* constants.  This requires adjusting
every copy of bits/signum.h to permit inclusion by sys/param.h as well
as signal.h.

While I was at it I moved the comment about sys/param.h being obsolete
from sysdeps/mach/hurd/bits/param.h, where it’s not likely to be seen,
to the top-level sys/param.h, and edited it to give more useful advice.

This patch partially fixes Hurd-specific bug 23088; sys/wait.h is now
conformant.

	* posix/sys/wait.h: Include bits/types/siginfo_t.h, not signal.h.
	* conform/data/sys/wait.h-data: Do not expect a definition of uid_t.

	* malloc/tst-mallocfork.c, nptl/tst-fork4.c, nptl/tst-getpid3.c
	* nptl/tst-mutex9.c, nptl/tst-rwlock12.c
	* resolv/tst-resolv-res_init-skeleton.c
	* sysdeps/unix/sysv/linux/s390/tst-ptrace-singleblock.c:
        Include signal.h.

	* nptl/tst-cancel4.c, rt/tst-mqueue1.c
	* support/tst-support_capture_subprocess.c
	* sysdeps/unix/sysv/linux/tst-align-clone.c:
	Include signal.h.  Sort includes.

	* misc/sys/param.h: Include bits/signum.h, not signal.h.
	Add comment explaining that this header is obsolete, based on
	a similar comment in Hurd bits/param.h.

	* bits/param.h: Add multiple inclusion guard and defensive #error.
	* sysdeps/mach/hurd/bits/param.h: Add multiple inclusion guard.
	Remove comment explaining that this header is obsolete (see above).
	* sysdeps/mach/i386/bits/mach/param.h: Add multiple inclusion guard.
	* sysdeps/unix/sysv/linux/bits/param.h: Add multiple inclusion guard.

	* bits/signum-generic.h, bits/signum.h
	* sysdeps/unix/bsd/bits/signum.h
	* sysdeps/unix/sysv/linux/alpha/bits/signum.h
	* sysdeps/unix/sysv/linux/bits/signum.h
	* sysdeps/unix/sysv/linux/hppa/bits/signum.h
	* sysdeps/unix/sysv/linux/mips/bits/signum.h:
	Allow inclusion by sys/param.h as well as signal.h.

	* scripts/check-obsolete-constructs.py (HEADER_ALLOWED_INCLUDES):
	Update.

        * sysdeps/mach/hurd/i386/Makefile: Remove XFAILs for sys/wait.h.

Add bits/types/ wrappers for stddef.h and stdarg.h types.

2020-01-08T18:41:32+00:00

We rely on the compiler's stddef.h and stdarg.h to define size_t,
ptrdiff_t, wchar_t, NULL, and __gnuc_va_list, and to implement a
convention that allows us to request the definition of a specific one:
for instance

    #define __need_size_t
    #include 

is expected to define size_t but not any of the other things stddef.h
defines.

This patch hides that convention behind a set of bits/types/ headers,
which allows check-obsolete-constructs.py to verify that none of our
headers include these headers unconditionally.  (Both of them define
at least one item in the user namespace that no other header is
supposed to expose.)  It will also facilitate coping with compilers
that don’t implement the __need convention.  (That scenario is not
hypothetical, see the next patch.)

Only public headers use the new bits headers.  Non-public headers and
.c files in our codebase, that were formerly defining __need macros,
now just include stddef.h and/or stdarg.h without any __need macros.
A few files didn’t need to be including stddef.h / stdarg.h at all.

Uses of NULL in public headers that aren’t expected to define NULL
are changed to a bare 0.  bits/NULL.h is only used by headers that
are expected to define NULL.

malloc.h and printf.h were, in fact, including stddef.h and/or
stdarg.h unconditionally; they no longer do that.  This broke a few of
our test cases, which are fixed by adding appropriate inclusions to
the relevant .c files.

	* stdlib/bits/NULL.h
	* stdlib/bits/types/__va_list.h
	* stdlib/bits/types/ptrdiff_t.h
	* stdlib/bits/types/size_t.h
	* stdlib/bits/types/va_list.h
	* stdlib/bits/types/wchar_t.h:
	New headers defining a single type or macro each.
	* stdlib/Makefile: Install new headers.
	* include/bits/NULL.h
	* include/bits/types/__va_list.h
	* include/bits/types/ptrdiff_t.h
	* include/bits/types/size_t.h
	* include/bits/types/va_list.h
	* include/bits/types/wchar_t.h:
	New wrapper headers.

	* malloc/malloc.h: Don’t include stdio.h or stddef.h.
	Include bits/NULL.h, bits/types/size_t.h, bits/types/ptrdiff_t.h,
	and bits/types/FILE.h.
	* stdio-common/printf.h: Don’t include stddef.h or stdarg.h.
	Include bits/types/size_t.h, bits/types/wchar_t.h, and
	bits/types/__va_list.h.  Use __gnuc_va_list instead of va_list
	in prototypes.

	* libio/bits/types/struct_FILE.h: Include bits/types/size_t.h.
	* misc/sys/param.h: Include features.h.

	* sysvipc/sys/msg.h: Include bits/msq.h after all bits/types/ headers.
	* sysvipc/sys/sem.h: Include bits/sem.h after all bits/types/ headers.
	* sysvipc/sys/shm.h: Include bits/shm.h after all bits/types/ headers.

	* hurd/hurd/signal.h: Don’t use NULL.
	* hurd/hurd/ioctl.h: Don’t include stdarg.h.
	* hurd/hurd/userlink.h: Don’t include stddef.h.  Don’t use NULL.
	* intl/libintl.h: Don’t include stddef.h.  Don’t use NULL.

	* intl/gettext.c, intl/ngettext.c: Include stddef.h
	unconditionally.  Don’t define any __need macros first.
	Don’t include stdlib.h.

	* sysdeps/posix/sigignore.c:, sysdeps/posix/sigset.c:
	Don’t include errno.h or string.h.

	* malloc/tst-malloc-thread-fail.c: Include stddef.h.
	* malloc/tst-malloc_info.c: Include stdio.h.
	* stdio-common/tst-vfprintf-user-type.c: Include stdarg.h.
	* string/tst-cmp.c: Include stdio.h.

	* debug/wcpcpy_chk.c, iconv/loop.c, iconv/skeleton.c
	* signal/sighold.c, signal/sigrelse.c, stdio-common/tempname.c
	* sysdeps/generic/ldsodefs.h, sysdeps/nptl/libc-lock.h
	* sysdeps/nptl/libc-lockP.h, sysdeps/posix/waitid.c
	* wcsmbs/wcstol_l.c, wcsmbs/wcstoll_l.c, wcsmbs/wcstoul_l.c
	* wcsmbs/wcstoull_l.c, sysdeps/posix/sigignore.c
	* sysdeps/posix/sigset.c: Don’t define __need macros before
	including stddef.h.

	* bits/socket.h, bits/types/stack_t.h, dirent/dirent.h
	* dlfcn/dlfcn.h, gmon/sys/profil.h, grp/grp.h, gshadow/gshadow.h
        * hurd/hurd/signal.h, hurd/hurd/sigpreempt.h, iconv/gconv.h
        * include/set-hooks.h, include/stdio.h, inet/aliases.h
        * io/sys/sendfile.h, libio/stdio.h, misc/bits/types/struct_iovec.h
	* misc/search.h, misc/sys/mman.h, misc/syslog.h, posix/glob.h
	* posix/sched.h, posix/sys/types.h, posix/unistd.h
	* posix/wordexp.h, pwd/pwd.h, shadow/shadow.h, signal/signal.h
        * socket/sys/socket.h, stdlib/alloca.h, stdlib/monetary.h
	* stdlib/stdlib.h, stdlib/sys/random.h, string/string.h
	* string/strings.h, sunrpc/rpc/netdb.h
	* sysdeps/htl/bits/types/struct___pthread_attr.h
	* sysdeps/mach/hurd/bits/socket.h
	* sysdeps/unix/sysv/linux/bits/socket.h
	* sysdeps/unix/sysv/linux/bits/types/stack_t.h
	* sysdeps/unix/sysv/linux/ia64/bits/sigcontext.h
	* sysdeps/unix/sysv/linux/mips/bits/types/stack_t.h
	* sysdeps/unix/sysv/linux/scsi/sg.h
	* sysdeps/unix/sysv/linux/sys/sysctl.h
	* sysvipc/sys/msg.h, sysvipc/sys/sem.h, sysvipc/sys/shm.h
        * time/time.h, wcsmbs/uchar.h, wcsmbs/wchar.h:
	Use bits/types/size_t.h instead of __need_size_t.

	* iconv/gconv.h, iconv/iconv.h, libio/libio.h
	* stdlib/inttypes.h, stdlib/stdlib.h, wcsmbs/wchar.h:
	Use bits/types/wchar_t.h instead of __need_wchar_t.

	* libio/stdio.h, locale/locale.h, misc/sys/param.h
	* posix/sched.h, posix/unistd.h, stdlib/stdlib.h
	* string/string.h, sysdeps/unix/sysv/linux/bits/sigcontext.h
	* time/time.h, wcsmbs/wchar.h: Use bits/NULL.h instead of __need_NULL.

	* libio/stdio.h, misc/err.h: Use bits/types/__va_list.h instead
	of __need___va_list.
	* libio/stdio.h: Use bits/types/va_list.h instead of manually
	defining va_list.

	* hurd/hurd/userlink.h, misc/sys/mman.h, posix/sched.h
	* sysdeps/mach/hurd/bits/socket.h
	* sysdeps/unix/sysv/linux/ia64/bits/sigcontext.h
	* wcsmbs/wchar.h: Reorganize includes; no semantic effect.
	* stdlib/stdlib.h: Normalize format of multiple include guard.
	* sysdeps/unix/sysv/linux/bits/sigcontext.h: Annotate workarounds
	for kernel header bugs.

	* sysdeps/unix/sysv/linux/aarch64/sys/user.h
	* sysdeps/unix/sysv/linux/arm/sys/user.h
	* sysdeps/unix/sysv/linux/m68k/sys/user.h
	* sysdeps/unix/sysv/linux/microblaze/sys/user.h
	* sysdeps/unix/sysv/linux/nios2/sys/user.h
	* sysdeps/unix/sysv/linux/s390/sys/user.h
	* sysdeps/unix/sysv/linux/x86/sys/user.h
	Include features.h.

	* sysdeps/unix/sysv/linux/alpha/sys/user.h
	* sysdeps/unix/sysv/linux/ia64/sys/user.h
	* sysdeps/unix/sysv/linux/mips/sys/user.h
	* sysdeps/unix/sysv/linux/powerpc/sys/user.h
	* sysdeps/unix/sysv/linux/sh/sys/user.h
	* sysdeps/unix/sysv/linux/sparc/sys/user.h
	Include features.h and bits/types/size_t.h, in that order.
	Include kernel headers, if any, after those two.
	Don’t include stddef.h or sys/types.h.

	* scripts/check-obsolete-constructs.py
	(UNIVERSAL_ALLOWED_INCLUDES): Remove stddef.h and stdarg.h.
	(HEADER_ALLOWED_INCLUDES): Update.

elf: Move vDSO setup to rtld (BZ#24967)

2020-01-03T14:22:07+00:00

This patch moves the vDSO setup from libc to loader code, just after
the vDSO link_map setup.  For static case the initialization
is moved to _dl_non_dynamic_init instead.

Instead of using the mangled pointer, the vDSO data is set as
attribute_relro (on _rtld_global_ro for shared or _dl_vdso_* for
static).  It is read-only even with partial relro.

It fixes BZ#24967 now that the vDSO pointer is setup earlier than
malloc interposition is called.

Also, vDSO calls should not be a problem for static dlopen as
indicated by BZ#20802.  The vDSO pointer would be zero-initialized
and the syscall will be issued instead.

Checked on x86_64-linux-gnu, i686-linux-gnu, aarch64-linux-gnu,
arm-linux-gnueabihf, powerpc64le-linux-gnu, powerpc64-linux-gnu,
powerpc-linux-gnu, s390x-linux-gnu, sparc64-linux-gnu, and
sparcv9-linux-gnu.  I also run some tests on mips.

Reviewed-by: Siddhesh Poyarekar

Update copyright dates not handled by scripts/update-copyrights.

2020-01-01T00:21:22+00:00

I've updated copyright dates in glibc for 2020.  This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.  As well as the usual annual
updates, mainly dates in --version output (minus libc.texinfo which
previously had to be handled manually but is now successfully updated
by update-copyrights), there is a fix to
sysdeps/unix/sysv/linux/powerpc/bits/termios-c_lflag.h where a typo in
the copyright notice meant it failed to be updated automatically.

Please remember to include 2020 in the dates for any new files added
in future (which means updating any existing uncommitted patches you
have that add new files to use the new copyright dates in them).

Update copyright dates with scripts/update-copyrights.

2020-01-01T00:14:33+00:00

Correct range checking in mallopt/mxfast/tcache [BZ #25194]

2019-12-05T21:46:37+00:00

do_set_tcache_max, do_set_mxfast:
Fix two instances of comparing "size_t < 0"
Both cases have upper limit, so the "negative value" case
is already handled via overflow semantics.

do_set_tcache_max, do_set_tcache_count:
Fix return value on error.  Note: currently not used.

mallopt:
pass return value of helper functions to user.  Behavior should
only be actually changed for mxfast, where we restore the old
(pre-tunables) behavior.

Reviewed-by: Carlos O'Donell

Base max_fast on alignment, not width, of bins (Bug 24903)

2019-10-31T02:59:11+00:00

set_max_fast sets the "impossibly small" value based on,
eventually, MALLOC_ALIGNMENT.  The comparisons for the smallest
chunk used is, eventually, MIN_CHUNK_SIZE.  Note that i386
is the only platform where these are the same, so a smallest
chunk *would* be put in a no-fastbins fastbin.

This change calculates the "impossibly small" value
based on MIN_CHUNK_SIZE instead, so that we can know it will
always be impossibly small.

Prefer https to http for gnu.org and fsf.org URLs

2019-09-07T09:43:31+00:00

Also, change sources.redhat.com to sourceware.org.
This patch was automatically generated by running the following shell
script, which uses GNU sed, and which avoids modifying files imported
from upstream:

sed -ri '
  s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g
  s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g
' \
  $(find $(git ls-files) -prune -type f \
      ! -name '*.po' \
      ! -name 'ChangeLog*' \
      ! -path COPYING ! -path COPYING.LIB \
      ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \
      ! -path manual/texinfo.tex ! -path scripts/config.guess \
      ! -path scripts/config.sub ! -path scripts/install-sh \
      ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \
      ! -path INSTALL ! -path  locale/programs/charmap-kw.h \
      ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \
      ! '(' -name configure \
            -execdir test -f configure.ac -o -f configure.in ';' ')' \
      ! '(' -name preconfigure \
            -execdir test -f preconfigure.ac ';' ')' \
      -print)

and then by running 'make dist-prepare' to regenerate files built
from the altered files, and then executing the following to cleanup:

  chmod a+x sysdeps/unix/sysv/linux/riscv/configure
  # Omit irrelevant whitespace and comment-only changes,
  # perhaps from a slightly-different Autoconf version.
  git checkout -f \
    sysdeps/csky/configure \
    sysdeps/hppa/configure \
    sysdeps/riscv/configure \
    sysdeps/unix/sysv/linux/csky/configure
  # Omit changes that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines
  git checkout -f \
    sysdeps/powerpc/powerpc64/ppc-mcount.S \
    sysdeps/unix/sysv/linux/s390/s390-64/syscall.S
  # Omit change that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline
  git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S

malloc: Various cleanups for malloc/tst-mxfast

2019-08-15T09:37:18+00:00

Add glibc.malloc.mxfast tunable

2019-08-09T18:04:03+00:00

* elf/dl-tunables.list: Add glibc.malloc.mxfast.
* manual/tunables.texi: Document it.
* malloc/malloc.c (do_set_mxfast): New.
(__libc_mallopt): Call it.
* malloc/arena.c: Add mxfast tunable.
* malloc/tst-mxfast.c: New.
* malloc/Makefile: Add it.

Reviewed-by: Carlos O'Donell