We rely on the compiler's stddef.h and stdarg.h to define size_t,
ptrdiff_t, wchar_t, NULL, and __gnuc_va_list, and to implement a
convention that allows us to request the definition of a specific one:
for instance

    #define __need_size_t
    #include <stddef.h>

is expected to define size_t but not any of the other things stddef.h
defines.

This patch hides that convention behind a set of bits/types/ headers,
which allows check-obsolete-constructs.py to verify that none of our
headers include these headers unconditionally.  (Both of them define
at least one item in the user namespace that no other header is
supposed to expose.)  It will also facilitate coping with compilers
that don’t implement the __need convention.  (That scenario is not
hypothetical, see the next patch.)

Only public headers use the new bits headers.  Non-public headers and
.c files in our codebase, that were formerly defining __need macros,
now just include stddef.h and/or stdarg.h without any __need macros.
A few files didn’t need to be including stddef.h / stdarg.h at all.

Uses of NULL in public headers that aren’t expected to define NULL
are changed to a bare 0.  bits/NULL.h is only used by headers that
are expected to define NULL.

malloc.h and printf.h were, in fact, including stddef.h and/or
stdarg.h unconditionally; they no longer do that.  This broke a few of
our test cases, which are fixed by adding appropriate inclusions to
the relevant .c files.

	* stdlib/bits/NULL.h
	* stdlib/bits/types/__va_list.h
	* stdlib/bits/types/ptrdiff_t.h
	* stdlib/bits/types/size_t.h
	* stdlib/bits/types/va_list.h
	* stdlib/bits/types/wchar_t.h:
	New headers defining a single type or macro each.
	* stdlib/Makefile: Install new headers.
	* include/bits/NULL.h
	* include/bits/types/__va_list.h
	* include/bits/types/ptrdiff_t.h
	* include/bits/types/size_t.h
	* include/bits/types/va_list.h
	* include/bits/types/wchar_t.h:
	New wrapper headers.

	* malloc/malloc.h: Don’t include stdio.h or stddef.h.
	Include bits/NULL.h, bits/types/size_t.h, bits/types/ptrdiff_t.h,
	and bits/types/FILE.h.
	* stdio-common/printf.h: Don’t include stddef.h or stdarg.h.
	Include bits/types/size_t.h, bits/types/wchar_t.h, and
	bits/types/__va_list.h.  Use __gnuc_va_list instead of va_list
	in prototypes.

	* libio/bits/types/struct_FILE.h: Include bits/types/size_t.h.
	* misc/sys/param.h: Include features.h.

	* sysvipc/sys/msg.h: Include bits/msq.h after all bits/types/ headers.
	* sysvipc/sys/sem.h: Include bits/sem.h after all bits/types/ headers.
	* sysvipc/sys/shm.h: Include bits/shm.h after all bits/types/ headers.

	* hurd/hurd/signal.h: Don’t use NULL.
	* hurd/hurd/ioctl.h: Don’t include stdarg.h.
	* hurd/hurd/userlink.h: Don’t include stddef.h.  Don’t use NULL.
	* intl/libintl.h: Don’t include stddef.h.  Don’t use NULL.

	* intl/gettext.c, intl/ngettext.c: Include stddef.h
	unconditionally.  Don’t define any __need macros first.
	Don’t include stdlib.h.

	* sysdeps/posix/sigignore.c:, sysdeps/posix/sigset.c:
	Don’t include errno.h or string.h.

	* malloc/tst-malloc-thread-fail.c: Include stddef.h.
	* malloc/tst-malloc_info.c: Include stdio.h.
	* stdio-common/tst-vfprintf-user-type.c: Include stdarg.h.
	* string/tst-cmp.c: Include stdio.h.

	* debug/wcpcpy_chk.c, iconv/loop.c, iconv/skeleton.c
	* signal/sighold.c, signal/sigrelse.c, stdio-common/tempname.c
	* sysdeps/generic/ldsodefs.h, sysdeps/nptl/libc-lock.h
	* sysdeps/nptl/libc-lockP.h, sysdeps/posix/waitid.c
	* wcsmbs/wcstol_l.c, wcsmbs/wcstoll_l.c, wcsmbs/wcstoul_l.c
	* wcsmbs/wcstoull_l.c, sysdeps/posix/sigignore.c
	* sysdeps/posix/sigset.c: Don’t define __need macros before
	including stddef.h.

	* bits/socket.h, bits/types/stack_t.h, dirent/dirent.h
	* dlfcn/dlfcn.h, gmon/sys/profil.h, grp/grp.h, gshadow/gshadow.h
        * hurd/hurd/signal.h, hurd/hurd/sigpreempt.h, iconv/gconv.h
        * include/set-hooks.h, include/stdio.h, inet/aliases.h
        * io/sys/sendfile.h, libio/stdio.h, misc/bits/types/struct_iovec.h
	* misc/search.h, misc/sys/mman.h, misc/syslog.h, posix/glob.h
	* posix/sched.h, posix/sys/types.h, posix/unistd.h
	* posix/wordexp.h, pwd/pwd.h, shadow/shadow.h, signal/signal.h
        * socket/sys/socket.h, stdlib/alloca.h, stdlib/monetary.h
	* stdlib/stdlib.h, stdlib/sys/random.h, string/string.h
	* string/strings.h, sunrpc/rpc/netdb.h
	* sysdeps/htl/bits/types/struct___pthread_attr.h
	* sysdeps/mach/hurd/bits/socket.h
	* sysdeps/unix/sysv/linux/bits/socket.h
	* sysdeps/unix/sysv/linux/bits/types/stack_t.h
	* sysdeps/unix/sysv/linux/ia64/bits/sigcontext.h
	* sysdeps/unix/sysv/linux/mips/bits/types/stack_t.h
	* sysdeps/unix/sysv/linux/scsi/sg.h
	* sysdeps/unix/sysv/linux/sys/sysctl.h
	* sysvipc/sys/msg.h, sysvipc/sys/sem.h, sysvipc/sys/shm.h
        * time/time.h, wcsmbs/uchar.h, wcsmbs/wchar.h:
	Use bits/types/size_t.h instead of __need_size_t.

	* iconv/gconv.h, iconv/iconv.h, libio/libio.h
	* stdlib/inttypes.h, stdlib/stdlib.h, wcsmbs/wchar.h:
	Use bits/types/wchar_t.h instead of __need_wchar_t.

	* libio/stdio.h, locale/locale.h, misc/sys/param.h
	* posix/sched.h, posix/unistd.h, stdlib/stdlib.h
	* string/string.h, sysdeps/unix/sysv/linux/bits/sigcontext.h
	* time/time.h, wcsmbs/wchar.h: Use bits/NULL.h instead of __need_NULL.

	* libio/stdio.h, misc/err.h: Use bits/types/__va_list.h instead
	of __need___va_list.
	* libio/stdio.h: Use bits/types/va_list.h instead of manually
	defining va_list.

	* hurd/hurd/userlink.h, misc/sys/mman.h, posix/sched.h
	* sysdeps/mach/hurd/bits/socket.h
	* sysdeps/unix/sysv/linux/ia64/bits/sigcontext.h
	* wcsmbs/wchar.h: Reorganize includes; no semantic effect.
	* stdlib/stdlib.h: Normalize format of multiple include guard.
	* sysdeps/unix/sysv/linux/bits/sigcontext.h: Annotate workarounds
	for kernel header bugs.

	* sysdeps/unix/sysv/linux/aarch64/sys/user.h
	* sysdeps/unix/sysv/linux/arm/sys/user.h
	* sysdeps/unix/sysv/linux/m68k/sys/user.h
	* sysdeps/unix/sysv/linux/microblaze/sys/user.h
	* sysdeps/unix/sysv/linux/nios2/sys/user.h
	* sysdeps/unix/sysv/linux/s390/sys/user.h
	* sysdeps/unix/sysv/linux/x86/sys/user.h
	Include features.h.

	* sysdeps/unix/sysv/linux/alpha/sys/user.h
	* sysdeps/unix/sysv/linux/ia64/sys/user.h
	* sysdeps/unix/sysv/linux/mips/sys/user.h
	* sysdeps/unix/sysv/linux/powerpc/sys/user.h
	* sysdeps/unix/sysv/linux/sh/sys/user.h
	* sysdeps/unix/sysv/linux/sparc/sys/user.h
	Include features.h and bits/types/size_t.h, in that order.
	Include kernel headers, if any, after those two.
	Don’t include stddef.h or sys/types.h.

	* scripts/check-obsolete-constructs.py
	(UNIVERSAL_ALLOWED_INCLUDES): Remove stddef.h and stdarg.h.
	(HEADER_ALLOWED_INCLUDES): Update.

Also, change sources.redhat.com to sourceware.org.
This patch was automatically generated by running the following shell
script, which uses GNU sed, and which avoids modifying files imported
from upstream:

sed -ri '
  s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g
  s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g
' \
  $(find $(git ls-files) -prune -type f \
      ! -name '*.po' \
      ! -name 'ChangeLog*' \
      ! -path COPYING ! -path COPYING.LIB \
      ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \
      ! -path manual/texinfo.tex ! -path scripts/config.guess \
      ! -path scripts/config.sub ! -path scripts/install-sh \
      ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \
      ! -path INSTALL ! -path  locale/programs/charmap-kw.h \
      ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \
      ! '(' -name configure \
            -execdir test -f configure.ac -o -f configure.in ';' ')' \
      ! '(' -name preconfigure \
            -execdir test -f preconfigure.ac ';' ')' \
      -print)

and then by running 'make dist-prepare' to regenerate files built
from the altered files, and then executing the following to cleanup:

  chmod a+x sysdeps/unix/sysv/linux/riscv/configure
  # Omit irrelevant whitespace and comment-only changes,
  # perhaps from a slightly-different Autoconf version.
  git checkout -f \
    sysdeps/csky/configure \
    sysdeps/hppa/configure \
    sysdeps/riscv/configure \
    sysdeps/unix/sysv/linux/csky/configure
  # Omit changes that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines
  git checkout -f \
    sysdeps/powerpc/powerpc64/ppc-mcount.S \
    sysdeps/unix/sysv/linux/s390/s390-64/syscall.S
  # Omit change that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline
  git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S

The function was never part of the malloc API.

As discussed previously on libc-alpha [1], this patch follows up the idea
and add both the __attribute_alloc_size__ on malloc functions (malloc,
calloc, realloc, reallocarray, valloc, pvalloc, and memalign) and limit
maximum requested allocation size to up PTRDIFF_MAX (taking into
consideration internal padding and alignment).

This aligns glibc with gcc expected size defined by default warning
-Walloc-size-larger-than value which warns for allocation larger than
PTRDIFF_MAX.  It also aligns with gcc expectation regarding libc and
expected size, such as described in PR#67999 [2] and previously discussed
ISO C11 issues [3] on libc-alpha.

From the RFC thread [4] and previous discussion, it seems that consensus
is only to limit such requested size for malloc functions, not the system
allocation one (mmap, sbrk, etc.).

The implementation changes checked_request2size to check for both overflow
and maximum object size up to PTRDIFF_MAX. No additional checks are done
on sysmalloc, so it can still issue mmap with values larger than
PTRDIFF_T depending on the requested size.

The __attribute_alloc_size__ is for functions that return a pointer only,
which means it cannot be applied to posix_memalign (see remarks in GCC
PR#87683 [5]). The runtimes checks to limit maximum requested allocation
size does applies to posix_memalign.

Checked on x86_64-linux-gnu and i686-linux-gnu.

[1] https://sourceware.org/ml/libc-alpha/2018-11/msg00223.html
[2] https://gcc.gnu.org/bugzilla//show_bug.cgi?id=67999
[3] https://sourceware.org/ml/libc-alpha/2011-12/msg00066.html
[4] https://sourceware.org/ml/libc-alpha/2018-11/msg00224.html
[5] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87683

	[BZ #23741]
	* malloc/hooks.c (malloc_check, realloc_check): Use
	__builtin_add_overflow on overflow check and adapt to
	checked_request2size change.
	* malloc/malloc.c (__libc_malloc, __libc_realloc, _mid_memalign,
	__libc_pvalloc, __libc_calloc, _int_memalign): Limit maximum
	allocation size to PTRDIFF_MAX.
	(REQUEST_OUT_OF_RANGE): Remove macro.
	(checked_request2size): Change to inline function and limit maximum
	requested size to PTRDIFF_MAX.
	(__libc_malloc, __libc_realloc, _int_malloc, _int_memalign): Limit
	maximum allocation size to PTRDIFF_MAX.
	(_mid_memalign): Use _int_memalign call for overflow check.
	(__libc_pvalloc): Use __builtin_add_overflow on overflow check.
	(__libc_calloc): Use __builtin_mul_overflow for overflow check and
	limit maximum requested size to PTRDIFF_MAX.
	* malloc/malloc.h (malloc, calloc, realloc, reallocarray, memalign,
	valloc, pvalloc): Add __attribute_alloc_size__.
	* stdlib/stdlib.h (malloc, realloc, reallocarray, valloc): Likewise.
	* malloc/tst-malloc-too-large.c (do_test): Add check for allocation
	larger than PTRDIFF_MAX.
	* malloc/tst-memalign.c (do_test): Disable -Walloc-size-larger-than=
	around tests of malloc with negative sizes.
	* malloc/tst-posix_memalign.c (do_test): Likewise.
	* malloc/tst-pvalloc.c (do_test): Likewise.
	* malloc/tst-valloc.c (do_test): Likewise.
	* malloc/tst-reallocarray.c (do_test): Replace call to reallocarray
	with resulting size allocation larger than PTRDIFF_MAX with
	reallocarray_nowarn.
	(reallocarray_nowarn): New function.
	* NEWS: Mention the malloc function semantic change.

	* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.

	* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.

The reallocarray function is an extension from OpenBSD.  It is an
integer-overflow-safe replacement for realloc(p, X*Y) and
malloc(X*Y) (realloc(NULL, X*Y)).  It can therefore help in preventing
certain security issues in code.

This is an updated version of a patch originally submitted by Rüdiger
Sonderfeld in May 2014 [1].

Checked on i686-linux-gnu and x86_64-linux-gnu.

[1] <https://sourceware.org/ml/libc-alpha/2014-05/msg00481.html>.

2017-05-30  Dennis Wölfing  <denniswoelfing@gmx.de>
            Rüdiger Sonderfeld  <ruediger@c-plusplus.de>

	* include/stdlib.h (__libc_reallocarray): New declaration.
	* malloc/Makefile (routines): Add reallocarray.
	(tests): Add tst-reallocarray.c.
	* malloc/Versions: Add reallocarray and __libc_reallocarray.
	* malloc/malloc-internal.h (check_mul_overflow_size_t): New inline
	function.
	* malloc/malloc.h (reallocarray): New declaration.
	* stdlib/stdlib.h (reallocarray): Likewise.
	* malloc/reallocarray.c: New file.
	* malloc/tst-reallocarray.c: New test file.
	* manual/memory.texi: Document reallocarray.
	* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Add reallocarray.
	* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
	Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist:
	Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/tilepro/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.