It is permissible to define macros prior to including a PCH, as long as
these definitions are disjoint from or identical to the macros in the
PCH. The PCH loading process replaces all libcpp data structures with those
from the PCH, so it is necessary to remember the extra macros separately and
then restore them after loading the PCH, which all is handled by
cpp_save_state() and cpp_read_state() in libcpp/pch.cc. The restoration
process consists of pushing a buffer containing the macro definition and
then lexing it from there, similar to how a command-line -D option is
processed. The current implementation does not attempt to set up the
line_map for this process, and so the locations assigned to the macros are
often not meaningful. (Similar to what happened in the past with lexing the
tokens out of a _Pragma string, lexing out of a buffer rather than a file
produces "sorta" reasonable locations that are often close enough, but not
reliably correct.)
Fix that up by remembering enough additional information (more or less, an
expanded_location for each macro definition) to produce a reasonable
location for the newly restored macros.
One issue that came up is the treatment of command-line-defined macros. From
the perspective of the generic line_map data structures, the command-line
location is not distinguishable from other locations; it's just an ordinary
location created by the front ends with a fake file name by convention. (At
the moment, it is always the string `<command-line>', subject to
translation.) Since libcpp needs to assign macros to that location, it
needs to know what location to use, so I added a new member
line_maps::cmdline_location for the front ends to set, similar to how
line_maps::builtin_location is handled.
This revealed a small issue, in c-opts.cc we have:
/* All command line defines must have the same location. */
cpp_force_token_locations (parse_in, line_table->highest_line);
But contrary to the comment, all command line defines don't actually end up
with the same location anymore. This is because libcpp/lex.cc has been
expanded (r6-4873) to include range information on the returned
locations. That logic has never been respecting the request of
cpp_force_token_locations. I believe this was not intentional, and so I have
corrected that here. Prior to this patch, the range logic has been leading
to command-line macros all having similar locations in the same line map (or
ad-hoc locations based from there for sufficiently long tokens); with this
change, they all have exactly the same location and that location is
recorded in line_maps::cmdline_location.
With that change, then it works fine for pch.cc to restore macros whether
they came from the command-line or from the main file.
gcc/c-family/ChangeLog:
PR preprocessor/105608
* c-opts.cc (c_finish_options): Set new member
line_table->cmdline_location.
* c-pch.cc (c_common_read_pch): Adapt linemap usage to changes in
libcpp pch.cc; it is now possible that the linemap is in a different
file after returning from cpp_read_state().
libcpp/ChangeLog:
PR preprocessor/105608
* include/line-map.h: Add new member CMDLINE_LOCATION.
* lex.cc (get_location_for_byte_range_in_cur_line): Do not expand
the token location to include range information if token location
override was requested.
(warn_about_normalization): Likewise.
(_cpp_lex_direct): Likewise.
* pch.cc (struct saved_macro): New local struct.
(struct save_macro_data): Change DEFNS vector to hold saved_macro
rather than uchar*.
(save_macros): Adapt to remember the location information for each
saved macro in addition to the definition.
(cpp_prepare_state): Likewise.
(cpp_read_state): Use the saved location information to generate
proper locations for the restored macros.
gcc/testsuite/ChangeLog:
PR preprocessor/105608
* g++.dg/pch/line-map-3.C: Remove xfails.
* g++.dg/pch/line-map-4.C: New test.
* g++.dg/pch/line-map-4.Hs: New test.
My changes for "Module Declarations Shouldn’t be Macros" paper broke the following testcase. The backup handling intentionally tries to drop CPP_PRAGMA_EOL token if things go wrong, which is desirable for the case where we haven't committed to the module preprocessing directive (i.e. changed the first token to the magic one). In that case there is no preprocessing directive start and so CPP_PRAGMA_EOL would be wrong. If there is a premature new-line after we've changed the first token though, we shouldn't drop CPP_PRAGMA_EOL, because otherwise we ICE in the FE. While clang++ and MSVC accept the testcase, in my reading it is incorrect at least in the C++23 and newer wordings and I think the changes have been a DR, https://eel.is/c++draft/cpp.module has no exception for new-lines and https://eel.is/c++draft/cpp.pre#1.sentence-2 says that new-line (unless deleted during phase 2 when after backslash) ends the preprocessing directive. The patch arranges for eol being set only in the not_module case. 2025-08-03 Jakub Jelinek <jakub@redhat.com> PR c++/120845 libcpp/ * lex.cc (cpp_maybe_module_directive): Move eol variable declaration to the start of the function, initialize to false and only set it to peek->type == CPP_PRAGMA_EOL in the not_module case. Formatting fix. gcc/testsuite/ * g++.dg/modules/cpp-21.C: New test.
Aaron mentioned in the PR that late in C23 N3124 was adopted and $@` are now part of basic character set. The paper has been implemented in GCC from what I can see, but we should allow for GNU23/2Y $@` in raw string delimiters as well, like they are allowed for C++26, because the delimiters can contain anything from basic character set but space, ()\, tab, form-feed, newline and backspace. 2025-04-24 Jakub Jelinek <jakub@redhat.com> PR c++/110343 * lex.cc (lex_raw_string): For C allow $@` in raw string delimiters if CPP_OPTION (pfile, low_ucns) i.e. for C23 and later. * gcc.dg/raw-string-1.c: New test.
The PR shows that we ICE after lexing an invalid unterminated raw string, because lex_raw_string() pops the main buffer unexpectedly. Resolve by handling this case the same way as for other directives. libcpp/ChangeLog: PR preprocessor/117118 * lex.cc (lex_raw_string): Treat an unterminated raw string the same way for a deferred pragma as is done for other directives. gcc/testsuite/ChangeLog: PR preprocessor/117118 * c-c++-common/raw-string-directive-3.c: New test. * c-c++-common/raw-string-directive-4.c: New test.
libcpp makes use of the cpp_buffer pfile->a_buff to store things while it is handling macros. It uses it to store pointers (cpp_hashnode*, for macro arguments) and cpp_macro objects. This works fine because a cpp_hashnode* and a cpp_macro have the same alignment requirement on either 32-bit or 64-bit systems (namely, the same alignment as a pointer.) When 64-bit location_t is enabled on a 32-bit sytem, the alignment requirement may cease to be the same, because the alignment requirement of a cpp_macro object changes to that of a uint64_t, which be larger than that of a pointer. It's not the case for x86 32-bit, but for example, on sparc, a pointer has 4-byte alignment while a uint64_t has 8. In that case, intermixing the two within the same cpp_buffer leads to a misaligned access. The code path that triggers this is the one in _cpp_commit_buff in which a hash table with its own allocator (i.e. ggc) is not being used, so it doesn't happen within the compiler itself, but it happens in the other libcpp clients, such as genmatch. Fix that up by ensuring _cpp_commit_buff commits a fully aligned chunk of the buffer, so it's ready for anything it may be used for next. Also modify CPP_ALIGN so that it guarantees to return an alignment at least the size of location_t. Currently it returns the max of a pointer and a double. I am not aware of any platform where a double may have smaller alignment than a uint64_t, but it does not hurt to add location_t here to be sure. libcpp/ChangeLog: * lex.cc (_cpp_commit_buff): Make sure that the buffer is properly aligned for the next allocation. * internal.h (struct dummy): Make sure alignment is large enough for a location_t, just in case.
This is an attempt to implement the https://wg21.link/p3034r1 paper, but I'm afraid the wording in the paper is bad for multiple reasons. I think I understand the intent, that the module name and partition if any shouldn't come from macros so that they can be scanned for without preprocessing, but on the other side doesn't want to disable macro expansion in pp-module altogether, because e.g. the optional attribute in module-declaration would be nice to come from macros as which exact attribute is needed might need to be decided based on preprocessor checks. The paper added https://eel.is/c++draft/cpp.module#2 which uses partly the wording from https://eel.is/c++draft/cpp.module#1 The first issue I see is that using that "defined as an object-like macro" from there means IMHO something very different in those 2 paragraphs. As per https://eel.is/c++draft/cpp.pre#7.sentence-1 preprocessing tokens in preprocessing directives aren't subject to macro expansion unless otherwise stated, and so the export and module tokens aren't expanded and so the requirement that they aren't defined as an object-like macro makes perfect sense. The problem with the new paragraph is that https://eel.is/c++draft/cpp.module#3.sentence-1 says that the rest of the tokens are macro expanded and after macro expansion none of the tokens can be defined as an object-like macro, if they would be, they'd be expanded to that. So, I think either the wording needs to change such that not all preprocessing tokens after module are macro expanded, only those which are after the pp-module-name and if any pp-module-partition tokens, or all tokens after module are macro expanded but none of the tokens in pp-module-name and pp-module-partition if any must come from macro expansion. The patch below implements it as if the former would be specified (but see later), so essentially scans the preprocessing tokens after module without expansion, if the first one is an identifier, it disables expansion for it and then if followed by . or : expects another such identifier (again with disabled expansion), but stops after second : is seen. Second issue is that while the global-module-fragment start is fine, matches the syntax of the new paragraph where the pp-tokens[opt] aren't present, there is also private-module-fragment in the syntax where module is followed by : private ; and in that case the colon doesn't match the pp-module-name grammar and appears now to be invalid. I think the https://eel.is/c++draft/cpp.module#2 paragraph needs to change so that it allows also that pp-tokens of a pp-module may also be : pp-tokens[opt] (and in that case, I think the colon shouldn't come from a macro and private and/or ; can). Third issue is that there are too many pp-tokens in https://eel.is/c++draft/cpp.module , one is all the tokens between module keyword and the semicolon and one is the optional extra tokens after pp-module-partition (if any, if missing, after pp-module). Perhaps introducing some other non-terminal would help talking about it? So in "where the pp-tokens (if any) shall not begin with a ( preprocessing token" it isn't obvious which pp-tokens it is talking about (my assumption is the latter) and also whether ( can't appear there just before macro expansion or also after expansion. The patch expects only before expansion, so #define F (); export module foo F would be valid during preprocessing but obviously invalid during compilation, but #define foo(n) n; export module foo (3) would be invalid already during preprocessing. The last issue applies only if the first issue is resolved to allow expansion of tokens after : if first token, or after pp-module-partition if present or after pp-module-name if present. When non-preprocessing scanner sees export module foo.bar:baz.qux; it knows nothing can come from preprocessing macros and is ok, but if it sees export module foo.bar:baz qux then it can't know whether it will be export module foo.bar:baz; or export module foo.bar:baz [[]]; or export module foo.bar:baz.freddy.garply; because qux could be validly a macro, which expands to ; or [[]]; or .freddy.garply; etc. So, either the non-preprocessing scanner would need to note it as possible export of foo.bar:baz* module partitions and preprocess if it needs to know the details or just compile, or if that is not ok, the wording would need to rule out that the expansion of (the second) pp-tokens if any can't start with . or : (colon would be only problematic if it isn't present in the tokens before it already). So, if e.g. defining qux above to . whatever is invalid, then the scanner can rely it sees the whole module name and partition. The patch below implements what is above described as the first variant of the first issue resolution, i.e. disables expansion of as many tokens as could be in the valid module name and module partition syntax, but as soon as it e.g. sees two adjacent identifiers, the second one can be macro expanded. If it is macro expanded though, the expansion can't start with . or :, and if it expands to nothing, tokens after it (whether they come from macro expansion or not) can't start with . or :. So, effectively: #define SEMI ; export module SEMI used to be valid and isn't anymore, #define FOO bar export module FOO; isn't valid, #define COLON : export module COLON private; isn't valid, #define BAR baz export module foo.bar:baz.qux.BAR; isn't valid, #define BAZ .qux export module foo BAZ; isn't valid, #define FREDDY :garply export module foo FREDDY; isn't valid, while #define QUX [[]] export module foo QUX; or #define GARPLY private module : GARPLY; etc. is. 2024-11-01 Jakub Jelinek <jakub@redhat.com> PR c++/114461 libcpp/ * include/cpplib.h: Implement C++26 P3034R1 - Module Declarations Shouldn’t be Macros (or more precisely its expected intent). (NO_DOT_COLON): Define. * internal.h (struct cpp_reader): Add diagnose_dot_colon_from_macro_p member. * lex.cc (cpp_maybe_module_directive): For pp-module, if module keyword is followed by CPP_NAME, ensure all CPP_NAME tokens possibly matching module name and module partition syntax aren't expanded and aren't defined as object-like macros. Verify first token after that doesn't start with open paren. If the next token after module name/partition is CPP_NAME defined as macro, set NO_DOT_COLON flag on it. * macro.cc (cpp_get_token_1): Set pfile->diagnose_dot_colon_from_macro_p if token to be expanded has NO_DOT_COLON bit set in flags. Before returning, if pfile->diagnose_dot_colon_from_macro_p is true and not returning CPP_PADDING or CPP_COMMENT and not during macro expansion preparation, set pfile->diagnose_dot_colon_from_macro_p to false and diagnose if returning CPP_DOT or CPP_COLON. gcc/testsuite/ * g++.dg/modules/cpp-7.C: New test. * g++.dg/modules/cpp-8.C: New test. * g++.dg/modules/cpp-9.C: New test. * g++.dg/modules/cpp-10.C: New test. * g++.dg/modules/cpp-11.C: New test. * g++.dg/modules/cpp-12.C: New test. * g++.dg/modules/cpp-13.C: New test. * g++.dg/modules/cpp-14.C: New test. * g++.dg/modules/cpp-15.C: New test. * g++.dg/modules/cpp-16.C: New test. * g++.dg/modules/cpp-17.C: New test. * g++.dg/modules/cpp-18.C: New test. * g++.dg/modules/cpp-19.C: New test. * g++.dg/modules/cpp-20.C: New test. * g++.dg/modules/pmp-4.C: New test. * g++.dg/modules/pmp-5.C: New test. * g++.dg/modules/pmp-6.C: New test. * g++.dg/modules/token-6.C: New test. * g++.dg/modules/token-7.C: New test. * g++.dg/modules/token-8.C: New test. * g++.dg/modules/token-9.C: New test. * g++.dg/modules/token-10.C: New test. * g++.dg/modules/token-11.C: New test. * g++.dg/modules/token-12.C: New test. * g++.dg/modules/token-13.C: New test. * g++.dg/modules/token-14.C: New test. * g++.dg/modules/token-15.C: New test. * g++.dg/modules/token-16.C: New test. * g++.dg/modules/dir-only-3.C: Expect an error. * g++.dg/modules/dir-only-4.C: Expect an error. * g++.dg/modules/dir-only-5.C: New test. * g++.dg/modules/atom-preamble-2_a.C: In export module malcolm; replace malcolm with kevin. Don't define malcolm macro. * g++.dg/modules/atom-preamble-4.C: Expect an error. * g++.dg/modules/atom-preamble-5.C: New test.
I've tried to build stage3 with -Wleading-whitespace=blanks -Wtrailing-whitespace=blank -Wno-error=leading-whitespace=blanks -Wno-error=trailing-whitespace=blank added to STRICT_WARN and that expectably resulted in about 2744 unique trailing whitespace warnings and 124837 leading whitespace warnings when excluding *.md files (which obviously is in big part a generator issue). Others from that are generator related, I think those need to be solved later. The following patch just fixes up the easy case (trailing whitespace), which could be easily automated: for i in `find . -name \*.h -o -name \*.cc -o -name \*.c | xargs grep -l '[ ]$' | grep -v testsuite/`; do sed -i -e 's/[ ]*$//' $i; done I've excluded files which I knew are obviously generated or go FE. Is there anything else we'd want to avoid the changes? Due to patch size, I've split it between gcc/ part and rest (include/, libiberty/, libgcc/, libcpp/, libstdc++-v3/; this part). 2024-10-24 Jakub Jelinek <jakub@redhat.com> include/ * dyn-string.h: Remove trailing whitespace. * libiberty.h: Likewise. * xregex.h: Likewise. * splay-tree.h: Likewise. * partition.h: Likewise. * plugin-api.h: Likewise. * demangle.h: Likewise. * vtv-change-permission.h: Likewise. * fibheap.h: Likewise. * hsa_ext_image.h: Likewise. * hashtab.h: Likewise. * libcollector.h: Likewise. * sort.h: Likewise. * symcat.h: Likewise. * hsa_ext_amd.h: Likewise. libcpp/ * directives.cc: Remove trailing whitespace. * mkdeps.cc: Likewise. * line-map.cc: Likewise. * internal.h: Likewise. * files.cc: Likewise. * init.cc: Likewise. * makeucnid.cc: Likewise. * system.h: Likewise. * include/line-map.h: Likewise. * include/symtab.h: Likewise. * include/cpplib.h: Likewise. * expr.cc: Likewise. * charset.cc: Likewise. * macro.cc: Likewise. * errors.cc: Likewise. * lex.cc: Likewise. * traditional.cc: Likewise. libgcc/ * crtstuff.c: Remove trailing whitespace. * libgcov.h: Likewise. * config/alpha/crtfastmath.c: Likewise. * config/alpha/vms-gcc_shell_handler.c: Likewise. * config/alpha/vms-unwind.h: Likewise. * config/pa/linux-atomic.c: Likewise. * config/pa/linux-unwind.h: Likewise. * config/pa/quadlib.c: Likewise. * config/pa/fptr.c: Likewise. * config/s390/32/_fixsfdi.c: Likewise. * config/s390/32/_fixunssfdi.c: Likewise. * config/s390/32/_fixunsdfdi.c: Likewise. * config/c6x/pr-support.c: Likewise. * config/lm32/_udivsi3.c: Likewise. * config/lm32/libgcc_lm32.h: Likewise. * config/lm32/_udivmodsi4.c: Likewise. * config/lm32/_mulsi3.c: Likewise. * config/lm32/_modsi3.c: Likewise. * config/lm32/_umodsi3.c: Likewise. * config/lm32/_divsi3.c: Likewise. * config/darwin-crt3.c: Likewise. * config/msp430/mpy.c: Likewise. * config/ia64/tf-signs.c: Likewise. * config/ia64/fde-vms.c: Likewise. * config/ia64/unwind-ia64.c: Likewise. * config/ia64/vms-unwind.h: Likewise. * config/ia64/sfp-exceptions.c: Likewise. * config/ia64/quadlib.c: Likewise. * config/ia64/unwind-ia64.h: Likewise. * config/rl78/vregs.h: Likewise. * config/arm/bpabi.c: Likewise. * config/arm/unwind-arm.c: Likewise. * config/arm/pr-support.c: Likewise. * config/arm/linux-atomic.c: Likewise. * config/arm/bpabi-lib.h: Likewise. * config/frv/frvend.c: Likewise. * config/frv/cmovw.c: Likewise. * config/frv/frvbegin.c: Likewise. * config/frv/cmovd.c: Likewise. * config/frv/cmovh.c: Likewise. * config/aarch64/cpuinfo.c: Likewise. * config/i386/crtfastmath.c: Likewise. * config/i386/cygming-crtend.c: Likewise. * config/i386/32/tf-signs.c: Likewise. * config/i386/crtprec.c: Likewise. * config/i386/sfp-exceptions.c: Likewise. * config/i386/w32-unwind.h: Likewise. * config/m32r/initfini.c: Likewise. * config/sparc/crtfastmath.c: Likewise. * config/gcn/amdgcn_veclib.h: Likewise. * config/nios2/linux-atomic.c: Likewise. * config/nios2/linux-unwind.h: Likewise. * config/nios2/lib2-mul.c: Likewise. * config/nios2/lib2-nios2.h: Likewise. * config/xtensa/unwind-dw2-xtensa.c: Likewise. * config/rs6000/darwin-fallback.c: Likewise. * config/rs6000/ibm-ldouble.c: Likewise. * config/rs6000/sfp-machine.h: Likewise. * config/rs6000/darwin-asm.h: Likewise. * config/rs6000/darwin-crt2.c: Likewise. * config/rs6000/aix-unwind.h: Likewise. * config/rs6000/sfp-exceptions.c: Likewise. * config/gthr-vxworks.c: Likewise. * config/riscv/atomic.c: Likewise. * config/visium/memcpy.c: Likewise. * config/darwin-crt-tm.c: Likewise. * config/stormy16/lib2funcs.c: Likewise. * config/arc/ieee-754/divtab-arc-sf.c: Likewise. * config/arc/ieee-754/divtab-arc-df.c: Likewise. * config/arc/initfini.c: Likewise. * config/sol2/gmon.c: Likewise. * config/microblaze/divsi3_table.c: Likewise. * config/m68k/fpgnulib.c: Likewise. * libgcov-driver.c: Likewise. * unwind-dw2.c: Likewise. * fp-bit.c: Likewise. * dfp-bit.h: Likewise. * dfp-bit.c: Likewise. * libgcov-driver-system.c: Likewise. libgcc/config/libbid/ * _le_td.c: Remove trailing whitespace. * bid128_compare.c: Likewise. * bid_div_macros.h: Likewise. * bid64_to_bid128.c: Likewise. * bid64_to_uint32.c: Likewise. * bid128_to_uint64.c: Likewise. * bid64_div.c: Likewise. * bid128_round_integral.c: Likewise. * bid_binarydecimal.c: Likewise. * bid128_string.c: Likewise. * bid_flag_operations.c: Likewise. * bid128_to_int64.c: Likewise. * _mul_sd.c: Likewise. * bid64_mul.c: Likewise. * bid128_noncomp.c: Likewise. * _gt_dd.c: Likewise. * bid64_add.c: Likewise. * bid64_string.c: Likewise. * bid_from_int.c: Likewise. * bid128.c: Likewise. * _ge_dd.c: Likewise. * _ne_sd.c: Likewise. * _dd_to_td.c: Likewise. * _unord_sd.c: Likewise. * bid64_to_uint64.c: Likewise. * _gt_sd.c: Likewise. * _sd_to_td.c: Likewise. * _addsub_td.c: Likewise. * _ne_td.c: Likewise. * bid_dpd.c: Likewise. * bid128_add.c: Likewise. * bid128_next.c: Likewise. * _lt_sd.c: Likewise. * bid64_next.c: Likewise. * bid128_mul.c: Likewise. * _lt_dd.c: Likewise. * _ge_td.c: Likewise. * _unord_dd.c: Likewise. * bid64_sqrt.c: Likewise. * bid_sqrt_macros.h: Likewise. * bid64_fma.c: Likewise. * _sd_to_dd.c: Likewise. * bid_conf.h: Likewise. * bid64_noncomp.c: Likewise. * bid_gcc_intrinsics.h: Likewise. * _gt_td.c: Likewise. * _ge_sd.c: Likewise. * bid128_minmax.c: Likewise. * bid128_quantize.c: Likewise. * bid32_to_bid64.c: Likewise. * bid_round.c: Likewise. * _td_to_sd.c: Likewise. * bid_inline_add.h: Likewise. * bid128_fma.c: Likewise. * _eq_td.c: Likewise. * bid32_to_bid128.c: Likewise. * bid64_rem.c: Likewise. * bid128_2_str_tables.c: Likewise. * _mul_dd.c: Likewise. * _dd_to_sd.c: Likewise. * bid128_div.c: Likewise. * _lt_td.c: Likewise. * bid64_compare.c: Likewise. * bid64_to_int32.c: Likewise. * _unord_td.c: Likewise. * bid128_rem.c: Likewise. * bid_internal.h: Likewise. * bid64_to_int64.c: Likewise. * _eq_dd.c: Likewise. * _td_to_dd.c: Likewise. * bid128_to_int32.c: Likewise. * bid128_to_uint32.c: Likewise. * _ne_dd.c: Likewise. * bid64_quantize.c: Likewise. * _le_dd.c: Likewise. * bid64_round_integral.c: Likewise. * _le_sd.c: Likewise. * bid64_minmax.c: Likewise. libgcc/config/avr/libf7/ * f7-renames.h: Remove trailing whitespace. libstdc++-v3/ * include/debug/debug.h: Remove trailing whitespace. * include/parallel/base.h: Likewise. * include/parallel/types.h: Likewise. * include/parallel/settings.h: Likewise. * include/parallel/multiseq_selection.h: Likewise. * include/parallel/partition.h: Likewise. * include/parallel/random_number.h: Likewise. * include/parallel/find_selectors.h: Likewise. * include/parallel/partial_sum.h: Likewise. * include/parallel/list_partition.h: Likewise. * include/parallel/search.h: Likewise. * include/parallel/algorithmfwd.h: Likewise. * include/parallel/random_shuffle.h: Likewise. * include/parallel/multiway_mergesort.h: Likewise. * include/parallel/sort.h: Likewise. * include/parallel/algobase.h: Likewise. * include/parallel/numericfwd.h: Likewise. * include/parallel/multiway_merge.h: Likewise. * include/parallel/losertree.h: Likewise. * include/bits/basic_ios.h: Likewise. * include/bits/stringfwd.h: Likewise. * include/bits/ostream_insert.h: Likewise. * include/bits/stl_heap.h: Likewise. * include/bits/unordered_map.h: Likewise. * include/bits/hashtable_policy.h: Likewise. * include/bits/stl_iterator_base_funcs.h: Likewise. * include/bits/valarray_before.h: Likewise. * include/bits/regex.h: Likewise. * include/bits/postypes.h: Likewise. * include/bits/stl_iterator.h: Likewise. * include/bits/localefwd.h: Likewise. * include/bits/stl_algo.h: Likewise. * include/bits/ios_base.h: Likewise. * include/bits/stl_function.h: Likewise. * include/bits/basic_string.h: Likewise. * include/bits/hashtable.h: Likewise. * include/bits/valarray_after.h: Likewise. * include/bits/char_traits.h: Likewise. * include/bits/gslice.h: Likewise. * include/bits/locale_facets_nonio.h: Likewise. * include/bits/mask_array.h: Likewise. * include/bits/specfun.h: Likewise. * include/bits/random.h: Likewise. * include/bits/slice_array.h: Likewise. * include/bits/valarray_array.h: Likewise. * include/tr1/float.h: Likewise. * include/tr1/functional_hash.h: Likewise. * include/tr1/math.h: Likewise. * include/tr1/hashtable_policy.h: Likewise. * include/tr1/stdio.h: Likewise. * include/tr1/complex.h: Likewise. * include/tr1/stdbool.h: Likewise. * include/tr1/stdarg.h: Likewise. * include/tr1/inttypes.h: Likewise. * include/tr1/fenv.h: Likewise. * include/tr1/stdlib.h: Likewise. * include/tr1/wchar.h: Likewise. * include/tr1/tgmath.h: Likewise. * include/tr1/limits.h: Likewise. * include/tr1/wctype.h: Likewise. * include/tr1/stdint.h: Likewise. * include/tr1/ctype.h: Likewise. * include/tr1/random.h: Likewise. * include/tr1/shared_ptr.h: Likewise. * include/ext/mt_allocator.h: Likewise. * include/ext/sso_string_base.h: Likewise. * include/ext/debug_allocator.h: Likewise. * include/ext/vstring_fwd.h: Likewise. * include/ext/pointer.h: Likewise. * include/ext/pod_char_traits.h: Likewise. * include/ext/malloc_allocator.h: Likewise. * include/ext/vstring.h: Likewise. * include/ext/bitmap_allocator.h: Likewise. * include/ext/pool_allocator.h: Likewise. * include/ext/type_traits.h: Likewise. * include/ext/ropeimpl.h: Likewise. * include/ext/codecvt_specializations.h: Likewise. * include/ext/throw_allocator.h: Likewise. * include/ext/extptr_allocator.h: Likewise. * include/ext/atomicity.h: Likewise. * include/ext/concurrence.h: Likewise. * include/c_compatibility/wchar.h: Likewise. * include/c_compatibility/stdint.h: Likewise. * include/backward/hash_fun.h: Likewise. * include/backward/binders.h: Likewise. * include/backward/hashtable.h: Likewise. * include/backward/auto_ptr.h: Likewise. * libsupc++/eh_arm.cc: Likewise. * libsupc++/unwind-cxx.h: Likewise. * libsupc++/si_class_type_info.cc: Likewise. * libsupc++/vec.cc: Likewise. * libsupc++/class_type_info.cc: Likewise. * libsupc++/vmi_class_type_info.cc: Likewise. * libsupc++/guard_error.cc: Likewise. * libsupc++/bad_typeid.cc: Likewise. * libsupc++/eh_personality.cc: Likewise. * libsupc++/atexit_arm.cc: Likewise. * libsupc++/pmem_type_info.cc: Likewise. * libsupc++/vterminate.cc: Likewise. * libsupc++/eh_terminate.cc: Likewise. * libsupc++/bad_cast.cc: Likewise. * libsupc++/exception_ptr.h: Likewise. * libsupc++/eh_throw.cc: Likewise. * libsupc++/bad_alloc.cc: Likewise. * libsupc++/nested_exception.cc: Likewise. * libsupc++/pointer_type_info.cc: Likewise. * libsupc++/pbase_type_info.cc: Likewise. * libsupc++/bad_array_new.cc: Likewise. * libsupc++/pure.cc: Likewise. * libsupc++/eh_exception.cc: Likewise. * libsupc++/bad_array_length.cc: Likewise. * libsupc++/cxxabi.h: Likewise. * libsupc++/guard.cc: Likewise. * libsupc++/eh_catch.cc: Likewise. * libsupc++/cxxabi_forced.h: Likewise. * libsupc++/tinfo.h: Likewise.
The following patch on top of the r15-4346 patch adds -Wleading-whitespace= warning option. This warning doesn't care how much one actually indents which line in the source (that is something that can't be easily done in the preprocessor without doing syntactic analysis), but just simple checks on what kind of whitespace is used in the indentation. I think it is still useful to get warnings about such issues early, while git diagnoses some of it in patches (e.g. the tab after space case), getting the warnings earlier might help avoiding such issues sooner. There are projects which ban use of tabs and require just spaces, others which require indentation just with horizontal tabs, and finally projects which want indentation with tabs for multiples of tabstop size followed by spaces (fewer than tabstop size), like GCC. For all 3 kinds the warning diagnoses indentation with '\v' or '\f' characters (unless line contains just whitespace), and for the last one also cases where a space in the indentation is followed by horizontal tab or where there are N or more consecutive spaces in the indentation (for -ftabstop=N). BTW, for additional testing I've enabled the warnings (without -Werror for them) in stage3. There are many warnings (both trailing and leading whitespace), some of them something that can be easily fixed in the headers or source files, but others with whitespace issues in generated sources, so if we enable the warnings, either we'd need to adjust the generators or disable the warnings in (some of the) generated files. 2024-10-23 Jakub Jelinek <jakub@redhat.com> libcpp/ * include/cpplib.h (struct cpp_options): Add cpp_warn_leading_whitespace and cpp_tabstop members. (enum cpp_warning_reason): Add CPP_W_LEADING_WHITESPACE. * internal.h (struct _cpp_line_note): Document new line note kinds. * init.cc (cpp_create_reader): Set cpp_tabstop to 8. * lex.cc (find_leading_whitespace_issues): New function. (_cpp_clean_line): Use it. (_cpp_process_line_notes): Handle 'L', 'S' and 'T' line notes. (lex_raw_string): Clear type on 'L', 'S' and 'T' line notes inside of raw string literals. gcc/ * doc/invoke.texi (Wleading-whitespace=): Document. gcc/c-family/ * c.opt (Wleading-whitespace=): New option. * c-opts.cc (c_common_post_options): Set cpp_opts->cpp_tabstop to global_dc->m_tabstop. gcc/testsuite/ * c-c++-common/cpp/Wleading-whitespace-1.c: New test. * c-c++-common/cpp/Wleading-whitespace-2.c: New test. * c-c++-common/cpp/Wleading-whitespace-3.c: New test. * c-c++-common/cpp/Wleading-whitespace-4.c: New test.
Trailing blanks is something even git diff diagnoses; while it is a coding
style issue, if it is so common that git diff diagnoses it, I think it could
be useful to various projects to check that at compile time.
Dunno if it should be included in -Wextra, currently it isn't, and due to
tons of trailing whitespace in our sources, haven't enabled it for when
building gcc itself either.
Note, git diff also diagnoses indentation with tab following space, wonder
if we couldn't have trivial warning options where one would simply ask for
checking of indentation with no tabs, just spaces vs. indentation with
tabs followed by spaces (but never tab width or more spaces in the
indentation). I think that would be easy to do also on the libcpp side.
Checking how much something should be exactly indented requires syntax
analysis (at least some limited one) and can consider columns of first token
on line, but what the exact indentation blanks were is something only libcpp
knows.
On Thu, Sep 19, 2024 at 08:17:24AM +0200, Richard Biener wrote:
> Generally I like diagnosing this early. For the above I'd say -Wtrailing-whitespace=
> with a set of things to diagnose (and a sane default - just spaces and tabs - for
> -Wtrailiing-whitespace) would be nice. As for naming possibly follow the
> is{space,blank,cntrl} character classifications? If those are a good
> fit, that is.
The patch currently allows blank (' ' '\t') and space (' ' '\t' '\f' '\v'),
cntrl not yet added, not anything non-ASCII, but in theory could
be added later (though, non-ASCII would be just for inside of comments,
say non-breaking space etc. in the source is otherwise an error).
2024-10-15 Jakub Jelinek <jakub@redhat.com>
libcpp/
* include/cpplib.h (struct cpp_options): Add
cpp_warn_trailing_whitespace member.
(enum cpp_warning_reason): Add CPP_W_TRAILING_WHITESPACE.
* internal.h (struct _cpp_line_note): Document 'W' line note.
* lex.cc (_cpp_clean_line): Add 'W' line note for trailing whitespace
except for trailing whitespace after backslash. Formatting fix.
(_cpp_process_line_notes): Emit -Wtrailing-whitespace diagnostics.
Formatting fixes.
(lex_raw_string): Clear type on 'W' notes.
gcc/
* doc/invoke.texi (Wtrailing-whitespace): Document.
gcc/c-family/
* c.opt (Wtrailing-whitespace=): New option.
(Wtrailing-whitespace): New alias.
* c.opt.urls: Regenerate.
gcc/testsuite/
* c-c++-common/cpp/Wtrailing-whitespace-1.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-2.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-3.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-4.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-5.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-6.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-7.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-8.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-9.c: New test.
* c-c++-common/cpp/Wtrailing-whitespace-10.c: New test.