1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2025-01-31 20:51:52 +01:00

134 Commits

Author SHA1 Message Date
Joel E. Denny
12c5f043b8 [FileCheck] Report captured variables
Report captured variables in input dumps and traces.  For example:

```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: goodbye [[WHAT]]

$ FileCheck -dump-input=always -vv check < input |& tail -8
<<<<<<
           1: hello world
check:1'0     ^~~~~~~~~~~
check:1'1           ^~~~~ captured var "WHAT"
           2: goodbye world
check:2'0     ^~~~~~~~~~~~~
check:2'1                   with "WHAT" equal to "world"
>>>>>>

$ FileCheck -dump-input=never -vv check < input
check2:1:8: remark: CHECK: expected string found in input
CHECK: hello [[WHAT:[a-z]+]]
       ^
<stdin>:1:1: note: found here
hello world
^~~~~~~~~~~
<stdin>:1:7: note: captured var "WHAT"
hello world
      ^~~~~
check2:2:8: remark: CHECK: expected string found in input
CHECK: goodbye [[WHAT]]
       ^
<stdin>:2:1: note: found here
goodbye world
^~~~~~~~~~~~~
<stdin>:2:1: note: with "WHAT" equal to "world"
goodbye world
^
```

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D83651
2020-07-28 19:15:18 -04:00
Joel E. Denny
7663136ac6 [FileCheck] Extend -dump-input with substitutions
Substitutions are already reported in the diagnostics appearing before
the input dump in the case of failed directives, and they're reported
in traces (produced by `-vv -dump-input=never`) in the case of
successful directives.  However, those reports are not always
convenient to view while investigating the input dump, so this patch
adds the substitution report to the input dump too.  For example:

```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: [[VERB]] [[WHAT]]

$ FileCheck -vv -DVERB=goodbye check < input |& tail -8
<<<<<<
           1: hello world
check:1       ^~~~~~~~~~~
           2: goodbye word
check:2'0     X~~~~~~~~~~~ error: no match found
check:2'1                  with "VERB" equal to "goodbye"
check:2'2                  with "WHAT" equal to "world"
>>>>>>
```

Without this patch, the location reported for a substitution for a
directive match is the directive's full match range.  This location is
misleading as it implies the substitution itself matches that range.
This patch changes the reported location to just the match range start
to suggest the substitution is known at the start of the match.  (As
in the above example, input dumps don't mark any range for
substitutions.  The location info in that case simply identifies the
right line for the annotation.)

Reviewed By: mehdi_amini, thopre

Differential Revision: https://reviews.llvm.org/D83650
2020-07-28 19:15:18 -04:00
Joel E. Denny
7fff66abb0 [FileCheck] Implement -dump-input-filter
This makes the input dump filtering implemented by D82203 more
configurable.  D82203 enables filtering out everything but the initial
input lines of error diagnostics (plus some context).  This patch
enables including any line with any kind of annotation.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83097
2020-07-10 11:02:11 -04:00
Joel E. Denny
ec018a8846 [FileCheck] In input dump, elide only if ellipsis is shorter
For example, given `-dump-input-context=3 -vv`, the following now
shows more leading context for the error than requested because a
leading ellipsis would occupy the same number of lines as it would
elide:

```
<<<<<<
         1: foo6
         2: foo5
         3: foo4
         4: foo3
         5: foo2
         6: foo1
         7: hello world
check:1     ^~~~~
check:2           X~~~~ error: no match found
         8: foo1
check:2     ~~~~
         9: foo2
check:2     ~~~~
        10: foo3
check:2     ~~~~
         .
         .
         .
>>>>>>
```

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83526
2020-07-10 11:02:11 -04:00
Joel E. Denny
82eeb7f9a7 [FileCheck] Implement -dump-input-context
This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

When input is dumped as specified by `-dump-input=fail`, this patch
filters the dump to show only input lines that are the starting lines
of error diagnostics plus the number of contextual lines specified
`-dump-input-context` (defaults to 5).

When `-dump-input=always`, there might be not be any errors, so all
input lines are printed, as without this patch.

Here's some sample output with `-dump-input-context=3 -vv`:

```
<<<<<<
           .
           .
           .
          13: foo
          14: foo
          15: hello world
check:1       ^~~~~~~~~~~
          16: foo
check:2'0     X~~ error: no match found
          17: foo
check:2'0     ~~~
          18: foo
check:2'0     ~~~
          19: foo
check:2'0     ~~~
           .
           .
           .
          27: foo
check:2'0     ~~~
          28: foo
check:2'0     ~~~
          29: foo
check:2'0     ~~~
          30: goodbye word
check:2'0     ~~~~~~~~~~~~
check:2'1     ?            possible intended match
          31: foo
check:2'0     ~~~
          32: foo
check:2'0     ~~~
          33: foo
check:2'0     ~~~
           .
           .
           .
>>>>>>
```

Reviewed By: mehdi_amini, arsenm, jhenderson, rsmith, SjoerdMeijer, Meinersbur, lattner

Differential Revision: https://reviews.llvm.org/D82203
2020-07-10 11:02:10 -04:00
Joel E. Denny
e330e469db [FileCheck] Improve -dump-input documentation
Document the default of `fail` in `-help`.  Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their full documentation.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D83091
2020-07-09 18:00:30 -04:00
Joel E. Denny
f5f29ea8b3 [FileCheck] Permit multiple -v or -vv
`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite has a FileCheck call that already specifies
`-v` or `-vv`, then that call will fail if `FILECHECK_OPTS` also
specifies it.

For `-vv`, this problem already exists:

`clang/test/CodeGen/aarch64-v8.2a-fp16-intrinsics-constrained.c`

It's not yet clear if the `-vv` in that test was intentional, but this
usage shouldn't fail anyway.  It's already true that FileCheck permits
`-vv` and `-v` together even though `-vv` implies `-v`.

Compare D70784, which fixed the same problem for `-dump-input`.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82601
2020-06-29 18:35:22 -04:00
Joel E. Denny
5366dbf89a [FileCheck][NFC] Clean up RUN style in verbose.txt test
Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82658
2020-06-29 18:35:11 -04:00
Joel E. Denny
cbbdc52957 [FileCheck][NFC] Fix typo in test comment 2020-06-24 14:49:23 -04:00
Joel E. Denny
3252abfb38 [FileCheck] Add missing %ProtectFileCheckOutput to FileCheck tests
This is a continuation of D65121 (committed at f471eb8e99b5).
2020-06-10 12:40:35 -04:00
Thomas Preud'homme
a5f3d35feb FileCheck [11/12]: Add matching constraint specification
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support for specifying the
matching constraint for a numeric expression, ie. how the value being
matched should relate to the numeric expression.

This commit only adds the equality constraint where the numeric value
matched must be equal to the numeric expression. It is the default
matching constraint used when not specified. It is added to provision
other matching constraint (e.g. inequality relations).

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D60391
2020-06-10 15:56:10 +01:00
Paul Walker
a8c59c6124 [FileCheck] Add function call support to numerical expressions.
This patch extends numerical expressions to allow calls to
predefined functions. These calls can be combined with the
existing numerical operators, which includes nesting calls.

The call syntax is:

  <func>(<args>)

Where <func> is a predefined string literal, currently limited to
one of add, max, min and sub. <arg> is a comma seperated list of
numerical expressions.

Subscribers: arichardson, hiraditya, thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79936
2020-06-10 09:42:00 +00:00
Mehdi Amini
ed76433842 Change filecheck default to dump input on failure
Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.

Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.

Differential Revision: https://reviews.llvm.org/D81422
2020-06-09 18:57:46 +00:00
Thomas Preud'homme
959d70cd6e FileCheck [10/12]: Add support for signed numeric values
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support signed numeric
values, thus allowing negative numeric values.

As such, the patch adds a new class to represent a signed or unsigned
value and add the logic for type promotion and type conversion in
numeric expression mixing signed and unsigned values. It also adds
the %d format specifier to represent signed value.

Finally, it also adds underflow and overflow detection when performing a
binary operation.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson

Reviewed By: jhenderson, arichardson

Subscribers: MaskRay, hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60390
2020-05-28 10:44:21 +01:00
Alex Richardson
aa7eac2e43 [FileCheck] Allow parenthesized expressions
With this change it is be possible to write FileCheck expressions such
as [[#(VAR+1)-2]]. Currently, the only supported arithmetic operators are
plus and minus, so this is not particularly useful yet. However, it our
CHERI fork we have tests that benefit from having multiplication in
FileCheck expressions. Allowing parenthesized expressions is the simplest
way for us to work around the current lack of operator precedence in
FileCheck expressions.

Reviewed By: thopre, jhenderson
Differential Revision: https://reviews.llvm.org/D77383
2020-05-27 16:31:39 +01:00
Jon Roelofs
d0f0991d73 Revert "[llvm][test] Add COM: directives before colon-less non-CHECKs in comments. NFC"
This reverts commit 183d6af081899973f00fc24aeafcfc32de732f02.

Revert pending further consensus building: https://reviews.llvm.org/D79963#2050521
2020-05-22 05:36:15 -06:00
Jon Roelofs
aae486e5a4 [llvm][test] Add COM: directives before colon-less non-CHECKs in comments. NFC
Differential Revision: https://reviews.llvm.org/D79963
2020-05-21 09:29:27 -06:00
Thomas Preud'homme
37b37d76ae [test] Improve FileCheck's numeric-expression.txt
Summary:
Various improvement for FileCheck's numeric-expression.txt test:
- remove unused values in USE DEF FMT IMPL MATCH section
- replace 14 literal for 0xe and 0xE to have example of hex literals
- rename variable to be more self-descriptive
- move CHECK as comment of the values being matched to help readability
- add conversion tests
- simplify test for use of several numeric variables by using existing
  variable
- adjust position of error message check to match the alignment of the
  error message wrt. the output matched by the previous check

Reviewed By: jhenderson, jdenny

Differential Revision: https://reviews.llvm.org/D79820
2020-05-16 00:09:24 +01:00
Joel E. Denny
d5b9bb2e37 [FileCheck] Fix isalpha/isalnum calls
D79276 caused the following builder to fail:

  http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489

Specifically, FileCheck dumped stack in the following tests:

  LLVM :: MC/Mips/micromips-jump-pc-region.s
  LLVM :: MC/Mips/mips-jump-pc-region.s

Those tests contained characters encoded as 160 but that render (at
least for me in vim) like a single space (32).  Those characters
appeared between the `#` and `RUN:` on several lines, and D79276
caused FileCheck to process those lines differently: `RUN:` is a
comment directive.  As a result, D79276 caused FileCheck to start
calling is `isalnum` on those characters.

The problem is that FileCheck calls `isalnum` on type `char` without
casting to `unsigned char` first, so it sign-extends 160 beyond what
`unsigned char` or `EOF` can represent.  C says that has undefined
behavior.  This problem is general to FileCheck's prefix parsing and
so exists independently of D79276.

524457edbc3d fixed the above tests.  This patch changes FileCheck to
use LLVM's replacements for `ctype.h` functions, and it adds tests for
cases that are representative with or without D79276.

Reviewed By: jhenderson, thopre, efriedma

Differential Revision: https://reviews.llvm.org/D79810
2020-05-14 20:24:09 -04:00
Joel E. Denny
f717458fdf [FileCheck] Support comment directives
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name.  The `COM:` directive makes it easy to do this.  For example,
you might have:

```
; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

  <http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear.  `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well.  Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276
2020-05-13 11:29:48 -04:00
Joel E. Denny
b000ffc696 [FileCheck] Make invalid prefix diagnostics more precise
This will prove especially helpful after D79276, which introduces
comment prefixes.  Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375
2020-05-11 21:11:58 -04:00
Joel E. Denny
cbd9bd8f7e Revert "[FileCheck] Make invalid prefix diagnostics more precise"
This reverts commit a78e13745d4ee4a42e41ebbe698159f651515fc5 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489
2020-05-11 19:41:22 -04:00
Joel E. Denny
2412893f8a Revert "[FileCheck] Support comment directives"
This reverts commit 9a9a5f9893c8db05cebc8818eb8485bff61f7c74 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489
2020-05-11 19:41:22 -04:00
Joel E. Denny
efde2f4800 [FileCheck] Support comment directives
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name.  The `COM:` directive makes it easy to do this.  For example,
you might have:

```
; X32: pinsrd_1:
; X32:    pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM:   X64: pinsrd_1:
; COM:   X64:    pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

  <http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear.  `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well.  Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276
2020-05-11 14:53:48 -04:00
Joel E. Denny
cfbaeca35c [FileCheck] Make invalid prefix diagnostics more precise
This will prove especially helpful after D79276, which introduces
comment prefixes.  Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375
2020-05-11 14:53:48 -04:00
Joel E. Denny
a7c6415ae4 [FileCheck] Fix --dump-input annotation sort per input line
Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line.  That seemed
fine because that's often identical to the order in which FileCheck
looks for matches for those directives.

The first problem is that an `--implicit-check-not` pattern has no
check-file line.  The logical equivalent is sorting in command-line
order, but that's not implemented.

The second problem is that, unlike a directive, an
`--implicit-check-not` pattern applies at many points, between many
different pairs of directives.  However, sorting in command-line order
gathers all its associated diagnostics together at one point in an
input line's list of annotations.

In general, it seems to be easier to understand FileCheck's logic when
annotations on a single input line are sorted in the order FileCheck
produced the associated diagnostics, so this patch makes that change.
As documented in the patch, the annotation sort order is also
especially relevant to `CHECK-LABEL`, `CHECK-NOT`, and `CHECK-DAG`, so
this patch updates or extends tests to check the sort makes sense for
them.  (However, the sort for `CHECK-DAG` annotations should not
actually be altered by this patch.)

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77607
2020-04-16 15:39:35 -04:00
Joel E. Denny
00108cdc64 [FileCheck] Fix --dump-input implicit pattern location
Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
       1: foo bar baz
not:1         !~~     error: no match expected
```

This patch changes that to:

```
          1: foo bar baz
not:imp1         !~~     error: no match expected
```

`imp1` indicates the first `--implicit-check-not` pattern.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77605
2020-04-16 15:39:35 -04:00
Georgii Rymar
ced5ee12e6 [FileCheck] - Fix the false positive when -implicit-check-not is used with an unknown -check-prefix.
Imagine we have the following invocation:

`FileCheck -check-prefix=UNKNOWN-PREFIX -implicit-check-not=something`

When the check prefix does not exist it does not fail.
This patch fixes the issue.

Differential revision: https://reviews.llvm.org/D78024
2020-04-16 15:00:50 +03:00
Thomas Preud'homme
a8ced4e1db [FileCheck] Better diagnostic for format conflict
Summary:
Improve error message in case of conflict between several implicit
format to mention the operand that conflict.

Reviewers: jhenderson, jdenny, probinson, grimar, arichardson, rnk

Reviewed By: jdenny

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77741
2020-04-15 13:46:45 +01:00
Joel E. Denny
9fb4af2f28 [FileCheck] Add missing %ProtectFileCheckOutput to FileCheck tests
I'm committing this fixup without review because it's an obvious
continuation of D65121 (committed at f471eb8e99b5).
2020-03-31 17:29:11 -04:00
David Bozier
3e5a3b434c Improve error message of FileCheck when stdin is empty
Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson, probinson, jdenny, grimar, arichardson

Reviewed By: jhenderson

Subscribers: thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73793
2020-02-04 11:14:55 +00:00
Thomas Preud'homme
d5ea9f022a FileCheck [9/12]: Add support for matching formats
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support for selecting a
matching format to match a numeric value against (ie. decimal, hex lower
case letters or hex upper case letters).

This commit allows to select what format a numeric value should be
matched against. The following formats are supported: decimal value,
lower case hex value and upper case hex value. Matching formats impact
both the format of numeric value to be matched as well as the format of
accepted numbers in a definition with empty numeric expression
constraint.

Default for absence of format is decimal value unless the numeric
expression constraint is non null and use a variable in which case the
format is the one used to define that variable. Conclict of format in
case of several variable being used is diagnosed and forces the user to
select a matching format explicitely.

This commit also enables immediates in numeric expressions to be in any
radix known to StringRef's GetAsInteger method, except for legacy
numeric expressions (ie [[@LINE+<offset>]] which only support decimal
immediates.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson

Reviewed By: jhenderson, arichardson

Subscribers: daltenty, MaskRay, hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60389
2020-01-24 14:15:28 +00:00
Joel E. Denny
b4b16598f6 [FileCheck] Given multiple -dump-input, prefer most verbose
Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-input=fail`.  However, some existing test suites have FileCheck
calls that already specify `-dump-input=fail` or `-dump-input=always`.
Without this patch, such tests fail under such a test runner because
FileCheck doesn't accept multiple occurrences of `-dump-input`.

Solution: This patch permits multiple occurrences of `-dump-input` by
assigning precedence to its values in the following descending order:
`help`, `always`, `fail`, and `never`.  That is, any occurrence of
`help` always obtains help, and otherwise the behavior is similar to
`-v` vs. `-vv` in that the option specifying the greatest verbosity
has precedence.

Rationale: My justification for the new behavior is as follows.  I
have not experienced use cases where, either as a test runner or as a
test author, I want to **limit** the permitted debugging verbosity
(except as a test author in FileCheck's or lit's test suites where the
FileCheck debugging output itself is under test, but the solution
there is `env FILECHECK_OPTS=`, and I imagine we should use the same
solution anywhere else this need might occur).  Of course, as either a
test runner or test author, it is useful to **increase** debugging
verbosity.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D70784
2019-12-03 14:21:13 -05:00
Joel E. Denny
dd1fd5b646 [FileCheck] Make FILECHECK_OPTS useful for its test suite
Without this patch, `FILECHECK_OPTS` isn't propagated to FileCheck's
test suite so that `FILECHECK_OPTS` doesn't inadvertently affect test
results by affecting the output of FileCheck calls under test.  As a
result, `FILECHECK_OPTS` is useless for debugging FileCheck's test
suite.

In `llvm/test/FileCheck/lit.local.cfg`, this patch provides a new
subsitution, `%ProtectFileCheckOutput`, to address this problem for
both `FILECHECK_OPTS` and the deprecated
`FILECHECK_DUMP_INPUT_ON_FAILURE`.  The rest of the patch uses
`%ProtectFileCheckOutput` throughout the test suite

Fixes PR40284.

Reviewed By: probinson, thopre

Differential Revision: https://reviews.llvm.org/D65121
2019-11-21 18:01:12 -05:00
Joel E. Denny
15ec0cb41e [FileCheck] Use lit's internal shell for the test suite
An advantage is that there are less portability concerns when writing
tests.  For example, `-u` is not supported by all implementations of
`env`, but lit's internal shell provides its own `env` that supports
`-u`.

A disadvantage is that some shell constructs, such as parentheses, are
not supported, but FileCheck's test suite currently doesn't require
such constructs.

For comparison, lit configures its test suite in the same manner.  See
`llvm/utils/lit/tests/lit.cfg`.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D70278
2019-11-19 12:08:54 -05:00
Kai Nacke
f65b9d0379 [FileCheck] Implement --ignore-case option.
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374538
2019-10-11 11:59:14 +00:00
Dmitri Gribenko
0c3e83e627 Revert "[FileCheck] Implement --ignore-case option."
This reverts commit r374339. It broke tests:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066

llvm-svn: 374359
2019-10-10 14:27:14 +00:00
Kai Nacke
28535e7629 [FileCheck] Implement --ignore-case option.
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374339
2019-10-10 13:15:41 +00:00
Thomas Preud'homme
393fb2cc2e [FileCheck] Forbid using var defined on same line
Summary:
Commit r366897 introduced the possibility to set a variable from an
expression, such as [[#VAR2:VAR1+3]]. While introducing this feature, it
introduced extra logic to allow using such a variable on the same line
later on. Unfortunately that extra logic is flawed as it relies on a
mapping from variable to expression defining it when the mapping is from
variable definition to expression. This flaw causes among other issues
PR42896.

This commit avoids the problem by forbidding all use of a variable
defined on the same line, and removes the now useless logic. Redesign
will be done in a later commit because it will require some amount of
refactoring first for the solution to be clean. One example is the need
for some sort of transaction mechanism to set a variable temporarily and
from an expression and rollback if the CHECK pattern does not match so
that diagnostics show the right variable values.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66141

llvm-svn: 370663
2019-09-02 14:04:00 +00:00
Joel E. Denny
86f2b361a4 [FileCheck] Move -dump-input diagnostic to first line
Without this patch, `-dump-input` prints a diagnostic at the end of
its marker range.  For example:

```
         1: Start.
check:1     ^~~~~~
         2: Bad.
next:2      X~~~
         3: Many lines
next:2      ~~~~~~~~~~
         4: of input.
next:2      ~~~~~~~~~
         5: End.
next:2      ~~~~ error: no match found
```

This patch moves it to the beginning like this:

```
         1: Start.
check:1     ^~~~~~
         2: Bad.
next:2      X~~~ error: no match found
         3: Many lines
next:2      ~~~~~~~~~~
         4: of input.
next:2      ~~~~~~~~~
         5: End.
next:2      ~~~~
```

The former somehow looks nicer because the diagnostic doesn't appear
to be somewhere within the marker range.  However, the latter is more
practical, especially when the marker range includes the remainder of
a very long dump.  First, in the case of an error, this patch enables
me to search the dump for `error:` and usually immediately land where
the detected error began.  Second, when trying to follow FileCheck's
logic, it's best to read top down, so this patch enables me to see
each diagnostic as soon as I encounter its marker.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65702

llvm-svn: 368786
2019-08-14 02:56:09 +00:00
Thomas Preud'homme
dce7717f79 [FileCheck]] Canonicalize caret location testing
Summary:
Testing of caret location in diagnostic message is currently made with
CHECK directive with the following general format:
CHECK: {{^         \^$}}

James Henderson suggested the following would be more readable:
CHECK: {{^}}         ^{{$}}

and when whole lines can be matched (as is the case for command-line
testing where error messages do not include path):
CHECK:         ^
using the option --match-full-lines.

This commit implements these 2 changes on all existing caret position
tests. It also aligns the caret to the character it is trying to match
in the above line.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64923

> llvm-svn: 366861

llvm-svn: 366898
2019-07-24 12:38:29 +00:00
Thomas Preud'homme
fddc968349 FileCheck [8/12]: Define numeric var from expr
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch lift the restriction for a
numeric expression to either be a variable definition or a numeric
expression to try to match.

This commit allows a numeric variable to be set to the result of the
evaluation of a numeric expression after it has been matched
successfully. When it happens, the variable is allowed to be used on
the same line since its value is known at match time.

It also makes use of this possibility to reuse the parsing code to
parse a command-line definition by crafting a mirror string of the
-D option with the equal sign replaced by a colon sign, e.g. for option
'-D#NUMVAL=10' it creates the string
'-D#NUMVAL=10 (parsed as [[#NUMVAL:10]])' where the numeric expression
is parsed to define NUMVAL. This result in a few tests needing updating
for the location diagnostics on top of the tests for the new feature.

It also enables empty numeric expression which match any number without
defining a variable. This is done here rather than in commit #5 of the
patch series because it requires to dissociate automatic regex insertion
in RegExStr from variable definition which would make commit #5 even
bigger than it already is.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60388

> llvm-svn: 366860

llvm-svn: 366897
2019-07-24 12:38:22 +00:00
Thomas Preud'homme
443542ddca Revert "FileCheck [8/12]: Define numeric var from expr"
This reverts commit 1b05977538d9487aa845ee2f3bec8b89c63c4f29.

llvm-svn: 366872
2019-07-24 07:32:34 +00:00
Thomas Preud'homme
cb2d51e486 Revert "[FileCheck]] Canonicalize caret location testing"
This reverts commit ba4e8d791342219c072a5c8de8a3f41fbb53c6be.

llvm-svn: 366871
2019-07-24 07:32:18 +00:00
Thomas Preud'homme
06b80a82f4 [FileCheck]] Canonicalize caret location testing
Summary:
Testing of caret location in diagnostic message is currently made with
CHECK directive with the following general format:
CHECK: {{^         \^$}}

James Henderson suggested the following would be more readable:
CHECK: {{^}}         ^{{$}}

and when whole lines can be matched (as is the case for command-line
testing where error messages do not include path):
CHECK:         ^
using the option --match-full-lines.

This commit implements these 2 changes on all existing caret position
tests. It also aligns the caret to the character it is trying to match
in the above line.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64923

llvm-svn: 366861
2019-07-23 22:41:46 +00:00
Thomas Preud'homme
f562841f29 FileCheck [8/12]: Define numeric var from expr
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch lift the restriction for a
numeric expression to either be a variable definition or a numeric
expression to try to match.

This commit allows a numeric variable to be set to the result of the
evaluation of a numeric expression after it has been matched
successfully. When it happens, the variable is allowed to be used on
the same line since its value is known at match time.

It also makes use of this possibility to reuse the parsing code to
parse a command-line definition by crafting a mirror string of the
-D option with the equal sign replaced by a colon sign, e.g. for option
'-D#NUMVAL=10' it creates the string
'-D#NUMVAL=10 (parsed as [[#NUMVAL:10]])' where the numeric expression
is parsed to define NUMVAL. This result in a few tests needing updating
for the location diagnostics on top of the tests for the new feature.

It also enables empty numeric expression which match any number without
defining a variable. This is done here rather than in commit #5 of the
patch series because it requires to dissociate automatic regex insertion
in RegExStr from variable definition which would make commit #5 even
bigger than it already is.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60388

llvm-svn: 366860
2019-07-23 22:41:38 +00:00
Thomas Preud'homme
a40c217d3c [FileCheck] Fix numeric variable redefinition
Summary:
Commit r365249 changed usage of FileCheckNumericVariable to have one
instance of that class per variable as opposed to one instance per
definition of a given variable as was done before. However, it retained
the safety check in setValue that it should only be called with the
variable unset, even after r365625.

However this causes assert failure when a non-pseudo variable is being
redefined. And while redefinition of @LINE at each CHECK line work in
the general case, it caused problem when a substitution failed (fixed in
r365624) and still causes problem when a CHECK line does not match since
@LINE's value is cleared after substitutions in match() happened but
printSubstitutions also attempts a substitution.

This commit solves the root of the problem by changing setValue to set a
new value regardless of whether a value was set or not, thus fixing all
the aforementioned issues.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64882

llvm-svn: 366434
2019-07-18 13:39:04 +00:00
Thomas Preud'homme
04e17bdb7c FileCheck [7/12]: Arbitrary long numeric expressions
Summary:
This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch extend numeric expression to
support an arbitrary number of operands, either variable or literals.

Copyright:
    - Linaro (changes up to diff 183612 of revision D55940)
    - GraphCore (changes in later versions of revision D55940 and
                 in new revision created off D55940)

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits, probinson, dblaikie, grimar, arichardson, tra, rnk, kristina, hfinkel, rogfer01, JonChesterfield

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60387

llvm-svn: 366001
2019-07-13 13:24:30 +00:00
Thomas Preud'homme
e54ac8e70d [FileCheck] Don't diagnose undef vars at parse time
Summary:
Diagnosing use of undefined variables takes place in
parseNumericVariableUse() and printSubstitutions() for numeric variables
but only takes place in printSubstitutions() for string variables. The
reason for the split location of diagnostics is that parsing is not
aware of the clearing of variables due to --enable-var-scope and thus
use of variables cleared in this way can only be catched by
printSubstitutions().

Beyond the code level inconsistency, there is also a user facing
inconsistency since diagnostics look different between the two
functions. While the diagnostic in printSubstitutions is more verbose,
doing the diagnostic there allows to diagnose all undefined variables
rather than just the first one and error out.

This patch create dummy variable definition when encountering a use of
undefined variable so that parsing can proceed and be diagnosed by
printSubstitutions() later. Tests that were testing whether parsing
fails in such case are thus modified accordingly.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64228

llvm-svn: 365219
2019-07-05 16:25:33 +00:00
Thomas Preud'homme
0520d355f2 [FileCheck] Factor some parsing checks out
Summary:
Both callers of parseNumericVariableDefinition() perform the same extra
check that no character is found after the variable name. This patch
factors out this check into parseNumericVariableDefinition().

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: JonChesterfield, rogfer01, hfinkel, kristina, rnk, tra, arichardson, grimar, dblaikie, probinson, llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64226

llvm-svn: 365191
2019-07-05 12:01:06 +00:00