Summary:
Calling WriteConsoleW is the most reliable way to print Unicode
characters to a Windows console.
If binary data gets printed to the console, attempting to re-encode it
shouldn't be a problem, since garbage in can produce garbage out.
This breaks printing strings in the local codepage, which WriteConsoleA
knows how to handle. For example, this can happen when user source code
is encoded with the local codepage, and an LLVM tool quotes it while
emitting a caret diagnostic. This is unfortunate, but well-behaved tools
should validate that their input is UTF-8 and escape non-UTF-8
characters before sending them to raw_fd_ostream. Clang already does
this, but not all LLVM tools do this.
One drawback to the current implementation is printing a string a byte
at a time doesn't work. Consider this LLVM code:
for (char C : MyStr) outs() << C;
Because outs() is now unbuffered, we wil try to convert each byte to
UTF-16, which will fail. However, this already didn't work, so I think
we may as well update callers that do that as we find them to print
complete portions of strings. You can see a real example of this in my
patch to SourceMgr.cpp
Fixes PR38669 and PR36267.
Reviewers: zturner, efriedma
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D51558
llvm-svn: 341433
On windows when raw_fd_ostream::write_impl calls write, a 32 bit input is required for character count. As a variable with size_t is used for this argument, on x64 integral demotion occurs. In the case of large files an infinite loop follows.
See: https://bugs.llvm.org/show_bug.cgi?id=37926
This fix allows the output of files larger than the previous int32 limit.
Differential Revision: https://reviews.llvm.org/D48948
llvm-svn: 339027
The standard library functions ::isprint/std::isprint have platform-
and locale-dependent behavior which makes LLVM's output less
predictable. In particular, regression tests my fail depending on the
implementation of these functions.
Implement llvm::isPrint in StringExtras.h with a standard behavior and
replace all uses of ::isprint/std::isprint by a call it llvm::isPrint.
The function is inlined and does not look up language settings so it
should perform better than the standard library's version.
Such a replacement has already been done for isdigit, isalpha, isxdigit
in r314883. gtest does the same in gtest-printers.cc using the following
justification:
// Returns true if c is a printable ASCII character. We test the
// value of c directly instead of calling isprint(), which is buggy on
// Windows Mobile.
inline bool IsPrintableAscii(wchar_t c) {
return 0x20 <= c && c <= 0x7E;
}
Similar issues have also been encountered by Julia:
https://github.com/JuliaLang/julia/issues/7416
I noticed the problem myself when on Windows isprint('\t') started to
evaluate to true (see https://stackoverflow.com/questions/51435249) and
thus caused several unit tests to fail. The result of isprint doesn't
seem to be well-defined even for ASCII characters. Therefore I suggest
to replace isprint by a platform-independent version.
Differential Revision: https://reviews.llvm.org/D49680
llvm-svn: 338034
On Windows when raw_fd_ostream::write_impl calls write, a 32 bit input
is required for character count. As a variable with size_t is used for
this argument on x64 integral demotion occurs. In the case of large
files an infinite loop follows.
See PR37926.
This fix allows the output of files larger than previous int32 limit.
Differential Revision: https://reviews.llvm.org/D48948
Patch by Owen Reynolds
Reviewed by: zturner
llvm-svn: 338027
This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition. The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum. The second controls more flags-like values.
This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before. This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.
llvm-svn: 334221
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of a reinvented one.
See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.
This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.
llvm-svn: 331127
Previously, we call write(2) for each 32767 byte chunk. That is not
efficient because Linux can handle much larger write requests.
This patch changes the chunk size on Linux to 1 GiB.
This patch also changes the default chunks size to SSIZE_MAX. I think
that doesn't in practice change this function's behavior on any operating
system because SSIZE_MAX on 64-bit machine is unrealistically large,
and writing 2 GiB (SSIZE_MAX on 32-bit) on a 32-bit machine by a single
call of write(2) is also unrealistic, as the userspace is usually
limited to 2 GiB. That said, it is in general a good thing to do because
a write larger than SSIZE_MAX is implementation-defined in POSIX.
Differential Revision: https://reviews.llvm.org/D39444
llvm-svn: 317015
Summary:
Previously, we would emit error messages like "IO failure on output
stream". This change causes use to include information about what
actually went wrong, e.g. "No space left on device".
Reviewers: sunfish, rnk
Reviewed By: rnk
Subscribers: mehdi_amini, llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D39203
llvm-svn: 316404
The full story is in the comments:
// Do not attempt to close stdout or stderr. We used to try to maintain the
// property that tools that support writing file to stdout should not also
// write informational output to stdout, but in practice we were never able to
// maintain this invariant. Many features have been added to LLVM and clang
// (-fdump-record-layouts, optimization remarks, etc) that print to stdout, so
// users must simply be aware that mixed output and remarks is a possibility.
NFC, I am just updating comments to reflect reality.
llvm-svn: 310016
This is a workaround for large file writes. It has been witnessed that
write(2) failing with EINVAL (22) due to a large value (>2G). Thanks to
James Knight for the help with coming up with a sane test case.
llvm-svn: 305846
We do not want to close STDOUT as there may have been several uses of it
such as the case: llc %s -o=- -pass-remarks-output=- -filetype=asm
which cause multiple closes of STDOUT_FILENO and/or use-after-close of it.
Using dup() in getFD doesn't work as we end up with original STDOUT_FILENO
open anyhow.
reviewed by Rafael Espindola
Differential Revision: https://reviews.llvm.org/D31505
llvm-svn: 299098
If raw_fd_ostream is constructed with the path of "-", it claims
ownership of the stdout file descriptor. This means that it closes
stdout when it is destroyed. If there are multiple users of
raw_fd_ostream wrapped around stdout, then a crash can occur because
of operations on a closed stream.
An example of this would be running something like "clang -S -o - -MD
-MF - test.cpp". Alternatively, using outs() (which creates a local
version of raw_fd_stream to stdout) anywhere combined with such a
stream usage would cause the crash.
The fix duplicates the stdout file descriptor when used within
raw_fd_ostream, so that only that particular descriptor is closed when
the stream is destroyed.
Patch by James Henderson!
llvm-svn: 297624
This introduces a new type-safe general purpose formatting
library. It provides compile-time type safety, does not require
a format specifier (since the type is deduced), and provides
mechanisms for extending the format capability to user defined
types, and overriding the formatting behavior for existing types.
This patch additionally adds documentation for the API to the
LLVM programmer's manual.
Mailing List Thread:
http://lists.llvm.org/pipermail/llvm-dev/2016-October/105836.html
Differential Revision: https://reviews.llvm.org/D25587
llvm-svn: 286682
This makes it possible to indent a binary blob by a certain
number of bytes, and also makes some things more idiomatic.
Finally, it integrates this binary blob formatter into ScopedPrinter
which used to have its own implementation of this algorithm.
Differential Revision: https://reviews.llvm.org/D26477
llvm-svn: 286495
This resubmits r284436 and r284437, which were reverted in
r284462 as they were breaking the AArch64 buildbot.
The breakage on AArch64 turned out to be a miscompile which is
still not fixed, but is actively tracked at llvm.org/pr30748.
This resubmission re-writes the code in a way so as to make the
miscompile not happen.
llvm-svn: 285483
This reverts commits 284436 and 284437 because they still break AArch64 bots:
Value of: format_number(-10, IntegerStyle::Integer, 1)
Actual: "-0"
Expected: "-10"
llvm-svn: 284462
This resubmits commits 284425 and r284428, which were reverted
in r284429 due to some infinite recursion caused by an incorrect
selection of function overloads. Reproduced the failure on Linux
using GCC 4.8.4, and confirmed that with the new patch the tests
path on GCC as well as MSVC. So hopefully this fixes everything.
llvm-svn: 284436
raw_ostream has not afforded a lot of flexibility in terms of
how to format numbers when outputting. Wrap this all up into
a set of low level helper functions that can be used to output
numbers with arbitrary precision, alignment, format, etc and
then update raw_ostream to use these functions.
This will be useful for upcoming improvements to llvm's string
formatting libraries, but are still useful independently.
Differential Revision: https://reviews.llvm.org/D25497
llvm-svn: 284425
Low level functionality to format numbers were embedded in the
implementation of raw_ostream. I have need to use these through
an interface other than the overloaded stream operators, so they
need to be raised to a level that they can be used from either
raw_ostream operators or other code.
llvm-svn: 283921
Summary:
Without this, this command
$ llvm-run llc -stop-after machine-cp -o - <( echo '' )
outputs an error, because we close stdout twice -- once when closing the
file opened for "-o", and again when closing outs().
Also clarify in the outs() definition that you can't ever call it if you
want to open your own raw_fd_ostream on stdout.
Reviewers: jroelofs, tstellarAMD
Subscribers: jholewinski, qcolombet, dsanders, llvm-commits
Differential Revision: http://reviews.llvm.org/D17422
llvm-svn: 261286
Apparently the preferred version is the incredibly complicated
VerifyVersionInfoW function.
Rename the function to avoid potential future name clashes.
llvm-svn: 257415
This patch is similar to the Python issue#11395. We need to cap the output
size to 32767 on Windows to work around the size limit of WriteConsole().
Reference: https://bugs.python.org/issue11395
Writing a test for this bug turns out to be harder than I thought. I am
still working on it (see phabricator review D15705).
Differential Revision: http://reviews.llvm.org/D15553
llvm-svn: 256892
It looks like the code this patch deletes is based on a misunderstanding of
what guarantees writev provides. In particular, writev with 1 iovec is
not "more atomic" than a write.
Testing on OS X shows that both write and writev from multiple processes
can be intermixed.
llvm-svn: 255837
