In D49466, sys::path::replace_path_prefix was used instead startswith for -f[macro/debug/file]-prefix-map options.
However those were reverted later (commit rG3bb24bf25767ef5bbcef958b484e7a06d8689204) due to broken Windows tests.
This patch restores those replace_path_prefix calls.
It also modifies the prefix matching to be case-insensitive under Windows.
Differential Revision : https://reviews.llvm.org/D76869
Currently, normalize() for posix replaces backslashes to slashes, except
that two backslashes in sequence are kept as-is.
clang calls normalize() to convert \ to / is microsoft compat mode. This
generally works well, but a path like "c:\\foo\\bar.h" with two
backslashes doesn't work due to the exception in normalize().
These paths happen naturally on Windows hosts with e.g.
`#include __FILE__`, and them not working on other hosts makes it
more difficult to write tests for this case.
The special case has been around without justification since this code
was added in r203611 (since then moved around in r215241 r215243). No
integration tests fail if I remove it.
Try removing the special case.
Differential Revision: https://reviews.llvm.org/D79265
This reverts commit fb5fd74685e728b1d5e68d33e9842bcd734b98e6.
Re-instates commit 53913a65b408ade2956061b4c0aaed6bba907403
The fix is to trim off trailing separators, as in `/foo/bar/` and
produce `/foo/bar`. VFS tests rely on this. I added unit tests for
remove_dots.
This reverts commit 53913a65b408ade2956061b4c0aaed6bba907403.
Breaks VFSFromYAMLTest.DirectoryIterationSameDirMultipleEntries
in SupportTests on non-Windows.
LLD calls this on every source file string in every object file when
writing PDBs, so it is somewhat hot.
Avoid rewriting paths that do not contain path traversal components
(./..). Use find_first_not_of(separators) directly instead of using the
path iterators. The path component iterators appear to be slow, and
directly searching for slashes makes it easier to find double separators
that need to be canonicalized.
I discovered that the VFS relies on remote_dots to not canonicalize
early slashes (/foo or C:/foo) on Windows, so I had to leave that
behavior behind with unit tests for it. This is undesirable, but I claim
that my change is NFC.
Added unit tests for 2 scenarios that were failing.
Made replace_path_prefix back to 3 parameters instead of 5, simplifying the implementation. The other 2 were always used with the default value.
This commit is intended to be the first of 3:
1) simplify/fix replace_path_prefix.
2) use it in the context of -fdebug-prefix-map and -fmacro-prefix-map (see D76869).
3) Make Windows version of replace_path_prefix insensitive to both case and separators (slash vs backslash).
Differential Revision: https://reviews.llvm.org/D77223
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
GCC 8 implements -fmacro-prefix-map. Like -fdebug-prefix-map, it replaces a string prefix for the __FILE__ macro.
-ffile-prefix-map is the union of -fdebug-prefix-map and -fmacro-prefix-map
Reviewed By: rnk, Lekensteyn, maskray
Differential Revision: https://reviews.llvm.org/D49466
This fixes a bug for making path with a //net style root absolute. I
discovered the bug while writing a test case for the VFS, which uses
these paths because they're both legal absolute paths on Windows and
Unix.
Differential revision: https://reviews.llvm.org/D65675
llvm-svn: 368053
When moving a temp file, explicitly set the file descriptor to -1 so we
can never accidentally close the moved-from TempFile.
Differential revision: https://reviews.llvm.org/D63087
llvm-svn: 363083
I added a diagnostic along the lines of `-Wpessimizing-move` to detect `return x = y` suppressing copy elision, but I don't know if the diagnostic is really worth it. Anyway, here are the places where my diagnostic reported that copy elision would have been possible if not for the assignment.
P1155R1 in the post-San-Diego WG21 (C++ committee) mailing discusses whether WG21 should fix this pitfall by just changing the core language to permit copy elision in cases like these.
(Kona update: The bulk of P1155 is proceeding to CWG review, but specifically *not* the parts that explored the notion of permitting copy-elision in these specific cases.)
Reviewed By: dblaikie
Author: Arthur O'Dwyer
Differential Revision: https://reviews.llvm.org/D54885
llvm-svn: 359236
It turns out that I mesread the man page and fcopyfile(3) does not
actually support COPYFILE_CLONE for files.
<rdar://problem/50148757>
llvm-svn: 359127
This change makes linking into .build-id atomic and safe to use.
Some users under particular workflows are reporting that this races
more than half the time under particular conditions.
llvm-svn: 356404
Moved the remove of the temporary file to after the close to avoid
remove failures caused by ETXTBSY errors.
This issue was seen when FileOutputBuffer falls back to an in memory
buffer due to the inability to mmap the on disk file. This occurred when
running LLD on an Ubuntu VM in VirtualBox on a Windows host attempting
to write the output to a VirtualBox shared folder.
Differential Revision: https://reviews.llvm.org/D57960
llvm-svn: 354017
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
The version of make_absolute which accepted a specific directory to use
as the "base" for the computation could never fail, even though it
returned a std::error_code. The reason for that seems to be historical
-- the CWD flavour (which can fail due to failure to retrieve CWD) was
there first, and the new version was implemented by extending that.
This removes the error return value from the non-CWD overload and
reimplements the CWD version on top of that. This enables us to remove
some dead code where people were pessimistically trying to handle the
errors returned from this function.
Reviewers: zturner, sammccall
Subscribers: hiraditya, kristina, llvm-commits
Differential Revision: https://reviews.llvm.org/D56599
llvm-svn: 351317
This is available on most platforms (Linux/Mac/Win/BSD) with no extra syscalls.
On other platforms (e.g. Solaris) we stat() if this information is requested.
This will allow switching clang's VFS to efficiently expose (path, type) when
traversing a directory. Currently it exposes an entire Status, but does so by
calling fs::status() on all platforms.
Almost all callers only need the path, and all callers only need (path, type).
Patch by sammccall (Sam McCall)
Differential Revision: https://reviews.llvm.org/D51918
llvm-svn: 342089
In r338216 / D49860 TempFile::keep was extended to allow keeping across
filesystems. The aim on Windows was to have this happen in rename_internal
using the existing system API. However, to fix an issue and preserve the
idea of "renaming" not being a move, put Windows keep-across-filesystem in
TempFile::keep.
Differential Revision: https://reviews.llvm.org/D50048
llvm-svn: 338841
Summary:
On Windows, TempFile::create() was prone to failing with permission
denied errors when a process created many tempfiles without providing
a model large enough to accommodate them. There was also a problem
with createUniqueEntity getting into an infinite loop when all names
permitted by the model are in use. This change fixes both of these
problems and adds a unit test for them.
Reviewers: pcc, rnk, zturner
Reviewed By: zturner
Subscribers: inglorion, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D50126
llvm-svn: 338745
Dsymutil's update functionality was broken on Windows because we tried
to rename a file while we're holding open handles to that file. TempFile
provides a solution for this through its keep(Twine) method. This patch
changes dsymutil to make use of that functionality.
Differential revision: https://reviews.llvm.org/D49860
llvm-svn: 338216
FileOutputBuffer creates a temp file and on commit atomically
renames the temp file to the destination file. Sometimes we
want to modify an existing file in place, but still have the
atomicity guarantee. To do this we can initialize the contents
of the temp file from the destination file (if it exists), that
way the resulting FileOutputBuffer can have only selective
bytes modified. Committing will then atomically replace the
destination file as desired.
llvm-svn: 335902
On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).
We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.
One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.
A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.
Differential Revision: https://reviews.llvm.org/D48051
llvm-svn: 334630
This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition. The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum. The second controls more flags-like values.
This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before. This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.
llvm-svn: 334221
There was only one place in the entire codebase where a non
default value was being passed, and that place was already hidden
in an implementation file. So we can delete the extra parameter
and all existing clients continue to work as they always have,
while making the interface a bit simpler.
Differential Revision: https://reviews.llvm.org/D47789
llvm-svn: 334046
Summary:
Various path functions were not treating paths consisting of slashes
alone consistently. For example, the iterator-based accessors decomposed the
path "///" into two elements: "/" and ".". This is not too bad, but it
is different from the behavior specified by posix:
```
A pathname that contains ***at least one non-slash character*** and that
ends with one or more trailing slashes shall be resolved as if a single
dot character ( '.' ) were appended to the pathname.
```
More importantly, this was different from how we treated the same path
in the filename+parent_path functions, which decomposed this path into
"." and "". This was completely wrong as it lost the information that
this was an absolute path which referred to the root directory.
This patch fixes this behavior by making sure all functions treat paths
consisting of (back)slashes alone the same way as "/". I.e., the
iterator-based functions will just report one component ("/"), and the
filename+parent_path will decompose them into "/" and "".
A slightly controversial topic here may be the treatment of "//". Posix
says that paths beginning with "//" may have special meaning and indeed
we have code which parses paths like "//net/foo/bar" specially. However,
as we were already not being consistent in parsing the "//" string
alone, and any special parsing for it would complicate the code further,
I chose to treat it the same way as longer sequences of slashes (which
are guaranteed to be the same as "/").
Another slight change of behavior is in the parsing of paths like
"//net//". Previously the last component of this path was ".". However,
as in our parsing the "//net" part in this path was the same as the
"drive" part in "c:\" and the next slash was the "root directory", it
made sense to treat "//net//" the same way as "//net/" (i.e., not to add
the extra "." component at the end).
Reviewers: zturner, rnk, dblaikie, Bigcheese
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D45942
llvm-svn: 331876
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
llvm-svn: 331184
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of a reinvented one.
See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.
This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.
llvm-svn: 331127
Summary:
This commit changes semantics of createUniqueFile and
createTemporaryFile variants that do not return file descriptors.
Previously they only checked if files exist, therefore being subject
to race conditions. Now they will create an empty file to avoid them.
Functions that do not create a file are now called
getPotentiallyUniqueTempFileName and getPotentiallyUniqueFileName.
Reviewers: klimek, bkramer, krasimir, JDevlieghere, espindola
Reviewed By: klimek
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36827
llvm-svn: 327851
Without this when lld failed to replace the output file it would leave
the temporary behind. The problem is that the existing logic is
- cancel the delete flag
- rename
We have to cancel first to avoid renaming and then crashing and
deleting the old version. What is missing then is deleting the
temporary file if the rename fails.
This can be an issue on both unix and windows, but I am not sure how
to cause the rename to fail reliably on unix. I think it can be done
on ZFS since it has an ACL system similar to what windows uses, but
adding support for checking that in llvm-lit is probably not worth it.
llvm-svn: 319786
We already allowed keep+discard. It is important to be able to discard
a temporary if a rename fail. It is also convenient as it allows the
use of RAII for discarding.
Allow discarding twice for similar reasons.
llvm-svn: 318867
This requires a small change to TempFile: allowing a discard after a
failed keep.
With this the cache now handles signals and reuses a fd instead of
reopening the file.
llvm-svn: 318322
This just adds a TempFile class and replaces the use in
FileOutputBuffer with it.
The only difference for now is better error handling. Followup work includes:
- Convert other user of temporary files to it.
- Add support for automatically deleting on windows.
- Add a createUnnamed method that returns a potentially unnamed
file. It would be actually unnamed on modern linux and have a
unknown name on windows.
llvm-svn: 318069
This allows clients to avoid an unnecessary fs::status() call on each
directory entry. Because the information returned by FindFirstFileEx
is a subset of the information returned by a regular status() call,
I needed to extract a base class from file_status that contains only
that information.
On my machine, this reduces the time required to enumerate a ThinLTO
cache directory containing 520k files from almost 4 minutes to less
than 2 seconds.
Differential Revision: https://reviews.llvm.org/D38716
llvm-svn: 315378
