Summary:
There was a subtle, but pretty important difference between the Slice
and regular versions of this function. The Slice function was
zero-initializing the rest of the buffer when the read syscall returned
less bytes than expected, while the regular function did not.
This patch removes the inconsistency by making both functions *not*
zero-initialize the buffer. The zeroing code is moved to the
MemoryBuffer class, which is currently the only user of this code. This
makes the API more consistent, and the code shorter.
While in there, I also refactor the functions to return the number of
bytes through the regular return value (via Expected<size_t>) instead of
a separate by-ref argument.
Reviewers: aganea, rnk
Subscribers: kristina, Bigcheese, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D66471
llvm-svn: 369627
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
llvm-svn: 369013
We built a StringRef from a string literal which we then converted to a
std::string to call c_str(). Just use a pointer to the string literal
instead of a StringRef.
No behavior change.
Differential Revision: https://reviews.llvm.org/D65890
llvm-svn: 368187
COPYFILE_CLONE is only defined on newer macOS versions, using it without
check breaks build on systems running legacy OS and toolchain.
Differential Revision: https://reviews.llvm.org/D65317
llvm-svn: 367084
While 'd_type' is a non-standard extension to `struct dirent`, only
glibc signals its presence with a macro '_DIRENT_HAVE_D_TYPE'.
However, any platform with 'd_type' also includes a way to convert to
mode_t values using the macro 'DTTOIF', so we can check for that alone
and still be confident that the 'd_type' member exists.
(If this turns out to be wrong, I'll go back and set up an actual
CMake check.)
I couldn't think of how to write a test for this, because I couldn't
think of how to test that a 'stat' call doesn't happen without
controlling the filesystem or intercepting 'stat', and there's no good
cross-platform way to do that that I know of.
Follow-up (almost a year later) to r342089.
rdar://problem/50592673
https://reviews.llvm.org/D64940
llvm-svn: 366486
There is currently an EPERM error when a regular user executes `llvm-objcopy a.o /dev/null`.
Worse, root can even change the mode bits of /dev/null.
Fix it by checking if the output file is special.
A new overload of llvm::sys::fs::setPermissions with FD as the parameter
is added. Users should provide `perm & ~umask` as the parameter if they
intend to respect umask.
The existing overload of llvm::sys::fs::setPermissions may be deleted if
we can find an implementation of fchmod() on Windows. fchmod() is
usually better than chmod() because it saves syscalls and can avoid race
condition.
Reviewed By: jakehehrlich, jhenderson
Differential Revision: https://reviews.llvm.org/D64236
llvm-svn: 365753
Summary:
On Windows, Posix integer file descriptors are a compatibility layer
over native file handles provided by the C runtime. There is a hard
limit on the maximum number of file descriptors that a process can open,
and the limit is 8192. LLD typically doesn't run into this limit because
it opens input files, maps them into memory, and then immediately closes
the file descriptor. This prevents it from running out of FDs.
For various reasons, I'd like to open handles to every input file and
keep them open during linking. That requires migrating MemoryBuffer over
to taking open native file handles instead of integer FDs.
Reviewers: aganea, Bigcheese
Reviewed By: aganea
Subscribers: smeenai, silvas, mehdi_amini, hiraditya, steven_wu, dexonsmith, dang, llvm-commits, zturner
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D63453
llvm-svn: 365588
Summary:
Previously, we'd pass a nullptr to std::string and crash().
This case happens when the binary is deleted while being used (e.g. rebuilding clangd).
Reviewers: kadircet
Subscribers: ilya-biryukov, kristina, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64068
llvm-svn: 364936
Summary: This patch changes fs::setPermissions to optionally set permissions while respecting the umask. It also adds the function fs::getUmask() which returns the current umask.
Reviewers: jhenderson, rupprecht, aprantl, lhames
Reviewed By: jhenderson, rupprecht
Subscribers: sanaanajjar231288, hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D63583
llvm-svn: 364621
Summary:
Emscripten's libc doesn't define MNT_LOCAL, thus causing a build
failure in the fallback path. However, to the best of my knowledge,
it also doesn't support remote file system mounts, so we may simply
return `true` here (as we do for e.g. Fuchsia). With this fix, the
core LLVM libraries build correctly under emscripten (though some
of the tools and utils do not).
Reviewers: kripken
Differential Revision: https://reviews.llvm.org/D63688
llvm-svn: 364143
Summary:
- Use alternative to MAP_ANONYMOUS for allocating mapped memory if it isn't available
- Use strtok_r instead of strsep as part of getting program path
- Don't try to find the width of a terminal using "struct winsize" and TIOCGWINSZ on POSIX builds. These aren't defined under POSIX (even though some platforms make them available when they shouldn't), so just check if we are doing a X/Open or POSIX compliant build first.
Author: daltenty
Reviewers: hubert.reinterpretcast, xingxue, andusy
Reviewed By: hubert.reinterpretcast
Subscribers: MaskRay, jsji, hiraditya, kristina, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61326
llvm-svn: 360898
This patch changes the return type of sys::Process::getPageSize to
Expected<unsigned> to account for the fact that the underlying syscalls used to
obtain the page size may fail (see below).
For clients who use the page size as an optimization only this patch adds a new
method, getPageSizeEstimate, which calls through to getPageSize but discards
any error returned and substitues a "reasonable" page size estimate estimate
instead. All existing LLVM clients are updated to call getPageSizeEstimate
rather than getPageSize.
On Unix, sys::Process::getPageSize is implemented in terms of getpagesize or
sysconf, depending on which macros are set. The sysconf call is documented to
return -1 on failure. On Darwin getpagesize is implemented in terms of sysconf
and may also fail (though the manpage documentation does not mention this).
These failures have been observed in practice when highly restrictive sandbox
permissions have been applied. Without this patch, the result is that
getPageSize returns -1, which wreaks havoc on any subsequent code that was
assuming a sane page size value.
<rdar://problem/41654857>
Reviewers: dblaikie, echristo
Subscribers: kristina, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59107
llvm-svn: 360221
Currently, clang's libTooling passes this function a fake argv0, which
means that no libTooling tools can find the standard headers on FreeBSD.
With this change, these will now work on any FreeBSD systems that have
procfs mounted. This isn't the right fix for the libTooling issue, but
it does bring the FreeBSD implementation of getExecutablePath closer to
the Linux and macOS implementations.
llvm-svn: 359427
It turns out that I mesread the man page and fcopyfile(3) does not
actually support COPYFILE_CLONE for files.
<rdar://problem/50148757>
llvm-svn: 359127
Summary:
`posix_fallocate` can fail if the underlying filesystem does not support
it; and, on AIX, such a failure is reported by a return value of
`ENOTSUP`. The existing code checks only for `EOPNOTSUPP`, which may
share the same value as `ENOTSUP`, but is not required to.
Reviewers: xingxue, sfertile, jasonliu
Reviewed By: xingxue
Subscribers: kristina, jsji, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60175
llvm-svn: 357662
Summary:
On AIX, we can determine whether a filesystem is remote using `mntctl`.
If the information is not found, then claim that the file is remote
(since that is the more restrictive case). Testing for the associated
interface is restored with a modified version of the unit test from
rL295768.
Reviewers: jasonliu, xingxue
Reviewed By: xingxue
Subscribers: jsji, apaprocki, Hahnfeld, zturner, krytarowski, kristina, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58801
llvm-svn: 357333
If the concatenation of arguments dir and bin has at least PATH_MAX
characters the call to snprintf will truncate. The result will usually
not exist, but if it does it's actually incorrect to return that the
path exists.
(Motivated by GCC compiler warning about format truncation.)
Differential Revision: https://reviews.llvm.org/D58835
llvm-svn: 356036
Use sysctl() to implement getMainExecutable() on NetBSD, rather than
trying to guess the correct path from argv[0]. This is one
of the fixes to recent clang-check-mac-libcxx-fixed-compilation-db.cpp
test failure on NetBSD.
This has been historically done on both FreeBSD and NetBSD in r303015,
and reverted in r303285 due to buggy implementation on FreeBSD.
However, FWIK the NetBSD implementation does not suffer from the same
bugs and is more reliable than playing with argv[0].
Differential Revision: https://reviews.llvm.org/D56975
llvm-svn: 355283
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
getLastAccessedTime() and getLastModificationTime() provided times in nanoseconds but with only 1 second resolution, even when the underlying file system could provide more precise times than that.
These changes add sub-second precision for unix platforms that support improved precision.
Also add some comments to make sure people are aware that the resolution of times can vary across different file systems.
Reviewers: labath, zturner, aaron.ballman, kristina
Reviewed By: aaron.ballman, kristina
Subscribers: lebedev.ri, mgorny, kristina, llvm-commits
Differential Revision: https://reviews.llvm.org/D54826
llvm-svn: 347530
In D54435 there was some discussion about the expand_tilde flag for
real_path that I wanted to expose through the VFS. The consensus is that
these two things should be separate functions. Since we already have the
code for this I went ahead and added a function expand_tilde that does
just that.
Differential revision: https://reviews.llvm.org/D54448
llvm-svn: 346776
On GNU/Hurd, llvm-config is returning bogus value, such as:
$ llvm-config-6.0 --includedir
/usr/include
while it should be:
$ llvm-config-6.0 --includedir
/usr/lib/llvm-6.0/include
This is because getMainExecutable does not get the actual installation
path. On GNU/Hurd, /proc/self/exe is indeed a symlink to the path that
was used to start the program, and not the eventual binary file. Llvm's
getMainExecutable thus needs to run realpath over it to get the actual
place where llvm was installed (/usr/lib/llvm-6.0/bin/llvm-config), and
not /usr/bin/llvm-config-6.0. This will not change the result on Linux,
where /proc/self/exe already points to the eventual file.
Patch by Samuel Thibault!
While making changes here, I reformatted this block a bit to reduce
indentation and match 2 space indent style.
Differential Revision: https://reviews.llvm.org/D53557
llvm-svn: 345104
This is available on most platforms (Linux/Mac/Win/BSD) with no extra syscalls.
On other platforms (e.g. Solaris) we stat() if this information is requested.
This will allow switching clang's VFS to efficiently expose (path, type) when
traversing a directory. Currently it exposes an entire Status, but does so by
calling fs::status() on all platforms.
Almost all callers only need the path, and all callers only need (path, type).
Patch by sammccall (Sam McCall)
Differential Revision: https://reviews.llvm.org/D51918
llvm-svn: 342089
In Bionic, open can be overloaded for _FORTIFY_SOURCE support, causing
compile errors of RetryAfterSignal due to overload resolution. Wrapping
the call in a lambda avoids this.
Based on a patch by Chih-Wei Huang <cwhuang@linux.org.tw>!
llvm-svn: 340751
Summary:
Add an overload to sys::fs::setLastModificationAndAccessTime that allows setting last access and modification times separately. This will allow tools to use this API when they want to preserve both the access and modification times from an input file, which may be different.
Also note that both the POSIX (futimens/futimes) and Windows (SetFileTime) APIs take the two timestamps in the order of (1) access (2) modification time, so this renames the method to "setLastAccessAndModificationTime" to make it clear which timestamp is which.
For existing callers, the 1-arg overload just sets both timestamps to the same thing.
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D50521
llvm-svn: 339628
On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).
We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.
One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.
A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.
Differential Revision: https://reviews.llvm.org/D48051
llvm-svn: 334630
It's been reported
<http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20180611/559616.html>
that template argument deduction for RetryAfterSignal fails if open is
not prefixed with "::".
This should help us build correctly on those platforms and explicitly
specifying the namespace is more correct anyway.
llvm-svn: 334403
O_CLOEXEC is the right default, but occasionally you don't
want this. This is especially true for tools like debuggers
where you might need to spawn the child process with specific
files already open, but it's occasionally useful in other
scenarios as well, like when you want to do some IPC between
parent and child.
llvm-svn: 334293
This one allows much more flexibility than the standard
openFileForRead / openFileForWrite functions. Since there is now
just one "real" function that does the work, all other implementations
simply delegate to this one.
llvm-svn: 334246
This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition. The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum. The second controls more flags-like values.
This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before. This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.
llvm-svn: 334221
Windows' CRT has a limit of 512 open file descriptors, and fds which are
generated by converting a HANDLE via _get_osfhandle count towards this
limit as well.
Regardless, often you find yourself marshalling back and forth between
native HANDLE objects and fds anyway. If we know from the getgo that
we're going to need to work directly with the handle, we can cut out the
marshalling layer while also not contributing to filling up the CRT's
very limited handle table.
On Unix these functions just delegate directly to the existing set of
functions since an fd *is* the native file type. It would be nice, very
long term, if we could convert most uses of fds to file_t.
Differential Revision: https://reviews.llvm.org/D47688
llvm-svn: 333945
These are necessary changes to support building LLVM for Fuchsia.
While these are not sufficient to run on Fuchsia, they are still
useful when cross-compiling LLVM libraries and runtimes for Fuchsia.
Differential Revision: https://reviews.llvm.org/D46345
llvm-svn: 331423
The autoconf manual: "This macro is obsolescent, as all current systems with
directory libraries have <dirent.h>. New programs need not use this macro."
llvm-svn: 328989
This is like MemoryBuffer (read-only) and WritableMemoryBuffer
(writable private), but where the underlying file can be modified
after writing. This is useful when you want to open a file, make
some targeted edits, and then write it back out.
Differential Revision: https://reviews.llvm.org/D44230
llvm-svn: 327057
According to the docs on opegroup.org, the function can return
EINVAL if:
The len argument is less than zero, or the offset argument is less
than zero, or the underlying file system does not support this
operation.
I'd say it's a peculiar choice (when EONOTSUPP is right there), but
let's keep POSIX happy for now. This was independently discovered
by Mark Millard (on FreeBSD/ZFS).
Quickly ack'ed by Rui on IRC.
llvm-svn: 317535
