1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-19 19:12:56 +02:00
llvm-mirror/lib/Support
Alexander Kornienko 73a92659e5 commandLineFitsWithinSystemLimits Overestimates System Limits
Summary:
The function `llvm::sys::commandLineFitsWithinSystemLimits` appears to be overestimating the system limits. This issue was discovered while attempting to enable response files in the Swift compiler. When the compiler submits its frontend jobs, those jobs are subjected to the system limits on command line length. `commandLineFitsWithinSystemLimits` is used to determine if the job's arguments need to be wrapped in a response file. There are some cases where the argument size for the job passes `commandLineFitsWithinSystemLimits`, but actually exceeds the real system limit, and the job fails.

`clang` also uses this function to decide whether or not to wrap it's job arguments in response files. See: https://github.com/llvm-mirror/clang/blob/master/lib/Driver/Driver.cpp#L1341. Clang will also fail for response files who's size falls within a certain range. I wrote a script that should find a failure point for `clang++`. All that is needed to run it is Python 2.7, and a simple "hello world" program for `test.cc`. It should run on Linux and on macOS. The script is available here: https://gist.github.com/dabelknap/71bd083cd06b91c5b3cef6a7f4d3d427. When it hits a failure point, you should see a `clang: error: unable to execute command: posix_spawn failed: Argument list too long`.

The proposed solution is to mirror the behavior of `xargs` in `commandLinefitsWithinSystemLimits`. `xargs` defaults to 128k for the command line length size (See: https://fossies.org/dox/findutils-4.6.0/buildcmd_8c_source.html#l00551). It adjusts this depending on the value of `ARG_MAX`.

Reviewers: alexfh

Reviewed By: alexfh

Subscribers: llvm-commits

Tags: #clang

Patch by Austin Belknap!

Differential Revision: https://reviews.llvm.org/D47795

llvm-svn: 334295
2018-06-08 15:19:16 +00:00
..
Unix commandLineFitsWithinSystemLimits Overestimates System Limits 2018-06-08 15:19:16 +00:00
Windows Clean up some code in Program. 2018-06-08 15:16:25 +00:00
Allocator.cpp Recover some overzealously removed includes. 2017-12-13 22:21:02 +00:00
AMDGPUMetadata.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
APFloat.cpp [APFloat] Set losesInfo on no-op convert 2018-05-11 09:45:42 +00:00
APInt.cpp Rename DEBUG macro to LLVM_DEBUG. 2018-05-14 12:53:11 +00:00
APSInt.cpp
ARMAttributeParser.cpp Test commit 2018-02-26 13:05:18 +00:00
ARMBuildAttrs.cpp Sort the remaining #include lines in include/... and lib/.... 2017-06-06 11:49:48 +00:00
ARMWinEH.cpp
Atomic.cpp Fix llvm-for-windows-on-linux build after LLVM r272701. 2017-08-03 20:10:47 +00:00
BinaryStreamError.cpp
BinaryStreamReader.cpp Add a BinarySubstreamRef, and a method to read one. 2017-06-23 16:38:40 +00:00
BinaryStreamRef.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
BinaryStreamWriter.cpp [BinaryStream] Support growable streams. 2017-11-27 18:48:37 +00:00
BlockFrequency.cpp Remove redundant includes from lib/Support. 2017-12-13 21:30:58 +00:00
BranchProbability.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
CachePruning.cpp Rename DEBUG macro to LLVM_DEBUG. 2018-05-14 12:53:11 +00:00
Chrono.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
circular_raw_ostream.cpp [Support] Remove unused raw_ostream::handle whose anchor role was superseded by anchor() 2018-06-02 06:00:35 +00:00
CMakeLists.txt [Support] Link libzircon.so when building LLVM for Fuchsia 2018-06-07 21:01:32 +00:00
CodeGenCoverage.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
COM.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
CommandLine.cpp [CommandLine] Error message for incorrect PositionalEatArgs usage 2018-05-14 23:26:06 +00:00
Compression.cpp
ConvertUTF.cpp Sort the remaining #include lines in include/... and lib/.... 2017-06-06 11:49:48 +00:00
ConvertUTFWrapper.cpp Sort the remaining #include lines in include/... and lib/.... 2017-06-06 11:49:48 +00:00
COPYRIGHT.regex
CrashRecoveryContext.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
DAGDeltaAlgorithm.cpp Rename DEBUG macro to LLVM_DEBUG. 2018-05-14 12:53:11 +00:00
DataExtractor.cpp [DWARF] Support for DW_FORM_strx3 and complete support for DW_FORM_strx{1,2,4} 2017-06-21 19:37:44 +00:00
Debug.cpp Rename DEBUG macro to LLVM_DEBUG. 2018-05-14 12:53:11 +00:00
DebugCounter.cpp [DebugCounter] Make -debug-counter cl::Hidden. 2018-04-01 22:16:52 +00:00
DeltaAlgorithm.cpp
DJB.cpp Make llvm::djbHash an inline function. 2018-03-02 22:00:38 +00:00
DynamicLibrary.cpp s/LLVM_ON_WIN32/_WIN32/, llvm 2018-04-29 00:45:03 +00:00
Errno.cpp Sort the remaining #include lines in include/... and lib/.... 2017-06-06 11:49:48 +00:00
Error.cpp [Support] Make llvm::Error and Expected faster. 2017-11-09 19:31:52 +00:00
ErrorHandling.cpp s/LLVM_ON_WIN32/_WIN32/, llvm 2018-04-29 00:45:03 +00:00
FileOutputBuffer.cpp Fix unused private variable. 2018-06-07 20:07:08 +00:00
FileUtilities.cpp
FoldingSet.cpp Revert commit 333506 2018-05-30 09:01:12 +00:00
FormattedStream.cpp Sort the remaining #include lines in include/... and lib/.... 2017-06-06 11:49:48 +00:00
FormatVariadic.cpp Remove unused variables. No functionality change. 2017-10-08 19:11:02 +00:00
GlobPattern.cpp [Support/GlobPattern] - Do not crash when pattern has characters with int value < 0. 2017-07-31 09:26:50 +00:00
GraphWriter.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
Hashing.cpp
Host.cpp [x86] invpcid LLVM intrinsic 2018-05-25 06:32:05 +00:00
InitLLVM.cpp Rename sys::Process::GetArgumentVector -> sys::windows::GetCommandLineArguments 2018-04-17 21:09:16 +00:00
IntEqClasses.cpp
IntervalMap.cpp
JamCRC.cpp
KnownBits.cpp [KnownBits][ValueTracking] Move the math for calculating known bits for add/sub into a static method in KnownBits object 2017-08-08 16:29:35 +00:00
LEB128.cpp
LineIterator.cpp
LLVMBuild.txt
Locale.cpp IWYU for llvm-config.h, removals. Also see r331184. 2018-04-30 15:26:01 +00:00
LockFileManager.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
LowLevelType.cpp [GlobalISel] Enable legalizing non-power-of-2 sized types. 2017-11-07 10:34:34 +00:00
ManagedStatic.cpp [NFC] Remove inaccurate comment 2018-05-12 19:39:34 +00:00
MathExtras.cpp
MD5.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
Memory.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
MemoryBuffer.cpp [FileSystem] Split up the OpenFlags enumeration. 2018-06-07 19:58:58 +00:00
Mutex.cpp Revert commit 333506 2018-05-30 09:01:12 +00:00
NativeFormatting.cpp Support: Add missing #include. 2018-01-18 20:49:33 +00:00
Options.cpp
Parallel.cpp Remove unused SyncExecutor and make it clearer that the whole file is only used if LLVM_ENABLE_THREADS 2018-05-11 15:25:38 +00:00
Path.cpp [FileSystem] Split up the OpenFlags enumeration. 2018-06-07 19:58:58 +00:00
PluginLoader.cpp
PrettyStackTrace.cpp Add more initializers to quiet a clang warning 2018-01-30 16:02:32 +00:00
Process.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
Program.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
RandomNumberGenerator.cpp Rename DEBUG macro to LLVM_DEBUG. 2018-05-14 12:53:11 +00:00
raw_os_ostream.cpp
raw_ostream.cpp [FileSystem] Split up the OpenFlags enumeration. 2018-06-07 19:58:58 +00:00
README.txt.system
regcomp.c Attempt to heal bots after r328970. 2018-04-02 13:49:35 +00:00
regengine.inc
regerror.c
regex2.h Support/reg*.h: Make headers include their dependencies 2017-10-26 20:23:11 +00:00
regex_impl.h
Regex.cpp Fix compilation on Darwin with expensive checks. 2018-03-12 11:01:05 +00:00
regexec.c
regfree.c
regstrlcpy.c
regutils.h
RWMutex.cpp Fix include of config.h that was incorrectly changed in r331184 2018-05-03 21:59:13 +00:00
ScaledNumber.cpp
ScopedPrinter.cpp Remove redundant includes from lib/Support. 2017-12-13 21:30:58 +00:00
SHA1.cpp Fix header comment on SHA1 code. 2018-03-09 00:23:35 +00:00
Signals.cpp Signal handling should be signal-safe 2018-05-16 17:25:35 +00:00
SmallPtrSet.cpp Revert commit 333506 2018-05-30 09:01:12 +00:00
SmallVector.cpp Revert commit 333506 2018-05-30 09:01:12 +00:00
SourceMgr.cpp [Support] Change std::sort to llvm::sort in response to r327219 2018-04-08 16:46:22 +00:00
SpecialCaseList.cpp Extend SpecialCaseList to allow users to blame matches on entries in the file. 2017-11-07 21:16:46 +00:00
Statistic.cpp Fix lock order inversion between ManagedStatic and Statistic 2018-04-17 23:37:18 +00:00
StringExtras.cpp [ADT] Make escaping fn conform to coding guidelines 2018-05-31 17:01:42 +00:00
StringMap.cpp Revert commit 333506 2018-05-30 09:01:12 +00:00
StringPool.cpp
StringRef.cpp Fix APFloat from string conversion for Inf 2017-12-19 04:27:39 +00:00
StringSaver.cpp
SystemUtils.cpp
TargetParser.cpp AArch64: Implement support for the shadowcallstack attribute. 2018-04-04 21:55:44 +00:00
TargetRegistry.cpp Add backend name to Target to enable runtime info to be fed back into TableGen 2017-11-15 23:55:44 +00:00
TarWriter.cpp [FileSystem] Split up the OpenFlags enumeration. 2018-06-07 19:58:58 +00:00
Threading.cpp s/LLVM_ON_WIN32/_WIN32/, llvm 2018-04-29 00:45:03 +00:00
ThreadLocal.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
ThreadPool.cpp Speculative build fix for lld on Linux after Michael's #include removals 2017-12-13 22:12:57 +00:00
Timer.cpp [Timers] TimerGroup: add constructor from StringMap<TimeRecord> 2018-05-16 18:16:01 +00:00
ToolOutputFile.cpp [Support] Rename tool_output_file to ToolOutputFile, NFC 2017-09-23 01:03:17 +00:00
TrigramIndex.cpp Sort the remaining #include lines in include/... and lib/.... 2017-06-06 11:49:48 +00:00
Triple.cpp Remove \brief commands from doxygen comments. 2018-05-01 15:54:18 +00:00
Twine.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
Unicode.cpp
UnicodeCaseFold.cpp Resubmit r325107 (case folding DJB hash) 2018-02-21 22:36:31 +00:00
Valgrind.cpp
Watchdog.cpp IWYU for llvm-config.h in llvm, additions. 2018-04-30 14:59:11 +00:00
WithColor.cpp [Support] Move header to WithColor header 2018-05-24 11:47:20 +00:00
xxhash.cpp
YAMLParser.cpp Remove @brief commands from doxygen comments, too. 2018-05-01 16:10:38 +00:00
YAMLTraits.cpp [YAML] Escape non-printable multibyte UTF8 in Output::scalarString. 2018-03-27 19:52:45 +00:00

Design Of lib/System
====================

The software in this directory is designed to completely shield LLVM from any
and all operating system specific functionality. It is not intended to be a
complete operating system wrapper (such as ACE), but only to provide the
functionality necessary to support LLVM.

The software located here, of necessity, has very specific and stringent design
rules. Violation of these rules means that cracks in the shield could form and
the primary goal of the library is defeated. By consistently using this library,
LLVM becomes more easily ported to new platforms since the only thing requiring
porting is this library.

Complete documentation for the library can be found in the file:
  llvm/docs/SystemLibrary.html
or at this URL:
  http://llvm.org/docs/SystemLibrary.html

While we recommend that you read the more detailed documentation, for the
impatient, here's a high level summary of the library's requirements.

 1. No system header files are to be exposed through the interface.
 2. Std C++ and Std C header files are okay to be exposed through the interface.
 3. No exposed system-specific functions.
 4. No exposed system-specific data.
 5. Data in lib/System classes must use only simple C++ intrinsic types.
 6. Errors are handled by returning "true" and setting an optional std::string
 7. Library must not throw any exceptions, period.
 8. Interface functions must not have throw() specifications.
 9. No duplicate function impementations are permitted within an operating
    system class.

To accomplish these requirements, the library has numerous design criteria that
must be satisfied. Here's a high level summary of the library's design criteria:

 1. No unused functionality (only what LLVM needs)
 2. High-Level Interfaces
 3. Use Opaque Classes
 4. Common Implementations
 5. Multiple Implementations
 6. Minimize Memory Allocation
 7. No Virtual Methods