Many of the MachO load commands can have data appended after the command structure. This data is frequently strings, but can actually be anything. This patch adds support for three optional fields on load command yaml descriptions.
The new PayloadString YAML field is populated with the data after load commands known to have strings as extra data.
The new ZeroPadBytes YAML field is a count of zero'd bytes after the end of the load command structure before the next command. This can apply anywhere in the file. MachO2YAML verifies that bytes are zero before populating this field, and YAML2MachO will add zero'd bytes.
The new PayloadBytes YAML field stores all bytes after the end of the load command structure before the next command if they are non-zero. This is a catch all for all unhandled bytes. If MachO2Yaml populates PayloadBytes it will not populate ZeroPadBytes, instead zero'd bytes will be in the PayloadBytes structure.
llvm-svn: 270115
This adds support for all the MachO *_command structures. The load_command payloads still are not represented, but that will come next.
llvm-svn: 269808
This adds support for all the MachO *_command structures. The load_command payloads still are not represented, but that will come next.
llvm-svn: 269782
This patch adds basic support for MachO::load_command. Load command types and sizes are encoded in the YAML and expanded back into MachO.
The YAML doesn't yet support load command structs, that is coming next. In the meantime as a temporary measure when writing MachO files the load commands are padded with zeros so that the generated binary is valid.
llvm-svn: 269442
I've added the reserved field as an "optional" in YAML, but I've added asserts in the yaml2macho code to enforce that the field is present in mach_header_64, but not in mach_header.
llvm-svn: 269320
Adding the initial files for adding MachO support to yaml2obj. Passing a MachO file will result in an error.
I will be implementing obj2yaml and yaml2obj for MachO in parallel so that one can be used to test the other.
llvm-svn: 269244
Removed some unused headers, replaced some headers with forward class declarations.
Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'
Patch by Eugene Kosov <claprix@yandex.ru>
Differential Revision: http://reviews.llvm.org/D19219
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
The section alignment field was marked optional but not provided a
default value: initialize it with 0.
While we are here, ensure that the section alignment is plausible.
llvm-svn: 263692
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html
"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi
Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark
Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D16471
llvm-svn: 258861
In this mode it just tries to tail merge the strings without imposing any other
format constrains. It will not, for example, add a null byte between them.
Also add support for keeping a tentative size and offset if we decide to
not optimize after all.
This will be used shortly in lld for merging SHF_STRINGS sections.
llvm-svn: 251153
Use AddressAlign field's value to properly align sections content in the
yaml2obj tool. Before this change the yaml2obj ignored AddressAlign and
always aligned section on 16 bytes boundary.
llvm-svn: 241674
SHT_NOBITS sections do not have content in an object file. Now the yaml2obj
tool does not accept `Content` field for such sections, and the obj2yaml
tool does not attempt to read the section content from a file.
Restore r241350 and r241352.
llvm-svn: 241377
r241350 broke lld tests.
r241352 depends on r241350.
Original messages:
"[ELFYAML] Fix handling SHT_NOBITS sections by obj2yaml/yaml2obj tools"
"[ELFYAML] Make the Size field for .bss section optional"
llvm-svn: 241354
SHT_NOBITS sections do not have content in an object file. Now yaml2obj
tool does not accept `Content` field for such sections, and obj2yaml
tool does not attempt to read the section content from a file.
llvm-svn: 241350
Now caller of ELFState::writeSectionContent() methods is responsible to check
a section type and selects an appropriate writeSectionContent method.
So unexpected section type inside writeSectionContent method indicates
a wrong usage of the method and should be guarded by assert.
llvm-svn: 236808
ARM32 ELF R_ARM_V4BX relocation format is a special relocation type
that records the location of an ARMv4t BX instruction to enable a
static linker to generate ARMv4 compatible instructions. This
relocation does not contain a reference symbol.
This patch enabled its creation by removing the requeriment of a
relocation symbol target in ELFState<ELFT>::writeSectionContent.
llvm-svn: 235513
MIPS64 ELF file has a very specific relocation record format. Each
record might specify up to three relocation operations. So the `r_info`
field in fact consists of three relocation type sub-fields and optional
code of "special" symbols.
http://techpubs.sgi.com/library/manuals/4000/007-4658-001/pdf/007-4658-001.pdf
page 40
The patch implements support of the MIPS64 relocation record format in
yaml2obj/obj2yaml tools by introducing new optional Relocation fields:
Type2, Type3, and SpecSym. These fields are recognized only if the
object/YAML file relates to the MIPS64 target.
Differential Revision: http://reviews.llvm.org/D7136
llvm-svn: 227044
utils/sort_includes.py.
I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.
llvm-svn: 225974
Having two ways to do this doesn't seem terribly helpful and
consistently using the insert version (which we already has) seems like
it'll make the code easier to understand to anyone working with standard
data structures. (I also updated many references to the Entry's
key and value to use first() and second instead of getKey{Data,Length,}
and get/setValue - for similar consistency)
Also removes the GetOrCreateValue functions so there's less surface area
to StringMap to fix/improve/change/accommodate move semantics, etc.
llvm-svn: 222319
While this program worked correctly with small example programs, larger
ones tickled this bug. I'm working on a reduction because my program is
quite large.
llvm-svn: 222078
In support of serializing executables, obj2yaml now records the virtual address
and size of sections. It also serializes whatever we strictly need from
the PE header, it expects that it can reconstitute everything else via
inference.
yaml2obj can reconstitute a fully linked executable.
In order to get executables correctly serialized/deserialized, other
bugs were fixed as a circumstance. We now properly respect file and
section alignments. We also avoid writing out string tables unless they
are strictly necessary.
llvm-svn: 221975
The ELF symbol `st_other` field might contain additional flags besides
visibility ones. This patch implements support for some MIPS specific
flags.
llvm-svn: 221491
I think it might make sense to make COFF::MaxNumberOfSections16 be a uint32_t, however, that may have wider-reaching implications in other projects, which is why I did not change that declaration.
llvm-svn: 220384
Teach yaml2obj how to make a bigobj COFF file. Like the rest of LLVM,
we automatically decide whether or not to use regular COFF or bigobj
COFF on the fly depending on how many sections the resulting object
would have.
This ends the task of adding bigobj support to LLVM.
N.B. This was tested by forcing yaml2obj to be used in bigobj mode
regardless of the number of sections. While a dedicated test was
written, the smallest I could make it was 36 MB (!) of yaml and it still
took a significant amount of time to execute on a powerful machine.
llvm-svn: 217858
Teach WinCOFFObjectWriter how to write -mbig-obj style object files;
these object files allow for more sections inside an object file.
Our support for BigObj is notably different from binutils and cl: we
implicitly upgrade object files to BigObj instead of asking the user to
compile the same file *again* but with another flag. This matches up
with how LLVM treats ELF variants.
This was tested by forcing LLVM to always emit BigObj files and running
the entire test suite. A specific test has also been added.
I've lowered the maximum number of sections in a normal COFF file,
VS "14" CTP 3 supports no more than 65279 sections. This is important
otherwise we might not switch to BigObj quickly enough, leaving us with
a COFF file that we couldn't link.
yaml2obj support is all that remains to implement.
Differential Revision: http://reviews.llvm.org/D5349
llvm-svn: 217812
Take a StringRef instead of a "const char *".
Take a "std::error_code &" instead of a "std::string &" for error.
A create static method would be even better, but this patch is already a bit too
big.
llvm-svn: 216393
Add header guards to files that were missing guards. Remove #endif comments
as they don't seem common in LLVM (we can easily add them back if we decide
they're useful)
Changes made by clang-tidy with minor tweaks.
llvm-svn: 215558
Now that we have a lib/MC/MCAnalysis, the dependency was there just because
of two helper classes. Move the two over to MC.
This will allow IRObjectFile to parse inline assembly.
llvm-svn: 212248
Input YAML file might contain multiple object file definitions.
New option `-docnum` allows to specify an ordinal number (starting from 1)
of definition used for an object file generation.
Patch reviewed by Sean Silva.
llvm-svn: 209967
Now the only method to configure ELF section's content and size is to assign
a hexadecimal string to the `Content` field. Unfortunately this way is
completely useless when you need to declare a really large section.
To solve this problem this patch adds one more optional field `Size`
to the `RawContentSection` structure. When yaml2obj generates an ELF file
it uses the following algorithm:
1. If both `Content` and `Size` fields are missed create an empty section.
2. If only `Content` field is missed take section length from the `Size`
field and fill the section by zero.
3. If only `Size` field is missed create a section using data from
the `Content` field.
4. If both `Content` and `Size` fields are provided validate that the `Size`
value is not less than size of `Content` data. Than take section length
from the `Size`, fill beginning of the section by `Content` and the rest
by zero.
Examples
--------
* Create a section 0x10000 bytes long filled by zero
Name: .data
Type: SHT_PROGBITS
Flags: [ SHF_ALLOC ]
Size: 0x10000
* Create a section 0x10000 bytes long starting from 'CA' 'FE' 'BA' 'BE'
Name: .data
Type: SHT_PROGBITS
Flags: [ SHF_ALLOC ]
Content: CAFEBABE
Size: 0x10000
The patch reviewed by Michael Spencer.
llvm-svn: 208995
We already do this for shstrtab, so might as well do it for strtab. This
extracts the string table building code into a separate class. The idea
is to use it for other object formats too.
I mostly wanted to do this for the general principle, but it does save a
little bit on object file size. I tried this on a clang bootstrap and
saved 0.54% on the sum of object file sizes (1.14 MB out of 212 MB for
a release build).
Differential Revision: http://reviews.llvm.org/D3533
llvm-svn: 207670
The patch implements support for both relocation record formats: Elf_Rel
and Elf_Rela. It is possible to define relocation against symbol only.
Relocations against sections will be implemented later. Now yaml2obj
recognizes X86_64, MIPS and Hexagon relocation types.
Example of relocation section specification:
Sections:
- Name: .text
Type: SHT_PROGBITS
Content: "0000000000000000"
AddressAlign: 16
Flags: [SHF_ALLOC]
- Name: .rel.text
Type: SHT_REL
Info: .text
AddressAlign: 4
Relocations:
- Offset: 0x1
Symbol: glob1
Type: R_MIPS_32
- Offset: 0x2
Symbol: glob2
Type: R_MIPS_CALL16
The patch reviewed by Michael Spencer, Sean Silva, Shankar Easwaran.
llvm-svn: 206017
and ContiguousBlobAccumulator classes. Pass ContiguousBlobAccumulator to
the handleSymtabSectionHeader function directly.
No functional changes.
llvm-svn: 205431
Summary:
The FileHeader mapping now accepts an optional Flags sequence that accepts
the EF_<arch>_<flag> constants. When not given, Flags defaults to zero.
Reviewers: atanasyan
Reviewed By: atanasyan
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D3213
llvm-svn: 205173
The current state of affairs has auxiliary symbols described as a big
bag of bytes. This is less than satisfying, it detracts from the YAML
file as being human readable.
Instead, allow for symbols to optionally contain their auxiliary data.
This allows us to have a much higher level way of describing things like
weak symbols, function definitions and section definitions.
This depends on D3105.
Differential Revision: http://llvm-reviews.chandlerc.com/D3092
llvm-svn: 204214
This compiles with no changes to clang/lld/lldb with MSVC and includes
overloads to various functions which are used by those projects and llvm
which have OwningPtr's as parameters. This should allow out of tree
projects some time to move. There are also no changes to libs/Target,
which should help out of tree targets have time to move, if necessary.
llvm-svn: 203083
* ELFTypes.h contains template magic for defining types based on endianess, size, and alignment.
* ELFFile.h defines the ELFFile class which provides low level ELF specific access.
* ELFObjectFile.h contains ELFObjectFile which uses ELFFile to implement the ObjectFile interface.
llvm-svn: 188022
Although in reality the symbol table in ELF resides in a section, the
standard requires that there be no more than one SHT_SYMTAB. To enforce
this constraint, it is cleaner to group all the symbols under a
top-level `Symbols` key on the object file.
llvm-svn: 184627
The improperly aligned section content in the output was causing
buildbot failures. This should fix them.
Incidentally, this results in a simpler and more robust API for
ContiguousBlobAccumulator.
llvm-svn: 184621
Previously we unconditionally enforced that section references in
symbols in the YAML had a name that was a section name present in the
object, and linked the references to that section. Now, permit empty
section names (already the default, if the `Section` key is not
provided) to indicate SHN_UNDEF.
llvm-svn: 184513
Instead, just have 3 sub-lists, one for each of
{STB_LOCAL,STB_GLOBAL,STB_WEAK}.
This allows us to be a lot more explicit w.r.t. the symbol ordering in
the object file, because if we allowed explicitly setting the STB_*
`Binding` key for the symbol, then we might have ended up having to
shuffle STB_LOCAL symbols to the front of the list, which is likely to
cause confusion and potential for error.
Also, this new approach is simpler ;)
llvm-svn: 184506
After this patch, the ELF file produced by
`yaml2obj-elf-symbol-basic.yaml`, when linked and executed on x86_64
(under SysV ABI, obviously; I tested on Linux), produces a working
executable that goes into an infinite loop!
llvm-svn: 184469
One of the key things that the YAML format abstracts over is the use of
section numbers for referencing sections. Instead, textual section names
are used, which yaml2obj then translates into appropriate section
numbers. (Technically ELF doesn't care about section names (only section
numbers), but since this is a testing tool, readability counts).
This simplifies using section names as symbolic references in various
parts of the code. An upcoming commit will use this to allow symbols to
reference sections.
llvm-svn: 184467
Previously, we would monkeypatch the vector of YAML::Section's in order
to ensure that the SHT_NULL entry is present. Now we just add it
unconditionally.
The proliferation of small numerical adjustments is beginning to
frighten me, but I can't think of a way having a single point of truth
for them without introducing a whole new layer of data structures (i.e.
lots of code and complexity) between the YAML and binary ELF formats.
llvm-svn: 184260
A bug in libObject will cause it to assert() if a symbol table's string
table and the section header string table are the same section, so we
need to ensure that we emit two different string tables (among other
things). The problematic code is the hardcoded usage of ".strtab"
(`dot_strtab_sec`) for looking up symbol names in
ELFObjectFile<ELFT>::getSymbolName.
I discussed this with Michael, and he has some local improvements to the
ELF code in libObject that, among other things, should fix our handling
of this scenario.
llvm-svn: 184161
I was spotting garbage in the output. I'd like to just zero the entire
ELFYAML::Section to be sure, but it contains non-POD types. (I'm also
trying to avoid bloating the ELFYAML::Foo classes with a bunch of
constructor code).
No test, since this is by its very nature unpredictable. I'm pretty sure
that one of the sanitizers would catch it immediately though.
llvm-svn: 184160
For consistency, change the address in the test case from 0xDEADBEEF to
0xCAFEBABE since 0xCAFEBABE that actually has a 2-byte alignment.
llvm-svn: 183962
Currently, only emitting the ELF header is supported (no sections or
segments).
The ELFYAML code organization is broadly similar to the COFFYAML code.
llvm-svn: 183711
Previously, yaml2coff.cpp had a writeHexData static helper function to
do this, but it is generally useful functionality.
Also, validate hex strings up-front to avoid running having to handle
errors "deep inside" the yaml2obj code (it also gives better diagnostics
than it used to).
llvm-svn: 183345
See the comment in yaml2obj.cpp for why this is currently needed.
Eventually we can get rid of this, but for now it is needed in order to
make forward progress with adding ELF support, and should be
straightforward to remove later.
Also, preserve the default of COFF, to avoid breaking existing tests.
This policy can easily be changed later though.
llvm-svn: 183332
The alignment is just a byte in the middle of Characteristics, not an
independent flag. Making it an independent field in the yaml
representation makes it more yamlio friendly.
llvm-svn: 181243
The COFFParser now contains only a COFFYAML::Object and the string table
(which is recomputed, not serialized).
The structs in COFFParser now all begin with a Header field with what is
actually on the COFF object. The other fields are things that are semantically
part of the struct (relocations in a section for exmaple), but are not actually
represented that way in the object file.
llvm-svn: 180134
Instead, use MappingNormalization to directly parse COFF::header. Also change
the naming convention of the helper classes to be a bit shorter.
llvm-svn: 179917
