1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-23 11:13:28 +01:00
llvm-mirror/docs
Yonghong Song 61809186ed [BPF] add new intrinsics preserve_{array,union,struct}_access_index
For background of BPF CO-RE project, please refer to
  http://vger.kernel.org/bpfconf2019.html
In summary, BPF CO-RE intends to compile bpf programs
adjustable on struct/union layout change so the same
program can run on multiple kernels with adjustment
before loading based on native kernel structures.

In order to do this, we need keep track of GEP(getelementptr)
instruction base and result debuginfo types, so we
can adjust on the host based on kernel BTF info.
Capturing such information as an IR optimization is hard
as various optimization may have tweaked GEP and also
union is replaced by structure it is impossible to track
fieldindex for union member accesses.

Three intrinsic functions, preserve_{array,union,struct}_access_index,
are introducted.
  addr = preserve_array_access_index(base, index, dimension)
  addr = preserve_union_access_index(base, di_index)
  addr = preserve_struct_access_index(base, gep_index, di_index)
here,
  base: the base pointer for the array/union/struct access.
  index: the last access index for array, the same for IR/DebugInfo layout.
  dimension: the array dimension.
  gep_index: the access index based on IR layout.
  di_index: the access index based on user/debuginfo types.

For example, for the following example,
  $ cat test.c
  struct sk_buff {
     int i;
     int b1:1;
     int b2:2;
     union {
       struct {
         int o1;
         int o2;
       } o;
       struct {
         char flags;
         char dev_id;
       } dev;
       int netid;
     } u[10];
  };

  static int (*bpf_probe_read)(void *dst, int size, const void *unsafe_ptr)
      = (void *) 4;

  #define _(x) (__builtin_preserve_access_index(x))

  int bpf_prog(struct sk_buff *ctx) {
    char dev_id;
    bpf_probe_read(&dev_id, sizeof(char), _(&ctx->u[5].dev.dev_id));
    return dev_id;
  }
  $ clang -target bpf -O2 -g -emit-llvm -S -mllvm -print-before-all \
    test.c >& log

The generated IR looks like below:

  ...
  define dso_local i32 @bpf_prog(%struct.sk_buff*) #0 !dbg !15 {
    %2 = alloca %struct.sk_buff*, align 8
    %3 = alloca i8, align 1
    store %struct.sk_buff* %0, %struct.sk_buff** %2, align 8, !tbaa !45
    call void @llvm.dbg.declare(metadata %struct.sk_buff** %2, metadata !43, metadata !DIExpression()), !dbg !49
    call void @llvm.lifetime.start.p0i8(i64 1, i8* %3) #4, !dbg !50
    call void @llvm.dbg.declare(metadata i8* %3, metadata !44, metadata !DIExpression()), !dbg !51
    %4 = load i32 (i8*, i32, i8*)*, i32 (i8*, i32, i8*)** @bpf_probe_read, align 8, !dbg !52, !tbaa !45
    %5 = load %struct.sk_buff*, %struct.sk_buff** %2, align 8, !dbg !53, !tbaa !45
    %6 = call [10 x %union.anon]* @llvm.preserve.struct.access.index.p0a10s_union.anons.p0s_struct.sk_buffs(
         %struct.sk_buff* %5, i32 2, i32 3), !dbg !53, !llvm.preserve.access.index !19
    %7 = call %union.anon* @llvm.preserve.array.access.index.p0s_union.anons.p0a10s_union.anons(
         [10 x %union.anon]* %6, i32 1, i32 5), !dbg !53
    %8 = call %union.anon* @llvm.preserve.union.access.index.p0s_union.anons.p0s_union.anons(
         %union.anon* %7, i32 1), !dbg !53, !llvm.preserve.access.index !26
    %9 = bitcast %union.anon* %8 to %struct.anon.0*, !dbg !53
    %10 = call i8* @llvm.preserve.struct.access.index.p0i8.p0s_struct.anon.0s(
         %struct.anon.0* %9, i32 1, i32 1), !dbg !53, !llvm.preserve.access.index !34
    %11 = call i32 %4(i8* %3, i32 1, i8* %10), !dbg !52
    %12 = load i8, i8* %3, align 1, !dbg !54, !tbaa !55
    %13 = sext i8 %12 to i32, !dbg !54
    call void @llvm.lifetime.end.p0i8(i64 1, i8* %3) #4, !dbg !56
    ret i32 %13, !dbg !57
  }

  !19 = distinct !DICompositeType(tag: DW_TAG_structure_type, name: "sk_buff", file: !3, line: 1, size: 704, elements: !20)
  !26 = distinct !DICompositeType(tag: DW_TAG_union_type, scope: !19, file: !3, line: 5, size: 64, elements: !27)
  !34 = distinct !DICompositeType(tag: DW_TAG_structure_type, scope: !26, file: !3, line: 10, size: 16, elements: !35)

Note that @llvm.preserve.{struct,union}.access.index calls have metadata llvm.preserve.access.index
attached to instructions to provide struct/union debuginfo type information.

For &ctx->u[5].dev.dev_id,
  . The "%6 = ..." represents struct member "u" with index 2 for IR layout and index 3 for DI layout.
  . The "%7 = ..." represents array subscript "5".
  . The "%8 = ..." represents union member "dev" with index 1 for DI layout.
  . The "%10 = ..." represents struct member "dev_id" with index 1 for both IR and DI layout.

Basically, traversing the use-def chain recursively for the 3rd argument of bpf_probe_read() and
examining all preserve_*_access_index calls, the debuginfo struct/union/array access index
can be achieved.

The intrinsics also contain enough information to regenerate codes for IR layout.
For array and structure intrinsics, the proper GEP can be constructed.
For union intrinsics, replacing all uses of "addr" with "base" should be enough.

The test case ThinLTO/X86/lazyload_metadata.ll is adjusted to reflect the
new addition of the metadata.

Signed-off-by: Yonghong Song <yhs@fb.com>

Differential Revision: https://reviews.llvm.org/D61810

llvm-svn: 365423
2019-07-09 01:51:36 +00:00
..
_ocamldoc
_static
_templates
_themes/llvm-theme
AMDGPU [AMDGPU][MC][DOC] Updated AMD GPU assembler syntax description. 2019-07-08 16:50:11 +00:00
CommandGuide [docs][llvm-readobj][llvm-readelf] Improve wording 2019-07-08 15:46:26 +00:00
Frontend
HistoricalNotes
PDB PDB docs: Delete trailing whitespace, wrap to 80 cols 2019-06-22 11:23:01 +00:00
Proposals [Documentation] Proposal to change variable names 2019-03-28 14:42:21 +00:00
TableGen [docs] Fix TableGen/LangRef typos 2019-06-13 18:42:30 +00:00
tutorial Fix missing arguments in tutorial 2019-04-12 08:23:28 +00:00
AddingConstrainedIntrinsics.rst [FPEnv] A missing crucial step was undocumented. 2019-06-25 16:09:39 +00:00
AdvancedBuilds.rst Use backquotes to avoid a sphinx unexpected error: 2018-12-16 14:19:39 +00:00
AliasAnalysis.rst
AMDGPUInstructionNotation.rst [AMDGPU][MC][DOC] Updated AMD GPU assembler description. 2018-12-28 11:48:23 +00:00
AMDGPUInstructionSyntax.rst [AMDGPU][MC][DOC] Updated AMD GPU assembler syntax description. 2019-07-08 16:50:11 +00:00
AMDGPUModifierSyntax.rst [AMDGPU][MC][DOC] Updated AMD GPU assembler syntax description. 2019-07-08 16:50:11 +00:00
AMDGPUOperandSyntax.rst [AMDGPU][MC][DOC] Updated AMD GPU assembler syntax description. 2019-07-08 16:50:11 +00:00
AMDGPUUsage.rst [AMDGPU][MC][DOC] Updated AMD GPU assembler syntax description. 2019-07-08 17:09:09 +00:00
ARM-BE-bitcastfail.png
ARM-BE-bitcastsuccess.png
ARM-BE-ld1.png
ARM-BE-ldr.png
Atomics.rst [docs][AtomicExpandPass] Document the alternate lowering strategy for part-word atomicrmw/cmpxchg 2018-11-30 09:23:24 +00:00
Benchmarking.rst
BigEndianNEON.rst
BitCodeFormat.rst
BlockFrequencyTerminology.rst
BranchWeightMetadata.rst [TI removal] Remove discussion of TerminatorInst from the LLVM 2018-10-18 07:40:24 +00:00
BugLifeCycle.rst Introduce bug life cycle documentation. 2018-11-07 08:49:36 +00:00
Bugpoint.rst
BuildingADistribution.rst [docs] Add new document on building distributions 2019-05-21 16:29:31 +00:00
CFIVerify.rst [cfi-verify] Abort on unsupported targets 2018-07-16 15:26:44 +00:00
CMake.rst Remove some autoconf references from docs and comments 2019-07-03 09:57:59 +00:00
CMakeLists.txt
CMakePrimer.rst
CodeGenerator.rst [WebAssembly] Implement tail calls and unify tablegen call classes 2019-06-26 16:17:15 +00:00
CodeOfConduct.rst
CodingStandards.rst Update the coding standards with the new file header. 2019-01-19 11:53:58 +00:00
CommandLine.rst [NFC] fix trivial typos in documents 2019-06-25 07:24:27 +00:00
CompileCudaWithLLVM.rst Adjust documentation for git migration. 2019-01-29 16:37:27 +00:00
CompilerWriterInfo.rst [Docs] Modernize references to macOS 2019-05-30 16:46:22 +00:00
conf.py [docs] Add support for Markdown documentation when creating man pages 2019-05-03 05:11:48 +00:00
Contributing.rst
Coroutines.rst
CoverageMappingFormat.rst
DebuggingJITedCode.rst [Docs] Modernize references to macOS 2019-05-30 16:46:22 +00:00
DeveloperPolicy.rst DeveloperPolicy: update toolchain with sample RFC / patch 2019-01-31 23:18:11 +00:00
Docker.rst [build] Rename clang-headers to clang-resource-headers 2019-03-04 21:19:53 +00:00
doxygen-mainpage.dox
doxygen.cfg.in [Docs] Increase Doxygen cache size 2019-05-22 00:56:42 +00:00
ExceptionHandling.rst
ExtendedIntegerResults.txt
ExtendingLLVM.rst Note ImmArg in documentation for adding intrinsics 2019-03-14 13:46:17 +00:00
Extensions.rst Add IR support, ELF section and user documentation for partitioning feature. 2019-05-29 03:29:01 +00:00
FAQ.rst Remove some autoconf references from docs and comments 2019-07-03 09:57:59 +00:00
FaultMaps.rst
FuzzingLLVM.rst [docs] Try to clarify the FuzzingLLVM docs 2018-08-16 21:55:09 +00:00
GarbageCollection.rst [GC docs] Update the gcroot documentation to reflect recent simplifcations to GCStrategy configurability 2018-11-12 20:30:50 +00:00
gcc-loops.png
GetElementPtr.rst [DOC] Fix load instructions' syntax, function definition. 2019-06-12 11:24:22 +00:00
GettingStarted.rst [docs] Add 'git llvm revert' to getting started guide 2019-06-10 17:00:49 +00:00
GettingStartedVS.rst
GlobalISel.rst [globalisel] Update the legalizer documentation 2019-05-01 16:52:29 +00:00
GoldPlugin.rst [docs][gold] Fix a typo 2018-08-21 21:00:54 +00:00
HowToAddABuilder.rst
HowToBuildOnARM.rst
HowToBuildWithPGO.rst HowToBuildWithPGO.rst: Fix a few details in the manual steps 2018-12-05 08:35:30 +00:00
HowToCrossCompileBuiltinsOnArm.rst [docs] Improve HowToCrossCompilerBuiltinsOnArm 2018-12-18 12:40:19 +00:00
HowToCrossCompileLLVM.rst
HowToReleaseLLVM.rst Remove some autoconf references from docs and comments 2019-07-03 09:57:59 +00:00
HowToSetUpLLVMStyleRTTI.rst
HowToSubmitABug.rst [docs] Prevent O0 optnone for opt input 2019-01-29 22:17:51 +00:00
HowToUseAttributes.rst
HowToUseInstrMappings.rst
InAlloca.rst
index.rst [docs] Add new document on building distributions 2019-05-21 16:29:31 +00:00
LangRef.rst [BPF] add new intrinsics preserve_{array,union,struct}_access_index 2019-07-09 01:51:36 +00:00
Lexicon.rst Update the coding standards and developer policy documentation surrounding whitespace. 2018-08-10 17:26:07 +00:00
LibFuzzer.rst [libFuzzer] document -len_control 2019-03-18 22:20:47 +00:00
LinkTimeOptimization.rst
linpack-pc.png
llvm-objdump.1 Update the file headers across all of the LLVM projects in the monorepo 2019-01-19 08:50:56 +00:00
LLVMBuild.rst
LLVMBuild.txt Update the file headers across all of the LLVM projects in the monorepo 2019-01-19 08:50:56 +00:00
make.bat
Makefile.sphinx
MarkdownQuickstartTemplate.md [docs] Reinstate r337730 - Add support for Markdown documentation in 2018-08-06 00:38:31 +00:00
MarkedUpDisassembly.rst
MCJIT-creation.png
MCJIT-dyld-load.png
MCJIT-engine-builder.png
MCJIT-load-object.png
MCJIT-load.png
MCJIT-resolve-relocations.png
MCJITDesignAndImplementation.rst
MeetupGuidelines.rst Add guidelines/recommendations for organizers of LLVM Socials 2019-05-14 07:20:58 +00:00
MemorySSA.rst
MergeFunctions.rst Updating MergeFunctions.rst 2018-08-18 20:17:19 +00:00
MIRLangRef.rst [MachineFunction] Base support for call site info tracking 2019-06-27 07:48:06 +00:00
NVPTXUsage.rst
OptBisect.rst [Docs] Remove hard tab character from code block in optbisect documentation 2018-07-27 06:54:13 +00:00
ORCv2DesignAndImplementation.rst Fix sphinx warning about missing footnote. 2019-05-30 09:40:35 +00:00
Packaging.rst
Passes.rst [Unroll/UnrollAndJam/Vectorizer/Distribute] Add followup loop attributes. 2018-12-12 17:32:52 +00:00
Phabricator.rst Update GettingStarted guide to recommend that people use the new 2019-01-14 22:27:32 +00:00
ProgrammersManual.rst Remove some autoconf references from docs and comments 2019-07-03 09:57:59 +00:00
Projects.rst
re_format.7
README.txt [NFC] Test commit, whitespace change 2019-06-06 22:07:14 +00:00
ReleaseNotes.rst Add lldb-mi deprecation to the release notes 2019-07-05 18:23:52 +00:00
ReleaseProcess.rst
ReportingGuide.rst
ScudoHardenedAllocator.rst [docs] Scudo: document error messages & their potential cause 2019-01-22 16:43:45 +00:00
SegmentedStacks.rst
SourceLevelDebugging.rst [DebugInfo][Docs] Document that prologue/epilogue variable location changes are ignored 2019-06-18 08:52:38 +00:00
speculative_load_hardening_microbenchmarks.png [x86/SLH] Add the design document for Speculative Load Hardening, 2018-07-18 14:05:14 +00:00
SpeculativeLoadHardening.md [docs] Fix a couple spelling errors. 2019-01-28 23:03:41 +00:00
SphinxQuickstartTemplate.rst
StackMaps.rst [docs] Clarify ELF section naming for StackMaps and fix a typo 2018-11-08 17:20:35 +00:00
StackSafetyAnalysis.rst Remove trailing empty line 2018-11-26 23:17:52 +00:00
Statepoints.rst [docs][statepoints] Reformulate open issues list 2018-11-09 17:09:16 +00:00
SupportLibrary.rst [Docs] [Support] Correct a missed reference and use up to date .inc examples. 2018-09-21 01:53:51 +00:00
SystemLibrary.rst [Docs] [Support] System Library to Support Library transition along with minor corrections to reflect it. 2018-09-18 18:05:38 +00:00
TableGenFundamentals.rst
TestingGuide.rst [Docs] Modernize references to macOS 2019-05-30 16:46:22 +00:00
TestSuiteGuide.md Adjust documentation for git migration. 2019-01-29 16:37:27 +00:00
TestSuiteMakefileGuide.rst Adjust documentation for git migration. 2019-01-29 16:37:27 +00:00
TransformMetadata.rst [Unroll/UnrollAndJam/Vectorizer/Distribute] Add followup loop attributes. 2018-12-12 17:32:52 +00:00
TypeMetadata.rst Update GettingStarted guide to recommend that people use the new 2019-01-14 22:27:32 +00:00
Vectorizers.rst Update GettingStarted guide to recommend that people use the new 2019-01-14 22:27:32 +00:00
WritingAnLLVMBackend.rst
WritingAnLLVMPass.rst Document legacy pass manager extension points 2019-07-04 14:03:11 +00:00
XRay.rst [Docs] Update of Xray page 2018-07-26 04:44:31 +00:00
XRayExample.rst [XRay][docs] Fix option name 2019-03-12 13:44:42 +00:00
XRayFDRFormat.rst
yaml2obj.rst
YamlIO.rst [Docs] Fix LLVM_YAML_IS_DOCUMENT_LIST_VECTOR 2018-07-23 14:17:43 +00:00

LLVM Documentation
==================

LLVM's documentation is written in reStructuredText, a lightweight
plaintext markup language (file extension `.rst`). While the
reStructuredText documentation should be quite readable in source form, it
is mostly meant to be processed by the Sphinx documentation generation
system to create HTML pages which are hosted on <http://llvm.org/docs/> and
updated after every commit. Manpage output is also supported, see below.

If you instead would like to generate and view the HTML locally, install
Sphinx <http://sphinx-doc.org/> and then do:

    cd <build-dir>
    cmake -DLLVM_ENABLE_SPHINX=true -DSPHINX_OUTPUT_HTML=true <src-dir>
    make -j3 docs-llvm-html
    $BROWSER <build-dir>/docs//html/index.html

The mapping between reStructuredText files and generated documentation is
`docs/Foo.rst` <-> `<build-dir>/docs//html/Foo.html` <-> `http://llvm.org/docs/Foo.html`.

If you are interested in writing new documentation, you will want to read
`SphinxQuickstartTemplate.rst` which will get you writing documentation
very fast and includes examples of the most important reStructuredText
markup syntax.

Manpage Output
===============

Building the manpages is similar to building the HTML documentation. The
primary difference is to use the `man` makefile target, instead of the
default (which is `html`). Sphinx then produces the man pages in the
directory `<build-dir>/docs/man/`.

    cd <build-dir>
    cmake -DLLVM_ENABLE_SPHINX=true -DSPHINX_OUTPUT_MAN=true <src-dir>
    make -j3 docs-llvm-man
    man -l >build-dir>/docs/man/FileCheck.1

The correspondence between .rst files and man pages is
`docs/CommandGuide/Foo.rst` <-> `<build-dir>/docs//man/Foo.1`.
These .rst files are also included during HTML generation so they are also
viewable online (as noted above) at e.g.
`http://llvm.org/docs/CommandGuide/Foo.html`.

Checking links
==============

The reachability of external links in the documentation can be checked by
running:

    cd docs/
    make -f Makefile.sphinx linkcheck

Doxygen page Output
==============

Install doxygen <http://www.stack.nl/~dimitri/doxygen/download.html> and dot2tex <https://dot2tex.readthedocs.io/en/latest>.

    cd <build-dir>
    cmake -DLLVM_ENABLE_DOXYGEN=On <llvm-top-src-dir>
    make doxygen-llvm # for LLVM docs
    make doxygen-clang # for clang docs

It will generate html in

    <build-dir>/docs/doxygen/html # for LLVM docs
    <build-dir>/tools/clang/docs/doxygen/html # for clang docs