mirror of
https://github.com/RPCS3/llvm-mirror.git
synced 2024-10-20 19:42:54 +02:00
a329086ea6
Introduction ============ This patch added intial support for bpf program compile once and run everywhere (CO-RE). The main motivation is for bpf program which depends on kernel headers which may vary between different kernel versions. The initial discussion can be found at https://lwn.net/Articles/773198/. Currently, bpf program accesses kernel internal data structure through bpf_probe_read() helper. The idea is to capture the kernel data structure to be accessed through bpf_probe_read() and relocate them on different kernel versions. On each host, right before bpf program load, the bpfloader will look at the types of the native linux through vmlinux BTF, calculates proper access offset and patch the instruction. To accommodate this, three intrinsic functions preserve_{array,union,struct}_access_index are introduced which in clang will preserve the base pointer, struct/union/array access_index and struct/union debuginfo type information. Later, bpf IR pass can reconstruct the whole gep access chains without looking at gep itself. This patch did the following: . An IR pass is added to convert preserve_*_access_index to global variable who name encodes the getelementptr access pattern. The global variable has metadata attached to describe the corresponding struct/union debuginfo type. . An SimplifyPatchable MachineInstruction pass is added to remove unnecessary loads. . The BTF output pass is enhanced to generate relocation records located in .BTF.ext section. Typical CO-RE also needs support of global variables which can be assigned to different values to different hosts. For example, kernel version can be used to guard different versions of codes. This patch added the support for patchable externals as well. Example ======= The following is an example. struct pt_regs { long arg1; long arg2; }; struct sk_buff { int i; struct net_device *dev; }; #define _(x) (__builtin_preserve_access_index(x)) static int (*bpf_probe_read)(void *dst, int size, const void *unsafe_ptr) = (void *) 4; extern __attribute__((section(".BPF.patchable_externs"))) unsigned __kernel_version; int bpf_prog(struct pt_regs *ctx) { struct net_device *dev = 0; // ctx->arg* does not need bpf_probe_read if (__kernel_version >= 41608) bpf_probe_read(&dev, sizeof(dev), _(&((struct sk_buff *)ctx->arg1)->dev)); else bpf_probe_read(&dev, sizeof(dev), _(&((struct sk_buff *)ctx->arg2)->dev)); return dev != 0; } In the above, we want to translate the third argument of bpf_probe_read() as relocations. -bash-4.4$ clang -target bpf -O2 -g -S trace.c The compiler will generate two new subsections in .BTF.ext, OffsetReloc and ExternReloc. OffsetReloc is to record the structure member offset operations, and ExternalReloc is to record the external globals where only u8, u16, u32 and u64 are supported. BPFOffsetReloc Size struct SecLOffsetReloc for ELF section #1 A number of struct BPFOffsetReloc for ELF section #1 struct SecOffsetReloc for ELF section #2 A number of struct BPFOffsetReloc for ELF section #2 ... BPFExternReloc Size struct SecExternReloc for ELF section #1 A number of struct BPFExternReloc for ELF section #1 struct SecExternReloc for ELF section #2 A number of struct BPFExternReloc for ELF section #2 struct BPFOffsetReloc { uint32_t InsnOffset; ///< Byte offset in this section uint32_t TypeID; ///< TypeID for the relocation uint32_t OffsetNameOff; ///< The string to traverse types }; struct BPFExternReloc { uint32_t InsnOffset; ///< Byte offset in this section uint32_t ExternNameOff; ///< The string for external variable }; Note that only externs with attribute section ".BPF.patchable_externs" are considered for Extern Reloc which will be patched by bpf loader right before the load. For the above test case, two offset records and one extern record will be generated: OffsetReloc records: .long .Ltmp12 # Insn Offset .long 7 # TypeId .long 242 # Type Decode String .long .Ltmp18 # Insn Offset .long 7 # TypeId .long 242 # Type Decode String ExternReloc record: .long .Ltmp5 # Insn Offset .long 165 # External Variable In string table: .ascii "0:1" # string offset=242 .ascii "__kernel_version" # string offset=165 The default member offset can be calculated as the 2nd member offset (0 representing the 1st member) of struct "sk_buff". The asm code: .Ltmp5: .Ltmp6: r2 = 0 r3 = 41608 .Ltmp7: .Ltmp8: .loc 1 18 9 is_stmt 0 # t.c:18:9 .Ltmp9: if r3 > r2 goto LBB0_2 .Ltmp10: .Ltmp11: .loc 1 0 9 # t.c:0:9 .Ltmp12: r2 = 8 .Ltmp13: .loc 1 19 66 is_stmt 1 # t.c:19:66 .Ltmp14: .Ltmp15: r3 = *(u64 *)(r1 + 0) goto LBB0_3 .Ltmp16: .Ltmp17: LBB0_2: .loc 1 0 66 is_stmt 0 # t.c:0:66 .Ltmp18: r2 = 8 .loc 1 21 66 is_stmt 1 # t.c:21:66 .Ltmp19: r3 = *(u64 *)(r1 + 8) .Ltmp20: .Ltmp21: LBB0_3: .loc 1 0 66 is_stmt 0 # t.c:0:66 r3 += r2 r1 = r10 .Ltmp22: .Ltmp23: .Ltmp24: r1 += -8 r2 = 8 call 4 For instruction .Ltmp12 and .Ltmp18, "r2 = 8", the number 8 is the structure offset based on the current BTF. Loader needs to adjust it if it changes on the host. For instruction .Ltmp5, "r2 = 0", the external variable got a default value 0, loader needs to supply an appropriate value for the particular host. Compiling to generate object code and disassemble: 0000000000000000 bpf_prog: 0: b7 02 00 00 00 00 00 00 r2 = 0 1: 7b 2a f8 ff 00 00 00 00 *(u64 *)(r10 - 8) = r2 2: b7 02 00 00 00 00 00 00 r2 = 0 3: b7 03 00 00 88 a2 00 00 r3 = 41608 4: 2d 23 03 00 00 00 00 00 if r3 > r2 goto +3 <LBB0_2> 5: b7 02 00 00 08 00 00 00 r2 = 8 6: 79 13 00 00 00 00 00 00 r3 = *(u64 *)(r1 + 0) 7: 05 00 02 00 00 00 00 00 goto +2 <LBB0_3> 0000000000000040 LBB0_2: 8: b7 02 00 00 08 00 00 00 r2 = 8 9: 79 13 08 00 00 00 00 00 r3 = *(u64 *)(r1 + 8) 0000000000000050 LBB0_3: 10: 0f 23 00 00 00 00 00 00 r3 += r2 11: bf a1 00 00 00 00 00 00 r1 = r10 12: 07 01 00 00 f8 ff ff ff r1 += -8 13: b7 02 00 00 08 00 00 00 r2 = 8 14: 85 00 00 00 04 00 00 00 call 4 Instructions #2, #5 and #8 need relocation resoutions from the loader. Signed-off-by: Yonghong Song <yhs@fb.com> Differential Revision: https://reviews.llvm.org/D61524 llvm-svn: 365503
278 lines
9.0 KiB
C++
278 lines
9.0 KiB
C++
//===-- BTF.h --------------------------------------------------*- C++ -*-===//
|
|
//
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
//
|
|
//===----------------------------------------------------------------------===//
|
|
///
|
|
/// \file
|
|
/// This file contains the layout of .BTF and .BTF.ext ELF sections.
|
|
///
|
|
/// The binary layout for .BTF section:
|
|
/// struct Header
|
|
/// Type and Str subsections
|
|
/// The Type subsection is a collection of types with type id starting with 1.
|
|
/// The Str subsection is simply a collection of strings.
|
|
///
|
|
/// The binary layout for .BTF.ext section:
|
|
/// struct ExtHeader
|
|
/// FuncInfo, LineInfo, OffsetReloc and ExternReloc subsections
|
|
/// The FuncInfo subsection is defined as below:
|
|
/// BTFFuncInfo Size
|
|
/// struct SecFuncInfo for ELF section #1
|
|
/// A number of struct BPFFuncInfo for ELF section #1
|
|
/// struct SecFuncInfo for ELF section #2
|
|
/// A number of struct BPFFuncInfo for ELF section #2
|
|
/// ...
|
|
/// The LineInfo subsection is defined as below:
|
|
/// BPFLineInfo Size
|
|
/// struct SecLineInfo for ELF section #1
|
|
/// A number of struct BPFLineInfo for ELF section #1
|
|
/// struct SecLineInfo for ELF section #2
|
|
/// A number of struct BPFLineInfo for ELF section #2
|
|
/// ...
|
|
/// The OffsetReloc subsection is defined as below:
|
|
/// BPFOffsetReloc Size
|
|
/// struct SecOffsetReloc for ELF section #1
|
|
/// A number of struct BPFOffsetReloc for ELF section #1
|
|
/// struct SecOffsetReloc for ELF section #2
|
|
/// A number of struct BPFOffsetReloc for ELF section #2
|
|
/// ...
|
|
/// The ExternReloc subsection is defined as below:
|
|
/// BPFExternReloc Size
|
|
/// struct SecExternReloc for ELF section #1
|
|
/// A number of struct BPFExternReloc for ELF section #1
|
|
/// struct SecExternReloc for ELF section #2
|
|
/// A number of struct BPFExternReloc for ELF section #2
|
|
/// ...
|
|
///
|
|
/// The section formats are also defined at
|
|
/// https://github.com/torvalds/linux/blob/master/include/uapi/linux/btf.h
|
|
///
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
#ifndef LLVM_LIB_TARGET_BPF_BTF_H
|
|
#define LLVM_LIB_TARGET_BPF_BTF_H
|
|
|
|
namespace llvm {
|
|
namespace BTF {
|
|
|
|
enum : uint32_t { MAGIC = 0xeB9F, VERSION = 1 };
|
|
|
|
/// Sizes in bytes of various things in the BTF format.
|
|
enum {
|
|
HeaderSize = 24,
|
|
ExtHeaderSize = 40,
|
|
CommonTypeSize = 12,
|
|
BTFArraySize = 12,
|
|
BTFEnumSize = 8,
|
|
BTFMemberSize = 12,
|
|
BTFParamSize = 8,
|
|
BTFDataSecVarSize = 12,
|
|
SecFuncInfoSize = 8,
|
|
SecLineInfoSize = 8,
|
|
SecOffsetRelocSize = 8,
|
|
SecExternRelocSize = 8,
|
|
BPFFuncInfoSize = 8,
|
|
BPFLineInfoSize = 16,
|
|
BPFOffsetRelocSize = 12,
|
|
BPFExternRelocSize = 8,
|
|
};
|
|
|
|
/// The .BTF section header definition.
|
|
struct Header {
|
|
uint16_t Magic; ///< Magic value
|
|
uint8_t Version; ///< Version number
|
|
uint8_t Flags; ///< Extra flags
|
|
uint32_t HdrLen; ///< Length of this header
|
|
|
|
/// All offsets are in bytes relative to the end of this header.
|
|
uint32_t TypeOff; ///< Offset of type section
|
|
uint32_t TypeLen; ///< Length of type section
|
|
uint32_t StrOff; ///< Offset of string section
|
|
uint32_t StrLen; ///< Length of string section
|
|
};
|
|
|
|
enum : uint32_t {
|
|
MAX_VLEN = 0xffff ///< Max # of struct/union/enum members or func args
|
|
};
|
|
|
|
enum TypeKinds : uint8_t {
|
|
#define HANDLE_BTF_KIND(ID, NAME) BTF_KIND_##NAME = ID,
|
|
#include "BTF.def"
|
|
};
|
|
|
|
/// The BTF common type definition. Different kinds may have
|
|
/// additional information after this structure data.
|
|
struct CommonType {
|
|
/// Type name offset in the string table.
|
|
uint32_t NameOff;
|
|
|
|
/// "Info" bits arrangement:
|
|
/// Bits 0-15: vlen (e.g. # of struct's members)
|
|
/// Bits 16-23: unused
|
|
/// Bits 24-27: kind (e.g. int, ptr, array...etc)
|
|
/// Bits 28-30: unused
|
|
/// Bit 31: kind_flag, currently used by
|
|
/// struct, union and fwd
|
|
uint32_t Info;
|
|
|
|
/// "Size" is used by INT, ENUM, STRUCT and UNION.
|
|
/// "Size" tells the size of the type it is describing.
|
|
///
|
|
/// "Type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT,
|
|
/// FUNC, FUNC_PROTO and VAR.
|
|
/// "Type" is a type_id referring to another type.
|
|
union {
|
|
uint32_t Size;
|
|
uint32_t Type;
|
|
};
|
|
};
|
|
|
|
// For some specific BTF_KIND, "struct CommonType" is immediately
|
|
// followed by extra data.
|
|
|
|
// BTF_KIND_INT is followed by a u32 and the following
|
|
// is the 32 bits arrangement:
|
|
// BTF_INT_ENCODING(VAL) : (((VAL) & 0x0f000000) >> 24)
|
|
// BTF_INT_OFFSET(VAL) : (((VAL & 0x00ff0000)) >> 16)
|
|
// BTF_INT_BITS(VAL) : ((VAL) & 0x000000ff)
|
|
|
|
/// Attributes stored in the INT_ENCODING.
|
|
enum : uint8_t {
|
|
INT_SIGNED = (1 << 0),
|
|
INT_CHAR = (1 << 1),
|
|
INT_BOOL = (1 << 2)
|
|
};
|
|
|
|
/// BTF_KIND_ENUM is followed by multiple "struct BTFEnum".
|
|
/// The exact number of btf_enum is stored in the vlen (of the
|
|
/// info in "struct CommonType").
|
|
struct BTFEnum {
|
|
uint32_t NameOff; ///< Enum name offset in the string table
|
|
int32_t Val; ///< Enum member value
|
|
};
|
|
|
|
/// BTF_KIND_ARRAY is followed by one "struct BTFArray".
|
|
struct BTFArray {
|
|
uint32_t ElemType; ///< Element type
|
|
uint32_t IndexType; ///< Index type
|
|
uint32_t Nelems; ///< Number of elements for this array
|
|
};
|
|
|
|
/// BTF_KIND_STRUCT and BTF_KIND_UNION are followed
|
|
/// by multiple "struct BTFMember". The exact number
|
|
/// of BTFMember is stored in the vlen (of the info in
|
|
/// "struct CommonType").
|
|
///
|
|
/// If the struct/union contains any bitfield member,
|
|
/// the Offset below represents BitOffset (bits 0 - 23)
|
|
/// and BitFieldSize(bits 24 - 31) with BitFieldSize = 0
|
|
/// for non bitfield members. Otherwise, the Offset
|
|
/// represents the BitOffset.
|
|
struct BTFMember {
|
|
uint32_t NameOff; ///< Member name offset in the string table
|
|
uint32_t Type; ///< Member type
|
|
uint32_t Offset; ///< BitOffset or BitFieldSize+BitOffset
|
|
};
|
|
|
|
/// BTF_KIND_FUNC_PROTO are followed by multiple "struct BTFParam".
|
|
/// The exist number of BTFParam is stored in the vlen (of the info
|
|
/// in "struct CommonType").
|
|
struct BTFParam {
|
|
uint32_t NameOff;
|
|
uint32_t Type;
|
|
};
|
|
|
|
/// Variable scoping information.
|
|
enum : uint8_t {
|
|
VAR_STATIC = 0, ///< Linkage: InternalLinkage
|
|
VAR_GLOBAL_ALLOCATED = 1, ///< Linkage: ExternalLinkage
|
|
VAR_GLOBAL_TENTATIVE = 2, ///< Linkage: CommonLinkage
|
|
VAR_GLOBAL_EXTERNAL = 3, ///< Linkage: ExternalLinkage
|
|
};
|
|
|
|
/// BTF_KIND_DATASEC are followed by multiple "struct BTFDataSecVar".
|
|
/// The exist number of BTFDataSec is stored in the vlen (of the info
|
|
/// in "struct CommonType").
|
|
struct BTFDataSec {
|
|
uint32_t Type; ///< A BTF_KIND_VAR type
|
|
uint32_t Offset; ///< In-section offset
|
|
uint32_t Size; ///< Occupied memory size
|
|
};
|
|
|
|
/// The .BTF.ext section header definition.
|
|
struct ExtHeader {
|
|
uint16_t Magic;
|
|
uint8_t Version;
|
|
uint8_t Flags;
|
|
uint32_t HdrLen;
|
|
|
|
uint32_t FuncInfoOff; ///< Offset of func info section
|
|
uint32_t FuncInfoLen; ///< Length of func info section
|
|
uint32_t LineInfoOff; ///< Offset of line info section
|
|
uint32_t LineInfoLen; ///< Length of line info section
|
|
uint32_t OffsetRelocOff; ///< Offset of offset reloc section
|
|
uint32_t OffsetRelocLen; ///< Length of offset reloc section
|
|
uint32_t ExternRelocOff; ///< Offset of extern reloc section
|
|
uint32_t ExternRelocLen; ///< Length of extern reloc section
|
|
};
|
|
|
|
/// Specifying one function info.
|
|
struct BPFFuncInfo {
|
|
uint32_t InsnOffset; ///< Byte offset in the section
|
|
uint32_t TypeId; ///< Type id referring to .BTF type section
|
|
};
|
|
|
|
/// Specifying function info's in one section.
|
|
struct SecFuncInfo {
|
|
uint32_t SecNameOff; ///< Section name index in the .BTF string table
|
|
uint32_t NumFuncInfo; ///< Number of func info's in this section
|
|
};
|
|
|
|
/// Specifying one line info.
|
|
struct BPFLineInfo {
|
|
uint32_t InsnOffset; ///< Byte offset in this section
|
|
uint32_t FileNameOff; ///< File name index in the .BTF string table
|
|
uint32_t LineOff; ///< Line index in the .BTF string table
|
|
uint32_t LineCol; ///< Line num: line_col >> 10,
|
|
/// col num: line_col & 0x3ff
|
|
};
|
|
|
|
/// Specifying line info's in one section.
|
|
struct SecLineInfo {
|
|
uint32_t SecNameOff; ///< Section name index in the .BTF string table
|
|
uint32_t NumLineInfo; ///< Number of line info's in this section
|
|
};
|
|
|
|
/// Specifying one offset relocation.
|
|
struct BPFOffsetReloc {
|
|
uint32_t InsnOffset; ///< Byte offset in this section
|
|
uint32_t TypeID; ///< TypeID for the relocation
|
|
uint32_t OffsetNameOff; ///< The string to traverse types
|
|
};
|
|
|
|
/// Specifying offset relocation's in one section.
|
|
struct SecOffsetReloc {
|
|
uint32_t SecNameOff; ///< Section name index in the .BTF string table
|
|
uint32_t NumOffsetReloc; ///< Number of offset reloc's in this section
|
|
};
|
|
|
|
/// Specifying one offset relocation.
|
|
struct BPFExternReloc {
|
|
uint32_t InsnOffset; ///< Byte offset in this section
|
|
uint32_t ExternNameOff; ///< The string for external variable
|
|
};
|
|
|
|
/// Specifying extern relocation's in one section.
|
|
struct SecExternReloc {
|
|
uint32_t SecNameOff; ///< Section name index in the .BTF string table
|
|
uint32_t NumExternReloc; ///< Number of extern reloc's in this section
|
|
};
|
|
|
|
} // End namespace BTF.
|
|
} // End namespace llvm.
|
|
|
|
#endif
|