1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-22 10:42:39 +01:00
Commit Graph

56 Commits

Author SHA1 Message Date
Valentin Churavy
a01ce5e73a Reland [Orc] Add verylazy example for C-bindings
This patch relands https://reviews.llvm.org/D104799, but fixes the
memory handling causing leak sanitizer failures.

This reverts commit a56fe117e04f7d4b953a4226af412dad59425fb5.
2021-07-18 21:17:49 +02:00
Valentin Churavy
814653b1a2 Revert "[Orc] Add verylazy example for C-bindings"
Broke ASAN buildbot, will reland with fixes

This reverts commit b5a6ad8c893a642bcb08ab81b251952c545405d9.
2021-07-18 16:21:37 +02:00
Valentin Churavy
3f3bee8a2c [Orc] Add verylazy example for C-bindings
Still WIP, based on the Kaleidoscope/BuildingAJIT/Chapter4.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D104799
2021-07-18 12:07:16 +02:00
Valentin Churavy
0b1b7443f1 [Orc] At CBindings for LazyRexports
At C bindings and an example for LLJIT with lazy reexports

Differential Revision: https://reviews.llvm.org/D104672
2021-07-01 21:52:05 +02:00
Lang Hames
a397416183 [ORC] Rename TargetProcessControl to ExecutorProcessControl. NFC.
This is a first step towards consistently using the term 'executor' for the
process that executes JIT'd code. I've opted for 'executor' as the preferred
term over 'target' as target is already heavily overloaded ("the target
machine for the executor" is much clearer than "the target machine for the
target").
2021-07-01 13:31:12 +10:00
Valentin Churavy
0c3065489c [Orc] Fix name of LLVMOrcIRTransformLayerSetTransform
In https://reviews.llvm.org/D103855 we added access to IRTransformLayer, but I
just noticed that the function name is following the wrong pattern.

Differential Revision: https://reviews.llvm.org/D104840
2021-06-30 21:43:34 +02:00
Stefan Gränitz
374d6c56df [Orc][examples] Temporarily disable LLJITWithRemoteDebugging build and test
The underlying TargetProcessControl API changes with D104694. Once it landed I can patch and re-enable the example.
2021-06-28 16:54:53 +02:00
Mara Sophie Grosch
27c4ea000e [Orc][examples] LLJITWithRemoteDebugger: fix CMake when utils are not built 2021-06-27 13:52:04 +02:00
Lang Hames
7afd72c69b [ORC][examples] Add missing library dependence 2021-06-19 14:48:34 +10:00
Lang Hames
b6ca0c60bb [ORC][C-bindings] Add access to LLJIT IRTransformLayer, ThreadSafeModule utils.
This patch was derived from Valentin Churavy's work in
https://reviews.llvm.org/D104480. It adds support for setting the transform on
an IRTransformLayer, and for accessing the IRTransformLayer in LLJIT. It also
adds access to the ThreadSafeModule::withModuleDo method for thread-safe
access to modules.

A new example has been added to show how to use these APIs to optimize a module
during materialization.

Thanks Valentin!

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D103855
2021-06-19 11:50:27 +10:00
Lang Hames
320507db5e [ORC][examples] Fix file name in comment. 2021-06-19 11:50:26 +10:00
Lang Hames
d2647ecc04 [ORC][C-bindings] Re-order object transform function arguments.
ObjInOut is an in-out parameter not a return value argument, so by convention
it should come after the context value (Ctx).
2021-06-18 22:12:39 +10:00
Lang Hames
04c9dcdc5d [ORC] Add support for dumping objects to the C API.
Provides ObjectTransformLayer APIs, a getter to access the
ObjectTransformLayer member of LLJIT, and the DumpObjects utility
to make construction of a dump-to-disk transform easy.

An example showing how the new APIs can be used has been added in
llvm/examples/OrcV2Examples/OrcV2CBindingsDumpObjects.
2021-06-18 20:56:45 +10:00
Stefan Gränitz
9c76412adb [Orc][examples] Add missing FileCheck for lit test and polish output 2021-04-07 11:12:20 +02:00
Stefan Gränitz
cd6dd68cfd [Orc][examples] Add missing dependency to OrcShared in LLJITWithRemoteDebugging 2021-03-28 17:48:28 +02:00
Stefan Gränitz
8130b8a5a1 [Orc][examples] Add LLJITWithRemoteDebugging example 2021-03-28 17:25:09 +02:00
Stefan Gränitz
934fb256bb [Orc][examples] Factor out make_error from parseExampleModule (NFC) 2021-03-26 18:49:07 +01:00
Stefan Gränitz
ecba2bfe16 [Orc][examples] Fix copy/paste issues in comments and inclue guards (NFC) 2021-03-26 18:49:07 +01:00
Lang Hames
0342e92559 [ORC] Fix some comments in the LLJITWithObjectLinkingLayerPlugin example. 2021-03-12 19:13:42 -08:00
Lang Hames
d7eedad739 [JITLink][ORC] Make the LinkGraph available to modifyPassConfig.
This makes the target triple, graph name, and full graph content available
when making decisions about how to populate the linker pass pipeline.

Also updates the LLJITWithObjectLinkingLayerPlugin example to show more
API use, including use of the API changes in this patch.
2021-03-12 18:42:51 -08:00
Stefan Gränitz
249736daff [Orc] Fix OrcV2Examples after D94690
Differential Revision: https://reviews.llvm.org/D94690
2021-01-15 13:45:46 +01:00
Stefan Gränitz
9bbc173276 [Orc] Allow LLJITBuilder's CreateObjectLinkingLayer to return errors
It can be useful for an ObjectLinkingLayerCreator to allow callee errors to get propagated to the builder. Specifically, this is the case when the ObjectLayer uses the EHFrameRegistrationPlugin, because it requires a TPCEHFrameRegistrar and instantiation for it may fail (e.g. if the required registration symbols are missing in the target process).

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D94690
2021-01-15 12:53:41 +01:00
Lang Hames
09080e305f [ORC][examples] Fix missing includes/dependencies in more examples. 2020-11-13 20:22:01 +11:00
Lang Hames
7103f74446 [ORC] Break up OrcJIT library, add Orc-RPC based remote TargetProcessControl
implementation.

This patch aims to improve support for out-of-process JITing using OrcV2. It
introduces two new class templates, OrcRPCTargetProcessControlBase and
OrcRPCTPCServer, which together implement the TargetProcessControl API by
forwarding operations to an execution process via an Orc-RPC Endpoint. These
utilities are used to implement out-of-process JITing from llvm-jitlink to
a new llvm-jitlink-executor tool.

This patch also breaks the OrcJIT library into three parts:
  -- OrcTargetProcess: Contains code needed by the JIT execution process.
  -- OrcShared: Contains code needed by the JIT execution and compiler
     processes
  -- OrcJIT: Everything else.

This break-up allows JIT executor processes to link against OrcTargetProcess
and OrcShared only, without having to link in all of OrcJIT. Clients executing
JIT'd code in-process should start linking against OrcTargetProcess as well as
OrcJIT.

In the near future these changes will enable:
  -- Removal of the OrcRemoteTargetClient/OrcRemoteTargetServer class templates
     which provided similar functionality in OrcV1.
  -- Restoration of Chapter 5 of the Building-A-JIT tutorial series, which will
     serve as a simple usage example for these APIs.
  -- Implementation of lazy, cross-target compilation in lli's -jit-kind=orc-lazy
     mode.
2020-11-13 17:05:13 +11:00
Lang Hames
242ff9475f [ORC] Break up C-API header Orc.h, and add JITEventListener support.
This patch breaks Orc.h up into Orc.h, LLJIT.h and OrcEE.h.

Orc.h contain core Orc utilities.
LLJIT.h contains LLJIT specific types and functions.
OrcEE.h contains types and functions that depend on ExecutionEngine.

The intent is that these headers should match future library divisions: Clients
who only use Orc.h should only need to link againt the Orc core libraries,
clients using LLJIT.h will also need to link against LLVM core, and clients
using OrcEE.h will also have to link against ExecutionEngine.

In addition to breaking up the Orc.h header this patch introduces functions to:
(1) Set the object linking layer creation function on LLJITBuilder.
(2) Create an RTDyldObjectLinkingLayer instance (particularly for use in (1)).
(3) Register JITEventListeners with an RTDyldObjectLinkingLayer.

Together (1), (2) and (3) can be used to force use of RTDyldObjectLinkingLayer
as the underlying JIT linker for LLJIT, rather than the platform default, and
to register event listeners with the RTDyldObjectLinkingLayer.
2020-10-19 01:59:04 -07:00
Lang Hames
dff52178bf [ORC] Add an OrcV2 ResourceTracker / Removable-code example. 2020-10-19 01:59:04 -07:00
Lang Hames
318f69487c [ORC] Add basic ResourceTracker support to the OrcV2 C Bindings.
Based on a patch by Andres Freund. Thanks Andres!
2020-10-19 01:59:04 -07:00
Lang Hames
e841390d1a [ORC] Rename LLVMOrcJITDylibDefinitionGeneratorRef.
The DefinitionGenerator class has been moved out of JITDylib. This updates
the C API type and function names to reflect that.
2020-10-19 01:59:04 -07:00
Lang Hames
efc9f3486a [ORC] Add support for resource tracking/removal (removable code).
This patch introduces new APIs to support resource tracking and removal in Orc.
It is intended as a thread-safe generalization of the removeModule concept from
OrcV1.

Clients can now create ResourceTracker objects (using
JITDylib::createResourceTracker) to track resources for each MaterializationUnit
(code, data, aliases, absolute symbols, etc.) added to the JIT. Every
MaterializationUnit will be associated with a ResourceTracker, and
ResourceTrackers can be re-used for multiple MaterializationUnits. Each JITDylib
has a default ResourceTracker that will be used for MaterializationUnits added
to that JITDylib if no ResourceTracker is explicitly specified.

Two operations can be performed on ResourceTrackers: transferTo and remove. The
transferTo operation transfers tracking of the resources to a different
ResourceTracker object, allowing ResourceTrackers to be merged to reduce
administrative overhead (the source tracker is invalidated in the process). The
remove operation removes all resources associated with a ResourceTracker,
including any symbols defined by MaterializationUnits associated with the
tracker, and also invalidates the tracker. These operations are thread safe, and
should work regardless of the the state of the MaterializationUnits. In the case
of resource transfer any existing resources associated with the source tracker
will be transferred to the destination tracker, and all future resources for
those units will be automatically associated with the destination tracker. In
the case of resource removal all already-allocated resources will be
deallocated, any if any program representations associated with the tracker have
not been compiled yet they will be destroyed. If any program representations are
currently being compiled then they will be prevented from completing: their
MaterializationResponsibility will return errors on any attempt to update the
JIT state.

Clients (usually Layer writers) wishing to track resources can implement the
ResourceManager API to receive notifications when ResourceTrackers are
transferred or removed. The MaterializationResponsibility::withResourceKeyDo
method can be used to create associations between the key for a ResourceTracker
and an allocated resource in a thread-safe way.

RTDyldObjectLinkingLayer and ObjectLinkingLayer are updated to use the
ResourceManager API to enable tracking and removal of memory allocated by the
JIT linker.

The new JITDylib::clear method can be used to trigger removal of every
ResourceTracker associated with the JITDylib (note that this will only
remove resources for the JITDylib, it does not run static destructors).

This patch includes unit tests showing basic usage. A follow-up patch will
update the Kaleidoscope and BuildingAJIT tutorial series to OrcV2 and will
use this API to release code associated with anonymous expressions.
2020-10-18 21:02:54 -07:00
Stefan Gränitz
bf37528089 [ORC][examples] Temporarily remove LLJITWithChildProcess until ORC TPC lands
This solves a phase ordering problem: OrcV2 remote process support depends on OrcV2 removable code, OrcV2 removable code depends on OrcV1 removal, OrcV1 removal depends on LLJITWithChildProcess migration, and LLJITWithChildProcess migration depends on OrcV2 TargetProcessControl support.
2020-10-01 10:25:13 +02:00
Lang Hames
cafb3faae9 [ORC][examples] Add missing library dependencies. 2020-09-22 16:18:00 -07:00
Lang Hames
6ab9d78a52 [ORC][examples] Add an OrcV2 example for IR optimization via IRTransformLayer.
Shows how to write a custom IR transform to apply a legacy::PassManager
pipeline.
2020-09-19 18:59:52 -07:00
Lang Hames
c221b22448 [examples] Fix dependencies for OrcV2Examples/LLJITWithThinLTOSummaries. 2020-08-25 11:51:20 -07:00
Stefan Gränitz
45e94e8a0b [ORC] Add a LLJITWithThinLTOSummaries example in OrcV2Examples
The example demonstrates how to use a module summary index file produced for ThinLTO to:
* find the module that defines the main entry point
* find all extra modules that are required for the build

A LIT test runs the example as part of the LLVM test suite [1] and shows how to create a module summary index file.
The code also provides two Error types that can be useful when working with ThinLTO summaries.

[1] if LLVM_BUILD_EXAMPLES=ON and platform is not Windows

Differential Revision: https://reviews.llvm.org/D85974
2020-08-23 14:02:10 +02:00
Stephen Neuendorffer
1e6f44130d [examples][cmake] build fix for examples with BUILD_SHARED_LIBS=on
Differential Revision: https://reviews.llvm.org/D85987
2020-08-14 11:39:04 -07:00
Stefan Gränitz
4cb29fd0d3 [ORC] Build LLJITWithChildProcess example only on UNIX host systems
Differential Revision: https://reviews.llvm.org/D85919
2020-08-14 18:09:08 +02:00
Stefan Gränitz
0de7dcb92e [ORC][NFC] Fix a header comment 2020-08-14 11:46:03 +02:00
Stefan Gränitz
92757f17c9 [ORC] Add JITLink-compatible remote memory-manager and LLJITWithChildProcess example
This adds RemoteJITLinkMemoryManager is a new subclass of OrcRemoteTargetClient. It implements jitlink::JITLinkMemoryManager and targets the OrcRemoteTargetRPCAPI.

Behavior should be very similar to RemoteRTDyldMemoryManager. The essential differnce with JITLink is that allocations work in isolation from its memory manager. Thus, the RemoteJITLinkMemoryManager might be seen as "JITLink allocation factory".

RPCMMAlloc is another subclass of OrcRemoteTargetClient and implements the actual functionality. It allocates working memory on the host and target memory on the remote target. Upon finalization working memory is copied over to the tagrte address space. Finalization can be asynchronous for JITLink allocations, but I don't see that it makes a difference here.

Differential Revision: https://reviews.llvm.org/D85919
2020-08-14 11:34:44 +02:00
Lang Hames
85d946b864 [ORC] Enable use of TargetProcessControl::getMemMgr with ObjectLinkingLayer.
This patch makes ownership of the JITLinkMemoryManager by ObjectLinkingLayer
optional: the layer can still own the memory manager but no longer has to.

Evevntually we want to move to a state where ObjectLinkingLayer never owns its
memory manager. For now allowing optional ownership makes it easier to develop
classes that can dynamically use either RTDyldObjectLinkingLayer, which owns
its memory managers, or ObjectLinkingLayer (e.g. LLJIT).
2020-07-23 16:18:57 -07:00
Simon Pilgrim
a834cda061 Fix implicit dependency on CommandLine.h 2020-07-23 12:15:17 +01:00
Lang Hames
6f874be76d [ORC] Add a TargetProcessControl-based dynamic library search generator.
TPCDynamicLibrarySearchGenerator uses a TargetProcessControl instance to
load libraries and search for symbol addresses in a target process. It
can be used in place of a DynamicLibrarySearchGenerator to enable
target-process agnostic lookup.
2020-07-22 16:19:24 -07:00
Lang Hames
757f986da8 [ORC] Add TargetProcessControl and TPCIndirectionUtils APIs.
TargetProcessControl is a new API for communicating with JIT target processes.
It supports memory allocation and access, and inspection of some process
properties, e.g. the target proces triple and page size.

Centralizing these APIs allows utilities written against TargetProcessControl
to remain independent of the communication procotol with the target process
(which may be direct memory access/allocation for in-process JITing, or may
involve some form of IPC or RPC).

An initial set of TargetProcessControl-based utilities for lazy compilation is
provided by the TPCIndirectionUtils class.

An initial implementation of TargetProcessControl for in-process JITing
is provided by the SelfTargetProcessControl class.

An example program showing how the APIs can be used is provided in
llvm/examples/OrcV2Examples/LLJITWithTargetProcessControl.
2020-07-16 15:09:13 -07:00
Eric Christopher
26606860a8 [docs/examples] As part of using inclusive language within the llvm
project, migrate away from the use of blacklist and whitelist.
2020-06-20 00:51:18 -07:00
Lang Hames
15668dbaec [examples] Fix an incomplete comment. 2020-05-01 11:06:41 -07:00
Lang Hames
950b4bd8f4 [examples] Call setProcessAllSections to ensure debug sections aren't droppped.
Calling setProcessAllSections(true) is required to make sure that all sections,
even those not marked as necessary for execution, are passed to the memory
manager.
2020-05-01 11:06:41 -07:00
Lang Hames
13d5cb59cb [ORC] Add example showing how to initialize/deinitialize a JITDylib with LLJIT. 2020-04-18 14:16:54 -07:00
Lang Hames
4315ffca85 [ORC] Require JITDylib to be specified when adding IR and objects in the C API. 2020-04-09 17:59:26 -07:00
Lang Hames
a737d1b399 [ORC] Add C API support for adding object files to an LLJIT instance. 2020-04-09 16:18:46 -07:00
Lang Hames
e2925289c9 [ORC] Expand the OrcV2 C API bindings.
Adds basic support for LLJITBuilder and DynamicLibrarySearchGenerator. This
allows C API clients to configure LLJIT to expose process symbols to JIT'd
code. An example of this is added in
llvm/examples/OrcV2CBindingsReflectProcessSymbols.
2020-04-09 16:18:46 -07:00
Stephen Neuendorffer
5f9c6088ec [examples] Fixes for BUILD_SHARED_LIBS=on 2020-03-23 15:21:45 -07:00