When asan is enabled, we poison slabs as we allocate them, and only unpoison the pieces
we need from the slab.
However, in Reset, we were failing to reset the state of the slab back to being poisoned.
Patch by b17 c0de.
llvm-svn: 257388
I added the poisoning back in r76891 (2009) because of some bugs in
Unladen Swallow, and then Evan Cheng added the setRangeWritable() call
in r81308. Profiling a Release+Asserts build on Windows shows that this
memory protection call is actually very expensive. 4 seconds of a 70
second Clang compilation are spent in VirtualQuery. These days we have
more reliable tools like ASan to find these kinds of bugs, so we can go
ahead and retire these checks.
llvm-svn: 235542
Instead of aligning and moving the CurPtr forward, and then comparing
with End, simply calculate how much space is needed, and compare that
to how much is available.
Hopefully this avoids any doubts about comparing addresses possibly
derived from past the end of the slab array, overflowing, etc.
Also add a test where aligning CurPtr would move it past End.
llvm-svn: 217330
In theory, alignPtr() could push a pointer beyond the end of the current slab, making
comparisons with that pointer undefined behaviour. Use an integer type to avoid this.
llvm-svn: 216973
In both Clang and LLVM, this is a common pattern:
Size = sizeof(DeclRefExpr) + SomeExtraStuff;
void *Mem = Context.Allocate(Size, llvm::alignOf<DeclRefExpr>());
return new (Mem) DeclRefExpr(...);
The annoying thing is that because the default placement-new operator has a
nothrow specification, the compiler will insert a null check of Mem before
calling the DeclRefExpr constructor. This null check is redundant for us,
because we expect the allocation functions to never return null.
By annotating the allocator functions with returns_nonnull, we can optimize
away these checks. Compiling clang with a recent version of Clang and measuring
with:
$ perf stat -r20 bin/clang.patch -fsyntax-only -w gcc.c && perf stat -r20 bin/clang.orig -fsyntax-only -w gcc.c
Shows a 2.4% speed-up (+- 0.8%).
The pattern occurs in LLVM too. Measuring with -O3 (and now using bzip2.c
instead, because it's smaller):
$ perf stat -r20 bin/clang.patch -O3 -w bzip2.c && perf stat -r20 bin/clang.orig -O3 -w bzip2.c
Shows 4.4 % speed-up (+- 1%).
If anyone knows of a similar attribute we can use for MSVC, or some other
technique to get rid off the null check there, please let me know.
Differential Revision: http://reviews.llvm.org/D4989
llvm-svn: 216192
We already handle the no-slabs case when checking whether the current slab
is large enough: if no slabs have been allocated, CurPtr and End are both 0.
alignPtr(0), will still be 0, and so "if (Ptr + Size <= End)" fails.
Differential Revision: http://reviews.llvm.org/D4943
llvm-svn: 215841
implementation of the SpecificBumpPtrAllocator -- we have to actually
move the subobject. =] Noticed when using this code more directly.
llvm-svn: 206582
because there is another (size_t, size_t) overload of Allocator, and the
only distinguishing factor is that one is a tempalte and the other
isn't. There was only one usage of this and that one was easily
converted to carry the alignment constraint in the type itself.
llvm-svn: 206325
by removing the MallocSlabAllocator entirely and just using
MallocAllocator directly. This makes all off these allocators expose and
utilize the same core interface.
The only ugly part of this is that it exposes the fact that the JIT
allocator has no real handling of alignment, any more than the malloc
allocator does. =/ It would be nice to fix both of these to support
alignments, and then to leverage that in the BumpPtrAllocator to do less
over allocation in order to manually align pointers. But, that's another
patch for another day. This patch has no functional impact, it just
removes the somewhat meaningless wrapper around MallocAllocator.
llvm-svn: 206267
allocation libraries, may allow more efficient allocation and
deallocation. It at least makes the interface implementable by the JIT
memory manager.
However, this highlights problematic overloading between the void* and
the T* deallocation functions. I'm looking into a better way to do this,
but as it happens, it comes up rarely in the codebase.
llvm-svn: 206265
overloads. This doesn't matter *that* much yet, but it will in
a subsequent patch. I had tested the original pattern, but not my
attempt to pacify MSVC. This at least appears to work. Still fixing the
rest of the fallout in the final patch that uses these overloads, but it
will follow shortly.
llvm-svn: 206259
'sizeof(T)' for T == void and produces a hard error. I cannot fathom why
this is OK. Oh well. switch to an explicit test for being the
(potentially qualified) void type, which is the only specific case I was
worried about. Hopefully this survives the libstdc++ build bots which
have limited type traits implementations...
llvm-svn: 206256
to types which we can compute the size of. The comparison with zero
isn't actually interesting here, it's mostly about putting sizeof into
a sfinae context.
This is particular important for Deallocate as otherwise the void*
overload can quickly become ambiguous.
llvm-svn: 206251
along with templated overloads much like we have for Allocate. These
will facilitate switching the Deallocate interface of all the Allocator
classes to accept the size by pre-filling it from the type size where we
can do so. I plan to convert several uses to the template variants in
subsequent patches prior to adding the Size parameter.
No functionality changed, WIP.
llvm-svn: 206230
rather than defining them (differently!) in both allocators. This also
serves as a basis for documenting and even enforcing some of the
LLVM-style "allocator" concept methods which must exist with various
signatures.
I plan on extending and changing the signatures of these to further
simplify our allocator model in subsequent commits, so I wanted to
factor things as best as I could first. Notably, I'm working to add the
'Size' to the deallocation method of all allocators. This has several
implications not the least of which are faster deallocation times on
certain allocation libraries (tcmalloc). It also will allow the JIT
allocator to fully model the existing allocation interfaces and allow
sanitizer poisoning of deallocated regions. The list of advantages goes
on. =] But by factoring things first I'll be able to make this easier by
first introducing template helpers for the deallocation path.
llvm-svn: 206225
declaration. GCC 4.7 appears to get hopelessly confused by declaring
this function within a member function of a class template. Go figure.
llvm-svn: 206152
abstract interface. The only user of this functionality is the JIT
memory manager and it is quite happy to have a custom type here. This
removes a virtual function call and a lot of unnecessary abstraction
from the common case where this is just a *very* thin vaneer around
a call to malloc.
Hopefully still no functionality changed here. =]
llvm-svn: 206149
slabs rather than embedding a singly linked list in the slabs
themselves. This has a few advantages:
- Better utilization of the slab's memory by not wasting 16-bytes at the
front.
- Simpler allocation strategy by not having a struct packed at the
front.
- Avoids paging every allocated slab in just to traverse them for
deallocating or dumping stats.
The latter is the really nice part. Folks have complained from time to
time bitterly that tearing down a BumpPtrAllocator, even if it doesn't
run any destructors, pages in all of the memory allocated. Now it won't.
=]
Also resolves a FIXME with the scaling of the slab sizes. The scaling
now disregards specially sized slabs for allocations larger than the
threshold.
llvm-svn: 206147
This patch is to fix the following warning when compiled with MSVC 64 bit.
warning C4334: '<<' : result of 32-bit shift implicitly converted to 64
bits (was 64-bit shift intended?)
llvm-svn: 205245
parameters rather than runtime parameters.
There is only one user of these parameters and they are compile time for
that user. Making these compile time seems to better reflect their
intended usage as well.
llvm-svn: 205143
BumpPtrAllocator significantly less strange by making it a simple
function of the number of slabs allocated rather than by making it
a recurrance. I *think* the previous behavior was essentially that the
size of the slabs would be doubled after the first 128 were allocated,
and then doubled again each time 64 more were allocated, but only if
every allocation packed perfectly into the slab size. If not, the wasted
space wouldn't be counted toward increasing the size, but allocations
over the size threshold *would*. And since the allocations over the size
threshold might be much larger than the slab size, this could have
somewhat surprising consequences where we rapidly grow the slab size.
This currently requires adding state to the allocator to track the
number of slabs currently allocated, but that isn't too bad. I'm
planning further changes to the allocator that will make this state fall
out even more naturally.
It still doesn't fully decouple the growth rate from the allocations
which are over the size threshold. That fix is coming later.
This specific fix will allow making the entire thing into a more
stateless device and lifting the parameters into template parameters
rather than runtime parameters.
llvm-svn: 204993
rewrite some of them to be more clear.
The terminology being used in our allocators is making me really sad. We
call things slab allocators that aren't at all slab allocators. It is
quite confusing.
llvm-svn: 204907
The problem with having DefaultSlabAllocator being a global static is that it is undefined if BumpPtrAllocator
will be usable during global initialization because it is not guaranteed that DefaultSlabAllocator will be
initialized before BumpPtrAllocator is created and used.
llvm-svn: 189433
AKA: Recompile *ALL* the source code!
This one went much better. No manual edits here. I spot-checked for
silliness and grep-checked for really broken edits and everything seemed
good. It all still compiles. Yell if you see something that looks goofy.
llvm-svn: 169133
