Custom STL Allocators

1
Custom STL Allocators

• Pete Isensee
• Xbox Advanced Technology Group
• pkisensee_at_msn.com

2
Topics

• Allocators What are They Good For?
• Writing Your First Allocator
• The Devil in the Details
• Allocator Pitfalls
• State
• Syntax
• Testing
• Case Study

3
Containers and Allocators

• STL containers allocate memory
• e.g. vector (contiguous), list (nodes)
• string is a container, for this talk
• Allocators provide a standard interface for
  container memory use
• If you dont provide an allocator, one is
  provided for you

4
Example

• Default Allocator
• listltintgt b
• // same as
• listlt int, allocatorltintgt gt b
• Custom Allocator
• include MyAlloc.h
• listlt int, MyAllocltintgt gt c

5
The Good

• Original idea abstract the notion of near and
  far memory pointers
• Expanded idea allow customization of container
  allocation
• Good for
• Size Optimizing memory usage (pools, fixed-size
  allocators)
• Speed Reducing allocation time (single-threaded,
  one-time free)

6
Example Allocators

• No heap locking (single thread)
• Avoiding fragmentation
• Aligned allocations (_aligned_malloc)
• Fixed-size allocations
• Custom free list
• Debugging
• Custom heap
• Specific memory type

7
The Bad

• No realloc()
• Requires advanced C compilers
• C Standard hand-waving
• Generally library-specific
• If you change STL libraries you may need to
  rewrite allocators
• Generally not cross-platform
• If you change compilers you may need to rewrite
  allocators

8
The Ugly

• Not quite real objects
• Allocators with state may not work as expected
• Gnarly syntax
• mapltint,chargt m
• mapltint,char,lessltintgt,
• MyAllocltpairltint,chargt gt gt m

9
Pause to Reflect

• Premature optimization is the root of all evil
  Donald Knuth
• Allocators are a last resort and low-level
  optimization
• Especially for games, allocators can be the
  perfect optimization
• Written correctly, they can be introduced w/o
  many code changes

10
Writing Your First Allocator

• Create MyAlloc.h
• include ltmemorygt
• Copy or derive from the default allocator
• Rename allocator to MyAlloc
• Resolve any helper functions
• Replace some code with your own

11
Writing Your First Allocator

• Demo
• Visual C Pro 7.0 (13.00.9466)
• Dinkumware STL (V3.100009)
• 933MHz PIII w/ 512MB
• Windows XP Pro 2002
• Launch Visual Studio

12
Two key functions

• Allocate
• Deallocate
• Thats all!

13
Conventions

• templatelt typename T gt
• class allocator
• typedef size_t size_type
• typedef T pointer
• typedef const T const_pointer
• typedef T value_type

14
Allocate Function

• pointer allocate( size_type n,
  allocatorltvoidgtconst_pointer p 0)
• n is the number of items T, NOT bytes
• returns pointer to enough memory to hold n
  sizeof(T) bytes
• returns raw bytes NO construction
• may throw an exception (stdbad_alloc)
• default calls operator new
• p is optional hint avoid

15
Deallocate function

• void deallocate( pointer p,
• size_type n )
• p must come from allocate()
• p must be raw bytes already destroyed
• n must match the n passed to allocate()
• default calls operator delete(void)
• Most implementations allow and ignore NULL p you
  should too

16
A Custom Allocator

• Demo
• Thats it!
• Not quite the devil is in the details
• Construction
• Destruction
• Example STL container code
• Rebind

17
Construction

• Allocate() doesnt call constructors
• Why? Performance
• Allocators provide construct function
• void construct(pointer p, const T t)
• new( (void)p ) T(t)
• Placement new
• Doesnt allocate memory
• Calls copy constructor

18
Destruction

• Deallocate() doesnt call destructors
• Allocators provide a destroy function
• void destroy( pointer p )
• ((T)p)-gtT()
• Direct destructor invocation
• Doesnt deallocate memory
• Calls destructor

19
Example Vector

• templatelt typename T, typename A gt
• class vector
• A a // allocator
• pointer pFirst // first object
• pointer pEnd // 1 beyond end
• pointer pLast // 1 beyond last

20
Example Reserve

• vectorreserve( size_type n )
• pointer p a.allocate( n, 0 )
• // loop on a.construct() to copy
• // loop on a.destroy() to tear down
• a.deallocate( pFirst, capacity() )
• pFirst p
• pLast p size()
• pEnd p n

21
Performance is paramount

• Reserve
• Single allocation
• Doesnt default construct anything
• Deals properly with real objects
• No memcpy
• Copy constructs new objects
• Destroys old objects
• Single deallocation

22
Rebind

• Allocators dont always allocate T
• listltObjgt ObjList // allocates nodes
• How? Rebind
• templatelttypename Ugt struct rebind
• typedef allocatorltUgt other
• To allocate an N given type T
• AllocltTgt a
• T t a.allocate(1) // allocs sizeof(T)
• AllocltTgtrebindltNgtother na
• N n na.allocate(1) // allocs sizeof(N)

23
Allocator Pitfalls

• To Derive or Not to Derive
• State
• Copy ctor and template copy ctor
• Allocator comparison
• Syntax issues
• Testing
• Case Study

24
To Derive or Not To Derive

• Deriving from stdallocator
• Dinkumware derives (see ltxdebuggt)
• Must provide rebind, allocate, deallocate
• Less code easier to see differences
• Writing from scratch
• Allocator not designed as base class
• Josuttis and Austern write from scratch
• Better understanding
• Personal preference

25
Allocators with State

• State allocator member data
• Default allocator has no data
• C Std says (paraphrasing 20.1.5)
• Vendors encouraged to support allocators with
  state
• Containers may assume that allocators dont have
  state

26
State Recommendations

• Be aware of compatibility issues across STL
  vendors
• listsplice() or Cswap()will indicate if your
  vendor supports stateful allocators
• Dinkumware yes
• STLport no
• Test carefully

27
State Implications

• Container size increase
• Must provide allocator
• Constructor(s)
• Default may be private if parameters required
• Copy constructor
• Template copy constructor
• Global comparison operators (, !)
• No assignment operators required
• Avoid static data generates one per T

28
Heap Allocator Example

• templatelt typename T gt
• class Halloc
• Halloc() // could be private
• explicit Halloc( HANDLE hHeap )
• Halloc( const Halloc ) // copy
• templatelt typename U gt // templatized
• Halloc( const HallocltUgt ) // copy

29
Template Copy Constructor

• Cant see private data
• templatelt typename U gt
• Halloc( const HallocltUgt a )
• m_hHeap( a.m_hHeap ) // error
• Solutions
• Provide public data accessor function
• Or allow access to other types U
• template lttypename Ugt
• friend class Halloc

30
Allocator comparison

• Example
• templatelt typename T, typename U gt
• bool operator( const AllocltTgt a,
• const AllocltUgt b )
• return a.state b.state
• Provide both and !
• Should be global fucns, not members
• May require accessor functions

31
Syntax Typedefs

• Prefer typedefs
• Offensive
• listlt int, Alloclt int gt gt b
• Better
• // .h
• typedef Alloclt int gt IAlloc
• typedef listlt int, IAlloc gt IntList
• // .cpp
• IntList v

32
Syntax Construction

• Containers accept allocators via ctors
• IntList b( IAlloc( x,y,z ) )
• If none specified, you get the default
• IntList b // calls IAlloc()
• Map/multimap requires pairs
• Alloclt pairlt K,T gt gt a
• maplt K, T, lessltKgt,
• Alloclt pairlt K,T gt gt gt
• m( lessltKgt(), a )

33
Syntax Other Containers

• Container adaptors accept containers via
  constructors, not allocators
• AllocltTgt a
• dequelt T, AllocltTgt gt d(a)
• stacklt T, dequeltT,AllocltTgt gt gt s(d)
• String example
• AllocltTgt a
• basic_stringlt T, char_traitsltTgt, AllocltTgt gt s(a)

34
Testing

• Test the normal case
• Test with all containers (dont forget string,
  hash containers, stack, etc.)
• Test with different objects T, particularly those
  w/ non-trivial dtors
• Test edge cases like listsplice
• Verify that your version is better!
• Allocator test framework www.tantalon.com/pete.ht
  m

35
Case Study

• In-place allocator
• Hand off existing memory block
• Dole out allocations from the block
• Never free
• Example usage
• typedef InPlaceAlloclt int gt IPA
• void p malloc( 1024 )
• listlt int, IPA gt x( IPA( p, 1024 ) )
• x.push_back( 1 )
• free( p )
• View code

36
In-Place Allocator

• Problems
• Fails w/ multiple concurrent copies
• No copy constructor
• Didnt support comparison
• Didnt handle containers of void
• Correct implementation
• Reference counted
• Copy constructor implemented
• Comparison operators
• Void specialization

37
In-Place Summary

• Speed
• Scenario add x elements, remove half
• About 50x faster than default allocator!
• Advantages
• Fast no overhead no fragmentation
• Whatever memory you want
• Disadvantages
• Proper implementation isnt easy
• Limited use

38
Recommendations

• Allocators a last resort optimization
• Base your allocator on ltmemorygt
• Beware porting issues (both compilers and STL
  vendor libraries)
• Beware allocators with state
• Test thoroughly
• Verify speed/size improvements

39
Recommendations part II

• Use typedefs to simplify life
• Dont forget to write
• Rebind
• Copy constructor
• Templatized copy constructor
• Comparison operators
• Void specialization

40
References

• C Standard section 20.1.5, 20.4.1
• Your STL implementation ltmemorygt
• GDC Proceedings References section
• Game Gems III
• pkisensee_at_msn.com
• www.tantalon.com/pete.htm