Embracing the C STL: Why Angle Brackets are Good for You Pete Isensee

1
Embracing the C STLWhy Angle Brackets
areGood for YouPete Isensee
2
Introduction

• STL Background History
• Key Concepts
• Containers, Iterators and Algorithms
• Efficiency and Thread Safety
• Tips and Tricks

3
Goals

• STL Newbie
• Convince you that its a good thing
• Help you avoid common mistakes
• STL Junkie
• Add new STL techniques to your toolbox
• Tricks and tips

4
Preliminaries

• Microsoft Visual C 6.0
• Dinkumware STL implementation
• Benchmarks on Pentium III - 550MHz
• Performance graphs show relative performance
  taller is better!

5
History

• Alex Stepanov Meng Lee, based on earlier work
  by Stepanov Musser
• Proposed to C committee late 93
• HP version released 94
• Accepted into Standard summer 94
• Standard froze 97, ratified 98

6
Advantages

• Standardized
• Thin efficient
• Little inheritance no virtual functions
• Small easy to learn
• Flexible and extensible
• Naturally open source

7
Disadvantages

• Template syntax
• Difficult to read decipher
• Poor or incomplete compiler support
• Code bloat potential
• No constraints on template types
• Limited container types

8
Key Concepts

• Generic algorithms
• Container classes
• Iterators container walkers for accessing
  container elements
• Iterators provide an abstraction of container
  access, which in turn allows for generic
  algorithms
• Iterator invalidation

9
Key Concepts (cont.)

• Ranges C/C past-the-end pointer
• T WallaceN
• T p (Wallace N) // valid pointer
• T w (Wallace N) // invalid dereference
• c.begin() (Wallace) // first element
• c.end() (Wallace N) // valid iterator
• c.end() // invalid dereference
• end() - begin() size()
• if (begin() end()) container is empty
• for (iter i begin() i ! end() i)

10
Key Concepts (cont.)

• Linear search example
• template ltclass InputItr, class Tgt InputItr
• find(InputItr bg, InputItr end, const T val)
• while (bg ! end bg ! val)
• bg
• return (bg)
• const int nSize 4
• int GromitnSize 5, 18, 23, 9
• int pFind find(Gromit, Gromit nSize, 23)
• vectorltintgt Preston
• vectorltintgtiterator i
• find(Preston.begin(), Preston.end(), 4)

11
Putting the STL into Action

• Include files have no .h
• Standard namespace
• include ltcstdiogt // new include method
• include ltvectorgt // vector container
• include ltalgorithmgt // STL algorithms
• using namespace std // assume std
• vectorltintgt Chuck // declare a growable array
• Chuck.push_back(1) // add an element
• find(Chuck.begin(), Chuck.end(), 1)

12
Containers

• Containers contain elements they own the
  objects
• Containers provide iterators that point to its
  elements.
• Containers provide a minimal set of operations
  for manipulating elements

13
Containers (cont.)

• Minimum container object requirements
• X() // default ctor
• X(const X) // copy ctor
• X operator (const X) // assignment op
• bool operator lt (const X) // comparison op
• bool operator (const X) // comparison op

14
Vector

• Dynamic array
• Fast ins/erase from end of vector
• reserve(), capacity()
• Contiguous block of memory
• Obliged to grow by some factor (2x) when size()
  exceeds capacity()
• Insert invals all iters if capacity change
  insert/erase invals all iters following

15
Deque

• Double-ended queue (deck)
• Fast ins/erase at begin and end
• Directory array of pointers to nodes, where each
  node is small array of T
• Insert invals all iters erase in middle invals
  all erase at begin/end on invals iter to
  begin/end

16
List

• Doubly-linked list
• Fast insert/erase no random access
• Special functions splice(), merge()
• Erase invals iters to erased elements insert
  never invals any iters

17
Set

• List of sorted elements
• Fast retrieval based on key (log N)
• Fast insert/erase (log N)
• Red-black tree (balanced 2-3-4 tree)
• Erase invals erased elems insert never invals
  any iters

18
Map

• Dictionary of sorted elements
• List of sorted key and value pairs
• Same implementation and characteristics of set
  container

19
Container Adaptors

• Example adapator code
• stackltint, dequeltintgt gt TechnoTrousers
• TechnoTrousers.push(1)
• int i TechnoTrousers.top()
• TechnoTrousers.pop()

20
Whats Missing?

• stack-based arrays (T aN)
• hash tables
• singly-linked lists
• some STL implementations include one or more of
  these non-standard containers

21
Container Efficiency

• Overhead is approx. per-element size in bytes
• Hash and slist containers included for comparison
  only. C/N indicates best/worst case times

22
Other C containers

• string
• similar to vectorltchargt, but includes
  string-specific functions
• valarray
• similar to vectorltTgt, where T is numeric, but
  optimized for numeric ops
• bitset
• similar to vectorltboolgt, but fixed size and
  includes bit operators

23
Iterators

• Typical iteration
• cltTgtiterator i
• for (i c.begin() i ! c.end() i) // forward
• T t i
• for (i c.rbegin() i ! c.rend() i) //
  backward
• T t i

24
Algorithms

• Approx. 60 standard algorithms
• searching (e.g. find())
• sorting (e.g. sort())
• mutating (e.g. transform())
• numerical (e.g. accumulate())
• Most functions take the form
• fn(c.begin(), c.end(), ...)

25
Algorithms (cont.)

• Examples
• include ltalgorithmgt
• // return num elements equal to 123
• int i count(c.begin(), c.end(), 123)
• // negate all elements
• transform(c.begin(), c.end(), negateltintgt())
• // print all elements
• for_each(c.begin(), c.end(), printltintgt())
• // shuffle the deck
• random_shuffle(deck.begin(), deck.end())

26
Function Objects (Functors)

• C objects that can be called like a function to
  implement callbacks
• Use C operator()(...)
• Simplest type is a function pointer
• bool StlStrComp(const char a, const char b)
• return (strcmp(a, b) -1)
• vectorltchargt v
• sort(v.begin(), v.end(), StlStrComp)

27
Functors (cont.)

• Functors that do ordering are called predicates
• struct StlStrPred // public class
• bool operator()(const char a, const char b)
• return (strcmp(a, b) -1)
• vectorltchargt v
• sort(v.begin(), v.end(), StlStrPred())

28
Efficiency

• Designed to be as fast as hand-coded routines
• Limiting factor is typically copy ctor and
  assignment operator

29
Efficiency (cont.)

• STL faster in some cases than standard C
  functions
• const char WestWallaby Gromit
• strchr(WestWallaby, m)
• find(WestWallaby, WestWallaby6, m)

30
Efficiency (cont.)

• Sorting (ints)
• int arrnElements
• qsort(arr, nElements, sizeof(int), IntComp)
• int arrnElements
• sort(arr, arr nElements) // STL int array sort
• vectorltintgt v
• sort(v.begin(), v.end()) // STL int vector sort

31
Efficiency (cont.)

• Sorting (strings)
• char arrnElements
• qsort(arr, nElements, sizeof(char), StrComp)
• sort(arr, arr nElements, StlStrComp)
• sort(v.begin(), v.end(), StlStrComp) // char
• sort(v.begin(), v.end(), StlStrComp) // string

32
STL Allocators

• Every STL container takes an allocator object as
  a template parameter
• template ltclass Tgt public AllocSpecialCheese
• public
• pointer allocate(size_type, const void)
• void deallocate(void, size_type)
• // ... other boilerplate code here
• setltintgt Camembert // default allocator
• // All Wensleydale allocations use special
  allocator
• setltint, AllocSpecialCheesegt Wensleydale

33
Template Partial Specialization

• // generic template function for swapping objects
• template ltclass Tgt void swap(T x, T y)
• T z(x) x y y z
• swap(v1, v2) // swapping vectors slow!
• v1.swap(v2) // swapping vectors fast!
• // template partial specialization
• template ltclass Tgt void swap(vectorltTgt x,
• vectorltTgt y)
• x.swap(y)
• swap(v1, v2) // fast!

34
Template Specialization part II

• // STL generic copy() algorithm
• templateltclass InItr, class OutItrgt OutItr
• copy(InItr bg, InItr end, OutItr val)
• for ( bg ! end val, bg)
• val bg
• return (val)
• // A fast version for simple memory chunks
• templateltgt char copy(const char bg,
• const char end, char val)
• size_t n end - bg
• memcpy(val, bg, n)
• return (val n)

35
Thread Safety

• Official answer STL has no thread safety
  obligations whatsoever
• One (bad) answer have STL handle all
  synchronization issues
• Typical answer make STL thread safe internally,
  but require users to insure no thread accesses a
  container when another thread modifies the
  container

36
Thread Safety (cont.)

• Current status of implementations
• Original HP version not thread safe
• SGI thread safe
• VC mostly thread safe (dinkumware.com/vc_fixes.ht
  ml)
• Typical STL implementation promises
• Multiple reads is thread safe
• Read/write across different containers, same
  objects, is thread safe
• Writes to a container by one thread and
  read/writes by another thread is not thread safe
  users must prevent

37
Exception Safety

• C standard requires the following
• destructors may not throw excptns
• valid iterator operations may not throw
• containers must survive excptns content
  unspecified, but still destructable
• an excptn thrown while inserting one element
  leaves the container unchanged
• an excptn thrown while inserting two elements
  leaves a list unchanged

38
Code Bloat

• Templates expand into different sets of code for
  each type T
• If different types have the same size and same
  comparison functions, the compiler can optimize
• Some STL implementations are optimized to
  minimize code bloat (XTL from DeltaLogic)

39
Compiler Warnings Errors

• VC warning C4786 identifier truncation
• pragma warning(disable 4786) // before headers!
• Errors/warnings in header files
• really means your code has a problem
• Interpreting gonzo error messages
• mapltstring, intgt m
• mapltstring, intgtconst_iterator i m.find(abc)
• m.erase(i) // big error here

40
Compiler Errors (cont.)

• error C2664 'class std_Treeltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt,struct
  stdpairltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt const ,intgt,struct stdmapltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt,int,struct
  stdlessltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt gt,class stdallocatorltintgt gt_Kfn,struct
  stdlessltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt gt,class stdallocatorltintgt gtiterator
  __thiscall stdmapltclass stdbasic_stringltchar,s
  truct stdchar_traitsltchargt,class
  stdallocatorltchargt gt,int,struct stdlessltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt gt,class
  stdallocatorltintgt gterase(class
  std_Treeltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt,struct stdpairltclass stdbasic_stringltchar,st
  ruct stdchar_traitsltchargt,class
  stdallocatorltchargt gt const ,intgt,struct
  stdmapltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt,int,struct stdlessltclass stdbasic_stringltcha
  r,struct stdchar_traitsltchargt,class
  stdallocatorltchargt gt gt,class stdallocatorltintgt
  gt_Kfn,struct stdlessltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt gt,class
  stdallocatorltintgt gtiterator)' cannot
  convert parameter 1 from 'class std_Treeltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt,struct
  stdpairltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt const ,intgt,struct stdmapltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt,int,struct
  stdlessltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt gt,class stdallocatorltintgt gt_Kfn,struct
  stdlessltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt gt,class stdallocatorltintgt gtconst_iterator'
  to 'class std_Treeltclass stdbasic_stringltchar,
  struct stdchar_traitsltchargt,class
  stdallocatorltchargt gt,struct stdpairltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt const
  ,intgt,struct stdmapltclass stdbasic_stringltchar
  ,struct stdchar_traitsltchargt,class
  stdallocatorltchargt gt,int,struct stdlessltclass
  stdbasic_stringltchar,struct stdchar_traitsltcha
  rgt,class stdallocatorltchargt gt gt,class
  stdallocatorltintgt gt_Kfn,struct
  stdlessltclass stdbasic_stringltchar,struct
  stdchar_traitsltchargt,class stdallocatorltchargt
  gt gt,class stdallocatorltintgt gtiterator'
• No constructor could take the source
  type, or constructor overload resolution was
  ambiguous

41
Extending the STL

• Designed for extension
• Not difficult to write new algorithms,
  containers, or iterators
• SGI implementation has many useful container
  extensions (hash tables, slist)

42
Our own Algorithm

• // Compute sum of all squares in range
• templateltclass InItr, class Tgt T
• sumsqrs(InItr bg, InItr end, T init)
• T ss(init)
• for ( bg ! end bg) ss (bg) (bg)
  
• return (ss)
• int CloseShave4 1, 2, 3, 4
• int x sumsqrs(CloseShave, CloseShave4, 0) //
  30
• dequeltdoublegt WrongTrousers // 1.1, 2.2, 3.3
• double y sumsqrs(WrongTrousers.begin(), //
  16.94
• WrongTrousers.end(), 0.0)

43
Our own Iterator

• template ltclass C, class T, class A
  allocatorltTgt gt
• struct MfcIt public _RanitltT,
  Adifference_typegt
• C mpC // MFC container
• int mI // index into container
• MfcIt(C pC 0) mpC(pC), mI(0)
• MfcIt(C pC, int n) mpC(pC), mI(n)
• MfcIt begin() return (MfcIt(mpC, 0))
• MfcIt end() return (MfcIt(mpC,
  mpC-gtGetSize()))
• T operator () const return
  ((mpC)mI)
• MfcIt operator () if (mI lt
  mpC-gtGetSize()) mI else mI 0
• return (this)
• MfcIt operator (difference_type n) const
  MfcIt tmp this
• 
  return (tmp n)
• bool operator (const MfcIt i) const
  return ((mI i.mI)
• 
  (mpC i.mpC))
• bool operator lt (const MfcIt i) const
  return ((mI lt i.mI) (mpC i.mpC))

44
Our own Iterator (cont.)

• Example of using the MFC/STL iterator
• // MFC container
• CStringArray arr
• arr.Add("xyz")
• arr.Add("abc")
• // Create STL iterator from MFC container
• MfcItltCStringArray, CStringgt mfc(arr)
• // Sort the MFC container using an STL algorithm!
• sort(mfc.begin(), mfc.end())

45
Common Mistakes

• Template of templates
• stackltvectorltintgtgt GoneWrong // need space gt
  gt
• Same algorithm, different container
• sort(Marmalade.begin(), Jelly.end()) // crash!
• Right iterator, wrong container
• listltintgtiterator i Feathers.begin()
• McGraw.erase(i) // i doesnt point into McGraw!

46
Common Mistakes (cont.)

• Invalid iterator
• vectorltintgtiterator i GrandDayOut.begin()
• GrandDayOut.push_back(1) // potential realloc
• TheCooker(i) // uh-oh! i might be invalid
• Adding elements with subscript op
• vectorltchargt Wendolene
• Wendolene0 W // crash!
• Wendolene.push_back(W) // the right way

47
Common Mistakes (cont.)

• Sorting problems
• e.g. lookups fail, a set gets duplicates
• Usually a problem with op lt or op
• Rules
• if xlty is true, ygtx is always false
• if xlty yltz are true, xltz is always false
• if xy, then xlty is always false
• if !(xlty) and !(yltx), x y

48
Hiding the Angle Brackets

• Not pretty
• // This is hard to read
• mapltstring, intgt Shaun
• Shaun.insert(pairltstring, intgt(abc, 31))
• mapltstring, intgtiterator i Shaun.find(abc)
• pairltstring, intgt pr i
• pr.first // abc
• pr.second // int

49
Hiding the Angle Brackets (cont.)

• Typedefs are your friend
• // Tuck these away in a header file
• typedef mapltstring, intgt ShaunMap
• typedef pairltstring, intgt ShaunPair
• typedef ShaunMapiterator ShaunItr
• // Same code, but no more angle brackets
• ShaunMap Shaun
• Shaun.insert(ShaunPair(abc, 31))
• ShaunItr i Shaun.find(abc)
• ShaunPair pr i

50
Storing Pointers in Containers

• Container will not delete the pointers when the
  container goes away
• Will sort on pointer, not what it contains,
  unless you tell it otherwise
• dequeltintgt Walkies // sorted by pointer
• sort(Walkies.begin(), Walkies.end())
• bool intpLess(const int a, const int j)
• return (a lt b) // sorted by int
• sort(Walkies.begin(), Walkies.end(), intpLess)

51
Vector Tips

• Use reserve() to set aside space when vector size
  is known in advance
• Can I safely take the address of a vector
  element, e.g. v21?
• According to Standard, no. According to
  practice, yes. Standard expected to adjust.
• Trimming unused space
• v.swap(vectorltTgt(v)) // vector, swap thyself!

52
Checking for Empty Container

• Write if (c.empty()) instead of if (c.size()
  0)
• empty() is guaranteed constant time
• size() is not guaranteed constant time (it
  usually is, but not always, e.g. lists)

53
Copying Containers

• The wrong way copy() cant add elems
• copy(v.begin(), v.end(), nv.begin()) // uh-oh
• Better and best
• nv.resize(v.size()) // size of nv matches v
• copy(v.begin(), v.end(), nv.begin())
• copy(v.begin(), v.end(), back_inserter(nv))
• copy(m.begin(), m.end(), insert_iteratorltTgt
• / map and set use this method / (nm,
  nm.begin())

54
Algorithms that Remove Elems

• Algorithms by themselves cant insert or delete
  elements, so they move them!
• unique() moves unique elems to the front and
  returns an iter to the new end of the container
• remove() similarly moves the unremoved elems to
  the front and returns an iter to the new end

55
Removing Elems (cont.)

• To actually get rid of the extra elems, you must
  call erase()
• // Removes duplicates and shrinks the container
• v.erase(unique(v.begin(), v.end()), v.end())
• // Removes the given elements and shrinks v
• v.erase(remove(v.begin(), v.end(), 1), v.end())

56
Iterator Troubleshooting

• Iter dereferenceable? (end() is not)
• Container invalidated the iter?
• Pair of iters a valid range? First really before
  last?
• Iter pointing to the right container?
• Pair of iters pointing to the same container?

57
Vectors vs C-style arrays

• Prefer vector or deque over arrays
• Dont have to know size in advance
• Dont have to keep track of size separately
• Little loss of efficiency
• Vector works well with legacy code that expect
  arrays

58
Deque vs Vector

• Almost identical usability characteristics
• Faster middle insertion because deque only needs
  to shift a chunk
• Constant-time insertion at the front
• Uses memory in a more operating system-friendly
  way
• Dont need to worry about reserve()

59
Vectorltboolgt

• Standard says vectorltboolgt stores boolean values
  as bits for space optimization
• Technically not a container cant be used
  reliably with generic algorithms
• Prefer dequeltboolgt or vectorltchargt unless space
  benefits outweigh

60
The Key Requirement

• Once a key has been inserted into a container,
  the key should not change
• Cant be enforced internally by the STL
• Example key is a filename comparison is based
  on file contents

61
Copy() vs Memcpy()

• copy() works reliably with all objects
• memcpy() unsafe on any object with a copy ctor
• copy() can copy a sequence whose length is not
  known in advance
• copy() returns an iter to the end of the copy
  can be used as concatenation tool
• good implementation will optimize

62
Wrap Up

• Algorithms, Containers, Iterators
• Efficient, Flexible, Extensible
• Angle Brackets... mmm

63
References

• Email me PKIsensee_at_msn.com
• Home page www.tantalon.com
• Books Generic Programming the STL (Austern),
  STL Tutorial Reference Guid (Musser)
• Websites sgi.com/Technology/STL, dinkumware.com,
  deltalogic.com, roguewave.com, stlport.org