The Closures Controversy

1
(No Transcript)
2
The Closures Controversy

• Joshua Bloch
• Chief Java Architect
• Google Inc.

3
Disclaimer Talk Represents My Opinion, Not
Googles!

• Google believes that the Java platform would
  likely benefit from some additional support for
  closures. Like the broader Java community, we are
  divided on what form this support should take.
  Some favor a lightweight approach that addresses
  the pain of anonymous inner classes without
  affecting the type system, VM, or libraries
  others favor a heavyweight approach designed to
  provide for programmer-defined control
  structures. We believe it is premature to launch
  a JSR that forces us down either path.

4
Outline

• I. Setting the Stage
• II. Why Enhance Support for Closures?
• III. BGGA Closures
• IV. A Lightweight Approach
• V. Where Do We Go From here?

5
OOPSLA Invited Talk October 8, 1996
Digitally reconstructed from the Internet
Archive (AKA the Wayback Machine) by Joshua
Bloch November 24, 2007
6
Oak

• Started as a reimplementation of C
• Always a tool, never an end itself
• Took on a life of its own
• The Web happened...
• serendipitous match!
• and it became Java

7
The Java Language

• Fusion of four kinds of programming
• Object Oriented like Simula/C/
  ObjectiveC
• Numeric like FORTRAN
• Systems like C
• Distributed like nothing else

8
Java - a language for a job

• Doing language research
• an anti-goal
• Started using C
• Broke down, needed
• Architecture neutral, portable, reliable, safe,
  long lived, multithreaded, dynamic, simple, ...

9
Practical, not theoretical

• Driven by what people needed
• (but hey, I spent too much time going to school!)
• Theory provides
• rigour
• cleanliness
• cohesiveness

10
No new ideas here

• Shamelessly ripped off ideas that worked in C,
  C, Objective C, Cedar/Mesa, Modula, Simula, ...
• (well, we slipped once or twice and invented
  something)

11
Dont fix it until it chafes

• To keep it simple...
• A procedural principle
• Require several real instances before including a
  feature
• i.e. nothing goes in because its nice
• ( Just Say No, until threatened with bodily
  harm)

12
Java feels...

• Hyped -(
• Playful
• Flexible
• Deterministic
• Non-threatening
• Rich
• Like I can just write code

Hey! I left out Object-Oriented!
13
IEEE Computer, June 1997
14
So How Are We Doing? Not So Well,
Unfortunately?

• EnumltE extends EnumltEgtgt ...
• ltT extends Object Comparablelt? super Tgtgt T
  Collections.max(Collectionlt? extends Tgt)
• public ltV extends Wrapperlt? extends
  ComparableltTgtgtgt
• ComparatorltVgt comparator() ...
• error equalTo(Boxltcapture of ?gt) in Boxltcapture
  of ?gt cannot be applied to (Boxltcapture of ?gt)
• equal unknownBox.equalTo(unknownBox)
• Arrays.asList(String.class, Integer.class) //
  Warning!
• See Angelia Langer's 427-page (!) Java Generics
  FAQ for more
• http//www.angelikalanger.com/GenericsFAQ/JavaGene
  ricsFAQ.pdf

15
What the Man on the Web is Saying

• I am completely and totally humbled. Laid low. I
  realize now that I am simply not smart at all. I
  made the mistake of thinking that I could
  understand generics. I simply cannot. I just
  can't. This is really depressing. It is the first
  time that I've ever not been able to understand
  something related to computers, in any domain,
  anywhere, period.
• I'm the lead architect here, have a PhD in
  physics, and have been working daily in Java for
  10 years and know it pretty well. The other guy
  is a very senior enterprise developer (wrote an
  email system that sends 600 million emails/year
  with almost no maintenance). If we can't get
  generics, it's highly unlikely that the
  average developer will ever in our lifetimes be
  able to figure this stuff out.

16
Where Does The Complexity Come From?
Feature Tuples (exponential)
Feature Pairs (quadratic)
Features (linear)
17
If The Feel of Java is to be Preserved...

• We simply cannot afford another wildcards
• Further language additions must be undertaken
  with extreme caution
• Minimal addition to conceptual surface area
• High power-to-weight ratio

18
Outline

• I. Setting the Stage
• II. Why Enhance Support for Closures?
• II. BGGA Closures
• III. A Lightweight Approach
• IV. Where Do We Go From here?

19
What is a Closure?

• One definition a function that is evaluated in
  an environment containing one or more bound
  variables Wikipedia
• In English a little snippet of code that can be
  passed around for subsequent execution
• Limited support for closures since JDK 1.1, in
  the form of anonymous classes

20
Why Are We Considering Better Support for
Closures?

• Fine-Grained Concurrecy - Passing snippets of
  code to fork-join frameworks using anonymous
  classes is a pain
• Resource Managememnt - Using try-finally blocks
  for resource management is a pain, and causes
  resource leaks

21
1. Fine-grained (fork-join) concurrency

• "It has to be easier to send snippets of code to
  frameworks for parallel execution otherwise no
  one will use them .
• Doug Lea, 2005

22
Heres How it Looks Today

• class StudentStatistics
• ParallelArrayltStudentgt students ...
• // ...
• public double getMaxSeniorGpa()
• return students.withFilter(isSenior).
• withMapping(gpaField).max()
• // helpers
• static final class IsSenior implements
  PredicateltStudentgt
• public boolean evaluate(Student s) return
  s.credits gt 90
• static final IsSenior isSenior new
  IsSenior()
• static final class GpaField implements
  MapperToDoubleltStudentgt
• public double map(Student s) return s.gpa
  
• static final GpaField gpaField new
  GpaField()

23
2. Automatic Resource Management

• The C destructor model is exactly the same as
  the Dispose pattern, except that it is far easier
  to use and a direct language feature and correct
  by default, instead of a coding pattern that is
  off by default and causing correctness or
  performance problems when it is forgotten.
• Herb Sutter, OOPSLA 2004

24
How it Looks TodayManual Resource Management

• static String readFirstLineFromFile(String path)
• throws IOException
• BufferedReader r null
• String s
• try
• r new BufferedReader(new
  FileReader(path))
• s r.readLine()
• finally
• if (r ! null)
• r.close()
• return s

25
Its Worse With Multiple Resources (Puzzler 41)

• static void copy(String src, String dest) throws
  IOException
• InputStream in null
• OutputStream out null
• try
• in new FileInputStream(src)
• out new FileOutputStream(dest)
• byte buf new byte1024
• int n
• while ((n in.read(buf)) gt 0)
• out.write(buf, 0, n)
• finally
• closeIgnoringException(in)
• closeIgnoringException(out)
• private static void closeIgnoringException(Closeab
  le c)
• if (c ! null)
• try c.close() catch (IOException ex)
  // ignore

26
Automatic Resource Management

• C Destructor
• String ReadFirstLineFromFile( String path )
• StreamReader r(path)
• return r.ReadLine()
• C using Block
• String ReadFirstLineFromFile( String path )
• using ( StreamReader r new StreamReader(path)
  )
• return r.ReadLine()

27
Outline

• I. Setting the Stage
• II. Why Enhance Support for Closures?
• III. BGGA Closures
• IV. A Lightweight Approach
• V. Where Do We Go From here?

28
Controversial Features in BGGA

• Function types
• Non-local return
• Non-local break and continue
• Unrestricted access to nonfinal local variables
• Design goal library-defined control constructs

29
Function Types

• The BGGA Spec says While the subtype rules for
  function types may at first glance appear arcane,
  they are defined this way for very good reason
  .... And while the rules seem complex, function
  types do not add complexity to Java's type system
  because function types and their subtype
  relations can be understood as a straightforward
  application of generics and wildcards (existing
  constructs). From the programmer's perspective,
  function types just do the right thing.

30
Function Types are Hard to Read

• static Pairlt gt int , gt int gt
  joinedCounters(int initial)
• return Pair.lt gt int , gt int gtof(
• gt initial , gt initial )
• interface BThunk extends gtboolean
• static final BThunk, gt void gt void wihle
• BThunk cond, gt void action gt
• while (cond.invoke()) action.invoke()
• static ltthrows Xgt gt void throws X gt void
  throws X foo()
• return gt void throws X block gt
  block.invoke()
• These examples come from test code that ships
  with BGGA Prototype

31
Function Types Encourage an Exotic Programming
Style

• static ltA1, A2, Rgt A1 gt A2 gt R curry(A1,
  A2 gt R fn)
• return A1 a1 gt A2 a2 gt fn.invoke(a1, a2)
• ltA1, A2, A3, Rgt A2, A3 gt R partial(A1, A2, A3
  gt R fn, A1 a1)
• static ltA1, A2, A3, Rgt A2, A3 gt R
• partial(A1, A2, A3 gt R fn, A1 a1)
• return A2 a2, A3 a3 gt fn.invoke(a1, a2,
  a3)
• From Mark Mahieu's blog
• Currying and Partial Application with Java
  Closures
• http//markmahieu.blogspot.com/2007/12/currying-an
  d-partial-application-with.html

32
Nominal Types are Rich Compared to Function Types
Name ? Known Implementations ? Documentation,
including semantic constraints
T, T gt T
33
Function Types Have Unexpected Interactions

• Arrays dont work
• Foo.java6 generic array creation
• gt int closures new gt intN
• Autoboxing doesnt work
• LoopBenchC.java10 ltE,XgtforEach(java.util.Collec
  tionltEgt, E gt void throws X) in LoopBenchC
  cannot be applied to (java.util.Listltjava.lang.Int
  egergt,int gt void)
• forEach(list, int i gt
• Wildcards produce difficult error messages
• NewtonWithClosures.java26 invoke(capture418
  of ? super double gt double) in capture418 of
  ? super double gt double gt capture928 of ?
  extends double gt double cannot be applied to
  (double)
• return fixedPoint(transform.invoke(guess))
  

34
Function Types Limit Interoperability With SAM
Types

• Closure conversion only works with interfaces
• Unfortunately, existing APIs sometimes use
  abstract classes for functions
• e.g., TimerTask, SwingWorker
• These APIs would become 2nd class citizens

35
Summary - Pros and Cons of Function Types

• Avoid need to define named interfaces
• Avoid incompatibility among SAM types with same
  signatures
• - Hard to read under moderate-to-heavy use
• - Encourage exotic style of programming
• - Don't reflect semantic constraints
• - Don't provide the same level of documentation
• - Dont interact well with autocompletion (or
  grep)
• - Limited interoperability may balkanize libraries

36
BGGA Closures Have Two Kinds of Returns

• static boolean test(boolean arg)
• boolean gt boolean closure boolean arg
  gt
• if (arg)
• return true // Non-local return
• false // local return
• return !closure.invoke(arg)
• return means something completely different in a
  BGGA closure and an anonymous class

37
What Does test() Return?

• static ltEgt Boolean contains(IterableltEgt seq,
  PredicateltEgt pred)
• for (E e seq)
• if (pred.invoke(e))
• return true
• return false
• static Boolean test()
• ListltCharactergt list Arrays.asList(
• 'h', 'e', 'l', 'l', 'o')
• return contains(list, new PredicateltCharactergt
  ()
• public Boolean invoke(Character c)
• return c gt 'j'
• )
• interface PredicateltTgt Boolean invoke(T t)

38
Now What Does test() Return? (BGGA)

• static ltEgt Boolean contains(IterableltEgt seq, E
  gt Boolean p)
• for (E e seq)
• if (p.invoke(e))
• return true
• return false
• static Boolean test()
• ListltCharactergt list Arrays.asList(
• 'h', 'e', 'l', 'l', 'o')
• return contains(list, Character c gt return
  c gt 'j' )

39
Now What Does test() Return? (BGGA)

• static ltEgt Boolean contains(IterableltEgt seq, E
  gt Boolean p)
• for (E e seq)
• if (p.invoke(e))
• return true
• return false
• static Boolean test()
• ListltCharactergt list Arrays.asList(
• 'h', 'e', 'l', 'l', 'o')
• return contains(list, Character c gt return
  c gt 'j' )
• Accidental non-local return due to cut-and-paste
  from anonymous class can cause insidious bug

40
Suppose You Wanted to Translate this Method to
BGGA

• static ltEgt PredicateltIterableltEgtgt contains(
• final PredicateltEgt pred)
• return new PredicateltIterableltEgtgt()
• public Boolean invoke(IterableltEgt seq)
• for (E e seq)
• if (pred.invoke(e))
• return true
• return false

41
Its Awkward, as Only One Local Return is
Permitted

• static ltEgt IterableltEgt gt Boolean contains(
• E gt Boolean pred)
• return IterableltEgt seq gt
• Boolean result false
• for (E e seq)
• if (pred.invoke(e))
• result true
• break
• result

42
Summary - Pros and Cons of Non-Local Returns

• Permits library-defined control structures
• - Having two kinds of returns is confusing
• - Meaning of return has changed
• - Unintentional non-local returns can cause bugs
• - Only one local return permitted per closure

43
What Does This Program Print?

• public class Test
• private static final int N 10
• public static void main(String args)
• Listlt gt intgt closures new
  ArrayListlt gt intgt()
• for (int i 0 i lt N i)
• closures.add( gt i )
• int total 0
• for ( gt int closure closures)
• total closure.invoke()
• System.out.println(total)

44
What does this program print?

• public class Test
• private static final int N 10
• public static void main(String args)
• Listlt gt intgt closures new
  ArrayListlt gt intgt()
• for (int i 0 i lt N i)
• closures.add( gt i )
• int total 0
• for ( gt int closure closures)
• total closure.invoke()
• System.out.println(total)
• It prints 100, not 45. The same loop variable is
  captured by all 10 closures and evaluated after
  the loop is finished!

45
Summary - Pros and Cons of Access to Nonfinal
Locals

• Permits library-defined control structures
• Eliminates some uses of final modifier
• - Semantics can be very confusing
• - Locals can persist after their scope is
  finished
• - Locals can be modified by other threads
• - Performance model for locals will change

46
Library-Defined Control Constructs

• What are the compelling use-cases?
• Custom loops
• Automatic resource management blocks
• Timer block

47
Custom for-loops (Example from BGGA Spec)

• ltK,V,throws Xgt
• void for eachEntry(MapltK,Vgt map, K,Vgtvoid
  throws X block)
• throws X
• for (Map.EntryltK,Vgt entry map.entrySet())
• block.invoke(entry.getKey(),
  entry.getValue())
• for eachEntry(String name, Integer value
  map)
• if ("end".equals(name)) break
• if (name.startsWith("com.sun."))
  continue
• System.out.println(name "" value)

48
What's Wrong With This example?

• BGGA loop competes with Java 5 for-each
• Don't define a construct just implement Iterable
• Only compelling use is multiple loop variables
• Last example doesn't offer power of for-each
• Loop variable cant be primitive (no
  auto-unboxing)
• Would require 81 (!) overloadings to do fix this

49
Loop syntax tailored to for awkward for while

• public static ltthrows Xgt void for myWhile(
• gt boolean throws X cond, gtvoid throws X
  block) throws X
• while (cond.invoke())
• block.invoke()
• for myWhile( gt i lt 7 )
• System.out.println(i)
• myWhile( gt i lt 7 , gt
• System.out.println(i)
• )

50
Automatic Resource Management Block (BGGA Spec)

• ltR, T extends Closeable, throws Xgt
• R with(T t, TgtR throws E block) throws X
• try
• return block.invoke(t)
• finally
• try t.close() catch (IOException ex)
  
• with (FileReader in makeReader()) //
  Requires nesting
• with (FileWriter out makeWriter())
• // code using in and out

51
Library-Defined Control Constructs are a
Double-Edged Sword

• A great feature of Java is programmer
  portability
• All Java code looks pretty much alike
• We can read and maintain each other's code with a
  minimum of effort
• Library-Defined Control Constructs
  fosterdialects, which hinder programmer
  portability

52
Performance is an Open Question
Time to iterate over 108 elements(Take these
numbers with a huge grain of salt)
for-each BGGA change
List 1.65 s 1.97 s 19
Array 0.18 s 1.11 s 519
Java HotSpot(TM) Server VM (build 1.6.0-b105,
mixed mode) Windows XP, Intel T2600 _at_ 2.16 GHz,
2GB RAM BGGA prototype closures-2007-11-30 (I
apologize)
53
Summary - Programmer-Defined Control Constructs

• They let you define new control constructs
• - They dont have the same syntax, semantics, or
  performance as built-in control constructs
• - Its not clear that you need the ability Java
  already has a rich set of control structures
• - Can lead to dialects
• - Responsible for much of the complexity in BGGA
• - Non-local return, break and continue
• - Unrestricted access to mutable local variables

54
Outline

• I. Setting the Stage
• II. Why Enhance Support for Closures?
• III. BGGA Closures
• IV. A Lightweight Approach
• V. Where Do We Go From here?

55
A Lightweight Approach

• There is a much simpler approach to reducing the
  verbosity of anonymous classes and manual
  resource management
• Attack the two problems head-on!
• Concise syntax for anonymous class instance
  creation
• Purpose-built construct for automatic resource
  management

56
Concise Instance Creation Expressions (CICE)

• This
• sort(list, ComparatorltStringgt(String s1,
  String s2)
• return s1.length() - s2.length()
• )
• Expands to this
• sort(list, new ComparatorltStringgt()
• public int compare(String s1, String s2)
• return s1.length() - s2.length()
• )
• I am not in love with this syntax we can
  probably do better.

57
Automatic Resource Management (ARM) Blocks

• // One resource - readFirstLineFromFile
• try (BufferedReader r new BufferedReader(new
  FileReader(path))
• String s r.readLine()
• // Multiple resources - copyFile
• try (InputStream in new FileInputStream(src)
• OutputStream out new FileOutputStream(dest)
  )
• byte buf new byte1024
• int n
• while ((n in.read(buf)) gt 0)
• out.write(buf, 0, n)

58
More Automatic Resource Management Blocks

• Perhaps this
• protected (lock)
• ... // access the resource protected by
  this lock
• Should be shorthand for this
• lock.lock()
• try
• // access the resource protected by this
  lock
• finally
• lock.unlock()

59
For More Information

• Automatic Resource Management Blocks (ARM)
• http//docs.google.com/View?dociddffxznxr_1nmsqkz
  
• Concise Instance Creation Expressions (CICE)
• http//docs.google.com/View?docidk73_1ggr36h
• Both of these docs are a bit sketchy ?

60
Outline

• I. Setting the Stage
• II. Why Enhance Support for Closures?
• III. BGGA Closures
• IV. Lightweight Closures
• V. Where Do We Go From here?

61
The Closures Spectrum
CICEARM
Bob Lee
62
The Big Questions

• Do we really want function types in Java?
• Do we really want library-defined control
  constructs in Java?
• We should keep in mind that theres already a
  fine programming language for the JVM that offers
  both of these things, and Java interoperability
  to boot Scala

63
The Medium-Sized Questions

• Should we support non-local return, break, and
  continue? If so
• Should it be the default?
• How do we prevent accidents?
• Should we allow access to non-final local
  variables from closures? If so
• Should it be the default?
• How do we prevent accidents?

64
Moving Forward, There are Two Approaches

• Gain more experience with prototypes
• When we come to an informed consensus, start a
  narrowly focused JSR
• This might take a couple of years
• Precedent established by generics
• Start a broadly focused JSR in the near future
• Must permit any point on the Closure Spectrum
• First task is to answer the two Big Questions

65
Closing Sermon

• We have a big decision to make!
• It will have a huge effect on the future of the
  Java platform
• We must take our time and do what is right
• We must not risk further damage tothe feel of
  Java

66
QA

• View JavaPolis talks _at_ www.parleys.com

67
Thank you for your attention