Celluloid

1
Celluloid

• An interactive media sequencing language

2
Today

• Overview of Celluloid
• Sample Code
• Syntax and Semantics
• Translator Architecture
• Framework Architecture
• Development Environment
• Testing
• Conclusion

3
Overview

• Interactive Media sequencing language
• Designed to allow users to easily construct their
  own media systems and allow for real time
  interaction with the programmer to dynamically
  change programs on the fly allowing programming
  to be elevated to a performance art
• Programs are translated into Java SE which can
  then be compiled into Java bytecode and run on
  any JVM

4
Motivation

• Celluloid was motivated by the growing role
  interactive media plays in our daily lives
• Weve focused on trying to build a language that
  targets a wide audience and remains simple and
  intuitive
• With Celluloid, you can build the technical and
  artistic developments that will power the future
  of the media age

5
Important Properties

• Reactive programming allows modified primitives
  to affect later primitives defined with them
• Celluloid is designed to be very human readable
• Provides facilities for abstracting away the
  hardware details of underlying input and output
  devices to increase portability

6
Visualizing a Celluloid System

• Multiple devices are connected and abstracted
  through the device type.
• Enables software mixing that was previously not
  possible.

7
Type System

• Primitive Types number, boolean, string, time.
• Timelines Controls the playback of inputs in the
  order specified in the source program
• Devices Encapsulate IO abstractions. Devices are
  responsible for providing the interface between
  external media and internal components
• Constraints Enforce a set of allowed
  operationson a device as well as share this
  definition across devices.
• Events Allows for simple message passing between
  devices and timelines.

8
Language Properties

• Compiled statically typed language.
• Compiled into Java code built on a custom
  framework.
• Allows for user defined functions and user
  defined types.
• Custom device, constraint, and event types are
  allowed.

9
Sample Program

• timeline audio
• input audio1 new AudioFile(audiofile1.mp3)
• input audio2 new AudioFile(audiofile2.mp3)
• input audio3 new AudioFile(audiofile3.mp3)
• output main new AudioOutput(Audio)
• in audio do
• play audio1 _at_start
• play audio2 _at_5s
• play audio3 _at_10s
• end
• in main do
• play audio _at_start
• end

10
Translator Architecture
source_code.cld
celluloidLexer.g
celluloid.g
target_code.java
celluloidParser.g
org.celluloidLang.
celluloid.java.stg
11
Framework Architecture

• Java Media Framework
• Implementation of Timelines
• Implementation of Devices
• Implementation of Constraints
• Announcements and Events
• Reactive Programming

12
Java Media Framework (JMF)

• Designed to manipulate audio, video and other
  time-based media
• Cross platform support
• Supports many popular formats
• Plugins available to support additional formats
• FOBS for JMF

13
Implementation of Timelines

• Controls the playback of inputs in the order
  specified in the source program
• Timer object keeps track of time and executes the
  appropriate action at the appropriate time
• System utilitzes 100 millisecond time-slices
• HashMap containing when-based events and the
  corresponding action to execute when the events
  condition(s) is satisfied

14
Implementation of Devices

• Two built-in devices for reading audio and video
  files are provided out-of-box JMFVideo and
  JMFAudio
• Framework provides the ability to implement
  real-time and static media
• Examples of real-time media midi devices,
  streaming media, instruments

15
Implementation of Constraints

• Constraint interfaces
• Enforce allowed operations on device classes in
  the framework
• Constraint functions
• Keeps track of what should be executed and when
  it should be executed
• Anonymous inner class keeps track of what is
  executed

16
Sample Constraint Translation
in timelineOne do play inputOne _at_0s end ...
would translate to ...
ConstraintFunction cf new ConstraintFunction
(inputOne, 0) public void execute()
input.play() //where (input inputOne)
timelineOne.addConstraintFunction(cf)
17
Reactive Programming

• Reactive programming involves the propagation of
  change among assignment expressions.
• Example
• A4
• B9
• C A B
• Print C //displays 13
• A10
• Print C //displays 19

18
Announcements and Events

• Announcements and events allow for simple message
  passing between devices and timelines.
• Implemented as a global event broker (i.e. event
  listener) available to relevant types in the
  framework
• Example
• when Audio5 audioGain gt 7 do
• play Video4
• end

19
Development Environment

• IDE Devolopment
• IntelliJ
• Eclipse
• ANTLRWorks
• Unit Testing
• JUnit
• Repository
• Git
• GitHub

20
Testing

• Goal Find faults and determine whether those
  faults are translation faults or target framework
  faults.
• Black Box
• White Box
• Unit Tests
• Full unit test suite for both the parser and the
  framework
• Integration Tests
• Testing hand of off parser to the framework

21
Conclusion

• What we learned
• Learned tools for developing language
• Implement language
• Worked well
• Repository, issue tracking, Google docs
• What to do different
• Start earlier
• Future development
• REPL- Read eval print loop

22
Questions?