Advanced Programming Techniques

1
Advanced Programming Techniques

• Lecture 3 I/O, Serialization

2
Java classes for IO

• Includes file IO, memory IO, networking IO,
  inter-process IO, and others
• Important for distributed systems
• All the classes are in package java.io
• Excellent example of OO design
• Very general and scalable
• Unfortunately, also obfuscates simple tasks

3
Stream

• What is a stream?
• Definition 1 Ordered sequence of information
  coming from a source, or going to a sink
• Definition 2 A sequence of bytes, generally
  external to the program
• Definition 3 Communication middleware between
  source and target
• Where do streams come from?
• The console System.in, System.out, System.err
• Files
• Network Connections
• Java programs byte array streams, piped streams,
  etc.
• Streams are designed to be platform-independent

4
IO Streams Basic Division
Java I/O is divided up based on directional flow

• Input Streams
• Through inheritance, all derivatives of
  InputStream class have the basic method read()
  to read a single byte or an array of bytes

• Output Streams
• Through inheritance, all derivatives of
  OutputStream class have the basic method
  write() to write a single byte or an array of
  bytes

Conceptually, the two are separate
5
IO Streams Basic Division
Java I/O is divided up based on the type of the
data

• Byte Streams
• Basic classes
• java.io.InputStream
• java.io.OutputStream
• Abstract classes

• Character Streams
• Basic classes
• java.io.Reader
• java.io.Writer
• Abstract classes

In Java, char!byte, thus, the two are separate
6
Java IO java.io Package

• The java.io package contains all of the IO
  classes.
• Many classes specialised for particular kinds of
  stream operations, e.g. file IO
• Reading/writing single bytes is quite limited
• java.io includes classes which provide extra
  functionality
• e.g. buffering, reading numbers and Strings (not
  only bytes, redirecting streams), etc.
• Results in large inheritance hierarchy, with
  separate trees for input/output stream and
  byte/character classes

7
Java IO InputStreams hierarchy
8
The Console

• The default destination for output written to
  System.out or System.err, and the default source
  of input for System.in.
• Converts written bytes to characters

9
Writing bytes to the console

• import java.io.
• public class WriteHello
• public static void main(String args) throws
  IOException
• byte hello 72, 101, 108, 108, 111, 32,
  87, 111, 114, 108, 100, 33, 10, 13
• System.out.write(hello)

10
Java IO Exceptions Handling

• We piped-up the exceptions handling in the
  previous example
• Almost all methods on the I/O classes (including
  constructors) can throw java.IOException or one
  of its subclasses
• Always wrap I/O code in trycatch blocks to
  handle errors

11
Writing bytes to the console

• import java.io.
• public class WriteHello
• public static void main(String args)
• try
• byte hello 72, 101, 108, 108, 111, 32,
  87, 111, 114, 108, 100, 33, 10, 13
• System.out.write(hello)
• catch (IOException e)//or simply (Exception e)
• e.printStackTrace()

12
The OutputStream class

• OutputStream is an abstract class.
• public abstract class OutputStream extends Object
• public abstract void write(int b) throws
  IOException
• public void write(byte data) throws IOException
  
• public void write(byte data, int offset, int
  length) throws IOException
• public void flush() throws IOException
• public void close() throws IOException
• The write() methods send raw bytes of data to
  whomever is listening to this stream.

13
Writing Bytes to Output Streams

• import java.io.
• public class HelloOutputStream
• public static void main(String args)
• String s "Hello World\n"
• // Convert s to a byte array
• byte b new bytes.length()
• s.getBytes(0, s.length()-1, b, 0)
• try
• System.out.write(b)
• catch (IOException e)
• System.err.println(e)

14
Flushing Output Streams

• Sometimes output streams are buffered by the
  operating system for performance.
• For non-buffered streams, flush() is not
  compulsory
• The flush() method forces the data to be written
  whether or not the buffer is full.
• Guarantees that the data reached the target side
• Calling flush() should empty both buffers.

15
Writing Bytes with Flushing

• import java.io.
• public class HelloOutputStream
• public static void main(String args)
• String s "Hello World\n"
• // Convert s to a byte array
• byte b new bytes.length()
• s.getBytes(0, s.length()-1, b, 0)
• try
• System.out.write(b)
• System.out.flush()
• catch (IOException e)
• System.err.println(e)

16
Closing Output Streams

• The close() method
• closes the stream,
• releases any resources associated with the
  stream.
• Once the stream is closed attempts to write to it
  will throw IOException.

17
The InputStream Class

• java.io.InputStream is an abstract class
• public abstract class InputStream extends Object
• public abstract int read() throws IOException
• public int read(byte data) throws IOException
• public int read(byte data, int offset, int
  length) throws IOException
• public long skip(long n) throws IOException
• public int available() throws IOException
• public void close() throws IOException
• public synchronized void mark(int readlimit)
• public synchronized void reset() throws
  IOException
• public boolean markSupported()

18
The read() method

• public abstract int read() throws IOException
• Reads a single unsigned byte of data
• Returns an int value of between 0 and 255.
• Returns -1 on end of stream
• May block

19
Example of the read() method

• import java.io.
• public class Echo
• public static void main(String args)
• echo(System.in)
• public static void echo(InputStream in)
• try
• while (true)
• int i in.read()
• if (i -1) break
• char c (char) i
• System.out.print(c)
• catch (IOException e)
• System.err.println(e)

20
Reading several bytes at once

• It's more efficient to read multiple bytes at a
  time public int read(byte data) throws
  IOException
• public int read(byte data, int offset, int
  length) throws IOException
• These methods block until there is some data
  available.
• They read as many bytes as they can into b, or
  until they've read length bytes.
• Each returns the number of bytes actually read or
  -1 on end of stream.

21
The FileOutputStream class

• public FileOutputStream(String name) throws
  IOException
• public FileOutputStream(String name, boolean
  append) throws IOException
• public FileOutputStream(File file) throws
  IOException
• public FileOutputStream(FileDescriptor fdObj)
• public native void write(int b) throws
  IOException
• public void write(byte data) throws IOException
• public void write(byte data, int offset, int
  length) throws IOException
• public native void close() throws IOException
• public final FileDescriptor getFD() throws
  IOException

22
Writing to Files - An Example

• import java.io.
• //Creates a file storing results of random tosses
  of a dice.
• class GenerateDiceData
• static final String FILENAME dice.dat
• static final int NUMBER_OF_TOSSES 100000
• public static void main(String args)
• try
• OutputStream output
• new FileOutputStream(FILENAME)
• for (long i0 iltNUMBER_OF_TOSSES i)
• int result (int)(Math.random()6)1
• output.write(result)
• output.close()
• catch (IOException ioe)
• System.err.println(Couldnt write to
  file)

23
The FileInputStream class

• public FileInputStream(String filename) throws
  FileNotFoundException
• public FileInputStream(File file) throws
  FileNotFoundException
• public FileInputStream(FileDescriptor fdObj)
• public native int read() throws IOException
• public int read(byte data) throws IOException
• public int read(byte data, int offset, int
  length) throws IOException
• public native long skip(long n) throws
  IOException
• public native int available() throws IOException
• public native void close() throws IOException
• public final FileDescriptor getFD() throws
  IOException

24
Reading from Files - An Example

• class CountOccurrences
• static final String FILENAME dice.dat
• static final int LOOK_FOR 6
• public static void main(String args)
• long count 0
• try
• InputStream input new
  FileInputStream(FILENAME)
• int result
• while ((result input.read()) ! -1)
• if (result LOOK_FOR)
• count
• input.close()
• System.out.println(count
  occurrences)
• catch (IOException ioe)
• System.err.println(Couldnt read from
  file)

25
Data Streams

• DataInputStream and DataOutputStream read and
  write primitive Java data types and Strings in a
  machine-independent way.
• IEEE 754 for floating point data
• Big-endian format for integer data. The most
  significant byte has the lowest address.
• Modified UTF-8 for Unicode data

26
DataOutputStream

• public DataOutputStream(OutputStream out)
• public synchronized void write(int b) throws
  IOException
• public synchronized void write(byte data, int
  offset, int length) throws IOException
• public final void writeBoolean(boolean b) throws
  IOException
• public final void writeByte(int b) throws
  IOException
• public final void writeShort(int s) throws
  IOException
• public final void writeChar(int c) throws
  IOException
• public final void writeInt(int i) throws
  IOException
• public final void writeFloat(float f) throws
  IOException
• public final void writeDouble(double d) throws
  IOException
• public final void writeBytes(String s) throws
  IOException
• public final void writeChars(String s) throws
  IOException
• public final void writeUTF(String s) throws
  IOException
• public void flush() throws IOException

27
DataInputStream

• public DataInputStream(InputStream in)
• public final int read(byte input) throws
  IOException
• public final int read(byte input, int offset,
  int length) throws IOException
• public final void readFully(byte input) throws
  IOException
• public final int skipBytes(int n) throws
  IOException
• public final boolean readBoolean() throws
  IOException
• public final byte readByte() throws IOException
• public final int readUnsignedByte() throws
  IOException
• public final short readShort() throws
  IOException
• public final int readUnsignedShort() throws
  IOException
• public final char readChar() throws IOException
• public final int readInt() throws IOException
• public final long readLong() throws IOException
• public final float readFloat() throws
  IOException
• public final double readDouble() throws
  IOException
• public final String readUTF() throws IOException
• public static final String readUTF(DataInput in)
  throws IOException

28
Decorator Classes

• Java uses different types of classes for
  different IO operations Filtering, Buffered IO,
  Console Streams, GZIP Streams
• Decorator objects are used to provide layers of
  functionality to IO classes
• A decorator wraps inner objects, all using the
  same interface.
• Pros/Cons Flexibility vs. the cost of complexity

29
Purpose of Decorator

• There are two issues when constructing IO library
• Where the IO is going (file, etc).
• How the data is represented (String, native type,
  etc.)
• Rather than create a class for each combination,
  decorator classes allow to mix and match, augment
  functionality of base classes.
• This is a bit confusing but allows high
  flexibility.

30
Streams Piping

• Streams are connected to another underlying
  streams (source streams, supplying the actual
  data) by passing the source stream to the stream
  constructor.
• FileInputStream fis new FileInputStream("ff.txt"
  )
• DataInputStream dis new DataInputStream(fis)
• It's not uncommon to combine these into one line,
  like this
• DataInputStream dos new DataInputStream(new
  FileInputStream(ff.txt))

31
Streams Piping
FileInputStream
ff.txt
DataInputStream
32
Example of DataOutputStream
import java.io. public class DataOutputEx
public static void main(String args) throws
Exception String file args0
double data 1.1,1.2,1.3,1.4,1.5
DataOutputStream dout new DataOutputStream
(new FileOutputStream(file))
for (int i 0 i lt data.length i) try
dout.writeDouble(datai) catch(IOExc
eption e) e.printStackTrace()
dout.close()
33
Example of DataInputStream
import java.io. public class DataInputEx
public static void main(String args) throws
Exception String file args0
DataInputStream din new DataInputStream(new
FileInputStream(file)) double data
try while (true)
data din.readDouble()
System.out.println(data)
catch (Exception e)e.printStackTRace()
din.close()
34
Unicode

• Each character in the ASCII character set fits
  into a single byte
• It is not enough for the wide variety of
  alphabets
• Need more than a single byte
• A Java character (char) is 2 bytes
• Java handles text using Unicode
• International standard character set, containing
  characters for almost all known languages
• And a few imaginary ones (Klingon, Elvish)
• Inside the JVM all text is held as Unicode

35
Java Text IO

• Because byte!char, using an Input/OutputStream
  requires converting bytes into chars
• Java provides java.io.Reader and java.io.Writer
  abstract classes performing this task
• public abstract class Reader extends Object
• public abstract class Writer extends Object
• These classes deal with streams of characters
• Read/write single character or array of
  characters
• As in case of byte streams, there are special
  sub-classes for particular purposes

36
The Writer class

• The methods of the java.io.Writer work with chars
• The basic write() method writes a single two-byte
  character with a value between 0 and 65535.
• public void write(int c) throws IOException
• public void write(char text) throws IOException
• public abstract void write(char text, int
  offset, int length) throws IOException
• public void write(String s) throws IOException
• public void write(String s, int offset, int
  length) throws IOException
• Like output streams, writers may be buffered. To
  force the write to take place, call flush()
• public abstract void flush() throws IOException

37
The Reader class

• The basic read() method reads a single character
  (may take between one and four bytes depending on
  the character set) and returns the character as
  integer between 0 and 65535.
• Returns -1 if the end of stream is seen.
• public int read() throws IOException
• public int read(char text) throws IOException
• public abstract int read(char text, int offset,
  int length) throws IOException
• All the read() methods block until some input is
  available.
• IO error occurs, or the end of the stream is
  reached.

38
FileReader FileWriter

• import java.io.
• // This class reads a text file and writes it
  into another
• // text file after converting all letters to
  uppercase.
• class ToUpper
• public static void main(String args)
• if (args.length!2)
• System.err.println(Invalid usage)
• String sourceName args0, targetName
  args1
• try
• Reader reader new FileReader(sourceName
  )
• Writer writer new FileWriter(targetNam
  e)
• int c
• while ((creader.read())!-1)
• c Character.toUpperCase((char)c)
• writer.write(c)
• catch (IOException ioe) e.printStackTrace()
  

39
PrintWriter

• A subclass of java.io.Writer allows you to use
  the familiar print() and println() methods
• Can be chained to other streams, usually to
  OutputStreamWriter.
• Automatic flushing is performed only when
  println() is invoked, not every time a newline
  character is seen.

40
Bridging the Gap

• Sometimes you need to bridge across the two
  hierarchies of bytes and characters
• Use InputStreamReader or OutputStreamWriter
• InputStreamReader
• Reads bytes from an InputStream, and turns them
  into characters using a character encoding
• OutputStreamWriter
• Turns characters sent to the Writer into bytes
  written by the OutputStream, again using a
  character encoding.

41
Buffered Streams

• IO operations are usually very slow
• More efficient IO operations needed
• BufferedInputStream and BufferedOutputStream
  classes perform bulk read and write by first
  storing the data in a buffer (internal array).
• The program reads the data from the array without
  calling the IO method until the buffer is empty.
• public BufferedInputStream(InputStream in)
• public BufferedInputStream(InputStream in, int
  size)
• public BufferedOutputStream(OutputStream out)
• public BufferedOutputStream(OutputStream out, int
  size)

42
Example Reading from Keyboard

• InputStreamReader wrapped with a decorator of
  BufferedReader
• Note strings are read, conversions are needed!
• InputStreamReader isr new InputStreamReader(Syst
  em.in)
• BufferedReader br new BufferedReader(isr)
• String str
• try
• str br.readLine()
• catch (Exception e)
• e.printStackTrace()

43
BufferedInputStream Example
import java.io. public class DataInputEx
public static void main(String args) throws
Exception String file args0
DataInputStream din new DataInputStream (new
BufferedInputStream (new FileInputStream(file)))
double data try
while (true) data
din.readDouble()
System.out.println(data)
catch (Exception e)e.printStackTrace()
din.close()
44
BufferedOutputStream example
import java.io. public class DataOutputEx
public static void main(String args) throws
Exception String file args0
double data 1.1,1.2,1.3,1.4,1.5
DataOutputStream dout new DataOutputStream (new
BufferedOutputStream(new FileOutputStream(file)))
for (int i 0 i lt data.length
i) try dout.writeDouble(dat
ai) catch (Exception e)e.printStackTrace()
dout.close()
45
Concurrency Issues

• Streams are not Thread Safe
• For example
• Do not use the same stream concurrently by
  multiple threads.
• Do not access the same file from 2 threads
• Sometimes (e.g., concurrent writing from
  different threads to the same file) it is blocked
  by the OS
• Use your own synchronization!

46
Security Issues

• IO is extremely limited in applets
• An applet cannot read a file.
• An applet cannot write a file.
• An applet cannot delete a file.
• An applet cannot make a network connection to
  most hosts.
• An applet cannot accept an incoming connection
  from an arbitrary host (except the host it is
  running from).

47
Serialization

• Storing data in text files is often sufficient
• But, sometimes we want to store objects as they
  are without converting to/from text
• Answer Serialization
• Convert a complex data object into serial form
  i.e., into a single stream of bytes
• Often called lightweight persistence

48
Serialization

• The Serializable interface provides this
  capabilities for lightweight persistence.
• The Serializable interface is lightweight because
  it is invoked manually to create and load
  persistent objects.
• How to use serialization?
• Make your data objects implement Serializable
• Use ObjectInputStream and ObjectOutputStream

49
Serialization

• Serializable is a special interface, no methods!
• It just tells Java that the class can be
  serialized
• All data contained in the class must also be
  Serializable
• e.g., if Car implements Serializable and Car
  contains fields of type Wheel, then Wheel must
  also be Serializable
• The compiler will NOT complain, but an exception
  will be thrown at runtime
• Built-in Java classes are usually Serializable
  already

50
To serialize or not to serialize?

• In some case we would like to serialize an
  object, but to omit certain features, e.g.,
  passwords or some other sensitive data.
• How does one keep data out of serialized objects?
• Solution use the transient keyword
• private transient String password pssst
• Data and methods identified as transient are not
  serialized.

51
Serialization - Summary

• When an object is written into ObjectOutputStream
  all his data members are written including
  internal objects which are referenced from the
  object
• Unless
• reference/data is marked as transient
• referenced objects do not implement Serializable
  interface (then NonSerializalizableException is
  thrown)
• reference/data member is static

52
ObjectOutputStream

• public void writeBoolean(boolean data) throws
  IOException
• public void writeByte(int data) throws
  IOException
• public void writeShort(int data) throws
  IOException
• public void writeChar(int data) throws
  IOException
• public void writeInt(long data) throws
  IOException
• public void writeFloat(long float) throws
  IOException
• public void writeDouble(double data) throws
  IOException
• public void writeBytes(String data) throws
  IOException
• public void writeChars(String data) throws
  IOException
• public final void writeObject(Object obj) throws
  IOException
• public void flush() throws IOException
• public void close() throws IOException

53
ObjectInputStream

• public boolean readBoolean() throws IOException
• public byte readByte() throws IOException
• public short readShort() throws IOException
• public int readUnsignedShort() throws IOException
• public int readInt() throws IOException
• public long readLong() throws IOException
• public float readFloat() throws IOException
• public double readDouble() throws IOException
• public final Object readObject() throws
  OptionalDataException, ClassNotFoundException,
  IOException

54
Reading Objects

• In order to read an object from ObjectInputStream
  it should implement java.io.Serializable
  interface
• readObject() returns reference to Object, thus
  casting is needed
• Do not forget checking type of object at the
  runtime using instanceOf reserved word.

55
Example

• public class SerializationDemo
• public static void main(String args) throws
  IOException
• Person p1new Person() Person p2new
  Person(222,"George W.")
• System.out.println(p1) System.out.println(p
  2)
• try
• FileOutputStream myFile new
  FileOutputStream("person.txt")
• ObjectOutputStream oos new ObjectOutputStream(
  myFile)
• oos.writeObject(p1)
• oos.writeObject(p2)
• oos.close()
• catch(Exception e) e.printStackTrace()
  
• Person p3 null Person p4 nulltry
• FileInputStream myFile new
  FileInputStream("person.txt")
• ObjectInputStream ois new
  ObjectInputStream(myFile)
• p3 (Person)ois.readObject()
• p4 (Person)ois.readObject()
  ois.close()
• catch (Exception error) error.printStackTr
  ace()

Try to add transient key word to one of the
fields, and re-run the application
56
ObjectOutputStream caveat

• If you (1) write a variable x, (2) change some
  field of x, and (3)write x again, then, Java
  writes the old value of the field
• This happens because Java remembers what it
  already wrote, so it just gives a pointer to the
  old serialized data rather than writing the data
  all over again
• This is how Java serializes objects pointing to
  each other.

e.g., Student x new Student() x.setName(
Alice ) oos.writeObject( x ) x.setName(
Bob ) oos.writeObject( x ) Later, when we
read the data that was written, we will get Alice
twice.
57
Solution Use reset()

• Call reset() on the ObjectOutputStream before
  calling writeObject()
• reset() causes the ObjectOutputStream to lose
  track of what it has written already so it writes
  the whole object again

e.g., Student x new Student() x.setName(
Alice ) oos.writeObject( x ) x.setName(
Bob ) oos.reset() oos.writeObject( x )
58
Class File

• Java provides access to the file system through
  the java.io.File object
• File objects does not necessarily refer to a
  single file. They can represent the name of a
  single file, or the names of a set of files.
• boolean isDirectory()
• String list()
• Information about the file can be obtained using
  File reference.

59
Using File to List A Directory

• public class Lister
• public static void main (String arg)
• try
• File path new File (arg0)
• String listing path.list()
• for(int i0 iltlisting.length i)
• System.out.println(listingi)
• catch(Exception ex) ex.printStackTrace()

60
Class File

• Well-featured, and intuitive API
• Has methods for OS-level operations with files
  and directories
• Making directories, listing directory contents
• Renaming and deleting, checking permissions, etc
• Information about the file can be obtained using
  File reference
• public String getName()
• public String getParent()
• public String getAbsolutePath()
• public boolean exists()
• public long lastModified()

61
Getting File Data

• public class FileDemo
• public static void main(String args) throws
  IOException
• File myFilemyFile new File("c\\autoexec.bat"
  )System.out.println("file name "
  myFile.getName())System.out.println("file path
  " myFile.getPath())System.out.println("file
  Absolute Path " myFile.getAbsolutePath())
  if (myFile.isAbsolute()) System.out.println("th
  e file is Absolute")else System.out.println("th
  e file isn't Absolute")