I/O with Java

Interacting with the World

Java I/O

The often not well-understood topic.

  • Two packages available
  • Classic I/O since 1.1: java.io
  • New I/O since 1.4: java.nio
  • NIO meant to be faster and logical including better interaction with real file systems
  • NIO more powerful but also complex
  • NIO is also an alternative to java.net
  • IO is often automatically mapped to NIO operations
  • Will cover java.io first
  • Reading/Writing from/to disk
  • Reading/Writing streams
  • Writer/Readers and the difference to streams

IO is were performance counts.

java.io.File

The entry point to the file system.

  • Immutable virtual representation
  • File or Directory
  • Might not exist!
  • Creating a file object does not create a or attach to a real file system object
  • Creating a file object does not create a or attach to a real file system object
  • OS specific tweaks
  • Interacts with environment settings, such as user.dir, temp.dir and more
  • equals works similarly to String.equals
  • Most operations are an OS call
  • Absolute, relative and canonical path definitions
  • Supports also URI aka file://
  • Do no attempt any locking implementations!

Path Names

Different path names and separators

Misunderstood Separators

  • pathSeparator ':' or ';'
  • separator '/' or '\'

Paths

  • Relative: foo/bar.txt or bar.txt
  • Relative is to user.dir mostly start dir of VM
  • Absolute: /tmp or c:\tmp
  • Absolute depends on the OS representation
  • Canonical: Unique OS dependent representation of an absolute path
  • e.g. /tmp//../tmp is /tmp

Don't assume that certain things will happen. There is OS and FS magic going on. This is especially valid for mounts and links. Don't forget, your Mac sucks, because it does not know anything about casing aka it is a little Windows-like.

Permissions and Exceptions

What can happen

SecurityException

  • Runtime Exception that communicates missing permission seen from the VM!
  • Does not indicate OS permission problems.
  • You are mostly unprepared for these.

IOException

  • Indicates problems when communicating with the file system
  • Does not tell you (in code) what went wrong
  • Checked exception hence you have to declare them in your code

FileNotFoundException

  • Trying to open a file that is not reachable by the pathname aka wrong pathname or does not exist

EOFException, InterruptedIOException and others

File in Detail

The classic version

// newFile object, NOT a new file or file connection
final File file = new File("/tmp/foobar.txt");

// create this new file
final boolean created = file.createNewFile();

// remove this file again
final boolean deleted = file.delete();

// create a temp file
final File tempFile = File.createTempFile("foo", ".txt")

// alternatively register for automatic deletion
tempFile.deleteOnExit();

The modern version

// get us a new path
final Path path = Paths.get("/tmp", "foobar.txt");

// create this new file
final Path file = Files.createFile(path);

// delete it
Files.delete(file);

// create a temp file
final Path tempFile = Files.createTempFile("foo", ".txt");

// register for automatic deletion
tempFile.toFile().deleteOnExit();

There are many more methods of course.

Byte Streams

A Byte at the time

InputStream

  • Interface
  • Byte-oriented processing
  • Reads from a source a byte at a time
  • Can be nested
  • No notion of the content aka bytes only

OutputStream

  • Interface
  • Byte-oriented processing
  • Writes to a sink a byte at a time
  • Can be nested
  • No notion of the content aka bytes only

Just bytes, nothing else.

Stream Classes

  • FileInputStream: Read from a file
  • ByteArrayInputStream: Read from an array
  • FilterInputStream: Filter a stream
  • ObjectInputStream: Read Java objects
  • PipedInputStream: Receiving end of pipe
  • SequenceInputStream: Read more than one stream and switch to the next automatically
  • FileOutputStream: Write to a file
  • ByteArrayOutputStream: Write to an array
  • FilterOutputStream: Filter a stream
  • ObjectOutputStream: Write Java objects
  • PipedOutputStream: Sending end of pipe

Most other streams are specialized versions of Filter*Stream.

Byte Streams Example

Copy file data

  • Copy data from one file to the other
  • Both must exist
  • try-with-resource to close properly
  • Reading byte by byte
  • Return value of read() is int and communicates also state (sigh)
public void copy()
{
    try (final InputStream in = new FileInputStream("/tmp/in.txt");
         final OutputStream out = new FileOutputStream("/tmp/out.txt");)
    {
        int b; // I will hold the read byte or status code

        // 0-255 indicate a byte, -1 indicates nothing to read anymore
        while ((count = in.read()) != -1)
        {
            out.write(count);
        }
    }
    catch(final FileNotFoundException e)
    {
        // cannot do anything
    } 
    catch (IOException e)
    {
        // something went wrong
    }

    // due to try-with-resource, we don't have to close anything explicitly
}

This is inefficient, but see later.

Byte Streams Example

Transition to NIO

public void copy()
{
    try (final InputStream in =
                    new FileInputStream("/tmp/in.txt");
         final OutputStream out =
                    new FileOutputStream("/tmp/out.txt");)
    {
        int b; // I will hold the read byte or status code

        // 0-255 indicate a byte, -1 indicates nothing to read anymore
        while ((count = in.read()) != -1)
        {
            out.write(count);
        }
    }
    catch(final FileNotFoundException e)
    {
        // cannot do anything
    } 
    catch (IOException e)
    {
        // something went wrong
    }

    // due to try-with-resource, we don't have to close anything explicitly
}
public void copy()
{
    try (final InputStream in =
                    Files.newInputStream(Paths.get("/tmp/in.txt"));
         final OutputStream out =
                    Files.newOutputStream(Paths.get("/tmp/out.txt"));)
    {
        int b; // I will hold the read byte or status code

        // 0-255 indicate a byte, -1 indicates nothing to read anymore
        while ((count = in.read()) != -1)
        {
            out.write(count);
        }
    }
    catch(final FileNotFoundException e)
    {
        // cannot do anything
    } 
    catch (IOException e)
    {
        // something went wrong
    }

    // due to try-with-resource, we don't have to close anything explicitly
}

Opening is more efficient due to NIO usage.

Byte Streams Example

Full NIO now

public void copy()
{
    try (final InputStream in =
                    Files.newInputStream(Paths.get("/tmp/in.txt"));
         final OutputStream out =
                    Files.newOutputStream(Paths.get("/tmp/out.txt"));)
    {
        int b; // I will hold the read byte or status code

        // 0-255 indicate a byte, -1 indicates nothing to read anymore
        while ((count = in.read()) != -1)
        {
            out.write(count);
        }
    }
    catch(final FileNotFoundException e)
    {
        // cannot do anything
    } 
    catch (IOException e)
    {
        // something went wrong
    }

    // due to try-with-resource, we don't have to close anything explicitly
}
/**
  Behaviorial not 100% identical
 */
public void copy()
{
    try
    {
        Files.copy(
            Paths.get("/tmp/in.txt"),
            Paths.get("/tmp/out.txt"),
            StandardCopyOption.REPLACE_EXISTING);
    } 
    catch (IOException e)
    {
        // something went wrong
    }
}

Opening is more efficient due to NIO usage.

Character Streams

A character at the time

Reader

  • Interface
  • Reads character-oriented
  • Delivers a character at a time
  • Can be nested
  • Requires knowledge of the character set of the source

Writer

  • Interface
  • Writes character-oriented
  • Consumes a character at a time
  • Can be nested
  • Requires decision about the sink encoding

Characters not bytes!

Character Streams

Typical character streams

Reader

  • BufferedReader: Read a character with internal buffering
  • CharArrayReader: Read from a char array
  • FilterReader: Filter a stream of characters
  • InputStreamReader: Translate from a byte stream
  • PipedReader: Read from a pipe
  • StringReader: Read from a string (odd, isn't it)

Writer

  • BufferedWriter: Write buffered to a sink
  • CharArrayWriter: Write to an array
  • FilterWriter: Filter a stream
  • OutputStreamWriter: Translate to a byte output stream
  • PipedWriter:
  • StringWriter:

Characters, not bytes!

Read a file

Just read lines from a file

  • Open a file reader
  • Wrap that in a buffered reader for line support
  • Read lines as long as available
public void readLines()
{
    try (final BufferedReader r =
                new BufferedReader(
                    new FileReader("/tmp/in.txt")))
    {
        String readLine = null;
        while ((readLine = r.readLine()) != null)
        {
            System.console().printf(readLine);
        }
    } 
    catch (FileNotFoundException e)
    {
        // handling here
    } 
    catch (IOException e)
    {
        // handling here
    }
}

A plain reader does not offer lines, only characters or character arrays.

Read a file NIO style

Everything is nicer in NIO land

  • Use the built-in lazy reading of file lines
  • Returns a stream of lines
  • Harvest each line lambda-like
public void readLines()
{
    try
    {
        Files.lines(Paths.get("/tmp", "in.txt"))
            .forEach(
                s -> System.console().printf(s)
            );
    } 
    catch (IOException e)
    {
        // handling here
    }
}

A plain reader does not offer lines, only characters or character arrays.

Nesting

Harvest the power of nesting

  • Streams can be nested
  • You can convert from byte to character and back
  • Make sure you close the outer stream
  • Closing propagates
  • Works great with all filters
BufferedWriter bw = null;        FileOutputStream fos = null;
CheckedOutputStream acos = null; CheckedOutputStream bcos = null;

try
{
    fos = new FileOutputStream("/tmp/test.gz");
    BufferedOutputStream bos = new BufferedOutputStream(fos);
    acos = new CheckedOutputStream(bos, new CRC32());
    GZIPOutputStream gzipos = new GZIPOutputStream(acos);
    bcos = new CheckedOutputStream(gzipos, new CRC32());
    OutputStreamWriter osw = new OutputStreamWriter(bcos);
    bw = new BufferedWriter(osw);
    
    bw.write("This is a test. This is a test.\n");
}
catch(IOException e)
{
    // handle here
}
finally
{
    if (bw != null)
    {
        try
        {
            bw.close();
            
            // we closed normally
            System.out.format("Checksum before compression %s and after %s", 
                            bcos.getChecksum().getValue(),
                            acos.getChecksum().getValue());
        } 
        catch (final IOException e)
        {
            // ok, make sure the file is closed at least
            try
            {
                if (fos != null)
                {
                    fos.close();
                }
            }
            catch(IOException ioe)
            {
                // really bad
            }
        }
    }
}

Place the buffer at the right position.

Warnings

Important considerations

  • Always close streams regardless of their type
  • Beware of deprecated, many methods or classes are not longer suitable for us
  • Not everything is IO with network or files
  • Check NIO for alternative and better versions
  • Be aware of file system differences
  • Permissions, permissions, permissions
  • Most IO is a switch from user to kernel-mode
  • Any operation is expensive
  • File system contention is easy to create
  • Buffers help to speed up things

Example: Read and duplicate

Read, filter, write

  • Read a file
  • Duplicate all bytes
  • Write the file
  • Optional: Duplication with Filter
  • Optional: Compression
  • Do copy of byte manually
  • Use FilterOutputStream for that

Example: List with filter

  • Classic
  • Modern Lamdbas
  • Delete a non-empty dir