Java NIO Buffer

Java NIO buffers are holders of information in an array-type structure. At its heart, NIO processing is about moving data in and out of buffers. Unlike traditional Java I/O that uses separate input and output streams, a Java NIO buffer is used for both read and writing. Buffers are abstractions that allow exchanging of data.

Working with the java.nio.Buffer API, however, is not straightforward as you need to understand low-level concepts such as position, limit, capacity and flipping. This article provides a detailed tutorial of how to work with the java.nio.Buffer API and sub-classes.

The Java NIO Buffer hierarchy

The java.nio.Buffer class is the high-level abstraction of a Java NIO buffer. This class defines the operations common to all buffer types, such as current position, limit, capacity, flipping, marking, and rewinding.

There are several subclasses of java.nio.Buffer, one for each primitive type:

Additionally, there is also a MappedByBuffer class that extends from ByteBuffer that is used to work with direct buffers. More on direct vs non-direct buffers later.

ByteBuffer is the most important buffer type, as operating systems work at byte level. The other buffer types provide a convenience interface to work with primitives such as integers, doubles or chars.  But when interacting with the operating system, we need to use byte buffers.

Creating Buffers

New buffers are created by either allocation or wrapping.

With allocation, you just specify the capacity of the buffer, and let the NIO framework create the internal structures to store the data. The current Java implementation use a backing array of the buffer type, but this is not guaranteed.

This is how you allocate a byte buffer of 100 bytes.

And this is how you create a char buffer of 100 chars:

With wrapping, you need to provide a backing array:

There are two types of byte buffers, direct and non-direct buffers.  Direct buffers are mapped outside of the Java heap and can be accessed by the operating system directly, thus they are faster than non-direct buffers backed by arrays in the Java heap.  Direct buffers are created with the allocateDirect() method:

Position, Limit and Capacity

The key to understanding Java NIO buffers is to understand buffer state and flipping. This section introduces the three main properties of buffer state:  position, limit and capacity.

Capacity is the maximum number of data items that the buffer can hold. For example, if you create a buffer with the backing array new byte[10], then the capacity is 10 bytes.  The capacity never changes after buffer creation.

Limit is the zero-based index that identifies the first data item that should not be read or written. Limit determines the data that can be read from the buffer.  Data between zero and limit (exclusive) is available for reading. Data items between limit (inclusive) and the capacity index are garbage.

Position is the zero-based index that identifies the next data item that can be read or written. As you read from or write into the buffer, the position index increases.

Java NIO Buffer state: position, limit and capacity

The following invariant must apply at all times:

0 <= position <= limit <= capacity

The java.nio.Buffer class provides generic methods to access the state:

There are also methods to set the position and the limit. Please note that we cannot change the buffer capacity after creation:

There is also a remaining() method to calculate the number of remaining data items available for consumption.  Remaining is calculated as limit() – position().

Reading and Writing Data

Each Buffer implementation provides several get and put methods to read from and write into the buffer.

ByteBuffer, for example, provides the following methods to read and write bytes:

CharBuffer provides methods to work with chars.

The position property plays a central role when reading from or writing into the buffer. The get() and put() methods read and write data for the index pointed by the current position property.  Then, the position index increases ready for the next operation.

NOTE: The remaining of this article will focus on ByteBuffer, as it is the most important Buffer implementation given the operation system works at byte level.

Buffer Life cycle: Fill, Flip, Drain, Clear

Java NIO buffers are structures that enable the exchange of data, and they are used for both reading and writing.  Conceptually, a Java NIO buffer has two modes of operation:

  • Filling mode – a producer writes into the buffer
  • Draining mode – a consumer reads from the buffer

In the typical life cycle of a Java NIO buffer, the buffer is created empty ready for a producer to fill it up with data. The buffer is in filling mode.  After the producer has finished writing data, the buffer is then flipped to prepare it for draining mode. At this point, the buffer is ready for the consumer to read the data.  Once done, the buffer is then cleared and ready for writing again.

Java NIO Buffer Life cycle

Fill the Buffer

Data is written into a Java NIO buffer using the put() method. The following code illustrates how to fill the buffer.

First, we create a buffer with capacity 6. The buffer is now empty ready to be filled. The limit and capacity properties are pointing at index 6, and position is pointing at 0.

Java NIO Buffer after allocation

The first put() writes a byte into index 0, and then increased the position to index 1. Then we add three more bytes into the buffer. After this, the position is pointing at index 4, with 2 remaining data slots in the buffer.

Java NIO Buffer after put

Flip the Buffer

Once the producer has finished writing data to the buffer, we need to flip it so that the consumer can start draining it.

Why do we need flipping? If we did not flip, the get() method would read data from the current position. In our example, we would be reading data from index 4, which is a position without data.

Java NIO Buffer after flip()

flip() is a method that prepares the buffer to retrieve its contents. It sets the limit property to the current position to mark the area of the buffer with data content, and the position is reset back to 0 so the get() operation can start consuming the data from the beginning of the buffer.

In practical terms, flip() is equivalent to the following

Drain the Buffer

After the buffer has been flipped, we are ready to start reading with the get() method:

This reads byte 10 from index 0, and increases the position to index 1.

Java NIO Buffer: Get data from buffer

We can also drain the buffer completely by checking the hasRemaining() method until we reach the buffer limit:

This is how the buffer looks like after draining:

Java NIO Buffer: Get data from buffer

Clear the Buffer

Once the buffer has been drained, the next step is to prepare the buffer for filling again.  This is done with the clear() method.

The clear() method sets the position back to 0 and the limit to same value as capacity. Please note clear() does not remove the data from the buffer, it just changes position and limit.

Java NIO Buffer clear() method

The clear() method is equivalent to the following:

You might wonder why we didn’t flip() the buffer instead of clear().  This is because flip() changes the limit property to the current position, thus it would not allow to fill the buffer to its full capacity.

Reading and Writing Data in Bulk

The buffer API provides methods to read and write data in bulk using arrays.

ByteBuffer has two bulk methods for put():

The first put() method writes the full content of the data array into the buffer starting at the current position. The position will be incremented by the length of the array. The second put() method writes the contents of the array starting at the offset position, and copying length bytes.  If we attempt to copy more bytes than remaining bytes in the buffer, we will get a BufferOverflowException.

Similarly, the ByteBuffer has two counterparts for reading in bulk:

The bulk get() method reads the contents of the buffer into the data array starting at the current position, until filling up the array completely.  Note that if the array provided is larger than the number of remaining bytes in the buffer, a BufferUnderflowException exception is thrown.

The following code illustrates how to read and write in bulk:

Direct vs Non-Direct Buffers

There are two types of buffers, direct and non-direct.

A non-direct buffer is backed by an array in the Java heap. The operating system, however, does not copy data in or out of this array directly.  Instead, it uses an intermediate buffer outside of the Java heap. This intermediate buffer is needed because the operating system cannot efficiently access objects in the Java heap, as objects might not be optimally page aligned or might be relocated by the Garbage Collector.

Non-direct buffers are created with the following allocate() method:

As an example, imagine an application needs to read data from a file in local disk. It will make a call to the operating system, which in turn will instructs the disk controller to copy bytes from storage to the intermediate buffer.  The data from this operating system buffer is then copied to the Java heap buffer. The application can then access data from this buffer via a java.nio.ByteBuffer instance.

Java NIO Non-Direct Buffer

A direct buffer is a byte buffer that access directly the memory used by the operating system to store the data from the I/O resource.  This buffer is outside of the reach of the Garbage Collector.

Direct buffers are created with the following allocateDirect() method:

Direct buffers are more efficient as I/O operations are performed directly on the buffer without the need of copying the information into memory first. However, creating a direct buffer is an expensive operation, and might even trigger a full Garbage Collection. Direct buffers are usually best suited when working with long-lived and large buffers, although performance gain should be measured before committing to using direct buffers.

Summary

This article has explained in great length the key concepts of NIO buffers. A Java NIO buffer is an array-like structured with three properties: position, limit and capacity.  The same buffer is used for both producers and consumers, thus conceptually we need to differentiate between two modes of operation: filling and draining modes.  To change from one mode to another we use the flip() and clear() methods. Finally, we explored the differences between direct and non-direct buffers.

Bibliography

The following two tabs change content below.

Eduard Manas

Eduard is a senior IT consultant with over 15 years in the financial sector. He is an experienced developer in Java, C#, Python, Wordpress, Tibco EMS/RV, Oracle, Sybase and MySQL.Outside of work, he likes spending time with family, friends, and watching football.

Latest posts by Eduard Manas (see all)

One thought on “Java NIO Buffer”

Leave a Reply