Build Your Own Netty — Move to NIO

Richard Feynman

In the previous post, we’ve implemented an Echo Server by the Blocking I/O (BIO) library and discussed some of the disadvantages of BIO:

1. Server thread cannot do anything but wait when accepting new connections.
2. Clients must wait in queue to be served, OR tens of thousands of threads must be created in high concurrency situation using one-client-one-thread model.
3. Switching thread contexts may be expensive under high concurrency situation.

In this post, we will introduce the Non-blocking I/O library (NIO for short) and use it to rewrite our Echo Server to resolve the problems that is inevitable in BIO implementation.

Non-blocking I/O

NIO is a new library introduced since Java 1.4, it was also called New I/O before but since it’s not new any longer, it’s usually considered as Non-blocking I/O later.

How Does It Work

We are going to rewrite the Echo Server with NIO in this post to resolve the problems listed above, but before that, let’s take a look at how NIO library manages to resolve them.

In the BIO implementation of Echo Server, serverSocket.accept(); blocks the server thread until there is connection coming in, so how does NIO solve this? Selector does the trick!

+-------+ +-------+ +-------+
|Channel| |Channel| |Channel| ...
+---+---+ +---+---+ +---+---+
    |         |         |
    |         |         |
    |         |         |
    +---------+---------+
              |
              | Register
              v
          +---+----+
          |Selector|
          +---+----+
              |
              |
    +---------+---------+
    |         |         |
  +-+-+     +-+-+     +-+-+
  |Key|     |Key|     |Key| ...
  +---+     +---+     +---+

  ------------------------>

       Iterate Over Keys

           +------+
           |Thread|
           +------+

The flow above shows how Selector helps to free the server thread from blocking when waiting new connection.

It’s perfectly right that the Selector works like a multiplexer:

1. First of all, we register a Channel to the Selector for interested events(read, write, accept), returning a Keyrepresenting this registry.
2. When our interested events happened(e.g. Channel is able to read, write or new connection can be accepted), the Selector changes the Keys' state.
3. We iterate over the Keys to find out Channels that are ready for our operations.

public class EchoServer {
  public void start() throws Exception {
    try (final ServerSocketChannel serverSocketChannel = ServerSocketChannel.open()) {
      final Selector selector = Selector.open();

      serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 8080));
      serverSocketChannel.configureBlocking(false);
      serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

      while (true) {
        if (selector.select(1000L) == 0) {
          // no connection yet, do some other staff
          continue;
        }
        // handle the selected keys (selector.selectedKeys())
      }
    }
  }
}

Here are some important notes that are different from the BIO implementation:

1. To make it possible for the ServerSocket to register to the Selector, ServerSocketChannel must be used instead of ServerSocket. Static method ServerSocketChannel.open() creates one for us.
2. Though ServerSocketChannel can be registered to a Selector, it works in blocking(synchronous) mode by default as ServerSocket does, in order to make it work in asynchronous mode, the methodserverSocketChannel.configureBlocking(false) must be called before registering to the Selector, otherwise the exception java.nio.channels.IllegalBlockingModeException would be thrown.
3. serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); registers our server socket channel to the selector and tells it that this channel is interested in accepting new connection(s) (SelectionKey.OP_ACCEPT).

Here comes the key reason why the server thread does not need to block: selector.select() tries to select out those Keys that are ready for our operations, and it will return the number of channels immediately without blocking the server thread, then we check the returned number to decide whether perform the READ, WRITE, ACCEPT operations or do other staff.

So far so good, the server thread don’t need to block any more, the first problem is solved. How about the other 2 problems? Can we handle the clients’ connection without creating too many threads?

The reason why we need to create so many threads is that the server thread cannot process so many clients fast enough, and the reason why the process is slow is that the process often includes many read/write operations, which are again blocking operations. So we’re going to use the NIO library again to make the client socket non-blocking:

public class EchoServer {
  public void start() throws Exception {
    // .....
    while (true) {
      if (selector.select(1000L) == 0) {
        continue;
      }
      for (final Iterator<SelectionKey> iterator = selector.selectedKeys().iterator(); iterator.hasNext(); iterator.remove()) {
        final SelectionKey key = iterator.next();
        if (key.isAcceptable()) {
          final ServerSocketChannel server = (ServerSocketChannel) key.channel();
          final SocketChannel client = server.accept();
          client.configureBlocking(false);
          client.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE, ByteBuffer.allocate(1024));
          LOGGER.info("client connected: " + client);
        }
        if (key.isReadable()) {
          readData(key);
        }
        if (key.isWritable()) {
          writeData(key);
        }
      }
    }
  }
}

The method accept() of ServerSocketChannel returns another SocketChannel that can be registered to a Selector too, so we configure it to work in non-blocking mode and register it back to the Selector again, one thing worth noting is that the interested events of this channel(client channel) is no longer SelectionKey.OP_ACCEPT, it's SelectionKey.OP_READ and SelectionKey.OP_WRITE instead, meaning that we are interested in the readability/writability of the client connection.

In this way, we put all the I/O operations from/to all the client connections into one thread, to avoid creating too many threads, and of course there is no cost to switch thread context among tens of thousands of threads.

Summarize

In this article, we rewrote the Echo Server by using the NIO (Non-blocking I/O) library. The Selector plays an important role in multiplexing the sockets. By multiplexing, the server thread can do other staff while waiting for new connections and handling client connections.

Putting all I/O operations into one thread is another important idea that helps to resolve the problems, and it’s also adopted in the Netty’s thread model, understanding this will help a lot in the future study of Netty’s source code.

In the next post, we’ll learn a design pattern that Netty uses, Reactor Pattern, and rewrote the Echo Server using this design pattern as well as the NIO library we learnt in this post.