IOCP Server Library

So I wrote C++ library that provides a scalable TCP server using Windows I/O Completion Port (IOCP).

Couple weeks ago, I started studying IOCP to improve the scalability of a C++ application that may handle thousands of TCP/IP data stream.

It didn’t take long for me to realize why IOCP has the reputation of being difficult to learn. IOCP tutorial online usually fall into the category of difficult to read, overly simplified, or just plain wrong.

Worse yet, Winsock2 contains a mix of confusing APIs that perform very similar functions with subtle differences. I spent a few days just to decide whether I should use WSAAccept, accept or AcceptEx to accept a connection.

Eventually, I stumbled onto two books that helped me out a great deal – Windows Via C++ and Network Programming For Windows.

The Library

The library interface is rather simple. It follows the Proactor design pattern where user supplies a completion handler and event notifications flow through the completion handler asynchronously.

Everyone uses echo server as tutorial. So what the heck, here’s mine. 🙂

class CEchoHandler : public CIocpHandler
{
public:
	virtual void OnReceiveData(
        uint64_t clientId,
        std::vector<uint8_t> const &data)
	{
        // echo data back directly to the connected client
		std::vector<uint8_t> d(data);
		GetIocpServer().Send(clientId, d);
	}
}
void main()
{
    // create a handler that echos data back
	boost::shared_ptr h(new CEchoHandler());
    try
    {
        // bind to port 50000 with the server
        CIocpServer *echoServer = new CIocpServer(50000,h);

        char c;
        std::cin >> c; // enter a key to exit

        delete echoServer;
    }
    // RAII constructor that throws different exceptions upon failure
    catch(...)
    {
    }
}

[10/27/2010 10:34AM EST]
Update: Moved “delete echoServer;” to within the try block per co-worker’s suggestion.

Focus

Of course, there are more to the IOCP server than the code snippet above.

Here are my area of focus when designing the library.

  1. Scalability – By ensuring that there are minimum number of critical section in the library.
  2. TCP Graceful shutdown – Allow user to perform TCP graceful shutdown and simplify the TCP half-closed state.
  3. RAII – A WYSIWYG constructor and a lenient destructor that allows you to do ungraceful shutdown.

Here is a screenshot of the CPU utilization of the echo server at 300 concurrent loopback file transfer.

IOCP Server scalability upon Intel I5-750 (quad-core)

 

License

IOCPServer is released under the Boost Software License 1.0.

Download

For latest version, please see the Projects page.

IOCPServer is tested under the following configurations.

OS: Window XP, Window 7.

Compiler: Visual Studio 2009 with Boost 1.40

Build Type: ANSI, Unicode.

3 thoughts on “IOCP Server Library

  1. Nice work.

    You can probably make it more scalable by removing the need to lock and access the connection manager collection to issue a send from within the context of a completion by using the ‘per connection’ data for data that relates to the connection (i.e. some per connected socket data that you can then pass into your recv callback and which you can then use for a send without needing to do a lock and lookup on your connection id map). At present you seem to be using the “per connection” data as a simply a link to your IOCP…

    Also the event you create in the OVERLAPPED structure isn’t needed and will probably cause a fractional perf degradation if present as the completion code will set it when it doesn’t need to.

    I’ve got some free IOCP client/server code that you might like to take a look at here: http://www.serverframework.com/products—the-free-framework.html which is more complex and built in a more ‘old school’ style than yours but which probably covers more edge cases which may or may not be of interest to you.

    • Len,

      You are definitely right that I should remove the mutex when accessing the connection. When I first started, I was afraid that I would lose track of the outstanding overlapped structure. It gave me a warm and fuzzy by going through that connection manager to verify that my overlapped structure has not gone out of wack. Now that I am more comfortable with the code, I will improve upon it.

      I will remove the event in the OVERLAPPED structure as you suggested.

      Thanks for the lesson. This gives me more idea for version 1.1. 🙂

      … Alan

Leave a Reply

Your email address will not be published. Required fields are marked *