In the past three weeks, I have been working on an old MFC application on my own time. The application uses CAsyncSocket to handles several hundred TCP data streams with somewhat high data rate. As much as I find MFC painful to work with, CAsyncSocket is not hard to use, and it fits in well with the MFC messaging framework.

I wrote all my automated testing in a small Python script to simulate the data streams. To my surprise, I found that the MFC application is missing data packets. Precisely, it is missing the last couple kilobytes of the stream.

I suspected that it is a TCP graceful shutdown issue (probably similar to the one with PuTTY). Very likely it has something to do with the OnClose() callback.

The MFC application treated the OnClose() callback as a graceful shutdown event after all packets are received. This might not be the correct assumption.

// Original implementation of the OnClose() function in the MFC app
// This implementation is leaking several kB of data.
void CMyAppAsyncSocket::OnClose(int nErrorCode)
{
	// ... do some app close stuff

	// Call the base class Close
	CAsyncSocket::OnClose(nErrorCode);
}

When Exactly Is CAsyncSocket::OnClose Called?

In MSDN, the CAsyncSocket::OnClose is described as the following:

Called by the framework to notify this socket that the connected socket is closed by its process.

This tells me nothing. There are tutorials on how OnReceive and OnSend should be written, but there is nothing for OnClose.

To find out what triggers the OnClose callback, I looked into the implementation of the CAsyncSocket.

In summary, it is nothing but a simple overlapped asynchronous I/O wrapper on WinSock API. And the OnClose function is invoked by the FD_CLOSE event from WSAGETSELECTEVENT.

[Update: CAsyncSocket does not use overlapped I/O. I misread the documentation, and my co-worker corrected me.]

// sockcore.cpp
void PASCAL CAsyncSocket::DoCallBack(WPARAM wParam, LPARAM lParam)
{
    // ... more code here
	switch (WSAGETSELECTEVENT(lParam))
	{
    // ... more cases here
	case FD_CLOSE:
		pSocket->OnClose(nErrorCode);
		break;
	}
}

Ah ha, I know FD_CLOSE fairly well. The Winsock graceful shutdown sequence is well described by MSDN.

(2) Receives FD_CLOSE, indicating graceful shutdown in progress and that all data has been received.

Upon FD_CLOSE, I am supposed to read all the remaining data from the socket. So to fix the problem, I modified the OnClose function to read the remaining data packets.

void CMyAppAsyncSocket::OnClose(int nErrorCode)
{
    CAsyncSocket::OnClose(nErrorCode);

	while(1)
	{
		// m_tempBuffer is my internal receive buffer
		int numBytes = Receive(m_tempBuffer, MESSAGE_BUFFER_LENGTH);
		if( (SOCKET_ERROR == numBytes) || (0 == numBytes) )
		{
			break;
		}
        // ... process the remaining data here
	}
    // .. more app close stuff here
}

With this slight modification, I have transferred hundreds of gigabytes of TCP streams without any data loss.

Conclusion

CAsyncsocket is a thin wrapper to the WinSock library.

To find out how to really handle the CAsyncsocket callbacks, it is recommended to look into its implementation to find the corresponding WSAAsyncSelect event.

I have been very busy recently. Aside from Starcraft II, I picked up another side programming project. I am reworking an old MFC app and see how far I can bring it into modern age with MFC9.0.

MFC is horrible in practice because it forces you to deal all the details, such as resizing and redrawing. But it suits my purpose because it forces me to learn all the details.

As I was testing the application across Window XP and Window 7, I noticed a strange error. The Create() function of CAsyncSocket returns an error 10022 on Window 7, and not Window XP.

// pueudocode

// Create a server socket, CMyAsyncSocket is derived from CAsyncSocket.
m_serverSocket = new CMyAsyncSocket();
BOOL hasError = m_serverSocket->Create(
   m_listenPort, // some port number to bind to
   SOCK_STREAM,
   FD_READ | FD_WRITE | FD_OOB | FD_ACCEPT | FD_CONNECT | FD_CLOSE,
   "");
if(TRUE == hasError)
{
   int error = GetLastError();
   ... // report and print error
}
... // more code here

Error 10022 is WSAEINVAL, where it indicates that an invalid argument was supplied.

Instead of reading the error code, my cognitive bias convinced me that the above code is flawless since it worked in XP. I jumped through the hoops to change firewall settings, network card settings, and tried to blame everything but the source.

Eventually I went back and inspect the API again, and realized that the third and fourth arguments are already defaulted to the appropriate values.

// CAsyncSocket signature
BOOL Create(
   UINT nSocketPort = 0,
   int nSocketType = SOCK_STREAM,
   long lEvent = FD_READ | FD_WRITE | FD_OOB | FD_ACCEPT | FD_CONNECT | FD_CLOSE,
   LPCTSTR lpszSocketAddress = NULL
);

So I cleaned up the function call to Create().

//...

m_serverSocket->Create(
   m_listenPort, // some port number to bind to
   SOCK_STREAM);

//...

Apparently the application was setting lpszSocketAddress to “” instead of NULL, it is an invalid argument (as indicated by the MFC for ~5000 times before it penetrated my thick head).

With the appropriate argument, everything works.