In the past three weeks, I have been working on an old MFC application on my own time. The application uses CAsyncSocket to handles several hundred TCP data streams with somewhat high data rate. As much as I find MFC painful to work with, CAsyncSocket is not hard to use, and it fits in well with the MFC messaging framework.

I wrote all my automated testing in a small Python script to simulate the data streams. To my surprise, I found that the MFC application is missing data packets. Precisely, it is missing the last couple kilobytes of the stream.

I suspected that it is a TCP graceful shutdown issue (probably similar to the one with PuTTY). Very likely it has something to do with the OnClose() callback.

The MFC application treated the OnClose() callback as a graceful shutdown event after all packets are received. This might not be the correct assumption.

// Original implementation of the OnClose() function in the MFC app
// This implementation is leaking several kB of data.
void CMyAppAsyncSocket::OnClose(int nErrorCode)
{
	// ... do some app close stuff

	// Call the base class Close
	CAsyncSocket::OnClose(nErrorCode);
}

When Exactly Is CAsyncSocket::OnClose Called?

In MSDN, the CAsyncSocket::OnClose is described as the following:

Called by the framework to notify this socket that the connected socket is closed by its process.

This tells me nothing. There are tutorials on how OnReceive and OnSend should be written, but there is nothing for OnClose.

To find out what triggers the OnClose callback, I looked into the implementation of the CAsyncSocket.

In summary, it is nothing but a simple overlapped asynchronous I/O wrapper on WinSock API. And the OnClose function is invoked by the FD_CLOSE event from WSAGETSELECTEVENT.

[Update: CAsyncSocket does not use overlapped I/O. I misread the documentation, and my co-worker corrected me.]

// sockcore.cpp
void PASCAL CAsyncSocket::DoCallBack(WPARAM wParam, LPARAM lParam)
{
    // ... more code here
	switch (WSAGETSELECTEVENT(lParam))
	{
    // ... more cases here
	case FD_CLOSE:
		pSocket->OnClose(nErrorCode);
		break;
	}
}

Ah ha, I know FD_CLOSE fairly well. The Winsock graceful shutdown sequence is well described by MSDN.

(2) Receives FD_CLOSE, indicating graceful shutdown in progress and that all data has been received.

Upon FD_CLOSE, I am supposed to read all the remaining data from the socket. So to fix the problem, I modified the OnClose function to read the remaining data packets.

void CMyAppAsyncSocket::OnClose(int nErrorCode)
{
    CAsyncSocket::OnClose(nErrorCode);

	while(1)
	{
		// m_tempBuffer is my internal receive buffer
		int numBytes = Receive(m_tempBuffer, MESSAGE_BUFFER_LENGTH);
		if( (SOCKET_ERROR == numBytes) || (0 == numBytes) )
		{
			break;
		}
        // ... process the remaining data here
	}
    // .. more app close stuff here
}

With this slight modification, I have transferred hundreds of gigabytes of TCP streams without any data loss.

Conclusion

CAsyncsocket is a thin wrapper to the WinSock library.

To find out how to really handle the CAsyncsocket callbacks, it is recommended to look into its implementation to find the corresponding WSAAsyncSelect event.