A reusable Windows socket server class

Servers that do nothing but echo a byte stream are rare, except as poor examples. Normally a server will be expecting a message of some kind, the exact format of the message is protocol specific but two common formats are a binary message with some form of message length indicator in a header and an ASCII text message with a predefined set of 'commands' and a fixed command terminator, often "\r\n". As soon as you start to work with real data you are exposed to a real-world problem that is simply not an issue for echo servers. Real servers need to be able to break the input byte stream provided by the TCP/IP socket interface into distinct commands. The results of issuing a single read on a socket could be any number of bytes up to the size of the buffer that you supplied. You may get a single, distinct, message or you may only get half of a message, or 3 messages, you just can't tell. Too often inexperienced socket developers assume that they'll always get a complete, distinct, message and often their testing methods ensure that this is the case during development.

One of the simplest protocols that a server could implement is a packet based protocol where the first X bytes are a header and the header contains details of the length of the complete packet. The server can read the header, work out how much more data is required and keep reading until it has a complete packet. At this point it can pass the packet to the business logic that knows how to process it. The code to handle this kind of situation might look something like this:

void CSocketServerWorkerThread::ReadCompleted(
   CSocketServer::Socket *pSocket,
   CIOBuffer *pBuffer)
{
   pBuffer = ProcessDataStream(pSocket, pBuffer);

   pSocket->Read(pBuffer);
}

CIOBuffer *CSocketServerWorkerThread::ProcessDataStream(
   CSocketServer::Socket *pSocket,
   CIOBuffer *pBuffer)
{
   bool done;

   do
   {
      done = true;

      const size_t used = pBuffer->GetUsed();

      if (used >= GetMinimumMessageSize())
      {
         const size_t messageSize = GetMessageSize(pBuffer);

         if (used == messageSize)
         {
            // we have a whole, distinct, message

            EchoMessage(pSocket, pBuffer);

            pBuffer = 0;

            done = true;
         }
         else if (used > messageSize)
         {
            // we have a message, plus some more data
            // allocate a new buffer, copy the extra data into it and try again...

            CIOBuffer *pMessage = pBuffer->SplitBuffer(messageSize);
            
            EchoMessage(pSocket, pMessage);

            pMessage->Release();

            // loop again, we may have another complete message in there...
      
            done = false;
         }
         else if (messageSize > pBuffer->GetSize())
         {
            Output(_T("Error: Buffer too small\nExpecting: ") + ToString(messageSize) +
               _T("Got: ") + ToString(pBuffer->GetUsed()) + _T("\nBuffer size = ") + 
               ToString(pBuffer->GetSize()) + _T("\nData = \n") + 
               DumpData(pBuffer->GetBuffer(), pBuffer->GetUsed(), 40));

            pSocket->Shutdown();

            // throw the rubbish away
            pBuffer->Empty();
            
            done = true;
         }
      }
   }
   while (!done);

   // not enough data in the buffer, reissue a read into the same buffer to collect more data
   return pBuffer;
}

The key points of the code above are that we need to know if we have at least enough data to start looking at the header, if we do then we can work out the size of the message somehow. Once we know that we have the minimum amount of data required we can work out if we have all the data that makes up this message. If we do, great, we process it. If the buffer only contains our message then we simply process the message and since processing simply involves us posting a write request for the data buffer we return 0 so that the next read uses a new buffer. If we have a complete message and some extra data then we split the buffer into two, a new one with our complete message in it and the old one which has the extra data copied to the front of the buffer. We then pass our complete message to the business logic to handle and loop to handle the data that we had left over. If we dont have enough data we return the buffer and the Read() that we issue in ReadCompleted() reads more data into the same buffer, starting at the point that we're at now.

Since we're a simple server we have a fairly important limitation, all our messages must fit into the IO buffer size that our server is using. Often this is a practical limitation, maximum message sizes can be known in advance and by setting our IO buffer size to be at least our maximum message size we avoid having to copy data around. If this isn't a viable limitation for your server then you'll need to have an alternative strategy here, copying data out of IO buffers and into something big enough to hold your whole message, or, processing the message in pieces...

In our simple server if the message is too big then we simply shutdown the socket connection, throw away the garbage data, and wait for the client to go away...

So how do we implement GetMinimumMessageSize() and GetMessageSize(), well, obviously it's protocol dependant, but for our packet echo server we do it like this:

   size_t CSocketServerWorkerThread::GetMinimumMessageSize() const
   {
      return 1;
   }

   size_t CSocketServerWorkerThread::GetMessageSize(CIOBuffer *pBuffer) const
   { 
      size_t messageSize = *pBuffer->GetBuffer();
   
      return messageSize;
   } 

You may have noticed that in the case where we had a message and some extra data we called SplitBuffer() to break the complete message out into its own buffer, and then, once we'd dealt with it, we called Release(). This is a little of the implementation of the socket server's buffer allocator poking through. The buffers are reference counted. The only time we need to worry about this is if we create a new buffer using SplitBuffer, or if we decide to call AddRef() on the buffer because we wish to pass it off to another thread for processing. We'll cover this in more detail in the next article, but the gist of it is that every time we post a read or a write the buffer's reference count goes up and every time a read or write completes the count goes down, when there are no outstanding references the buffer goes back into the pool for reuse.

A packet based echo server is available for download in SocketServer2.zip. The server expects to receive packets of up to 256 bytes which have a 1 byte header. The header byte contains the total length of the packet (including the header). The server reads complete packets and echos them back to the client. You can test the echo server by using telnet, if you're feeling clever ;) Simply telnet to localhost on port 5001 (the port that the sample uses by default) and type stuff and watch it get typed back at you. (Hint, CTRL B is 2 which is the smallest packet that contains data).

Some of the common internet protocols, such as RFC 1939 (POP3), use a crlf terminated ASCII text stream command structure. An example of how such a server might be implemented using the CSocketServer classes presented here can be found in SocketServer3.zip