November 2010 Archives

My approach to bugs

| 0 Comments
As the recent spate of bug fix and patch releases shows I'm not scared of talking about the bugs that I find in the code of The Server Framework and pushing fixes out quickly. It's my belief that the most important thing to get out of a bug report is an improved process which will help prevent similar bugs from occurring in future and the only way to achieve that is to be open about the bugs you find and equally open about how you then address them and try and prevent similar issues. Every bug is an opportunity to improve. Sometimes I wish I had fewer opportunities...

Bug fix for 6.3.x CServiceManager.cpp

| 0 Comments
There's a stupidly obvious, should have been caught at numerous points during pre-release testing, bug in CServiceManager.cpp.

The code below that starts at line 158:


Should actually look like this, note the inclusion of the missing break; and the exception source correction:


What I find especially annoying about this particular bug is that:
   a) this piece of code has pretty good test coverage but we don't cover this particular case. 
   b) the black box server tests that run as part of the release process on the build machines run the service examples using the /debug switch to run them as normal executables to avoid the requiring admin rights on the build machines.
   c) it's the most common use case for this particular piece of code which means that the client that reported it to me last night is the only person using 6.3 or later with single instance services and all of the service development that I've done recently must have been multiple instance.

The fix above will be released in 6.3.3 which will occur some time next week. Hopefully this is all that it will include, but many clients have upgraded to 6.3.2 from much earlier versions and so there may be a few more issued lurking in there.  

New client profile: Desktop Sharing Company

| 0 Comments
We have a new client profile available here for a client that we've had since 2006 in the desktop sharing market. Their system, built on The Server Framework, runs on more than 120 servers worldwide and handles more than 200,000 desktop sharing sessions each day!

Latest release of The Server Framework: 6.3.2

| 0 Comments | 1 TrackBack
Version 6.3.2 of The Server Framework was released today.

This release is purely a bug fix release and includes the following fixes.

  • Fixes to JetByteTools::OpenSSL::CAsyncConnection to remove the possibility of deadlock due to a lock inversion.
  • Fixes to JetByteTools::SSPI::SChannel::CAsyncConnection to remove the possibility of deadlock due to a lock inversion.
  • Fixes to JetByteTools::SSPI::Negotiate::CAsyncConnection to remove the possibility of deadlock due to a lock inversion.
  • Fixes to JetByteTools::CLRHosting::CCLREventSink and JetByteTools::CLRHosting::CCLRHost to remove a race condition during host shutdown which could have caused a purecall due to events being fired after the event sink has been destroyed.

There's no need for a documentation update so the latest docs available online will be for the 6.3 release. Likewise the server examples have not changed and code that built with 6.3 will build with 6.3.2 without changes.
I found this article recently whilst discussing a question about socket reuse using DisconnectEx() over on StackOverflow. It's a useful collection of the various configuration settings that can affect the number of concurrent TCP connections that a server can support, complete with links to the KB articles that discuss the settings in more detail. It's a bit out of date, but it's probably a good starting point if you want to understand the limits involved.

Note that I've never had to tune or tweak the default settings (apart from setting MaxUserPort to allow more outbound connections).
I've been improving my pre-release testing system and now run a lock inversion detector as part of my build machine's build and test cycle for the socket server examples. This lock inversion detector can detect the potential to deadlock without the code ever needing to actually deadlock, so it's a pretty powerful tool. It has detected a lock inversion in the async connectors used by the OpenSSL, SChannel and SSPI Negotiate libraries. I've fixed these problems and there will be a 6.3.2 release next week. 

At present I haven't finished changing all of the build scripts for all of the servers to include the lock inversion detection phase into the test runs and so there may be more issues yet to be discovered.

Note that these lock inversions have not, as far as I know, ever actually caused a deadlock, but the possibility is there if the right timing occurs between concurrent read and write operations.

Latest release of The Server Framework: 6.3.1

| 0 Comments
Version 6.3.1 of The Server Framework was released today.

This release is purely a bug fix release and includes the following fixes.

  • Fixes to JetByteTools::OpenSSL::CStreamSocketConnectionFilter to prevent buffer reference leaks which result in memory leaks during secure connection processing; see here for more details.
  • Fixes to JetByteTools::OpenSSL::CAsyncConnector to prevent socket reference leaks in situations where connections are reset or where unexpected SSL errors occur. These socket reference leaks can cause delays and hangs when shutting down a server or connection manager.
  • Fixes to JetByteTools::SSPI::SChannel::CStreamSocketConnectionFilter to prevent buffer reference leaks which result in memory leaks during secure connection processing.
  • Fixes to JetByteTools::SSPI::Negotiate::CStreamSocketConnectionFilter to prevent buffer reference leaks which result in memory leaks during secure connection processing.

There's no need for a documentation update so the latest docs available online will be for the 6.3 release. Likewise the server examples have not changed and code that built with 6.3 will build with 6.3.1 without changes.
There's a bug in the OpenSSL stream socket filter which results in I/O buffers being leaked on both inbound and outbound data flow. This causes the memory used by the client or server to grow throughout the life of the process. The bug has been present since release 6.2 when the structure of the filtering code was redesigned.

Note that due to the fact that the filtering code was all redone at that time I expect that the same bug is likely to be present in the SChannel, SSPI and compression filters.

I've fixed the problem in the OpenSSL code and will be running this through my test harnesses over the next few days. If you want the fix now then please get in touch, if you can wait then I expect 6.3.1 will be released next week.

This shows that my pre release testing could still do with some work, automated unit tests and server stress tests can show that the code works but don't, in themselves, necessarily catch everything and of course they're only as good as the test coverage in any case. I discovered this issue whilst building some new performance tests which use perfmon to monitor the process under test and produce reports at the end. I guess I now have to extend the scope of this project so that I can monitor expected max and min values from the counters I'm monitoring - so I can see if the memory is ballooning during my automated tests...

Follow us on Twitter: @ServerFramework

About this Archive

This page is an archive of entries from November 2010 listed from newest to oldest.

October 2010 is the previous archive.

December 2010 is the next archive.

I usually write about the development of The Server Framework, a super scalable, high performance, C++, I/O Completion Port based framework for writing servers and clients on Windows platforms.

Find recent content on the main index or look in the archives to find all content.