We've made a point of mentioning performance at nearly every opportunity lately, and it's time we expand on that a bit to fill in some details. We recently completed a series of benchmark tests that pitted Symlabs Virtual Directory Server against the two other leading virtual directories in the market, and the actual measured results from that testing not only back up our claim to the title of "fastest virtual directory server", it also lets us point how just how much faster we really are. Bear in mind that our products are built on a common core engine (it's possible this may also be true for others), so while we tested Symlabs Virtual Directory Server, the inference is that Symlabs LDAP Proxy delivers similar performance (but not necessarily all the same features).

We set a design goal of superior performance from the very beginning of our work on the product, so of course we knew from development testing and customer feedback that we had top-notch speed, but we were curious (and also prompted by customers) to see the actual numbers in a head-to-head comparison. Before I discuss the results though, let me note that we readily admit a couple points to consider when reviewing them.

First, we're a vendor testing our own product against the competition - how can that be trusted? Well, we did our best to create a level playing field for all the participants just as an independent lab would do by using the latest generally available version of all products, identical hardware that is typical of a normal customer installation and specified by all vendors, and by configuring all products in the same way as much as their configurable parameters allowed. We used the SLAMD Distributed Load Generation Engine to generate increasing loads under several scenarios that are typical of actual customer deployments. SLAMD is an open, Java-based, industry standard benchmarking suite specifically designed for stress-testing LDAP directory servers and analyzing their performance. Most important, we've published full details of the test platform, the products, plus the test scenarios in our report - we encourage you to download a copy of Symlabs Virtual Directory Server Competitive Benchmarks, duplicate the tests (you can also download an evaluation copy of Symlabs Virtual Directory Server), and see for yourself.

Second, we tested without enabling cache in any of the products. Yes, it sounds suspicious because we have taken a position on caching, and you can read our earlier post Virtual Directory Servers and Cache that discusses it, but testing sans cache was not done with a bias, it is a practical necessity to be able to see raw performance. There is a detailed explanation of that in the report, but if you care to look into it further, you'll find that disabling cache is a standard practice for test environments seeking to evaluate underlying input/output speed. So, while we're sure some may complain, we're happy to defend it, and we're confident that no independent evaluation would consider results based on caching to be representative of the true raw I/O performance of the product being tested.

All that being said, here's what we found ... Symlabs Virtual Directory Server outperformed the competition, and by quite a bit on average. In some tests, the results were similar at the start with a minimal load of 10 clients, but by the time the load increased to 100 clients we were way out in front. In other tests, we started out well in front and just increased our lead throughout. Here's an example of that, showing the response time of Symlabs Virtual Directory Server versus one competitor in a straightforward test of throughput using a pass-through configuration. Symlabs is the lower line in red, and you can see that in this basic I/O test we finish with a 40% faster response time when the load becomes significant. That's a pretty meaningful advantage in a real IT environment. Against the other major competitor (not graphed), even at starting point we are well over twice as fast, and maintain that lead plus a slight increase as client loads are added.

In a more complex test scenario, where we added a load balancing function to the responsibility of each virtual directory server, the results demonstrated an even bigger advantage for the Symlabs core engine. To realize the significance of this, think about a distributed identity infrastructure with multiple directory servers, perhaps organized by division or business function. A great many medium to large enterprises have such an environment, as do others such as service providers. Load balancing can be critical for overall performance (and also for adding redundancy) in these infrastructures, and the performance of the virtual directory server while handling load balancing is the key to achieving that, so we felt it was a valid addition to our test scenarios.

You can read the rest of the details in the full report, so I won't expand on them further now - here's the link: Download Symlabs Virtual Directory Server Competitive Benchmarks

As I mentioned earlier, if you are currently planning to implement a virtual directory in your infrastructure, or if you currently have one and would like to upgrade for better performance, we encourage you to duplicate these tests for yourself. After you download the report, stay to look around the Symlabs website where you'll find a wealth of helpful information and download an evaluation copy of our product to test. Finally, since this may become the subject of further discussion for a while, you can check out the links in our blogroll to see what everyone is saying, and be sure to check back here periodically for updates.

Jeff Zukowski