Why is performance so important for a virtual directory? And, what does performance actually mean, anyway? There are quite a few metrics you can find to choose from. We always claim that our Virtual Directory Server is the fastest product out there, but why should you care? Let me try to explain ...

First of all, what do we mean by performance? It's obviously about being fast, and it's also about being scalable. Fast usually means that you want responses to your queries as quickly as possible. Scalable means that you want the system to be powerful enough to handle hundreds, thousands, or even tens of thousands of operations per second without flinching. Ideally, the virtual directory should be faster than the actual directory infrastructure. We typically use two metrics: the time that it takes to handle a request, and the number of requests that can be handled per second.

Before I dive any further into this, I must point out one very important factor. Virtual directories are virtual front-ends to existing identity infrastructure. Often, virtual directories are deployed as front-ends to other LDAP servers. When a request is made to the virtual directory, the virtual directory will turn around and query the actual back-end identity store (LDAP directory, database or other type of data store). So, when we are talking about the time required to handle a request, we must be more specific: do we mean the total time for the request to be handled end-to-end? Or, do we mean just the time that is needed by the virtual directory?

The same thing is true when discussing the number of operations per second. If a virtual directory is the front-end for a LDAP server that can only handle 500 requests per second, then that is the maximum that you are going to get, regardless of what the virtual directory by itself can do. Therefore, by definition the virtual directory is very much dependent on the back-end infrastructure. Of course there are some ways to overcome this - either by load-balancing between multiple replicas or by caching. But, I won't go there - at least not today.

So, if the virtual directory is influenced so much by the performance of the back-ends, how do we take meaningful measurements? We typically run our products in idealized environments where the back-ends are not an issue. For example, instead of running our Virtual Directory Server against a "real" LDAP directory server, we run it against another version of Virtual Directory Server that acts as a "dummy LDAP server" - it responds to each query with a pre-configured reply that is always the same - but it does so very quickly. This allows us to see what the maximum values are for our products. So, can you get this type of performance in your deployment? Yes - if your back-ends can keep up. Or, if you configure Virtual Directory Server to load-balance or cache data, to help them out when they can't.

In ideal environments, Virtual Directory Server can handle many thousands of packets per second. Your mileage can vary, of course, depending on the speed of your CPUs, your network, and how fast your operating system can dispatch packets to and from Virtual Directory Server. We've even noticed some differences in the internal network stacks of certain operating systems, and some are definitely more optimized than others, so they will run Virtual Directory Server faster on the same CPU. This is because the underlying operating system has a better optimized TCP/IP stack in some cases, and can therefore shuffle packets faster.

So, why should you care? First of all, think of Virtual Directory Server as the front-door to your identity data. Applications make requests to Virtual Directory Server, and it acts as a virtual data switch. You obviously want to have as much flexibility here as possible, so that you can "grow" as needed. But there's an additional upside to having a scalable product. Unlike some of our competitors, who use a heavyweight architecture and therefore rely on caching results (because re-calculating them on the fly would just be too slow), we have none of these restrictions. We’ve designed our software from the ground up for performance and massive scalability without having to rely on caching. This gives you more flexibility and imposes fewer restrictions.

I hope this discussion is helpful to you. I also plan to write something soon that will examine in a little more detail what makes us so unique when it comes to performance and scalability.

A few months ago Volker Scheuber from Novell started a bit of a twirl when he blogged about why he doesn't really like virtualization and virtual directories. Volker is a well respected guy in Identity Management, and he's been with Novell for many years. He is intrinsically involved with their Dir-XML meta-directory product. In the old days, meta-directories used to be the only solution available for integrating multiple directory servers. But ever since virtual directories were born, there have been alternatives to meta-directories. So perhaps it's inevitable that the “meta-directory guys” feel threatened by the “virtual directory guys”. After all, we're a competing technology ... but are we really?

Several years ago, Gerry Gebel from the Burton group authored an excellent article on virtual directories, introducing the technology and giving an overview of the companies that were in the field. Since we were one of the pioneers of virtual directory technology, I've had the honor to brief him on our product. He was so kind as to send me the report after it was published, and ever since the Burton Group has been updating the report regularly. What I found very important to note was that Gerry didn't think meta-directories were dead and announce “long live virtual directories”. Instead, he saw the technologies as complimentary.

Gerry was completely right back then, but nowadays meta-directories are just about dead anyway. Let me explain why ... meta-directories use a synchronization approach. They work by propagating changes from multiple places into one place, and then optionally pushing those changes out again to other sources. So it's all about synchronization. Virtual directories, on the other hand work, by pulling data “on the fly” and transposing it “on the fly”.

Virtual directory technology has been around for about five years, and innovative companies like Vodafone and Boeing that saw the light right away were among the earliest adopters. Virtual directory may not be fully mainstream today, but it's a well respected technology, and its advantages and disadvantages relative to the good old (but slightly rusty) meta-directory approach are now well known. So why are meta-directories just about dead? It's because virtual directories have moved on. Nowadays, virtual directory servers such as our Virtual Directory Server include the functionality for synchronization as well, so that users can do the same tasks with a virtual directory that previously only meta-directories could do.

It's all about having the right tools for the right job. Virtualization is an exciting technology that promises very easy integration without the “yet another”  factor (having to create “yet another directory” to copy into). Virtual directories are non-intrusive and can typically be deployed in a fraction of the time that it takes to deploy a meta-directory. Synchronization, on the other hand, is a tool that has advantages in the few cases where virtualization is lacking - such as when data really does need to be copied, or if a data source cannot be relied upon to be permanently available.

So, as virtual directories such as Symlabs Virtual Directory Server take on more and more synchronization features, I can't think of a reason why anyone would want a pure meta-directory any more. Except for those very smart guys who grew up with meta-directory technology, but unfortunately think of virtualization being a "threat" to their good old technology. Move on guys! The future lies with virtual directories, or rather virtual meta-directories - engines that handle both virtualization and synchronization to the benefit of companies and their integrators that deploy them.