Identity Matters

The time has come to finish up this discussion of the new Remote Administration Server (RAS) in version 4.0 of Symlabs LDAP Proxy and Symlabs Virtual Directory Server by describing how it actually brings a lot of benefit to a production environment. This, of course, is what our customers care about more than the technology itself (and far more than a philosophy debate with the MOTCL* (*see last post)).

Refreshing your memory from last time, before RAS (i.e., before version 4.0) each instance of Symlabs LDAP Proxy or Symlabs Virtual Directory Server and its associated instance of DSGUI were tightly coupled, one-to-one. For example, every server running Symlabs LDAP Proxy was also running it's DSGUI interface, and while this provided a nice graphical configuration and management tool, you had to access it through that server, and you could only manage instances running on the same server when you did. As production environments routinely started having lots of instances on lots of servers, our customers started asking for a way to simplify and consolidate their management capabilities.

Now with RAS, the architecture has been re-worked so that there are several options for managing complex environments, allowing customers to create the one that makes it easiest for them. The "core engine" in Symlabs LDAP Proxy or Symlabs Virtual Directory Server no longer needs its tightly-coupled graphical environment for configuration and management (as it was before version 3.0, when MOTCL roamed the earth). Instead, RAS allows an instance of DSGUI running on any machine to configure and manage an instance of the core engine running in any server, using fully secure communication of course. One ring to rule them all.

Using the RAS server is fully optional, so if you want run a local DSGUI in the server along the instance of its core engine, you can still do so ... and start managing multi-server configurations the moment you need to, and not an instant before. The rapid prototyping that DSGUI users are used to has not been lost, in fact you haven't lost the flexibility to manage any instance of Symlabs LDAP Proxy or Symlabs Virtual Directory Server from the command line, if that's what works for you. You have simply gained the flexibility to configure and manage them all easily from one place, or from several places using whatever division of responsibility and toolkit matches your organization needs.

Also, with the introduction of RAS we changed the way in which configurations are stored, so they are now platform independent. If you have several RAS instances running on different types of servers, you can simply copy and move the configurations among the servers with just a mouse click. This should come in very handy, especially in production environments where several instances have to be kept current, such as when fail-over scenarios or server replications are managed. It will also be useful where different environments are maintained for preproduction and production, or where different OS are used, for example initial testing done on a Windows desktop with production running Solaris. Now configurations can easily be created on technician's desktop, verified there, and  moved to a preproduction environment to begin load and performance testing in seconds, all without having to worry about changing environments, desktop sharing, or other cumbersome annoyances.

I can keep on talking about implementation details for hours, but at this point you should get the picture, so the next step is to prove it to yourself. Just download a free evaluation version from http://symlabs.com and check out how useful this new feature is. We are always interested in opinions (including from MOTCL) to help us keep improving the features offered in our products, so after you try it, any feedback you want to send us will be greatly appreciated.

Fernando García Vegas

At last we've wrapped everything up, and the new version 4.0 of Symlabs Virtual Directory Server and Symlabs LDAP Proxy is now official, so I can finally take a moment to elaborate on the Remote Administration Server (RAS) feature that I briefly mentioned last time.

"In the beginning ... was the command line" (an interesting, but a bit outdated essay by famous author Neal Stephenson) is the best way to describe how our family of products started. A long time ago (in a galaxy far away) Symlabs began with an extremely fast and robust multi-protocol proxy engine, designed to give large LDAP deployments functionalities that existing LDAP servers could not provide. It was impressive by itself, and it has become the "core engine" of our products today, since its extensive programming capability has allowed us to keep on building new features and functions. Even now, we're pretty sure that we have barely scratched the surface of what can be done with that engine.

But, let's face it, it was not the easiest tool to configure and work with - its extreme "command line" approach was bucking the trend that most enterprises were following. That's why we created DSGUI, our name for a Java-based graphical user interface that makes managing configurations much easier. DSGUI allows end users to start working with both Symlabs LDAP Proxy and Symlabs Virtual Directory Server in a matter of minutes. This feature has allowed us to serve more than the "big IT & Telco" shops that had the resources to work without a GUI, and has been a success from the start for a wide range of customers.

But, the addition of DSGUI was not without some resistance, as a few developers (let's call them "Masters of the command-line", from now on - MOTCL) still hold the idea that graphical interfaces are for the weak and feeble. Still, DSGUI's success helped demonstrate that MOTCL are not always right (some may say never, but that's another story), so after we shipped it we decided to take the next step and listen to more customer feedback about how to continue improving the usability of our products. And, that's how our Remote Administration Server (RAS) functionality came to be.

RAS lets us take full advantage of the graphical user interface and at the same time adapts our products to fit in all possible environments, even those that do not have a graphical environment for some reason. It gives end users the ability to manage Symlabs LDAP Proxy and Symlabs Virtual Directory Server configurations regardless of where they are installed, and also allows them to deal with several instances at the same time. So, if an environment has six different instances of Symlabs LDAP Proxy running, let's say four in the local data center and two in different parts of the country, RAS allows them all to be managed from one place.

Think of RAS as a "connector" between the core engine I described earlier and the DSGUI graphical configuration utility. It works as a daemon process running on the server along with the core engine, communicating between any instance of the core engine in Symlabs LDAP Proxy or Symlabs Virtual Directory Server, and any instance of DSGUI.

OK, so that's a bit about where RAS came from and basically what it is. Next time, I'll finish this discussion with a more in-depth explanation of how to actually use RAS and DSGUI to simplify configuration and management chores in a complex environment. Meanwhile, I'll refer to my earlier comment and recommend that you fill some spare time with Neal Sthephenson's book "Cryptonomicon", which should be mandatory reading for anyone working in the security and identity management field.

Fernando García Vegas

Since a recent Symlabs press release mentioned that the new version 4.0 of both Symlabs Virtual Directory Server and Symlabs LDAP Proxy are nearing their availability date, I thought it would be good to give everyone an update on how the work to wrap everything up is proceeding. And, the answer is ... VERY well indeed!

We're on still target to meet our general availability date, which will be very soon now, and more important than that we're on target to deliver all the benefits that this new release was designed to offer. We're getting good feedback which is telling us that the performance improvements are going to be as dramatic in real-world implementations as we predicted they would be. Speaking as the Product Manager and also one of the developers, this is the best news we could get, since every small percentage increase in the speed and throughput of such communications-intensive processes is a prize we fight very hard to win. But, as our customers insist, this is a championship caliber prize because it's the key to making their whole infrastructure responsive - and to them that means customer satisfaction, efficient operation, and fewer problems overall.

Not to diminish the importance of usability by putting it last, but version 4.0 also gives Symlabs Virtual Directory Server and Symlabs LDAP Proxy an improved user interface to make them much nicer, meaning faster and easier to configure into working applications, plus a powerful new feature to centralize administration of multiple instances on different servers. I'll write more on that later, but already I can see that we are getting very good reports on this added usability. And, since this means we're giving our colleagues in the IT department a shiny new set of tools, that makes me feel pretty good, too.

Fernando García Vegas

Why is performance so important for a virtual directory? And, what does performance actually mean, anyway? There are quite a few metrics you can find to choose from. We always claim that our Virtual Directory Server is the fastest product out there, but why should you care? Let me try to explain ...

First of all, what do we mean by performance? It's obviously about being fast, and it's also about being scalable. Fast usually means that you want responses to your queries as quickly as possible. Scalable means that you want the system to be powerful enough to handle hundreds, thousands, or even tens of thousands of operations per second without flinching. Ideally, the virtual directory should be faster than the actual directory infrastructure. We typically use two metrics: the time that it takes to handle a request, and the number of requests that can be handled per second.

Before I dive any further into this, I must point out one very important factor. Virtual directories are virtual front-ends to existing identity infrastructure. Often, virtual directories are deployed as front-ends to other LDAP servers. When a request is made to the virtual directory, the virtual directory will turn around and query the actual back-end identity store (LDAP directory, database or other type of data store). So, when we are talking about the time required to handle a request, we must be more specific: do we mean the total time for the request to be handled end-to-end? Or, do we mean just the time that is needed by the virtual directory?

The same thing is true when discussing the number of operations per second. If a virtual directory is the front-end for a LDAP server that can only handle 500 requests per second, then that is the maximum that you are going to get, regardless of what the virtual directory by itself can do. Therefore, by definition the virtual directory is very much dependent on the back-end infrastructure. Of course there are some ways to overcome this - either by load-balancing between multiple replicas or by caching. But, I won't go there - at least not today.

So, if the virtual directory is influenced so much by the performance of the back-ends, how do we take meaningful measurements? We typically run our products in idealized environments where the back-ends are not an issue. For example, instead of running our Virtual Directory Server against a "real" LDAP directory server, we run it against another version of Virtual Directory Server that acts as a "dummy LDAP server" - it responds to each query with a pre-configured reply that is always the same - but it does so very quickly. This allows us to see what the maximum values are for our products. So, can you get this type of performance in your deployment? Yes - if your back-ends can keep up. Or, if you configure Virtual Directory Server to load-balance or cache data, to help them out when they can't.

In ideal environments, Virtual Directory Server can handle many thousands of packets per second. Your mileage can vary, of course, depending on the speed of your CPUs, your network, and how fast your operating system can dispatch packets to and from Virtual Directory Server. We've even noticed some differences in the internal network stacks of certain operating systems, and some are definitely more optimized than others, so they will run Virtual Directory Server faster on the same CPU. This is because the underlying operating system has a better optimized TCP/IP stack in some cases, and can therefore shuffle packets faster.

So, why should you care? First of all, think of Virtual Directory Server as the front-door to your identity data. Applications make requests to Virtual Directory Server, and it acts as a virtual data switch. You obviously want to have as much flexibility here as possible, so that you can "grow" as needed. But there's an additional upside to having a scalable product. Unlike some of our competitors, who use a heavyweight architecture and therefore rely on caching results (because re-calculating them on the fly would just be too slow), we have none of these restrictions. We’ve designed our software from the ground up for performance and massive scalability without having to rely on caching. This gives you more flexibility and imposes fewer restrictions.

I hope this discussion is helpful to you. I also plan to write something soon that will examine in a little more detail what makes us so unique when it comes to performance and scalability.

A few months ago Volker Scheuber from Novell started a bit of a twirl when he blogged about why he doesn't really like virtualization and virtual directories. Volker is a well respected guy in Identity Management, and he's been with Novell for many years. He is intrinsically involved with their Dir-XML meta-directory product. In the old days, meta-directories used to be the only solution available for integrating multiple directory servers. But ever since virtual directories were born, there have been alternatives to meta-directories. So perhaps it's inevitable that the “meta-directory guys” feel threatened by the “virtual directory guys”. After all, we're a competing technology ... but are we really?

Several years ago, Gerry Gebel from the Burton group authored an excellent article on virtual directories, introducing the technology and giving an overview of the companies that were in the field. Since we were one of the pioneers of virtual directory technology, I've had the honor to brief him on our product. He was so kind as to send me the report after it was published, and ever since the Burton Group has been updating the report regularly. What I found very important to note was that Gerry didn't think meta-directories were dead and announce “long live virtual directories”. Instead, he saw the technologies as complimentary.

Gerry was completely right back then, but nowadays meta-directories are just about dead anyway. Let me explain why ... meta-directories use a synchronization approach. They work by propagating changes from multiple places into one place, and then optionally pushing those changes out again to other sources. So it's all about synchronization. Virtual directories, on the other hand work, by pulling data “on the fly” and transposing it “on the fly”.

Virtual directory technology has been around for about five years, and innovative companies like Vodafone and Boeing that saw the light right away were among the earliest adopters. Virtual directory may not be fully mainstream today, but it's a well respected technology, and its advantages and disadvantages relative to the good old (but slightly rusty) meta-directory approach are now well known. So why are meta-directories just about dead? It's because virtual directories have moved on. Nowadays, virtual directory servers such as our Virtual Directory Server include the functionality for synchronization as well, so that users can do the same tasks with a virtual directory that previously only meta-directories could do.

It's all about having the right tools for the right job. Virtualization is an exciting technology that promises very easy integration without the “yet another”  factor (having to create “yet another directory” to copy into). Virtual directories are non-intrusive and can typically be deployed in a fraction of the time that it takes to deploy a meta-directory. Synchronization, on the other hand, is a tool that has advantages in the few cases where virtualization is lacking - such as when data really does need to be copied, or if a data source cannot be relied upon to be permanently available.

So, as virtual directories such as Symlabs Virtual Directory Server take on more and more synchronization features, I can't think of a reason why anyone would want a pure meta-directory any more. Except for those very smart guys who grew up with meta-directory technology, but unfortunately think of virtualization being a "threat" to their good old technology. Move on guys! The future lies with virtual directories, or rather virtual meta-directories - engines that handle both virtualization and synchronization to the benefit of companies and their integrators that deploy them.