OpenDS (open source directory server)

I am always deep lover of Directory Server and technology behind it. I was very happy when I saw that Sun (I have worked mostly with Sun LDAP and I love it) is aggressively working on a open source directory server project OpenDS which will take the directory server to new levels. I was reading about the intension of this project and why they have not supported the current open source projects like openLDAP,Apache directory etc and the reason they feel is because they want to redesign it based on current client's requirements.

Below are the features in the openDS:

Performance Lots of features are important, but performance is almost always near the top of the list. It needs to be extremely fast, outperforming all other servers wherever possible.

Upward Vertical Scalability. It needs to be capable of handling billions of entries in a single instance on appropriately-sized hardware. It should be able to make effective use of multi-CPU, multi-core machines with hundreds of gigabytes of memory.

Downward Vertical Scalability. It needs to be capable of running adequately in low-memory environments so that all essential components can be functional on edge devices like cell phones and PDAs.

Horizontal Scalability. It needs be possible to use synchronization to achieve higher levels of read scalability by adding servers to the directory service. In addition, it needs to be possible to use data distribution in conjunction with synchronization to achieve horizontal read and write scalability to achieve deployments into the billions.

Supportability. The server should be easy to support and maintain. Administration should be intuitive, and wherever possible the server should provide sufficient information and notifications to enable corrective actions, even predictively.

Security. The server must provide extensive security in areas like access control, encryption, authentication, auditing, and password and account management.

Extensibility. Virtually every aspect of the server should be customizable. It needs a safe and simple plugin API that delivers additional points of extensibility, including, but not limited to, password validation algorithms, password generators, monitor information providers, logging subsystems, backend repositories, protocol handlers, administrative tasks, SASL mechanisms, extended operations, attribute syntaxes, and matching rules.

Synchronization. The server must support data synchronization between instances, including not only total data synchronization but also partial synchronization (with fractional, filtered, and subtree capabilities), and must also provide a means of synchronizing with other applications and data repositories.
Availability. The server must be robust enough to continue running properly even if serious errors are encountered.

Portability. The server needs to be written entirely in Java so that it can run on any platform.

Reliability. A directory service is one of the most critical components of a business infrastructure. It is absolutely essential that the service function despite hostile or unexpected events and that the data it delivers be trusted.

Compatibility. The Sun Java System Directory Server will continue to be maintained over time and will not be immediately replaced by Sun products based on OpenDS. However, OpenDS must provide support for virtually all existing features of the Sun Java System Directory Server. Migration from other directory server implementations should also be taken into consideration when applicable.

DSML Introduction

Directory Services Markup Language (DSML) provides a mechanism for accessing directory servers as XML document. It evolved with version 1.0 which just gave flexibility to read the directory servers but version 2.0 came with more flexibility. Using version 2.0 directory server entries can be updated.

DSML functionality is motivated by scenarios including:

1) Hand held device which does not have LDAP client.
2) Applicaion programs developed using XML tools and application needs to access LDAP.
3) A program needs to access a directory across the firewall and firewall does not allow LDAP ports.
Click on the below image to view the animation.

4) Integrating two different vendor directory servers.

Most of the existing directory server vendors support the DSML but it needs to be enabled explicitely. Below are some links specific to Sun DS.

How to enable DSML support

Accessing Directory using DSML

DSML tools are also available for integration with existing applications.

Technorati : dsml introduction

How to enable special character support on Sun Directory Server?

Organizations in today's world are spreaded across the globe and they want to have a support of all the character set in the applications so there user's can access them at any part of the globe in there local language.

To enable Special character support in Sun DS follow below steps

1) Login into the directory manager console
2) Open the instance on which you want to enable special character support
3) Go to Configuration Tab
4) Expend the plugins
5) Select "7 bit check" plugin and disable it. By default it is enabled.
6) Restart the server and now you are all set to dump special characters in the Directory Server.

Same activity can be done using the command line but I am not posting that part here.

Static Vs Dynamic LDAP Groups

LDAP directory servers contain information about people: users, employees, customers, partners, and others. Many times, it makes sense to associate entries together in groups. A group is basically a collection of entries. These entries can be statically assigned to a group or can have a set of common attributes on which they can form a dynamic groups.

1) Static Group

A static group defines each member individually using the structural objectclass groupOfNames, groupOfUniqueNames, etc depending on Directory Server implementation. These objectclasses require the attribute member (or uniqueMember in the case of groupOfUniqueNames). These groups are good if the number of users in a group is not large because group contains an entry for each user who belong to this group. The more number of people assigned to the group more complicated the task to manage that group.

2) Dynamic Group

Dynamic groups allow you to use a LDAP URL to define a set of rules that match only for group members. For Dynamic Groups, the members do share a common attribute or set of attributes that are defined in the memberURL filter. These are good if the number of users in the group are very large. It's a much better choice for a dynamic group than a static group because the set of members will be automatically adjusted as new users are added, existing users are removed

Example :

dn: cn=Austin Users,ou=Groups,dc=example,dc=com
objectClass: top
objectClass: groupOfURLs
cn: Austin Users
memberURL: ldap:///ou=People,dc=example,dc=com??sub?(&(l=Austin)(st=Texas))

In the above example all the users who belong to Location as Austin or State as Texas belongs to Austin Users.

Roles

Roles are a similar to groups, but work differently. Groups are effectively listings of members. In order to find out, for example, which groups "David" belongs to, you would need to look at every group and see if it contains "David". Roles, on the other hand, are associations that are stored in users' entries themselves.

As a member of a role, you have the authority to do what is needed for the role in order to accomplish a job. Unlike a group, a role comes with an implicit set of permissions. There is not a built-in assumption about what permissions are gained (or lost) by being a member of a group.

Technorati : LDAP groups

How to see LDAP protocol working over the network???

Many network protocols like HTTP and SMTP are text-based, which means that
it is relatively simple to decode that information if it is intercepted over the wire.
LDAP, however, is binary protocol that uses the ASN.1 basic encoding rules
specification to encode all communication. While some components of LDAP
communication (e.g., distinguished names) may be decipherable, it is
significantly more difficult to interpret other data elements.
To address this problem, the LDAPDecoder utility provides a means of
interpreting LDAP communication and displaying it in a human-readable form.
This can be very useful for debugging problems with the interaction between
LDAP clients and a directory server, or to simply gain a better understanding of
the structure of LDAP traffic.

LDAPDecoder can be downloaded from http://www.slamd.com/download.shtml location.

Unpack the LDAPDecoder.jar and the howto guide from the compressed file in a folder.

for running LDAPDecoder use following command

java -jar LDAPDecoder.jar -L
-f

once it is listening you can run the ldapsearch/ldapmodify and check the log file to see how LDAP server responds to the request.

I am giving the request response sequence for ldapsearch

New client connection from 127.0.0.1:1298
-- Read data from the clientDecoded Data from Client:
LDAP Bind Request
Message ID: 1
LDAP Bind Request Protocol Op
LDAP Version: 3
Bind DN: cn=admin
Authentication Data:
Authentication Type: Simple
Bind Password: password

Read data from the serverDecoded Data from Server:
LDAP Bind Response Message ID: 1 LDAP Bind Response Protocol Op Result Code: 0 (Success)

Read data from the clientDecoded Data from Client:
LDAP Search Request
Message ID: 2 LDAP Search Request Protocol Op Base DN: dc=am,dc=sony,dc=com Scope: 2 (wholeSubtree) Deref Aliases: 0 (neverDerefAliases) Size Limit: 0 Time Limit: 0 Types Only: false Filter: (uid=abcd) Attributes:

Read data from the serverDecoded Data from Server:

LDAP Search Result Entry Message ID: 2 LDAP Search Result Entry Protocol Op dn: uid=abcd,ou=users,l=america,dc=am,dc=sony,dc=com
mail: Jack.Bauer@sun.com
cn: Bauer Jack
sn: Jack
givenName: Bauer
uid: abcd
objectClass: top
objectClass: person
objectClass: organizationalperson
objectClass: inetorgperson
objectClass: inetuser
objectClass: sonyperson
objectClass: americasonyperson
userPassword: {SSHA}7h3HPwNNIYAecfrYbigXsQinNqW2N/gqGxECLw==

Technorati : Inside LDAP Protocol

Tips for Developing efficient LDAP Applications

I took this topic from sun blog but added some example code to show what the topic means so part of credit goes to them.

* Make sure to use LDAPv3 rather than LDAPv2. Some APIs still default to LDAPv2, but LDAPv2 doesn't support features like controls, extended operations, referrals, SASL authentication, and multiple binds on the same connection.

* Use at least minimal caching to avoid repeating the same queries. If you include a list of attributes to return, then make sure that you include all attributes you may need rather than performing different queries to retrieve the same entry with different attribute lists.

For example :

use this
String RETURN_ATTRIBUTES[] = { "mail","givenname","sn" };
Attributes attrs = ctx.getAttributes(DN, RETURN_ATTRIBUTES);
insted of

String RETURN_ATTRIBUTES[] = { "mail","givenname","mail" };
Attributes attrs = ctx.getAttributes(DN, RETURN_ATTRIBUTES);

String RETURN_ATTRIBUTES[] = { "mail","givenname","sn" };
Attributes attrs1 = ctx.getAttributes(DN, RETURN_ATTRIBUTES);

String RETURN_ATTRIBUTES[] = { "mail","givenname","givenname" };
Attributes attrs2 = ctx.getAttributes(DN, RETURN_ATTRIBUTES);


* Design your application to allow for loose consistency in replication and the possibility that reads and writes may happen on different systems without the application's knowledge. Avoid read-after-write behavior because it can have inconsistent results.

What I understood : In a load balanced environment when application uses connection pooling it may get connections from different servers. Some times if write operation is performed on the consumer, it may redirect the request to the master. Master may take some time to replicate the change to all consumers. If you try to do read after write there may be some inconsistencies.

* Don't treat the Directory Server like a relational database. Avoid splitting data into separate pieces so that you need to retrieve multiple entries to get all the information about a given entity.

* If you generate search filters, then do so intelligently. If you have compound filters, then use a form like "(&(a=b)(c=d)(e=f))" rather than "(&(&(&(a=b))(c=d))(e=f))" to avoid unnecessary nesting.

Unnecessary nesting may make the filter more complicated to understand and take more time to perform the operation as in the above case LDAP server will have to process 3 AND operators in the second query instead of 1 AND in the smart query. Softerra browser has a very good interface for building queries.

* Unbind connections when they're no longer needed. It's generally best to re-use connections as much as possible, but whenever you're done with a connection make sure it gets closed.

* Don't litter your code with hard-coded attribute/objectclass names, base DNs, server addresses/ports, usernames/passwords, etc. If you need to change something later, it can be hard to make sure that everything gets updated properly. You should centralize all such values in a constants class or a properties file so that they are simple to change if necessary.

* Where possible, maintain a set of persistent connections to the server (i.e., connection pools) rather than connecting and disconnecting for each operation. This will be much more efficient, especially when using SSL. In order to avoid leaking connections and duplicating large amounts of code, it may be a good idea to code the various types of operations into the connection pool itself so that those operations will check out a connection, perform the operation and any necessary error handling, and make sure the connection is put back into the pool.

I promise I will add a topic to explain how to write code for connection pooling in LDAP.

* Design your application to be able to handle the different kinds of failures that may arise: server down, network outage, DS backlogged or unresponsive, DS returning unexpected responses (e.g., unavailable or busy). Don't assume that a lost connection means the server is down -- it could be that the connection was closed due to the idle timeout or some other constraint.

Technorati : Efficient LDAP programming

Structural Vs Auxillary Object class ??

What Are Objectclasses?
Objectclasses are prototypes for entries that will actually exist in directory server. The objectclass definition (uses ASN.1 syntax) specifies which attributes may or must be used by LDAP entries declared as instances of a particular objectclass.

When schema needs to be designed for custom implementation people do have confusion when to use structural and when to use auxillary object classes.

Thumb Rule: Every LDAP entry must use a STRUCTURAL objectclass. Furthermore, it can have only one STRUCTURAL objectclass. It can have any number of Auxillary object classes. If the auxillary object class is attached to the entry attributes attached to that auxillary object class (MAY/MUST) will also be attached.

For example in an organization there are multiple applications which needs specific attributes based on applications assigned to the user or not. All the applications can be represented using Auxillary object class. This way whatever application is attached to the user, attributes specific to that attributes will be assigned to the user.


I will give example on defining Structural Vs Auxillary object class but lets just read once the definition of each type of object class from SUN documentation

Abstract: defined only as a superclass or template for other (structural) object classes. An abstract object class is a way of collecting a set of attributes that will be common to a set of structural object classes, so that these classes may be derived as subclasses of the abstract class rather than being defined from scratch. An entry may not belong to an abstract object class.

--> Think of these as abstract class in java. They are just a helper classes so structural object classes can simply extend them for common attributes. For example top is a abstract object class.

Structural: indicates the attributes that the entry may have and where each entry may occur in the DIT. This object class represents the corresponding real world object. Entries must belong to a structural object class, so most object classes are structural object classes.


Auxiliary: indicates the attributes that the entry may have. An auxiliary object class does not represent a real world object, but represents additional attributes that can be associated with a structural object class to supplement its specification. Each entry may belong to only a single structural object class, but may belong to zero or more auxiliary object classes.


As promised here are example of how you can define object classes.

objectClasses: ( 1.3.6.1.4.1.20502.1.2.12 NAME 'ABCAPSMUser' SUP top AUXILIARY MAY absAPSMId X-ORIGIN 'user defined' )

objectClasses: ( 1.3.6.1.4.1.20502.1.2.8 NAME 'ABCuser' SUP inetorgperson STRUCTURAL MUST ABCuserstatus MAY ( gender $ absuserpassword $ inetUserStatus $ personalTitle $ mailAlternateAddress $ mailHost $ mailRoutingAddress ) X-ORIGIN 'user defined' )

How to rename LDAP DN ?

Often a times I hear from my friends that they have a requirement to rename the user id in LDAP/Active Directory. They tried to modify the DN attribute directly but that does not help as DN is an operational attribute which can not be directly modified.

Correct approach is to call the rename API from JNDI and that takes care of the job. I am pasting a sample code ( changing the which I feel will help some one who is also looking for similar functionality.

public static boolean changeId(DirContext ctx, String p_oldID, String p_NewID)
{
String RETURN_ATTRIBUTES[] = { "uid","objectclass","modifytimestamp"};
String DN = null, LDAPuid = null ;
boolean status = true;
int i = 0;
Attribute attr = null;

String newid = "\"cn=" + p_NewID +",OU=Users,OU=Test\"";

String empNo = null;

try
{
// Make LDAP connection
//ModificationItem[] mods = new ModificationItem[1];
String SEARCH_FILTER = "(cn=" +p_oldID+")";

SearchControls constraints = new SearchControls();
String newline = System.getProperty("line.separator");
// Set search scope to subtree
constraints.setSearchScope(SearchControls.SUBTREE_SCOPE);
NamingEnumeration results = ctx.search("", SEARCH_FILTER, constraints);

while ( results != null && results.hasMore() )
{
//String s = "";
SearchResult sr = (SearchResult) results.next();
DN = sr.getName();
System.out.println("DN is" + DN);
i++;
Attributes attrs = ctx.getAttributes(DN, RETURN_ATTRIBUTES);

attr = attrs.get("uid");
LDAPuid = (String)attr.get();
System.out.println("UID is " + LDAPuid + newline);

ctx.rename(DN,newid);

System.out.println("Employee Number | New User ID");
// System.out.println(empNo + " " + p_NewID );

}
System.out.println("Total no of records are " + i);
System.out.println("Now changing the userid in LMS ....");


}

catch(Exception e)
{
System.out.println("In the exception block" );
e.printStackTrace();
return false;
}
return status;
}

Technorati : LDAP rename entry