Java to LDAP Tutorial (Including How to Install an LDAP Server / Client)

This tutorial will show you how to write Java code to interact with a LDAP. But before we can do that we will need to setup an LDAP server and client on our machine.

If at this point you are not sure of exactly what LDAP is, I recommend this post which provides an excellent definition with examples. (In a nutshell it helps to think of an LDAP server as a specialised database).

Installing an LDAP Server

I'm running on a MBP. After looking around for a while I found that the easiest LDAP Server to install was ApacheDirectory which you can download from here. (To install and start the server should take less than 5 minutes)

Once it's installed it automatically starts the daemon. You can then run the server with this command.

sudo launchctl start org.apache.directory.server 

For further installation instructions see here.

(If you need to uninstall you will find the application installed at /usr/local/apacheds-2.0.0-M20 just delete that directory and it will be gone)

LDAP Client

You will want to view the contents of your LDAP Server.  The easiest LDAP client to install is Apache Directory Studio which can be downloaded from here.

Once it is downloaded you need to create a connection to the server - the instructions for which are contained here.

When connected your Apache Directory Studio should look something like this:

Now to access LDAP from a Java program. The best way to show you how to do this is through an example program. The program will perform the following tasks:

• Create an new LDAP object
• View an LDAP object
• Add a new attribute to an LDAP object
• Modify an attribute on an LDAP object
• Remove an attribute on an LDAP object
• Delete an LDAP object
• Search for all LDAP objects in a specific domain

Note:  This class cleans up after itself i.e. It leaves the LDAP Server in the state in which it was found.  If you want to see the various tasks in action just run one of the tasks and take a look at the LDAP Object through the LDAP Client. Don't forget you can modify the object in the LDAP Client and test in that way.

The code is below and should be self explanatory.

JAXB - XML to Java in 2 lines of Code!

A couple of weeks ago I was presented with a bunch of xml files for which I was also given the xsd document.  The task was to calculate some metrics based on properties of the xml files.

Not too complicated.  All I had to do was:

1. Convert the xml files to Java objects.
2. Write some code to perform a calculation on said objects.

I hadn't actually tried this before but I challenged myself to do this (at least the boilerplate of point 1) in 2 lines of code. Here's how I did it:

Step 1 - Create a Java data model

I created a project in IntelliJ and dropped my xsd document in a resources directory.

I then highlighted the xsd file and clicked Tools->JAXB->Generate Java Code

I was presented with a dialogue box as below

Click OK and hey presto your whole data model has been created into Java objects.

Step 2 - Deserialise the xml file into Java Objects  

//line 1 create an Unmarshaller for the object type you are reading from the xml file

Unmarshaller um = JAXBContext.newInstance(DataObject.class).createUnmarshaller();

//line 2 deserialise the xml file into a java object

DataObject dataOject = (DataObject)um.unmarshal(new FileReader("DataObject.xml"));

//now run the calculation on dataObject

So that's it - only took 2 lines!

Chronicle-Bytes vs nio ByteBuffer: A worked example

Chronicle-Bytes is an Open Source project under the Chronicle group of technologies.

It has got some really interesting features and I would definitely recommend it if:

• you find yourself at all frustrated with java.nio.ByteBuffer 
• you are in low latency environment and want to avoid allocation

Some of the extensions over java.nio.ByteBuffer are explained in the README for the project.

This is a worked example comparing Chronicle-Bytes with the standard java.nio.ByteBuffer.

The main class in Chronicle-Bytes is  net.openhft.chronicle.bytes.Bytes it extends the functionality found in java.nio.ByteBuffer.

Let's see this in action with the example I introduced in this post where we saw how to prepend into Chronicle Bytes. 

This is the scenario:

You have a variable length message that needs to be written to a buffer. You need to prepend the message with its length so you know how far to read.  The length of that message needs to be written out to the buffer in text not binary (typical for FIX style message formats).

e.g.  The resulting buffer would look like this 11 hello world or 5 hello


Let's first look at the working this example through using java.nio.ByteBuffer.

Interspersed through the code we'll debug example Hello World in red.  We'll track the progress of the buffer through the program as follows:

• hyphens - signify empty bytes, 
• single | mark the position

//For the sake of this example we will use a StringBuilder with "Hello World"

//but assume this StringBuilder can be passed into a function and could contain

//any amount of text

StringBuilder sb = new StringBuilder("Hello World");

//For the sake of this example we have allocated a buffer of 20 bytes but in reality

//if the string buffer could be any size we would have to be careful how we sized

//the buffer.  This is one of the advantages of the elasticByteBuffer in Chronicle-Bytes
//which automatically resizes. 

ByteBuffer buffer = ByteBuffer.allocateDirect(20);

//Make sure the buffer is cleared from the last run - assuming this is called in a loop

bytes.clear();
|--------------------

//We need to convert the string into a byte array to add to the buffer later

//NOTE: This line will be a serious source of garbage

byte[] bytes = sb.toString().getBytes(StandardCharsets.UTF_8);

//Now work out the length which is the first thing to be written into the buffer
//Again in this line we will see garbage being produced.  
//Chronicle-Bytes has a direct method for writing numbers as text. 

buffer.put(Integer.toString(bytes.length).getBytes());
11|------------------

//Add a separator between the length and the message
buffer.put((byte) ' ');
11 |-----------------
//Add the bytes for the message

buffer.put(bytes);
11 Hello World|------
//Flipping is required to set into read mode.  You will see that in Chronicle-Bytes
//none of this is sort of thing is required because it has the concept of readPosition()
//and writePosition()

buffer.flip();

|11 Hello World------

//Now read the contents of the buffer

int number = 0;
//You have no option but to read the buffer byte by byte until you hit the
//space which is the end of the length.

byte b = buffer.get();

while(b >= '0' && b<='9'){

  number *= 10;

  number += b - '0';

  b = buffer.get();

}
11 |Hello World------
//We know how many bytes to read until we reach the end of the message

byte[] dst = new byte[number];

buffer.get(dst);
11 Hello World|------

//There is no way to read them directly into a String so we have no alternative
//but to create a new String for the bytes.

System.out.println(number + ":" + new String(dst, StandardCharsets.UTF_8));

This is the standard Java NIO way of working through the example.  It's slightly clunky in places but worst of all it's going to produce a lot of garbage.  As we know (and I've written about many times in this blog) garbage is the enemy of real time latencies.

Now let's work through the example with Chronicle-Bytes.

We will use the exactly the same example and follow the progress of the buffer as we step through the code. This time however we have both a readPosition and a writePosition:

• hyphens - signify empty bytes, 
• single | mark the write position 
• double || mark the read position.

//For the sake of this example we will use a StringBuilder with "Hello World"

//but assume this StringBuilder can be passed into a function and could contain

//any amount of text

StringBuilder sb = new StringBuilder("Hello World");

//Using Bytes you don't need to worry about the length of your buffer - it will resize dynamically
//This really makes life easier.
Bytes bytes = Bytes.elasticByteBuffer();
|||--------------------
//Clear and pad allows you to leave room in your buffer so that you can write into the start of the
//buffer. For more information on prepending see here.
bytes.clearAndPad(8);
--------|||------------
//Bytes supports writing UTF8 from a CharSequence like a StringBuilder.  No need for
//any object creation here!
bytes.appendUtf8(sb);
--------||Hello World|-
//Effectively moves the readPosition backwards
bytes.prewriteByte((byte) ' ');
-------|| Hello World|-
//Bytes allows you write numbers directly as text!
bytes.prepend(sb.length());
-----||11 Hello World|-
//Note how there is no flip() required


//Bytes allows you to read text direct as a long - no object creation necessary
length = (int)bytes.parseLong();
-----11 ||Hello World|-
//Bytes allows you to read bytes straight into a CharSequence as UTF8
bytes.parseUTF(sb, length);
-----11 Hello World|||-
System.out.println(length + ":" + sb);


How do they Perform

I created a JMH benchmark to compare to compare Chronicle-Bytes with ByteBuffer (the full code listing for this is at the end of the post). 

As you can see from below Chronicle-Bytes runs almost twice as fast.

Benchmark                           Mode     Cnt    Score   Error  Units
BytesPerfTest.testByteBuffer      sample  451723  302.436 ± 8.260  ns/op

BytesPerfTest.testChronicleBytes  sample  418022  179.129 ± 1.307  ns/op

If you run the test with -verbosegc you will notice that whilst there are hundreds of collections triggered by testByteBuffer, testChronicleBytes produces triggers no collections at all.  This means that were you to account for coordinated omission in your tests the results would be far worse for ByteBuffer (for more details on coordinated omission see here).  

Conclusion

You should be able to see that Chronicle-Bytes is not only a great way to avoid object creation but is actually also simpler and more intuitive to use the the standard java.nio.ByteBuffer.

The Anatomy of a Google Community: Introducing the Chronicle User's Group

2 years ago Peter Lawrey CEO of Chronicle created the Performance Java User's Group. This is a Google community dedicated to sharing information in the form of blog posts, videos or presentations related to extreme Java performance.

The community has exceeded expectations in terms of both the number of people who have joined (currently standing at over 2000) and in the quality of the contributions received from its members.

So what are Google communities?  And why do we need yet another forum of sharing data.

Google communities are not question and answer forums or blogs - they have their own characteristics which I'll try and explain, at least from my experience.

Question and answer groups are aimed at solving individual's problems.  An individual will reach out to the community for help on a particular subject. This might turn out to be useful to other people but for most of the time, most members won't be interested in seeing most of the contributions to the forum.  You can describe this as a one to few relationship. One individual raises a subject which is interesting to a few within the community. StackOverflow would be an example of this.

Blogs are an individual's thoughts and perspectives on a particular subject. Subscribers will be interested in all the content but it is coming from only one individual.  Whether these blogs are from an individual or a company they are presenting a very curated moderated stream of data which by definition will lack a holistic communal perspective. You can characterise this as a one to many community. An individual reaches out with information which should interest all their community.  This blog or a FaceBook page would be an example of this.

Communities present information, which unlike Q&A groups should be interesting to all its members.  However communities unlike blogs share information from the community about a subject.  This leads to a much broader perspective and richer pool of data from which can be drawn.  This can be characterised as a many to many relationship.

Today we announce the Chronicle User's Group a forum to discuss Chronicle technology. We have bootstrapped it with about 20/30 relevant articles.  We will continue to update it with interesting information about Chronicle but are intending that users of Chronicle technology will also share their experiences on the group.  

We often come across Chronicle users happy to blog about how they implemented Chronicle into their projects.  This would be an ideal place to share these posts.  They would be of great benefit, not only us as stewards of Chronicle but to the rest of the community.

Even if you haven't actually used Chronicle software we will share some of the techniques behind our software. You might well find useful and can apply to other software projects especially if you are involved in performance Java applications.

We welcome you all to come and join this community.

Chronicle-Bytes: New Feature Prepending Bytes

Chronicle-Bytes is an Open Source project under the Chronicle group of technologies.

It has got some really interesting features and I would definitely recommend it if:

• you find yourself at all frustrated with java.nio.ByteBuffer 
• you are in low latency environment and want to avoid allocation

Some of the extensions over java.nio.ByteBuffer are explained in the README for the project.

If you are new to Chronicle-Bytes and want to see a worked example comparing Chronicle-Bytes with java NIO ByteBuffer see this article.

Chronicle-Bytes just introduced a new feature - 'prepending' bytes.

Chronicle-Bytes is one of the key building blocks we used in creating Chronicle-FIX. You can read more about it and the performance we achieved here. This is one of the features we added to Chronicle-Bytes whilst building Chronicle-FIX which allowed us to hit the numbers quoted in the article.

Let's say you have the following scenario (You find this sort of construct in FIX messages).

• You need to write a variable length message into a buffer. 
• Preceding the message you need to write the message length to the buffer. 
• The length must be written in text

e.g.  11 hello world or 5 hello

The problem here is that until you've written the message into the buffer you can't know how long the message is. So you don't know how much space will be required to write out the length which must precede the message.  e.g. If the message is less than 10 bytes the space required for the length will be 1 byte. If the message length is from 10-99 bytes the space required for the length will be 2 bytes etc. 

Before this new feature this is the code you would have had to have written:

(Example debug Hello World in red 

hyphens - signify empty bytes, 

single | mark the write position 

double || mark the read position).

StringBuilder sb = new StringBuilder("Hello World);

Bytes bytes = Bytes.elasticByteBuffer();

bytes.clear();

|||---------------------

//leave 8 bytes for writing the length

bytes.writeLong(0);

||--------|-------------

//remember the position at which you started writing

long pos = bytes.writePosition();

pos=8

//write the string to the buffer

bytes.appendUtf8(sb);

||--------Hello World|--

//remember the position you finished writing

long pos2 = bytes.writePosition();

pos2=19

//work out how many bytes you require to store the length

int sblen = sb.length();

int numlen = 1;

while(sblen > 9){

   numlen++;

   sblen /= 10;

}

sblen=2

bytes.writePosition(pos - numlen - 1);

||-----|---Hello World--

bytes.append(sb.length());

||-----11|-Hello World--

bytes.append(' ');

||-----11 |Hello World--

bytes.writePosition(pos2);

||-----11 Hello World|--

bytes.readPosition(pos - numlen - 1);

-----||11 Hello World|--

Job done - but a lot of work!

Now look at the alternative when you can use prepending.

bytes.clearAndPad(8);

//moves the read and write position to 8

--------|||-------------

bytes.appendUtf8(sb);

//normal append to buffer

--------||Hello World|--

bytes.prewriteByte((byte) ' ');

//prewriteByte writes the byte before the read position

//and then moves the read position back the number of bytes.

-------|| Hello World|--

bytes.prepend(sb.length());

//prepend writes the long (in text) backwards before the read //position it then moves the read position back the number of bytes.

-----||11 Hello World|--

That's a really massive saving of effort allowing you to very efficiently write to byte buffers.