Monday, 21 September 2015

Chronicle-Bytes: New Feature Prepending Bytes

Chronicle-Bytes is an Open Source project under the Chronicle group of technologies.

It has got some really interesting features and I would definitely recommend it if:
  • you find yourself at all frustrated with java.nio.ByteBuffer 
  • you are in low latency environment and want to avoid allocation
Some of the extensions over java.nio.ByteBuffer are explained in the README for the project.


If you are new to Chronicle-Bytes and want to see a worked example comparing Chronicle-Bytes with java NIO ByteBuffer see this article.

Chronicle-Bytes just introduced a new feature - 'prepending' bytes.

Chronicle-Bytes is one of the key building blocks we used in creating Chronicle-FIX. You can read more about it and the performance we achieved hereThis is one of the features we added to Chronicle-Bytes whilst building Chronicle-FIX which allowed us to hit the numbers quoted in the article.

Let's say you have the following scenario (You find this sort of construct in FIX messages).
  • You need to write a variable length message into a buffer. 
  • Preceding the message you need to write the message length to the buffer. 
  • The length must be written in text
e.g.  11 hello world or 5 hello

The problem here is that until you've written the message into the buffer you can't know how long the message is. So you don't know how much space will be required to write out the length which must precede the message.  e.g. If the message is less than 10 bytes the space required for the length will be 1 byte. If the message length is from 10-99 bytes the space required for the length will be 2 bytes etc. 

Before this new feature this is the code you would have had to have written:
(Example debug Hello World in red 
hyphens - signify empty bytes, 
single | mark the write position 
double || mark the read position).

StringBuilder sb = new StringBuilder("Hello World);
Bytes bytes = Bytes.elasticByteBuffer();

bytes.clear();
|||---------------------
//leave 8 bytes for writing the length
bytes.writeLong(0);
||--------|-------------
//remember the position at which you started writing
long pos = bytes.writePosition();
pos=8
//write the string to the buffer
bytes.appendUtf8(sb);
||--------Hello World|--
//remember the position you finished writing
long pos2 = bytes.writePosition();
pos2=19
//work out how many bytes you require to store the length
int sblen = sb.length();
int numlen = 1;
while(sblen > 9){
   numlen++;
   sblen /= 10;
}
sblen=2
bytes.writePosition(pos - numlen - 1);
||-----|---Hello World--
bytes.append(sb.length());
||-----11|-Hello World--
bytes.append(' ');
||-----11 |Hello World--
bytes.writePosition(pos2);
||-----11 Hello World|--
bytes.readPosition(pos - numlen - 1);
-----||11 Hello World|--

Job done - but a lot of work!

Now look at the alternative when you can use prepending.

bytes.clearAndPad(8);
//moves the read and write position to 8
--------|||-------------
bytes.appendUtf8(sb);
//normal append to buffer
--------||Hello World|--
bytes.prewriteByte((byte) ' ');
//prewriteByte writes the byte before the read position
//and then moves the read position back the number of bytes.
-------|| Hello World|--
bytes.prepend(sb.length());
//prepend writes the long (in text) backwards before the read //position it then moves the read position back the number of bytes.
-----||11 Hello World|--

That's a really massive saving of effort allowing you to very efficiently write to byte buffers.


No comments:

Post a Comment