Monday 30 March 2015

ChronicleMap - Java Architecture using Off Heap Memory

My last post was written a couple of weeks ago and after some valid feedback I'd like to clarify a couple of points as a preface to this article.

The main takeaway from 'Creating millions of objects with Zero Garbage' should be that with Chronicle you are not 'limited' to using jvm allocated on-heap memory when writing a Java program. Maybe the article would have been more aptly titled 'Creating Millions of Objects using Zero Heap'. Another point I wanted to bring out was that when you have no heap memory you cause no GC activity. 

A source of confusion came from the fact I used the term 'garbage' to describe the objects allocated on the heap. The objects allocated were actually not garbage though they caused GC activity. 

I contrived an example to demonstrate, one, that ChronicleMap does not use heap memory whilst ConcurrentHashMap does, and two, that when you use heap memory you can't ignore the GC. At the very least you need to tune your system carefully to ensure that you don't end up suffering from long GC pauses. This does not mean that there are no issues with allocating from off heap (see the end of this post) and it also does not mean that you can't tune your way through an on heap solution to eliminate GC. Going off heap is by no means a panacea to all Java performance issues but for very specific solutions it can deliver interesting opportunities some of which I will discuss in this post.

Let's examine this problem:

You have multiple JVMs on your machine and need to share data between them. You would also like that data to be persisted so that when the JVMs exits the data does not disappear.

Let's simplify for now and say that you have two JVMs running on the same machine, either or both of which would like to see updates from the other. Each Java program has a ConcurrentHashMap which it updates, those updates are stored and are of course available to it later. But how does the program get the updates applied by the other Java program to its map? And how do we prevent the data from evaporating when the programs exit?

Fundamentally, JDK on-heap collections such as HashMap and ConcurrentHashMap can't be shared directly between JVMs.  This is because heap memory is contained by the JVM through which it was allocated. Additionally when the JVM exits, the memory is released and the data is no longer available, there is no implicit way of persisting the memory outside the lifetime of the JVM. So you need to find some other mechanism to share the data between the JVMs and persist the data.  

From an architectural perspective you need accomplish these two tasks.
  1. A mechanism for sharing the data between the JVMs so that when one process updates the other will be informed of the action.
  2. A mechanism for storing the data so that when one or both JVMs exit the data does not just disappear.
Typically you might use a database as an external sharable store and messaging service to send the data updates to other processes to notify them that some data has been updated.

This results in the following architecture:



The problem with this architecture is that use lose the in-memory speeds of a HashMap, especially if writing to your database is not that fast and you want the write to be persisted before you send the message out over the messaging service. Also many solutions will involve TCP calls which can again be a source of latency. There are of course much faster ways to persist data than writing to a fully fledged database using mechanisms like journaling to disk, for example using a product like ChronicleQueue or similar.  But if you did use a journal you'd still have to build all the logic to recreate a Map data structure on restart not to mention having to keep a Map type structure up-to-date on another JVM.  In addition to the latency introduced by this architecture there is the complication of having to deal with the extra code and configuration for the database and messaging service. 

Even accepting that this sort of functionality can be wrapped up in frameworks, wouldn't it be great if your in memory Map was actually visible outside your JVM.  The Map should be able to implicitly persist the data so that its data is available independently of the life time of the JVM. It should allow access with the same 'memory' speeds as you might achieve using an on heap Map.

This is where ChronicleMap comes in.  ChronicleMap is an implementation of java.util.ConcurrentMap but critically it uses off heap memory which is visible outside the JVM to any other process running on the machine. (For a discussion about on-heap vs off-heap memory see here). 

Each JVM will create a ChronicleMap pointing at the same memory mapped files. When one process writes into its ChronicleMap the other process can instantly (~40 nanoseconds) see the update in its ChronicleMap. Since the data is stored in memory outside the JVM, a JVM exit will not cause any data to be lost.  The data will be held in memory (assuming there was no need for it to be paged out) and when the JVM restarts it can map it back in extremely quickly.  The only way data can be lost is if the OS crashes whilst it has dirty pages that haven't been persisted to disk. The solution to this is use replication which Chronicle supports but is beyond the scope of this post.

The architecture for this is simply this:



For a code example to get started with ChronicleMap see my last post or see the official ChronicleMap tutorial here.

There are a number of caveats and trade-offs to consider before diving into ChronicleMap.
  • The ChronicleMap entries have to be Serializable.  For systems that are very sensitive to performance you will need to implement the custom serialisation provided by Chronicle known as BytesMarshallable. Whilst this is pretty easy to implement it is not something that is necessary with an on-heap map. (Having said that storing data into a database will of course also require some method of serialisation.) 
  • Even with BytesMarshallable serialisation, the overhead of any serialisation might be significant to some systems. In such a scenario it is possible to employ a zero copy technique supported by Chronicle (see my last blog post for more details) to minimise the costs of serialisation. It is however a little trickier to implement than using 'normal' Java.  On the other hand in latency sensitive programs it will have the huge benefit of not creating any objects that might then later need to be cleaned up by the GC.
  • A ChronicleMap does not resize and must therefore be sized up front. This might be an issue if you have no idea how many items to expect.  It should be noted however, that oversizing, at least on Linux, is not a huge problem as Linux passively allocates memory. 
  • Chronicle relies on the OS to asynchronously flush to disk. If you want to be absolutely sure that data has actually been written to disk (as opposed to just being held in memory) you will need to replicate to another machine. In truth any mission critical system should be replicating to another machine so this might not be a big issue in adopting Chronicle.
  • ChronicleMap will be subject to OS memory paging issues. If memory is paged out and has to be swapped back in latency will be introduced into the system. Therefore even though you will be able to create ChronicleMaps with sizes well in excess of main memory, you will have to be aware that paging might occur depending on your access patterns on the data. 



49 comments:

  1. Thanks for the article.It is useful.

    ReplyDelete
  2. Technical courses of bachelor in architecture and masters in architecture are the usual degrees that are on offer to be studied.
    showboxdownloadsapp.com

    ReplyDelete
  3. Technical courses of bachelor in architecture and masters in architecture are the usual degrees that are on offer to be studied.
    Lucky Patcher
    teclast t10

    ReplyDelete
  4. pleasant post, stay aware of this fascinating work. It truly regards realize that this subject is being secured likewise on this site so cheers for setting aside time to talk about this! www.martynpattie.co.uk

    ReplyDelete
  5. The reasoning and rationale behind this blogs makes it a brilliant read.
    go to my site

    ReplyDelete
  6. Why do only so much written on this subject? Here you see more.
    clicking here

    ReplyDelete
  7. Use dual account with GBwhatsapp Latest Application With free of cost. Gbwhatsapp

    ReplyDelete
  8. Download Morpheus TV App and watch all movies and shows with great quality.

    ReplyDelete
  9. Even accepting that this sort of functionality can be wrapped up in frameworks, wouldn't it be great if your in memory Map was actually visible outside your JVM. The Map should be able to implicitly persist the data so that its data is available independently of the life time of the JVM. It should allow access with the same 'memory' speeds as you might achieve using an on heap Map. GBWhatsapp APK Download Latest Version

    ReplyDelete
  10. I am looking for and I love to post a comment that "The content of your post is awesome" Great work!
    Hardwood Flooring

    ReplyDelete
  11. This comment has been removed by the author.

    ReplyDelete
  12. My heartiest congrats to the author on composing this.
    southend roofing

    ReplyDelete
  13. thanks for sharing this great about java. keep it up.
    APENTAL CALC APP
    Apental calc app. It is a wonderful app that lets you get hundreds of followers on Facebook for free. The application work to promote your Facebook account by providing likes and comments on your post. Download it now.
    apental calc
    wefbee apk latest version

    ReplyDelete
  14. Most of the time I don’t make comments on websites, but I'd like to say that this article really forced me to do so. Really nice post! 62 niche related backlink blog comment

    ReplyDelete
  15. Thanks a lot for sharing us about this update. Hope you will not get tired on making posts as informative as this. off page seo

    ReplyDelete
  16. that soon only the rich and super rich will be able to afford even those items considered necessities, it is wise to save and get out of debt right now. triple twin bunk bed

    ReplyDelete
  17. Hey! I just wanna say that you are sharing a quality and useful content with us. This kind of content is difficult to find it and you are sharing FMWhatsApp is very good. Keep up the high work.

    ReplyDelete
  18. Thanks for sharing this very use full information. Also check Download GBWhatsApp APK

    ReplyDelete
  19. Thank you for sharing this. It was helpful. Keep sharing such things.
    Also I like to play.SolidWorks Premium

    ReplyDelete
  20. I was looking for Java Architecture using Off Heap Memory. thank you for the complete information. COC Mod APK

    ReplyDelete

  21. Wow! very beautiful, great, and good post. I like it very much.
    soft-organizer-pro

    ReplyDelete
  22. Many thanks for the shared this informative and interesting post with me.Driver Talent Pro

    ReplyDelete
  23. I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.
    Microsoft Office 2007

    ReplyDelete
  24. A good blog always brings new and exciting information, and as I read, I feel that this blog really has all those qualities that qualify a blog to be one.
    Product Key for Microsoft Office 2007

    ReplyDelete
  25. That's an outstanding piece of work!I look forward to seeing more!
    mirillis patch

    ReplyDelete
  26. I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post. crackcool.com

    ReplyDelete
  27. Really very nice information on this site. Thanks for sharing this nice information. I hope you'll continue to write like this in the future.
    SAM BroadCaster

    ReplyDelete
  28. You explained & solved this question very clearly. It was so easy to get through this blog. Thank you. fmwhatsapp

    ReplyDelete
  29. That is one of the most crucial pieces of information for me. And I've enjoyed reading your article.
    However, I'd like to make a few general remarks: The website's design is fantastic, and the posts are truly good : D.
    Best of luck with your endeavours.
    bulk image downloader crack
    netbalancer activation code download
    nero burning rom crack
    ntlite license key latest version

    ReplyDelete
  30. Many thanks for the shared this informative and interesting post with me.

    ReplyDelete

  31. Most of the people who have used this software around the world love works of art. This software was released 29 years ago. It is the most powerful, innovative and creative feature that professionals need. Powerful design software built for photographers and artists. GTA Vice City Highly compressed Ispoofer Crack boris fx license key crack wondershare mobiletrans cracked Alien skin blow up Crack Best Korean Action Movies macdrive Pro ZModeler Free macdrive Pro snapgene license keysoftware is essentially designed for graphic designers. Adobe Photoshop CC is photo editing software designed for professionals and artist-designers. This software has a creative cloud service and creative tools to enhance your images. About this software, we provide information on the fact that it has many cutting-edge features and also get into learning applications with the learning panel.

    ReplyDelete
  32. It is really enjoyable to visit your website because you have such an amazing writing style.
    Neural Fortin Nameless Crack

    ReplyDelete
  33. Yes you heard it right I almost used every WhatsApp mod and checked all mods themes features and I found Amazing themes those are better than other WhatsApp Mods in FMWhatsApp.

    ReplyDelete
  34. Your Blog is awesome. Really Admire your work!
    Veeam Backup & Replication Activation Key is amazing software for professional and personal use. Thanks to its simple and intuitive interface, anyone can use it with little knowledge.
    I can say that this software did not disappoint you at all. Anyone can download and use it for free.

    ReplyDelete
  35. I like your all post. You have done really good work. Thank you for the information you provide, it helped me a lot. I hope to have many more entries or so from you.
    Very interesting blog.https://vstmania.net/
    cracxfree.com Crack
    MacDrive Crack
    ShieldApps Webcam Blocker Premium Crack
    VyprVPN Crack

    ReplyDelete
  36. Hi dear,It is really enjoyable to visit your website because you have such an amazing writing style.
    Minitool Partition Wizard Crack

    ReplyDelete
  37. I like your all post. You have done really good work. Thank you for the information you provide, it helped me a lot. I hope to have many more entries or so from you.
    Very interesting blog.(https://crackplus.org/adobe-acrobat-pro-dc-crack/)
    https://crackplus.org/ibm-spss-statistics-crack/
    Macdrive Pro Crack
    Acronis True Image Crack
    Loaris Trojan Remover Crack
    Alien Skin Blow Up Crack
    McAfee LiveSafe Crack

    ReplyDelete
  38. I must say it is very informative as well as interesting. Thanks to the author of this post/page for writing such wonderful lines. abelssoft-filefusion-crack/

    ReplyDelete
  39. MY BLOG support of this software, users no longer have to register on several MY LINK websites at exactly the same time to control investments, and they don’t have to think about a large number of passwords that keep theirwebsite information safe. With the entire program, customers can quickly access their savings, credit card information, investments, loans or retirement accounts, all from one place. Website My

    ReplyDelete
  40. I am very happy to read this article. Thanks for giving us Amazing info. Fantastic post.
    Thanks For Sharing such an informative article, Im taking your feed also, Thanks.outotec-hsc-chemistry-crack/

    ReplyDelete
  41. The drawback of this design is that you give up a HashMap's in-memory performance, which is an issue if you need to write to a database quickly and want the write to be saved before sending the message via the messaging service. Additionally, a lot of solutions will entail TCP calls, which once again might add delay. There are, of course, many quicker methods to persist data than by employing mechanisms like journaling to disk, for instance by using a program like ChronicleQueue or something such. However, if you utilized a journal, you'd also need to implement all the logic necessary to construct a Map data structure upon restart and maintain a Map type structure.
    gta 5 apk

    ReplyDelete