Monday 27 April 2015

FileSystemMap: A File System backed Map

As part of a project I'm working on at the moment I've been looking at creating a FileSystemMap. I've started a very small GitHub project here to host the code. 

Essentially this map implementation is will allow the user to interact with a directory on their file system as if it were a java.util.Map. Each entry in the map will be a file in that directory, the key will be the file name and the value will be the contents of the file.

This code builds a FileServiceMap and adds five entries:

Map map = new FileSystemMap("/tmp/filetests");
  map.put("one", "one");
  map.put("two", "two");
  map.put("three", "three");
  map.put("four", "four");
  map.put("five", "five");

This results in a directly structure like this:

/tmp/filetests/
 |----- five
 |----- four
 |----- one
 |----- three
 |----- two

Adding and removing entries will change the files in your directory. Changing the value of the entry will cause the file to be re-written with the new value as its contents. For more examples see the code in testMapMethods.

Additionally the FileSystemMap has been designed for two way interaction. Any programmatic updates to it are reflected on the file system and any updates to the file system will be picked up by the map and fired as events.  

This code registers changes to the file system and prints them out:

Map map = new FileSystemMap("/tmp/filetests");
map.registerForEvents(System.out::println);

This is some example output:

FPMEvent{eventType=NEW, programmatic=true, key='one', value='one'}

The eventType is one of: 

  • NEW - a file has been created
  • UPDATE - a file has been modified
  • DELETE - a file has been deleted
The programmatic flag indicates whether it was the FileSystemMap itself that caused the event to be fired. e.g. if put() is called, a file will be created which in turn will cause an event to fired. To avoid feedback it can be useful to know whether it was an operation on the FileSystemMap that triggered the event.

The key is the name of the file that changed.

The value is the latest value associated with the file that has changed.  Note: this may or may not be the value that actually triggered the change.  For example if there were two very fast changes to the entry it is entirely possible that the value for the first event will pick up the value after the second update has already taken place.

e.g. 

map.put("one", "1");
map.put("one", "2");

could produce this output:

FPMEvent{eventType=NEW, programmatic=true, key='one', value='2'}


The first event (triggered by setting to "one" to '1') is picked up but by the time the program has checked the contents of the file the file has changed to '2'. The second event is then picked up (triggered by setting to "one" to '2') but this is suppressed because the value has not changed.  

Some notes:

  1. Files starting with a '.' are ignored. The reason for this is that many editors (including vi) create temporary files which should not be picked up by the FileServiceMap.
  2. If you look at the code you will notice that the WatchService (since Java7) was used to monitor changes to the file system. It is important to understand that the WatchService is OS specific. In particular it doesn't work that well on Mac OSX. This thread discusses some of the issues.  Updates from the WatchService are slow and and fast flowing events can be dropped. In my testing Ubuntu performed significantly better than MacOSX.  However if you're mainly interested in changes to the file system carried out by hand even Mac OSX will be fine.
  3. The Map only supports Strings.

It goes without saying that this class is designed for its specific utility rather than any sort of performance. 

All contributions to this project are welcome!


No comments:

Post a Comment