Exceptions in Lambdas: An Elegant Solution to a Bit of a Mess

Consider the following function for writing to a file:

The idea behind the method is to allow the user to pass in different implementations of InputStream to the method so that writeToFile can be called for example with a GZIPOuputStream, SnappyOuputStream (fast compression) or simply a plain FileInputStream.


It's a neat function which can be called like this:


Unfortunately as pointed out in the comment this does not compile!  The reason it doesn't compile is because the GZIPOutputStream throws an IOException in its constructor. What would have been nice was if the IOException was thrown out of the lambda and could then be dealt with in the try catch block - but that's not how lambdas work :-(

This is in fact how you have to deal with the exception to get the code to compile:


Not only is this ugly but you are left with the rather awkward problem of what to do with the IOException. In this case we have just re-packaged inside an AssertionError. See my previous post 'Cheating with Exceptions' on the correct way to handle this scenario. 

But there is a solution to this problem. Rather than using a java.util.function.Function that takes a value and returns a value, we can create a custom function that takes a value returns a value and throws an Exception. In this way the client code of writeToFile is nice and clean and can deal with the exception in a natural way. Moreover, lambdas are now used in the way they were intended to make our code prettier and easier to understand.

See full code listing below:

Cheating with Exceptions - Java 8 Lambdas

Leaving aside the religious debate about Checked vs Runtime exceptions, there are times where due to poorly constructed libraries, dealing with checked examples can drive you insane.

Consider this snippet of code which you might want to write:

public void createTempFileForKey(String key) {
  Map<String, File> tempFiles = new ConcurrentHashMap<>();
  //does not compile because it throws an IOException!!
  tempFiles.computeIfAbsent(key, k -> File.createTempFile(key, ".tmp"));
}

For it to compile you need to catch the exception which leaves you with this code:

public void createTempFileForKey(String key) {
    Map<String, File> tempFiles = new ConcurrentHashMap<>();
    tempFiles.computeIfAbsent(key, k -> {
        try {
            return File.createTempFile(key, ".tmp");
        }catch(IOException e) {
            e.printStackTrace();
            return null;
        }
    });
}

Although this compiles, the IOException has effectively been swallowed.  The user of this method should be informed that an Exception has been thrown.

To address this you could wrap the IOException in a generic RuntimeException as below:

public void createTempFileForKey(String key) throws RuntimeException {
    Map<String, File> tempFiles = new ConcurrentHashMap<>();
    tempFiles.computeIfAbsent(key, k -> {
        try {
            return File.createTempFile(key, ".tmp");
        }catch(IOException e) {
            throw new RuntimeException(e);
        }
    });
}

This code does throw an Exception but not the actual IOException which was intended to be thrown by the code. It's possible that those in favour of RuntimeExceptions only would be happy with this code especially if the solution could be refined to created a customised IORuntimeException.  Nevertheless the way most people code, they would expect their method to be able to throw the checked IOException from the File.createTempFile method.   

The natural way to do this is a little convoluted and looks like this:

public void createTempFileForKey(String key) throws IOException{
        Map<String, File> tempFiles = new ConcurrentHashMap<>();
        try {
            tempFiles.computeIfAbsent(key, k -> {
                try {
                    return File.createTempFile(key, ".tmp");
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            });
        }catch(RuntimeException e){
            if(e.getCause() instanceof IOException){
                throw (IOException)e.getCause();
            }
        }
}

From inside the lambda, you would have to catch the IOException, wrap it in a RuntimeException and throw that RuntimeException. The lambda would have to catch the RuntimeException unpack and rethrow the IOException. All very ugly indeed!

In an ideal world what we need is to be able to do is to throw the checked exception from within the lambda without having to change the declaration of computeIfAbsent. In other words, to throw a check exception as if it were an runtime exception. But unfortunately Java doesn't let us do that...

That is not unless we cheat! Here are two methods for doing precisely what we want, throwing a checked exception as if it were a runtime exception.

Method 1 - Using generics:

    public static void main(String[] args){
        doThrow(new IOException());
    }

    static void doThrow(Exception e) {
        CheckedException.<RuntimeException> doThrow0(e);
    }

    static <E extends Exception>
      void doThrow0(Exception e) throws E {
          throw (E) e;
    }

Note that we have create and thrown an IOException without it being declared in the main method.

Method 2 - Using Unsafe:

public static void main(String[] args){
        getUnsafe().throwException(new IOException());
    }

    private static Unsafe getUnsafe(){
        try {
            Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe");
            theUnsafe.setAccessible(true);
            return (Unsafe) theUnsafe.get(null);
        } catch (Exception e) {
            throw new AssertionError(e);
        }
    }

Again we have managed to throw an IOException without having declared it in the method.

Whichever method you prefer we are now free to write the original code in this way:

public void createTempFileForKey(String key) throws IOException{
        Map<String, File> tempFiles = new ConcurrentHashMap<>();

        tempFiles.computeIfAbsent(key, k -> {
            try {
                return File.createTempFile(key, ".tmp");
            } catch (IOException e) {
                throw doThrow(e);
            }
        });
    }
    
    private RuntimeException doThrow(Exception e){
        getUnsafe().throwException(e);
        return new RuntimeException();
    }

    private static Unsafe getUnsafe(){
        try {
            Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe");
            theUnsafe.setAccessible(true);
            return (Unsafe) theUnsafe.get(null);
        } catch (Exception e) {
            throw new AssertionError(e);
        }
    }

The doThrow() method would obviously be encapsulated in some utility class leaving your code in createTempFileForKey() pretty clean.

Java 8 Lambdas in One Line

If you understand this line, or better still can write this code you can pretty much say that you have understood the essence of Java 8 Lambdas. Certainly in as much as they can be used with collections.

I found this in a recent presentation by Peter Lawrey.  (Definitely worth watching the whole presentation when you have a spare hour.)

Anyway the task was to find the 20 most frequent words in a file:

As you can see, with Java 8 this can actually be done in a single (albeit rather long) line. 

If you're not used to lambdas the code might seem a little scary but actually it's pretty declarative and when you get past the logic reads fairly easily.

Java8 Multi-threading ForkJoinPool: Dealing with exceptions

One of the main motivations behind the introduction of Java8 lambdas was the ability to be able to use multicores as easily as possible (see Mastering Lambdas: Java Programming in a Multicore World).  By simply changing your code from collection.stream()... to collection.parallelStream()... you have instant multi-threading at your disposal which brings with it all the CPU power on your machine.  (Let's ignore contention at this point.)

If you print out the names of the threads used by parallelStream you will notice that they are the same threads used by the ForkJoin framework and look something like this:

[ForkJoinPool.commonPool-worker-1]
[ForkJoinPool.commonPool-worker-2]

See Benjamin Winterberg's blog for a nicely worked example of this.

Now in Java 8 you can use this commonPool directly with the new method on ForkJoinPool commonPool().  This returns an instance of ForkJoinPool (which is an ExecutorService) with the commonPool of threads - the same ones that are used in parallelStream. This means that any work you do directly with the commonPool will play very nicely with work done in parallelStream especially the thread scheduling and work stealing between threads.

Let's work through an example of how you use ForkJoin especially in dealing with the tricky subject of exceptions.

Firstly obtain an instance of the commonPool by calling ForkJoin.commonPool().  You can submit tasks to it using the submit() method. Because we are using Java8 we can pass in lambda expressions which is really neat.  As with all ExecutorService implementations you can pass either instances of Runnable or Callable into submit().  When you pass a lambda into the submit method it will automatically turn it into a Runnable or a Callable by inspecting the method signature.

This leads to an interesting problem which highlights how lambdas work.  Supposing that you have a method of return type void (like a Runnable) but throws a checked exception (like a Callable).  See the method throwException() in the code listing below for such an example.  If you write this code it won't compile.

Future task1 = commonPool.submit(() -> {
            throwException("task 1");
        });

The reason for this is that the compiler assumes, because of the void return type, that you are trying to create a Runnable.  Of course a Runnable can't throw an Exception.  To get around this problem you need to force the compiler to understand that you are creating a Callable which is allowed to throw an Exception using this code trick.

Future task1 = commonPool.submit(() -> {
            throwException("task 1");
            return null;
        });

This is a bit messy but does the job. Arguably, the compiler, could have worked this out itself.

Two more things to highlight in the full code listing below.  One, the fact that you can see how many threads are going to be available in the pool using commonPool.getParallelism(). This can be adjusted with the parameter '-Djava.util.concurrent.ForkJoinPool.common.parallelism'. Two, notice how you can unwrap the ExecutionException so that your code can just present an IOException to its callers rather a rather non-specific ExecutionException.  Also note that this code fails on the first exception.  If you want to collect all the exceptions you would have to structure the code appropriately, possibly returning a List of Exceptions.  Or maybe more neatly throwing a custom exception containing a list of underlying exceptions. 

public class ForkJoinTest {
    public void run() throws IOException{
        ForkJoinPool commonPool = ForkJoinPool.commonPool();

        Future task1 = commonPool.submit(() -> {
            throwException("task 1");
            return null;
        });
        Future task2 = commonPool.submit(() -> {
            throwException("task 2");
            return null;
        });

        System.out.println("Do something while tasks being " +
                "executed on " + commonPool.getParallelism()
                + " threads");

        try {
            //wait on the result from task2
            task2.get();
            //wait on the result from task1
            task1.get();
        } catch (InterruptedException e) {
            throw new AssertionError(e);
        } catch (ExecutionException e) {
            Throwable innerException = e.getCause();
            if (innerException instanceof RuntimeException) {
                innerException = innerException.getCause();
                if(innerException instanceof IOException){
                    throw (IOException) innerException;
                }
            }
            throw new AssertionError(e);
        }
    }

    public void throwException(String message) throws IOException,
            InterruptedException {
        Thread.sleep(100);
        System.out.println(Thread.currentThread() 

            + " throwing IOException");
        throw new IOException("Throw exception for " + message);
    }

    public static void main(String[] args) throws IOException{
        new ForkJoinTest().run();
    }
}

Book review: Mastering Lambdas: Java Programming in a Multicore World (Maurice Naftalin)

Java 8 with its introduction of lambda expressions is the biggest change to the Java language since it was created some 20 years ago. No escaping from it - if you want to stay relevant as a Java programmer you will need to know about lambda expressions and it is certainly worth the effort to really get to grips with the new semantics and paradigms that Java 8 delivers.

Naftalin's book is actually the 3rd book I've read on the subject. The first 2, whilst reasonably well written, didn't really deliver much more than I could have learnt by reading the numerous tutorials out there on the web. Any Java 8 tutorial will explain how to use Lambdas in your code and if that's all you want then you don't really want to buy a book at all. When I buy a book I expect a lot more than that. I expect a beginning, a middle and an end. And to this expectation Naftalin delivers beautifully. He explains the rationals and motivations of Lambda expressions. Are they just a sprinkling of syntactic sugar? No, he explains, they are much more than that. He sets the scene, as is hinted to in the title of his book 'a multicore world' and explains how Java needs to adapt to continue to hold its own so that it will still be one of the most important languages for the next 20 years.

I've been using Java 8 for the last 6 months and am fairly well aquatinted with the syntax but when I read his description of the syntax I could only wish that I had this book 6 months ago! Naftalin has clearly put a huge amount of effort and consideration into how he delivers his ideas, which is why, perhaps, it wasn't one of the first books on the market to greet Java 8 as it appeared in the early part of last year.

Some books go off on a tangent trying to introduce their readers to functional programming through lambdas. I think that is a mistake. I've been a Java programmer since the language emerged and have no wish whatsoever to change my programming style into a functional style. If I would want to do that I would use a functional language like Erlang etc. I want to continue using Java but to introduce some new concepts that can be delivered through Lambdas. The declarative nature of the Lambdas are extremely nice but we don't have to throw out the proverbial baby with the bath water and ditch everything that is good about OOP. I believe that Naftalin shares this opinion as is evident by the way in which he introduces as to practical uses of lambdas in our code.

In conclusion, I would highly recommend recommend this book to any Java developer that wants to get a real understanding of lambdas. The book is extremely readable, not particularly long (all the best books are short) but will tell you everything you will ever need to know about the subject. If you digest all the information in this short book you will be a far better programmer than someone who has just read a few tutorials on the web!

Java8 Lambdas Tip - Collect SortedGroupingBy

Java8 introduces a great new feature which easily allows your code to decompose a List of objects into a Map of Lists of objects keyed on a particular attribute.

This is best shown by example.
Let's say you have a list of Books as below:

public class Book{
    String author;
    Date published;
    int copiesSold;
    String catagory;

    public String getAuthor() {
        return author;
    }

    public Date getPublished() {
        return published;
    }

    public int getCopiesSold() {
        return copiesSold;
    }

    public String getCatagory() {
        return catagory;
    }
}

To group them into a map of authors to books used to be a little bit painful but is now a one liner!

Map<String, List<Book>> authorsToBooks = books
       .stream()
       .collect(Collectors.groupingBy(Book::getAuthor));

The only problem with this that you might have is that the default Map implementation returned is a HashMap which of course is unordered and you might well want to order by the key, in this example by the author.  Of course, you could always sort the Map in a second step but there's a way to do it in one line.

To fix that let's introduce this static utility function:

public static <T, K extends Comparable<K>> Collector<T, ?, TreeMap<K, List<T>>> 
   sortedGroupingBy(Function<T, K> function) {
        return Collectors.groupingBy(function, 
           TreeMap::new, Collectors.toList());
}

We can call it like this:

Map<String, List<Book>> authorsToBooks = books
       .stream()
       .collect(sortedGroupingBy(Book::getAuthor));

This time the map implementation is a TreeMap which has returned the Map of authors to their books in alphabetical order. 

Java 8 Lambdas: Map Reduce the correct way to sum a stream of BigDecimals

Let's start with a simple HashMap of Strings to Doubles as below and put in some values.

Map<String, Double> map = new HashMap<>();
map.put("one", 1.0);
map.put("two", 2.0);
map.put("three", 3.0);

Finding the sum using Java 8 lambdas is pretty straight forward

double sum = map.entrySet().stream().mapToDouble
   (e->e.getValue()).sum();

or using method references
double sum = map.entrySet().stream().mapToDouble
   (Map.Entry::getValue).sum();
What if you wanted to do the same thing for a map of String to BigDecimals?
Not being from a functional background my first instinct (based on the example above) was to do this:
Map<String, BigDecimal> map = new HashMap<>();
map.put("one", BigDecimal.ONE);
map.put("two", BigDecimal.valueOf(2));
map.put("three", BigDecimal.valueOf(3));
BigDecimal sum = BigDecimal.valueOf(map.entrySet().stream().mapToDouble(e->e.getValue().doubleValue()).sum());//DON'T DO THIS!!
This code is hideous, not to mention wrong because you are not actually using the add 
method of the BigDecimal which might be important especially if you are multiplying and 
dividing the values in the stream.
The correct way to sum this stream of BigDecimals is to use the map reduce pattern:
BigDecimal sum = map.entrySet().stream().map
   (e->e.getValue()).reduce(BigDecimal.ZERO, BigDecimal::add);
This is much more elegant.  It does exactly what you want by using the method add on 
the BigDecimal and not just re-creating a BigDecimal after doing the work on doubles.