Java Iterator backed by network connection

Question

With some frequency I find myself writing APIs that offer an Iterator<Foo> which is backed by a network connection. The implementation opens a network connection, reads information off the stream, and deserializes that information into Foos to pass on to the caller. Unfortunately there is always the possibility of an IOException and also the need to close the network connection gracefully (that can be made automatic when the caller reads the last Foo, but what if that doesn't happen?).

There are already a couple of questions (here and here) about how to deal with checked exceptions that would be thrown in an implementation of Iterator, and the accepted advice is "wrap them in unchecked RuntimeExceptions". Meanwhile to allow closing the network connection we can implement Closeable. So we end up with something like this for a well-behaved exception-checking caller:

Iterator<Foo> iter = null;
try {
    iter = getFooIterator();
    while(iter.hasNext()) {
        Foo foo = iter.next();
        // do something with foo
    }
}
catch(RuntimeException e) {
    if(e.getCause() instanceof IOException) {
       // do something with the IOException
    }
    else throw e;
}
finally {
    if(iter instanceof Closeable) try { ((Closeable)iter).close(); } catch(IOException e) {}
}

And it seemed like such a nice idea to implement Iterator. Is there a better way?

If you're working with Java 7 you could also consider using AutoCloseable and open the resources in the try statement, e.g. try(iter = getFooIterator();) { ... } (the iterator would have to implement AutoCloseable though). — Thomas
– Thomas, Commented Jan 24, 2012 at 22:06
You might also want to consider closing the connection in the iterator's methods (e.g. in next() or hasNext()) and then throw an exception. That way the user of the iterator would not have to worry about closing the iterator (and thus the underlying connection) and just would have to handle the exception. — Thomas
– Thomas, Commented Jan 24, 2012 at 22:10
Have to admit that I am not a big Iterator guy (I use Collections and simple arrays a ton). Why do you want to return an Iterator, with these open-ended questions, instead of a Collection, which would be a simpler API? You get back the Collection, and everything is closed and any Exceptions that are going to happen happen. I think I know the answer, but want to ask... — user949300
– user949300, Commented Jan 24, 2012 at 22:49
With an Iterator the caller can get to work on the Foos right away. That could make a significant difference if there are 1000s of Foos coming over a slow network connection. — Robert Tupelo-Schneck
– Robert Tupelo-Schneck, Commented Jan 25, 2012 at 15:48
That's what I thought. You could return a Collection that is a BlockingQueue. But not sure if that would make much of a difference compared to an Iterator. — user949300
– user949300, Commented Jan 26, 2012 at 3:14

Dave Newton · Accepted Answer · 2012-01-24 22:23:00Z

1

IMO the first step would be to wrap it up in an implementation- or app-specific exception, eliminating the need for catching generic RuntimeExceptions or checking the root cause.

I'd consider a specific implementation to avoid the closeable check and to wrap up the IOException.

NetworkIterator<Foo> iter = null;
try {
    iter = getFooIterator();
    while (iter.hasNext()) {
        Foo foo = iter.next();
        // do something with foo
    }
} catch (NetworkIteratorExceptiom e) {
    // do something with the IOException
} finally {
    iter.close();
}

I probably wouldn't give it method to make the boilerplate go away, but I'd be tempted; roughly:

NetworkIterator<Foo> iter = new IteratorThang<Foo>() {
    @Override public void each(Foo foo) {
        // Do something with foo
    }
};

edited Jan 24, 2012 at 22:23

answered Jan 24, 2012 at 22:12

Dave Newton

161k27 gold badges263 silver badges310 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Robert Tupelo-Schneck Over a year ago

Good ideas. In fact, I wonder if there is any real benefit to even implementing java.util.Iterator, instead of having a specific Iterator-alike which can throw checked exceptions. Are there many consumers of Iterators?

Dave Newton Over a year ago

@RobertTupelo-Schneck All sorts of things consume iterators; consider something like Guava's any method--there are lots of usecases for lazy iteration like this. It's basically the same as long computations that don't generate subsequent values until they're actually required (possibly caching)--it's just from the network instead of math (or whatever).

Collectives™ on Stack Overflow

Java Iterator backed by network connection

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related