2

I have the following nested hashmap:

HashMap<Foo1, HashMap<Foo2, TObjectDoubleHashMap<Foo1>>> my_map;

There are over a million entries in the TObjectDoubleHashmap and takes about a minute to generate. What are my options to save it to file so it won't need to be generated every time?

So far I've saved the actual keys and values to a text file then load it into the hashmap again, but that is not saving that much time. Is there a faster way such as saving it as an object using serialization? If so, what do I have to do to make my keys serializable?

Improve this question
0

2 Answers 2

Reset to default
2

Is there a faster way such as saving it as an object using serialization?

Serialisation of the map is merely writing out the keys and the matching values. It won't be faster to use serialisation than doing it yourself long-hand. The HashMap's description of its serialised data is:

The capacity of the HashMap (the length of the bucket array) is emitted (int), followed by the size (an int, the number of key-value mappings), followed by the key (Object) and value (Object) for each key-value mapping. The key-value mappings are emitted in no particular order.

You can see the code here.

What do I have to do to make my keys serializable?

To make an object serialisable, implement the Serializable interface. You probably also want to specify the serialVersionUID. Add a field:

private static final long serialVersionUID = 1L;
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

@gewizz Well there are different serialization libraries out there that you could research using, like kryo for example. I don't have the experience to say whether any of them would help but it might be worth investigating. There is also this google group for a project dedicated to benchmarking these different libraries: groups.google.com/group/java-serialization-benchmarking
@PaulBellora: One way these other serialisation libraries get gains from turning off the ability to write object references, which means you don't have to track whether or not you've seen the object in the stream. When you start hitting millions of objects this can have an impact. If you have references that should point to the same object however, this is unsuitable.
@GregKopff Ah okay - I definitely don't know enough about those libraries work or differ from standard Java serialization but just wanted to mention they existed.
@PaulBellora: yep - it's certainly worth a look - Kryo also has a serialiser that's very compact by not writing out any object metadata -- but this means it can't evolve with changes to the class, and you have to register the classes that you want to serialise (it writes out a numeric id instead of the classname). This makes for an efficient representation on disk / on the wire, but it's a bit fragile.
0

Are you using a Map<Foo1, Map<Foo2, Value>> because you want to use two keys for each value?

If so you can try using Guava's Table instead of nested maps, this might improve performance. This is designed for associating two keys to a value. My guess is that creating a whole lot of hash maps is what is taking most of the initialization time.

Improve this answer

1 Comment

Guava team member here: I wouldn't say that using Table will improve performance. (Indeed, on the backend HashBasedTable is implemented much like the OP's code, with a HashMap<Foo1, HashMap<Foo2, Value>>. (Not that using Table would hurt, especially not wrt readability.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.