If you need multiple values for specific keys,
Map<Integer, List<Integer>>
check (Guava's Multimap). You could find there quite nice classes or utility classes for transformation.
If this is not suitable for you, you definitions is not that bad, but if there is single key=value pair, this could be nicer:
List<Map.Entry<Integer, Integer>>
Entry is simple inner class inside map, which stores Key/Value pairs.
Also, if you stick with your approach (could be complicate) or you switch to Entry, it is not that difficult to change (in case you need always Keys or always only Values.
List<Entry<Integer, Integer>> values; // filled, initialised
// Getting only Keys
values.stream().map(Entry::getKey); // Java8 stuff - method reference
// Getting only Values
values.stream().map(Entry::getValue); // Java8 stuff - method reference
Here read about method reference