When we create a new object, 'new' operator returns the memory address of that object it creates; which in java we say it returns reference more generally.
Well, a reference certainly doesn't have to be an address. It's a way of referring to an object - the exact value of the bits depends on the JVM implementation.
What I want to know is, is this the same as the value returned by the hashCode()?
Not necessarily. You certainly shouldn't try to make any assumptions on that front. Again, it's an implementation detail.
It's worth remembering that while the garbage collector can move objects around in memory (updating references as it goes), the hash code must not change based on this, which is an argument against your suggestion.
But then again, when we have more than 2^32 objects and given hashCode() returns an integer (2^32 different numbers) there will be collisions all over and when we pass objects it would be a real mess.
There will be hash code collisions, yes. That's unavoidable - anything using hash codes needs to take the possibility of collisions into account. There can't be reference collisions though - a JVM which is able to support more than 232 values concurrently clearly can't just make the hash code and the reference have the same value.