The function range creates a lazy sequence. Lazy sequences create their elements only on-demand. If the beginning of the lazy seq is tied to some variable (either locally with let or globally with def), the seq will cache all its elements and thus consume memory. This is commonly referred to as "holding onto the head of a sequence".
That's why your second example consumes a lot of memory.
I would expect that an analysis with jvisualvm will show that it is not clojure itself that consumes the CPU cycles but the JVM trying to cope with the increasing memory usage.
Finally, count traverses the whole sequence, thereby realizing all elements. In case of just (range) the sequence would never end, it gets stopped when no more memory is available.
countdoes not retain data while it counts. But withdefthe materialized data is retained by thexvariable.