It is in fact possible to define numbers without ANY native primitives. There are many ways, but the simplest is:
data Peano = Z | S Peano
Then you can define instance Num for this type using pattern-matching. The second common representation of numbers is so called Church encoding using only functions (all numbers will be represented by some obscure functions, and + will 'add' two functions together to form third one).
Very interesting encodings are possible indeed. For example, you can represent arbitrary precision reals in [0,1) using sequences of bits:
data RealReal = RealReal Bool RealReal | RealEnd
In GHC of course it is defined in a machine-specific way by using either primitives or FFI.