Return to Answer

So if your units are metres, you'll lose millimetre precision around the 16 484 - 32 768 band (about 16-33 km from the origin).

So if your units are metres, you'll lose millimetre precision around the 16 384 - 32 768 range (about 16-33 km from the origin).

As an example, for i = 0, the smallest number in this range is (2^0) * 1 = 1. The next smallest number is (2^0) * (1 + 2^-23). If you wanted to represent 1 + 2^-24, you'll have to round up or down, for an error of 2^-24 either way.

As an example, for \$i = 0\$, the smallest number in this range is \$(2^0) \cdot 1 = 1\$. The next smallest number is \$(2^0) \cdot (1 + 2^{-23})\$. If you wanted to represent \$1 + 2^{-24}\$, you'll have to round up or down, for an error of \$2^{-24}\$ either way.

That means each number is represented as 1.xxx xxx xxx xxx xxx xxx xxx xx times some power of 2, where each x is a binary digit (either 0 or 1)

So in the range from 2^i and 2^(i+1), you can represent any number to within an accuracy of ±2^(i - 24)

In this range:           You get accuracy within:
-----------------------------------------------
0.25 - 0.5               2^-26 = 1.49011611938477E-08
0.5 - 1                  2^-25 = 2.98023223876953E-08
1 - 2                    2^-24 = 5.96046447753906E-08
2 - 4                    2^-23 = 1.19209289550781E-07
4 - 8                    2^-22 = 2.38418579101562E-07
8 - 16                   2^-21 = 4.76837158203125E-07
16 - 32                  2^-20 = 9.5367431640625E-07
32 - 64                  2^-19 = 1.9073486328125E-06
64 - 128                 2^-18 = 0.000003814697265625
128 - 256                2^-17 = 0.00000762939453125
256 - 512                2^-16 = 0.0000152587890625
512 - 1024               2^-15 = 0.000030517578125
1024 - 2048              2^-14 = 0.00006103515625
2048 - 4096              2^-13 = 0.0001220703125
4096 - 8192              2^-12 = 0.000244140625
8192 - 16384             2^-11 = 0.00048828125
16384 - 32768            2^-10 = 0.0009765625
32768 - 65536            2^-9 = 0.001953125
65536 - 131072           2^-8 = 0.00390625
131072 - 262144          2^-7 = 0.0078125
262144 - 524288          2^-6 = 0.015625
524288 - 1048576         2^-5 = 0.03125
1048576 - 2097152        2^-4 = 0.0625
2097152 - 4194304        2^-3 = 0.125
4194304 - 8388608        2^-2 = 0.25
8388608 - 16777216       2^-1 = 0.5
16777216 - 33554432      2^0 = 1

So if your units are metres, you'll lose millimetre precision around in the 16484-32768 band (about 16-33 km from the origin).

• If we use centimetres as our unit, we lose millimetre precision at the 1048576-2097152 band (10-21 km from the origin)

• If we use hectametres as our unit, we lose millimetre precision at the 128-256 band (13-26 km from the origin)

... so changing the unit over four orders of magnitude still ends up with a loss of millimetre precision somewhere in the range of tens of kilometers. All we're shifting is where exactly in that band it hits (due to the mismatch between base-10 and base-2 numbering) not drastically extending our playable area.

If you know the level of accuracy you need (say, a span of 0.01 units maps to about 1 px at your typical viewing/interaction distance, and any smaller offset is invisible), you can use the table above to find where you lose that accuracy, and step back a few orders of magnitude for safety.

That means each number is represented as 1.xxx xxx xxx xxx xxx xxx xxx xx times some power of 2, where each x is a binary digit, either 0 or 1. (With the exception of extremely small denormalized numbers less than \$2^{-126}\$ - they start with 0. instead of 1., but I'll ignore them for what follows)

So in the range from \$2^i\$ and \$2^{(i+1)}\$, you can represent any number within an accuracy of \$\pm 2^{(i - 24)}\$

In this range:                You get accuracy within:
-----------------------------------------------
         0.25   -     0.5    2^-26 = 1.490 116 119 384 77 E-08
         0.5    -     1      2^-25 = 2.980 232 238 769 53 E-08
         1     -      2      2^-24 = 5.960 464 477 539 06 E-08
         2     -      4      2^-23 = 1.192 092 895 507 81 E-07
         4     -      8      2^-22 = 2.384 185 791 015 62 E-07
         8     -     16      2^-21 = 4.768 371 582 031 25 E-07
        16     -     32      2^-20 = 9.536 743 164 062 5  E-07
        32     -     64      2^-19 = 1.907 348 632 812 5  E-06
        64     -    128      2^-18 = 0.000 003 814 697 265 625
       128    -     256      2^-17 = 0.000 007 629 394 531 25
       256    -     512      2^-16 = 0.000 015 258 789 062 5
       512    -   1 024      2^-15 = 0.000 030 517 578 125
     1 024    -   2 048      2^-14 = 0.000 061 035 156 25
     2 048    -   4 096      2^-13 = 0.000 122 070 312 5
     4 096    -   8 192      2^-12 = 0.000 244 140 625
     8 192   -   16 384      2^-11 = 0.000 488 281 25
    16 384   -   32 768      2^-10 = 0.000 976 562 5
    32 768   -   65 536      2^-9  = 0.001 953 125
    65 536   -  131 072      2^-8  = 0.003 906 25
   131 072   -  262 144      2^-7  = 0.007 812 5
   262 144   -  524 288      2^-6  = 0.015 625
   524 288 -  1 048 576      2^-5  = 0.031 25
 1 048 576 -  2 097 152      2^-4  = 0.062 5
 2 097 152 -  4 194 304      2^-3  = 0.125
 4 194 304 -  8 388 608      2^-2  = 0.25
 8 388 608 - 16 777 216      2^-1  = 0.5
16 777 216 - 33 554 432      2^0   = 1

So if your units are metres, you'll lose millimetre precision around the 16 484 - 32 768 band (about 16-33 km from the origin).

• If we use centimetres as our unit, we lose millimetre precision at the 1 048 576-2 097 152 band (10-21 km from the origin)

• If we use hectametres as our unit, we lose millimetre precision at the 128-256 band (13-26 km from the origin)

...so changing the unit over four orders of magnitude still ends up with a loss of millimetre precision somewhere in the range of tens of kilometers. All we're shifting is where exactly in that band it hits (due to the mismatch between base-10 and base-2 numbering) not drastically extending our playable area.

If you know the level of accuracy you need (say, a span of 0.01 units maps to about 1 px at your typical viewing/interaction distance, and any smaller offset is invisible), you can use the table above to find where you lose that accuracy, and step back a few orders of magnitude for safety in case of lossy operations.