1

I have a bunch of 32-bit floating point values in a Python script, need to store them to disc and load them in a C++ tool. Currently they are written in human-readable format. However the loss of precision is too big for my (numeric) application.

How do I best store (and load) them without loss?

Improve this question
4
  • json only has one way to represent numbers, so that pretty much limits your options, no? json.org I would recommend using a serialization library, however. Don't try to figure this stuff out yourself -- it can get complicated. Commented Sep 26, 2017 at 9:33
  • @xaxxon Thanks for the remark. I can use base64 to store any binary format in json. I edit my question accordingly. Commented Sep 26, 2017 at 10:03
  • The JSON part is completely meaningless to your question at that point, since you can literally store anything as a base-64 encoded string. I suggest just removing that part entirely as it doesn't add anything except potential confusion. Commented Sep 26, 2017 at 10:06
  • @xaxxon You are right. Thanks. I removed it. Commented Sep 26, 2017 at 10:13

2 Answers 2

Reset to default
2

You can use float.hex in python to get the hexadecimal representation of your number, then read it using the std::hexfloat stream manipulator in C++.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Cool, thanks. Do you know if this is portable between different systems? The documentation for std::hexfloat does not mention IEEE etc.
According to the c++14 standard, std::hexfloat is equivalent to a call of str.setf(ios_base::fixed | ios_base::scientific, ios_base::floatfield), and ios_base::fixed | ios_base::scientific is a format equivalent to stdio's %A. According to the C11 standard, the result is correctly rounded if FLT_RADIX is a power of 2.
Also, the Python documentation states that float.hex and float.fromhex are compatible with stdio's %a/%A.
Well, this is better than decimal representation because avoids expensive conversion.
2

Currently they are written in the default human-readable format.

That's the reason. If require e.g. "%.9e" or "%.9g", it will print float32 values with precision enough to restore (provided there is no errors in conversions).

Also, for Python3, repr() gives a shortest decimal representation for a value that is converted back to the same binary value; but this is not true for Python2.

Answering a question from comments,

If I understand correctly, there are values that have a finite floating point representation in base 2, but the base 10 representation is infinite,

No, it is finite anyway, but just longer. For example (Python3)

>>> math.pi
3.141592653589793
>>> '%.60g' % math.pi
'3.141592653589793115997963468544185161590576171875'

but, all digits after the final significant one do not matter (remember Python float is double precision):

>>> float('3.1415926535897931')
3.141592653589793
>>> float('3.1415926535897932')
3.141592653589793
>>> float('3.1415926535897933')
3.141592653589793
>>> float('3.1415926535897933') - float('3.1415926535897931')
0.0

Finally, hexadecimal is more reliable if available in your code. I expect your Python and C++ versions fresh enough to support %a (stdio), float.hex (Python).

Improve this answer

7 Comments

Thanks, but I would like to store them without any loss. I edited my question accordingly.
Yep. For float32, 8 significant decimal digits are enough for no loss.
If I understand correctly, there are values that have a finite floating point representation in base 2, but the base 10 representation is infinite, i.e. one would need an infinite number of digits after the decimal separator to represent them. Short: No, there should always be a loss in precision when using your approach.
@Netch: you need 9 significant digits, not 8. For example, 0x1.fffffep+9 = 1023.99993896484375 and 0x1.fffffcp+9 = 1023.9998779296875 are distinct exactly-representable float32 values, but their decimal representations agree when rounded to 8 significant digits: both give 1023.9999.
@Netch Ah OK, thanks. I think I now understand. 2 is a prime factor of 10 (base in decimal), so one can express every binary float in decimal. Is it just the other way that can make problems, since for example 1/5 can be expressed cleanly in decimal but not in binary since 5 is a prime factor of 10 but not of 2.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.