Certain mathematical operations, especially on data read from hardware drivers, can depend on fixed width of the data type. Example: bitwise shift. What is the Pythonic way of creating integer variables with fixed width (e.g. uint32, int16 etc.) that would overflow/shift accordingly?
2
-
3There is the fixedint library. They handle typical bit-fiddling and bitset operations.Cory Kramer– Cory Kramer2015-06-30 17:59:48 +00:00Commented Jun 30, 2015 at 17:59
-
fixedint looks pretty good! Thanks! I think that's a comment that should become an answer.Andrzej Pronobis– Andrzej Pronobis2015-06-30 18:01:51 +00:00Commented Jun 30, 2015 at 18:01
Add a comment
|
2 Answers
For interfacing with hardware we normally use the struct standard library - specifically struct.pack and struct.unpack not only are fixed widths handled but also endianess issues. See the python 2 or python 3 library manuals.
Once you have retrieved your data from your hardware and unpacked it if you are going to be performing any heavy duty mathematics on it you would usually assign it to numpy data types which will:
- Provide just about any operation that you could need,
- Behave as expected for the data type
- If you have lots of data stored in arrays or matrices provide vector methods for handling the data quickly & simply.
import numpy as np
x = np.uint32(42)
print(x << 20) # 44040192
print(x << 32) # 180388626432 - Looks like it has been promoted
print(type(x << 32)) # numpy.int64 - it has
So if you are doing something such as reading a set of A-D readings from registers and then performing a lot of maths on them so as to produce a graph, etc. then this is fine but if you are reading a value from one register, doing some math, then writing the results back to a register you need to take care using anything like numpy.
Note that there is also the ctypes library available for interfacing with hardware and DLLs but it will always expect the values to be in "native" format - so, for example, if you are reading a 16 bit number from big endian hardware on a little endian machine then you will get problems.
5 Comments
Andrzej Pronobis
That's a useful package to remember as well. Thanks!
wick
how you do integer arithmetic with packs :-)
Steve Barnes
@wick - Unpack the data from the hardware, perform arithmetic, and if needed try to pack, deal with any issues such as overflows then send good packed data back to hardware.
wick
@SteveBarnes aren't we losing all performance gains by doing all that? My thinking is performance being the whole point of using integers aligned with CPU registers...
Steve Barnes
@wick The question was about correctly interfacing with hardware drivers not CPU performance. One of the mantras of the python world is to avoid premature optimisation. Python integers are not CPU aligned but very powerful and often fast enough.
I would suggest the fixedint library. The classes in that library are named in the following convention:
[Mutable][U]Int<N>
So for your two examples, the classes would be
# C++ Python fixedint
std::uint32 UInt32
std::uint16 UInt16
This supports things like bit-shifting, etc
>>> a = fixedint.UInt32(14)
>>> a
UInt32(14)
>>> a << 2
UInt32(56)
1 Comment
Alexandro Sánchez
Another option is nativetypes, which additionally supports floating-point values and custom type aliases.