working with bytes in python

Question

I am trying to port a processing.org app into python and have some difficulties with it.

i need to write this in python:

int[][] elevation; // elevations in meters
float[][] felevation; // scaled to range [0,1] where 1=max
int maxheight;

void setup(){
size(600,600,P2D);

// read srtm binary file
elevation=new int[1201][1201];
felevation=new float[1201][1201];
byte b[] = loadBytes("N30W091.hgt"); // THIS IS A BINARY FILE
int ix=0;
maxheight=0;
for (int row=0;row<1201;row++) {
  for (int col=0;col<1201;col++) {
    // bytes are signed, from -128 to 127, converts to unsigned...
    int hi = b[ix] & 0xff; 
    int lo = b[ix+1] & 0xff; 
    int el=(int)((hi<<8)|lo); // big endian!
    elevation[row][col]=el;
    if (el>maxheight && el<32000) maxheight=el; 
    ix+=2;
   }
}

... and so on

what i have made so far is this:

elevation = [[],[]]
maxheight=0

b = open("C:\\Users\\CNA\\sketchbook\\_SRTM\\data\\N59E010.hgt","rb")
fin = b.read(1)
print(len(fin))
ix = 0
for row in range(0,1201):
    for col in range(0,1201):
        hi = (fin[ix]   + 0xff)
        lo = (fin[ix+1] + 0xff)

and i always get

Traceback (most recent call last):

  File "C:\Users\CNA\workspace\Revitter\PatternAsignment.py", line 16, in <module>

TypeError: unsupported operand type(s) for +: 'str' and 'int'

any ideas? .. i am new to python and i dont have experience in working with bytes...

if you're sure that what you've read from the file is convertible to int, use int(fin[ix])+0xff etc — ev-br
– ev-br, Commented May 5, 2012 at 13:18

Karl Knechtel · Accepted Answer · 2012-05-05 14:25:39Z

An idiomatic translation is going to work in a completely different way.

In the original code, you do a bunch of bit-twiddling to convert two byte values into a single numeric value. In Python, there is built-in functionality for this: use the struct module. As it turns out, this module is already built for reading several values at a time.

Also, use forward slashes for file names - it's easier, and guaranteed to work. Use a with-block to ensure that the file is properly closed automatically, and a list comprehension to simplify the looping - stop trying to tell Python how to build a list, and just ask for the list you want.

This gives us:

import struct
with open('C:/Users/CNA/sketchbook/_SRTM/data/N59E010.hgt', 'rb') as data:
    elevation = [
        list(struct.unpack('>1201H', data.read(1201 * 2)))
        for row in range(1201)
    ]
maxheight = max(max(cell for cell in row if cell < 32000) for row in elevation)

and you're done. Welcome to Python :)

Michał Zieliński · Accepted Answer · 2012-05-05 13:20:11Z

2

In Python values like 'hello'[2] are also strings (in this case == 'l'). You need to convert them to integers using ord and back to string using chr.

elevation = [[],[]]
maxheight=0

b = open("C:\\Users\\CNA\\sketchbook\\_SRTM\\data\\N59E010.hgt","rb")
fin = b.read() # you probably need to read more than 1 byte, this will read whole file
print(len(fin))
ix = 0
for row in range(0,1201):
    for col in range(0,1201):
        hi = (ord(fin[ix])   + 0xff) # ord returns unsigned integer, so you probably don't need to convert it
        lo = (ord(fin[ix+1]) + 0xff)
        el = (hi << 8) | lo

See: http://docs.python.org/library/functions.html

answered May 5, 2012 at 13:20

Michał Zieliński

1,34511 silver badges13 bronze badges

Comments

Hugh Bothwell · Accepted Answer · 2012-05-05 15:06:13Z

0

I am not sure if I have the .reshape syntax right (will test further) - but something very much like this should serve your needs:

import numpy

def readHgtFile(fname, h=1201, w=1201):
    with open(fname, 'rb') as inf:
        return numpy.fromfile(inf, dtype=[('height', '>u2')], count=h*w).reshape((h,w))

def main():
    elevation = readHgtFile('N30W091.hgt')
    maxheight = elevation.max()

if __name__=="__main__":
    main()

edited May 5, 2012 at 15:06

answered May 5, 2012 at 15:00

Hugh Bothwell

57k9 gold badges91 silver badges103 bronze badges

Collectives™ on Stack Overflow

working with bytes in python

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related