I am trying to get binary data from text and converting to numpy arrays

Question

I am trying to get binary data from text and convert to numpy arrays. I tried these codes:

import numpy as np
import matplotlib.pylab as pl
'''a=open ('rawdata.txt ','r',)
print (a.read())
a.close()'''
data=np.genfromtxt ('rawdata.txt', delimiter=',')
print (data)
time =data [1:,][:,0]
print (time)
voltage=(data [1:,][:,1])
print(voltage)

It is working but there are some problems. It just printes my binary numbers like float:

[[             nan              nan              nan]
 [  0.00000000e+00   0.00000000e+00   1.01101010e+08]
 [  5.00000000e-04   1.10001000e+05   1.01011000e+08]
 [  1.00000000e-03   1.10000000e+06   1.01011000e+08]
 [  1.50000000e-03   1.00011010e+07   1.00101110e+08]
 [  2.00000000e-03   1.01101110e+07   1.00010111e+08]
 [  2.50000000e-03   1.10111000e+07   0.00000000e+00]
 [  3.00000000e-03   1.11111000e+07   1.01110000e+04]
 [  3.50000000e-03   1.00010101e+08   1.01110000e+05]
 [  4.00000000e-03   1.00101000e+08   1.00010000e+06]
 [  4.50000000e-03   1.00110011e+08   1.01100000e+06]
 [  5.00000000e-03   1.00110111e+08   1.10101000e+06]
 [  5.50000000e-03   1.00110011e+08   1.11100100e+06]
 [  6.00000000e-03   1.00101000e+08   1.00001100e+07]
 [  6.50000000e-03   1.00010101e+08   1.00011110e+07]
 [  7.00000000e-03   1.11111000e+07   1.00101000e+07]
 [  7.50000000e-03   1.10111000e+07   1.00101100e+07]
 [  8.00000000e-03   1.01101110e+07   1.00101000e+07]
 [  8.50000000e-03   1.00011010e+07   1.00011110e+07]
 [  9.00000000e-03   1.10000000e+06   1.00001100e+07]
 [  9.50000000e-03   1.10001000e+05   1.11100100e+06]
 [  1.00000000e-02   0.00000000e+00   1.10101000e+06]
 [  1.05000000e-02   1.00011000e+09   1.01100000e+06]
 [  1.10000000e-02   1.00110000e+09   1.01100000e+06]
 [  1.15000000e-02   1.01000110e+09   1.01100000e+06]
 [  1.20000000e-02   1.01011011e+09   1.01110000e+04]
 [  1.25000000e-02   1.01101110e+09   0.00000000e+00]
 [  1.30000000e-02   1.01111110e+09   1.00010111e+08]
 [  1.35000000e-02   1.10001010e+09   1.00101110e+08]
 [  1.40000000e-02   1.10010100e+09   1.01000100e+08]
 [  1.45000000e-02   1.10011001e+09   1.01011000e+08]
 [  1.50000000e-02   1.10011011e+09   1.01101010e+08]
 [  1.55000000e-02   1.10011001e+09   1.01111001e+08]
 [  1.60000000e-02   1.10010100e+09   1.10000110e+08]
 [  1.65000000e-02   1.10001010e+09   1.10001111e+08]
 [  1.70000000e-02   1.01111110e+09   1.10010100e+08]
 [  1.75000000e-02   1.01101110e+09   1.10010110e+08]
 [  1.80000000e-02   1.01011011e+09   1.10010100e+08]
 [  1.85000000e-02   1.01000110e+09   1.10001111e+08]
 [  1.90000000e-02   1.00110000e+09   1.10000110e+08]
 [  1.95000000e-02   1.00011000e+09   1.01111001e+08]
 [  2.00000000e-02   0.00000000e+00   1.01101010e+08]]

How can I deal with it? My actual rawdata.txt text file:

time(s) ,voltage(V) ,current(A)
0       ,0000000000 ,101101010
0.0005  ,0000110001 ,101011000
0.001   ,0001100000 ,101011000
0.0015  ,0010001101 ,100101110
0.002   ,0010110111 ,100010111
0.0025  ,0011011100 ,000000000
0.003   ,0011111100 ,000010111
0.0035  ,0100010101 ,000101110
0.004   ,0100101000 ,001000100
0.0045  ,0100110011 ,001011000
0.005   ,0100110111 ,001101010
0.0055  ,0100110011 ,001111001
0.006   ,0100101000 ,010000110
0.0065  ,0100010101 ,010001111
0.007   ,0011111100 ,010010100
0.0075  ,0011011100 ,010010110
0.008   ,0010110111 ,010010100
0.0085  ,0010001101 ,010001111
0.009   ,0001100000 ,010000110
0.0095  ,0000110001 ,001111001
0.01    ,0000000000 ,001101010
0.0105  ,1000110001 ,001011000
0.011   ,1001100000 ,001011000

I assume you are reading binary data from a lab instrument? I think your lab instrument is outputting data in the IEEE 754 floating point format. Can you confirm if this format is 32-bit or 64-bit IEEE 754? — Kit
– Kit, Commented May 26, 2016 at 5:38
Actually I got this data from my teacher. I must ask him what he used to get this data. — Suhrob Kamolov
– Suhrob Kamolov, Commented May 26, 2016 at 5:58
I just saw your raw data. What you need to do is to split out each line so that you have (time, voltage_b, current_b). Then you can treat them accordingly. Treat time as a float, voltage_b and current_b as str then convert them to int. — Kit
– Kit, Commented May 26, 2016 at 6:24

Neapolitan · Accepted Answer · 2016-05-26 06:29:25Z

You can use pandas to read it:

import pandas as pd
binary = lambda s: int(s, 2)
df = pd.read_csv('rawdata.txt', converters={'voltage(V) ': binary, 'current(A)': binary})
print(df)

This allows you to specify separate converters for each column based on the name of the column. In this case, we use int(s, 2) to interpret an ascii string as a base 2 number.

Output:

    time(s)   voltage(V)   current(A)
0     0.0000            0         362
1     0.0005           49         344
2     0.0010           96         344
3     0.0015          141         302
4     0.0020          183         279
5     0.0025          220           0
6     0.0030          252          23
7     0.0035          277          46
8     0.0040          296          68
9     0.0045          307          88
10    0.0050          311         106
11    0.0055          307         121
12    0.0060          296         134
13    0.0065          277         143
14    0.0070          252         148
15    0.0075          220         150
16    0.0080          183         148
17    0.0085          141         143
18    0.0090           96         134
19    0.0095           49         121
20    0.0100            0         106
21    0.0105          561          88
22    0.0110          608          88

You can get the data as an array using df.values or individual columns using the column name, such as df['voltage(V) '].

Note the extra space in 'voltage(V) '. This is because your file format has a space before the comma, and so it is considered to be part of the column name. You have several options.

Ignore the column names in the file and set your own:

pd.read_csv('rawdata.txt', converters={'voltage(V)': binary, 'current(A)': binary}, 
                 names='time(s) voltage(V) current(A)'.split(), skiprows=1)

Use " ," as the column delimiter:

pd.read_csv('rawdata.txt', converters={'voltage(V)': binary, 'current(A)': binary}, delimiter=" ,", engine='python')

Rename the columns after loading:

df = pd.read_csv('rawdata.txt', converters={'voltage(V) ': binary, 'current(A)': binary})
df.columns = [v.strip() for v in df.columns]

Serenity · Accepted Answer · 2016-05-26 05:31:11Z

2

You have to specify a type for cols you want to read, f.e.:

data1 = np.loadtxt(flname, delimiter=',', unpack = True, skiprows=1, usecols=(0,1), dtype='float')
data2 = np.loadtxt(flname, delimiter=',', unpack = True, skiprows=1, usecols=(2,), dtype='int')
for i in xrange(data1.shape[1]):
     print data1[0][i], data1[1][i], data2[i]

edited May 26, 2016 at 5:31

answered May 26, 2016 at 5:22

Serenity

37.1k21 gold badges125 silver badges117 bronze badges

2 Comments

rmorshea Over a year ago

he wants the binary data to be converted to ints, this won't do that.

Suhrob Kamolov Over a year ago

Thanks it works, but column 2 are binary values. it doesn't converts them to int.

rmorshea · Accepted Answer · 2016-06-04 03:27:29Z

2

Using the answer by Stanley R, just add a little bit of logic to convert the binary data to integers.

base10 = np.loadtxt(filename, delimiter=',', unpack = True, skiprows=1, usecols=(0,), dtype=float)
base02 = np.loadtxt(filename, delimiter=',', unpack = True, skiprows=1, usecols=(1,2,), dtype=int)
bin2int = lambda n: int(str(n), base=2)
for i, col in enumerate(base02):
    base02[i] = np.array(map(bin2int, col))
print(base02)

The trick here is to use the optional base argument in int which normally defaults to 10 and instead set it to 2 so you can pass it a binary string. Other useful functions I use are enumerate and map.

edited Jun 4, 2016 at 3:27

answered May 26, 2016 at 5:32

rmorshea

8521 gold badge7 silver badges26 bronze badges

Comments

Suhrob Kamolov · Accepted Answer · 2016-05-26 20:28:40Z

Thank you guys for your answers. I finally got found the way:

import numpy as np
import pylab as pl
I=[]
V=[]
with open("rawdata.txt") as file:
    rawdata=[line.split()for line in file if not line.isspace()]
    data=np.array(rawdata)
    t=data[1:,0]
    I_b=data[1:,1] # I_b current in binary
    V_b=data[1:,2] # V_b voltage in binary

    def sign(value):
        if value[0] == '1':
            value = "-"+value[1:] 
        return int(value,base=2)
    for i in I_b:
        x=sign(i)
        I.append(x)
    for j in V_b:
        y=sign(j)
        V.append(y)
print(I)
print(V)

It prints what I want:

[0, 49, 96, 141, 183, 220, 252, 277, 296, 307, 311, 307, 296, 277, 252, 220, 183, 141, 96, 49, 0, -49, -96, -141, -183, -220, -252, -277, -296, -307, -311, -307, -296, -277, -252, -220, -183, -141, -96, -49, 0]
[-106, -88, -88, -46, -23, 0, 23, 46, 68, 88, 106, 121, 134, 143, 148, 150, 148, 143, 134, 121, 106, 88, 88, 88, 23, 0, -23, -46, -68, -88, -106, -121, -134, -143, -148, -150, -148, -143, -134, -121, -106]

Collectives™ on Stack Overflow

I am trying to get binary data from text and converting to numpy arrays

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related