1

This has been killing me all day/night and I cannot seem to come up with a solution. Basically, I have a text file containing a 2D vector (generated from a C++ program) in doubles. I need to read this into a 2D array in Python so I can plot a spectrogram. Here is what the data looks like:

-18.2258 -18.3581 -18.7323 -19.2183 -19.8016 -20.6132 -21.8101 -22.5386 -21.8071    
-20.9063 -20.4136 -20.3022 -20.3428 -20.4091 -20.6703 -21.0293 -21.5167 -22.1915    
-23.0438 -23.9086 -24.5955 -26.2508 -26.0188 -22.2163 -19.933 -18.6816 -18.1048
-18.0222 18.3233 -19.0456 -20.3134 -22.7954 -25.8716 -21.4845 -19.1923 -17.9268 
-17.4657 -17.3888 -16.9999 -16.4006 -15.9175 -15.8319 -16.1705 -16.6967 -17.0734 


-7.92685 -10.8266 -16.392 -12.4901 -13.0831 -17.7215 -17.5159 -14.1485 -12.9897 -12.0444   
-11.8363 -12.6952 -12.9652 -14.3788 -13.8465 -17.529 -17.4747 -11.9521 -12.545 -13.8976 
-12.4176 -15.3273 -14.8081 -19.4117 -17.9596 -16.2607 -16.7505 -15.8918 -16.5602 
-17.2225 -16.9048 -15.1381 -17.37 -16.43 -14.9437 -14.9821

Each block of data is separated by 2 lines within the text file.

I have tried the following:

with open('spec.txt') as file:
    array2d = [[float(digit) for digit in line.split()] for line in file]

However, this does not work and I just seem to be getting a lot of arrays generated.

Anyone have any ideas to solve this?

P.S. each block is of the same size. However, to shorten this question, I just included a sample.

Improve this question
4
  • What are you using to plot? And what exactly do you mean by '2d array'? A list of lists? Or a numpy array? Commented Jan 10, 2014 at 21:37
  • @hpaulj Hey, I'm using matplotlib to plot.. I've tried Ashwini Chaudhary implementation, but, it doesn't work. The results are kind of messed up Commented Jan 10, 2014 at 21:37
  • It would easier to test if your second block had the same number of numbers as the first. That way the resulting list of lists can be fed directly to numpy.array to create a 2d array. Commented Jan 10, 2014 at 21:54
  • This data file would be easier to load if it was written with one block per line. Then np.loadtxt(filename) would be all you need. Commented Jan 10, 2014 at 22:15

3 Answers 3

Reset to default
2 
raw_text = """-18.2258 -18.3581 -18.7323 -19.2183 -19.8016 -20.6132 -21.8101 -22.5386 -21.8071    
-20.9063 -20.4136 -20.3022 -20.3428 -20.4091 -20.6703 -21.0293 -21.5167 -22.1915    
-23.0438 -23.9086 -24.5955 -26.2508 -26.0188 -22.2163 -19.933 -18.6816 -18.1048
-18.0222 18.3233 -19.0456 -20.3134 -22.7954 -25.8716 -21.4845 -19.1923 -17.9268 
-17.4657 -17.3888 -16.9999 -16.4006 -15.9175 -15.8319 -16.1705 -16.6967 -17.0734 


-7.92685 -10.8266 -16.392 -12.4901 -13.0831 -17.7215 -17.5159 -14.1485 -12.9897 -12.0444   
-11.8363 -12.6952 -12.9652 -14.3788 -13.8465 -17.529 -17.4747 -11.9521 -12.545 -13.8976 
-12.4176 -15.3273 -14.8081 -19.4117 -17.9596 -16.2607 -16.7505 -15.8918 -16.5602 
-17.2225 -16.9048 -15.1381 -17.37 -16.43 -14.9437 -14.9821"""
#in your example raw_text = open(some_file).read()
blocks = raw_text.split("\n\n\n")
split_blicks = [[float(v) for v in block.split()] for block in blocks]

is that what you want?

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

what ? no see the comment line
0

You could reduce it quite a bit with list comprehensions.

with open('myfile') as f:
   return ([float(x) for x in l.split() if l] for l in (raw.strip() for raw in f))

Note the outer parens makes this return a generator rather than processing the entire file before returning anything.

Improve this answer

Comments

0

Split the data at empty lines:

def split_at_empty_lines(filename):
    with open(filename) as f:
        arr = []
        for line in f:
            #If the line is empty and arr is not empty, means it's
            #time to return the collected items and set `arr` back to [].
            if not line.strip() and arr:
                yield arr
                arr = []
            #If the line is not empty then simply collect the items in `arr`
            elif line.strip(): 
                arr.extend(float(x) for x in line.split())
            #Ignore the case of empty line and empty `arr`

        #Check if arr is not empty or not, if not empty returns its content.
        if arr: yield arr
...         
>>> list(split_at_empty_lines('abc1.txt'))
[
 [-18.2258, -18.3581, -18.7323, -19.2183, -19.8016, -20.6132, -21.8101, -22.5386, -21.8071, -20.9063, -20.4136, -20.3022, -20.3428, -20.4091, -20.6703, -21.0293, -21.5167, -22.1915, -23.0438, -23.9086, -24.5955, -26.2508, -26.0188, -22.2163, -19.933, -18.6816, -18.1048, -18.0222, 18.3233, -19.0456, -20.3134, -22.7954, -25.8716, -21.4845, -19.1923, -17.9268, -17.4657, -17.3888, -16.9999, -16.4006, -15.9175, -15.8319, -16.1705, -16.6967, -17.0734],
 [-7.92685, -10.8266, -16.392, -12.4901, -13.0831, -17.7215, -17.5159, -14.1485, -12.9897, -12.0444, -11.8363, -12.6952, -12.9652, -14.3788, -13.8465, -17.529, -17.4747, -11.9521, -12.545, -13.8976, -12.4176, -15.3273, -14.8081, -19.4117, -17.9596, -16.2607, -16.7505, -15.8918, -16.5602, -17.2225, -16.9048, -15.1381, -17.37, -16.43, -14.9437, -14.9821]
]
Improve this answer

2 Comments

Thanks! But how could I store the result of this method inside another array so it can be plotted?
@user1326876 I don't understand, simply call list() on it(as shown in code) and you'll get a list of lists.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.