retrieving data from a .txt file using python?

Question

In the following you can see data from a ephemeris.txt file. Now I want to retrieve several columns(say, for example the column starting with 00:00, 27.69 and 44.1) and name the array as x,y,z. What do I have to do?

I tried this

x, y, z = numpy.loadtxt("ephemeris.txt", unpack=True)

And this get this error

"ValueError: could not convert string to float: Date__(UT)__HR:MN"

Could you also help me in converting that HR:MN into minute only?

Date__(UT)__HR:MN     R.A.__(a-apparent)__DEC\
**********************************************\
 2013-Jan-01 00:00 *   14 31 27.69 -12 29 44.1\
 2013-Jan-01 00:01 *   14 31 27.71 -12 29 44.1\
 2013-Jan-01 00:02 *   14 31 27.72 -12 29 44.2\
 2013-Jan-01 00:03 *   14 31 27.73 -12 29 44.2\
 2013-Jan-01 00:04 *   14 31 27.75 -12 29 44.3\
 2013-Jan-01 00:05 *   14 31 27.76 -12 29 44.3\
 2013-Jan-01 00:06 *   14 31 27.77 -12 29 44.4\
 2013-Jan-01 00:07 *   14 31 27.78 -12 29 44.4\
 2013-Jan-01 00:08 *   14 31 27.80 -12 29 44.4\
 2013-Jan-01 00:09 *   14 31 27.81 -12 29 44.5\

thanks in advance

Sorry, I didn't noticed you are already using numpy.loadtxt. Take a look at my answer, then :o) — heltonbiker
– heltonbiker, Commented Apr 9, 2013 at 22:30

heltonbiker · Accepted Answer · 2013-04-09 22:35:13Z

1

You can use some more arguments of the loadtxt function.

The error you are getting most probably is due to the first two header lines, so skip them with the skiprows=2 argument;

Also, each row contains data in a different format, separated by space. Use delimiter=' ' just in case, and you can opt between dtype=string and dtype=object.

a = numpy.loadtxt("ephemeris.txt", delimiter=' ', dtype=string, skiprows=2)

This should give you a single array from where you can perform many kinds of "conversions": split one array per column, create a list of rows, etc.

x,y,z,etc = numpy.hsplit(a, a.shape[1])
x = x.astype(datetime)

# or
x = a[:,0].astype(datetime)
y = a[:,1].astype(some_type)

or something along these lines...

Hope this helps, and please elaborate more in the comments if needed.

edited Apr 9, 2013 at 22:35

answered Apr 9, 2013 at 22:30

heltonbiker

27.7k30 gold badges151 silver badges270 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

user22180 Over a year ago

a = np.loadtxt("ephemeris.txt", delimiter=' ', dtype=string, skiprows=2) gives the following error: "name 'string' is not defined" a.shape=11 x,y,z,etc = numpy.hsplit(a, a.shape) gives the error: ValueError: too many values to unpack

heltonbiker Over a year ago

1) sorry, it should be dtype='string' (the type "string" must be between quotes). 2) x,y,z,etc wasn't supposed to be meant literally, the number of arrays you are going to create depends on how many columns you have in your txt file. Supposing you have 8 columns, you would do b,c,d,e,f,g,h,i = numpy.hsplit(a, 8). At last, I have the impression that you are not familiar with python or programming in general, are you really UNDERSTANDING the explanation, or just copy-pasting code and trying to run it blindly? (no offense, but perhaps you need a DEEPER help).

user22180 Over a year ago

Probably you are saying this by seeing 'etc', but I tried 'string' also and it gives ValueError: cannot set an array element with a sequence.After that I tried object instead of 'string' and got no error. But the following b,c,d,e,f,g,h,i,j,k,l = numpy.hsplit(a, a.shape) gives the error "ValueError: need more than 2 values to unpack" One thing I agree with you I need a better help to understand the errors. Say from the error "need more than 2 values to unpack", I have understood nothing and that is why I couldn't correct the error.Is there any site which explains the different type of errors

heltonbiker Over a year ago

You should have used numpy.hsplit(a, a.shape[1]). Without the [1] part it shouldn't work, because the second argument would be seen as a sequence of length two (the two dimensions of the array). With the index, it would be seen as an integer, corresponding to the second dimension of the array, that is, its number of columns.

user22180 Over a year ago

Thank you, Now I have figured out the problem with "string". Actually there was a space before the first column, that is the column starting with 2013-Jan-01.

|

optimusfrenk · Accepted Answer · 2013-04-09 19:46:25Z

0

You can also split each line, setting a character separator. Then you can access on each (string) token using indexes:

def prova():
    f = open('/home/frenk/Desktop/ephemeris.txt')
    l = []
    for line in f:
        l = line.split(" ")
        print "date: " + l[1]

Second, if you want to convert a string like "31" to integer 31, you can simply write:

x = int('31')

Note that you can select a slice of string using slice notation:

string = "This is a slice of string"
print string[10:15]

answered Apr 9, 2013 at 19:46

optimusfrenk

1,3212 gold badges17 silver badges35 bronze badges

Comments

jazzpi · Accepted Answer · 2013-04-09 19:56:26Z

0

import re
f = open("ephemeris.txt")
for line in f.readlines():
    r = re.search("(\d{4})\-(\w{3})-(\d{2}) (\d{2}):(\d{2}) \*   (.*?)\\\n", line)
    if r:
        print "Year: "+r.group(1)
        print "Month: "+r.group(2)
        print "Day: "+r.group(3)
        print "Hour: "+r.group(4)
        print "Minute: "+r.group(5)
        print "Data: "+r.group(6)

This will read every line of the file, check if it matches the pattern and if it does so, print all the data it could retrieve.

edited Apr 9, 2013 at 19:56

answered Apr 9, 2013 at 19:14

jazzpi

1,43113 silver badges19 bronze badges

3 Comments

user22180 Over a year ago

sorry , this shows the following error: error: bogus escape (end of line)

jazzpi Over a year ago

@user22180: Fixed, forgot a \n at the end of the pattern :(

user22180 Over a year ago

could you be more specific?

Collectives™ on Stack Overflow

retrieving data from a .txt file using python?

3 Answers 3

10 Comments

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

10 Comments

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related