0

I have two columns in excel. Date and temp. They look like this:

  date             temp
20130102           34.20
20130102           34.42
20130102           34.23
20130102           34.12
20130102           34.84
20130103           34.48
20130103           34.42
20130103           33.77
20130103           33.62
20130103           33.94
20130103           33.45

when I extract them into python using numpy, i get 2 arrays like this:

date = [20130102,20130102,20130102,20130102,20130102,20130103,20130103,20130103,20130103]
temp = [34.20,34.42,34.23.....,33.45]

How do I convert this into 1 array of arrays by combining 1 date with all corresponding temps for that date.

dataarray = [[20130102,34.20,34.42,34.23,34.12,34.84],[20130103,34.48,34.42,33.77,33.62,33.94,33.45]]
Improve this question
2
  • Didn't you already get an answer to this in your previous question? If the issue is that you got a lists of lists rather than a 2D NumPy array there… well, of course you did. A NumPy array is square, it can't be jagged (where different rows have different lengths). Commented Feb 17, 2014 at 23:46
  • Aside: it's not clear to me that's a particularly useful data format. If you want to do something with the daily temperatures (such as get statistics on them, or whatever) then I think you're going about this the wrong way. Commented Feb 17, 2014 at 23:53

4 Answers 4

Reset to default
2

To get the exact format that you specify (a list of lists), I'd use itertools.groupby and then a couple of list comprehensions to unpack the group generators:

import itertools
groups = itertools.groupby(zip(date, temp), lambda dv:dv[0])
list_of_lists = [[d] + [v[1] for v in values] for d, values in groups]

I'm assuming, based on your example, that your data is available in sorted order. If not, the groupby function won't do what you need it to do, and you'd be better off with a dictionary (see below).

But my guess is, if you want to actually do anything with this data after you've unpacked it, it'll be more convenient to have it in a dictionary by date, in which case you'll want to go with the defaultdict strategy in Joran's answer.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

In his previous question he specifically said he wanted this list-of-list formats when I suggested a dict would be better. Of course he also asked this question after already getting an equivalent answer there, so… who knows what he's after.
figured out my question. Thanks for providing the perfect answer. this is exactly what i was looking for
@user1681664 Looks like your input data contains a bunch of blank lines, maybe? I'd trim them out when you're reading the data from whatever your data source is (e.g. if you're reading from a file, for line in file: if not line.strip(): continue). That way you don't waste time processing garbage data.
2 
d = {}
for date,temp in zip(dates,temps):
   try:
       d[date].append(temp)
   except KeyError:
       d[date] = [temp]

print d

you could use defaultdict to improve it slightly

from collections import defaultdict
d = defaultdict(list)
for date,temp in zip(dates,temps):
   d[date].append(temp)
Improve this answer

9 Comments

Is that widely accepted practice to use try in such a way? I would personally rather do "if date in d" rather than try but I don't really have a reason beyond "it looks better" :-). Plus, your way is likely faster anyway.
@CodeMonkey: See Easier to Ask Forgiveness than Permission (and the link to LBYL, and the entry on duck typing) in the glossary for why this is widely accepted practice.
@JoranBeasley: No idea why someone downvoted this; I already up voted it. Although I think using setdefault would be clearer than either a try statement or an if in this case.
yeah I dont disagree ... I just would do it with a defaultdict normally ... I dont think ive ever used the setdefault feature
@JoranBeasley: The way I see it, the choice between defaultdict and setdefault is entirely on how you intend to use the thing after building it. In many cases, you intend to use it exactly once to create some output, so it doesn't matter. But otherwise, it usually matters whether you get a KeyError or [] for looking up a missing key. (As a side note, setdefault is one of the few cases that can't easily be handled by the upcoming except-expression PEP, not yet published, but see the thread starting here.)
|
1

Try a dictionary where the key is the date and the result is an array that you append to. So you'd simply parse through the pairs and calling dataArray[date[i]].append(temp[i]) or the like. Don't forget to create the array if the key is currently null though.

Improve this answer

Comments

0

If you really want to have the array format, here is Joran's answer edited to give you the dataarray:

d = {}
for date,temp in zip(dates,temps):
   try:
       d[date].append(temp)
   except KeyError:
       d[date] = [temp]

dataarray = []
for year, temps in d.iteritems():
    tmp = [year]
    tmp.extend(temps)
    dataarray.append(tmp)
print dataarray
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.