concatenate multiple columns based on index in pandas

Question

As a follow up to this post, I would like to concatenate a number of columns based on their index but I am encountering some problems. In this example I get an Attribute error related to the map function. Help around this error would be appreciated as would code that does the equivalent concatenation of columns.

    #data
    df = DataFrame({'A':['a','b','c'], 'B':['d','e','f'], 'C':['concat','me','yo'], 'D':['me','too','tambien']})

    #row function to concat rows with index greater than 2
    def cnc(row):
        temp = []
        for x in range(2,(len(row))):
            if row[x] != None:
                temp.append(row[x])
        return map(concat, temp)

    #apply function per row
    new = df.apply(cnc,axis=1)

    #Expected Output
    new

    concat me
    me too
    yo tambien

thanks, zach cp

Please post the error you receive, as well as your expected output. — BrenBarn
– BrenBarn, Commented Aug 19, 2012 at 22:07

DSM · Accepted Answer · 2012-08-19 22:36:21Z

8

How about something like this?

>>> from pandas import *
>>> df = DataFrame({'A':['a','b','c'], 'B':['d','e','f'], 'C':['concat','me','yo'], 'D':['me','too','tambien']})
>>> df
   A  B       C        D
0  a  d  concat       me
1  b  e      me      too
2  c  f      yo  tambien
>>> df.columns[2:]
Index([C, D], dtype=object)
>>> df[df.columns[2:]]
        C        D
0  concat       me
1      me      too
2      yo  tambien
>>> [' '.join(row) for row in df[df.columns[2:]].values]
['concat me', 'me too', 'yo tambien']
>>> df["new"] = [' '.join(row) for row in df[df.columns[2:]].values]
>>> df
   A  B       C        D         new
0  a  d  concat       me   concat me
1  b  e      me      too      me too
2  c  f      yo  tambien  yo tambien

If you have None objects floating around, you could handle that too. For example:

>>> df["C"][1] = None
>>> df
   A  B       C        D
0  a  d  concat       me
1  b  e    None      too
2  c  f      yo  tambien
>>> rows = df[df.columns[2:]].values

In near-English:

>>> new = [' '.join(word for word in row if word is not None) for row in rows]
>>> new
['concat me', 'too', 'yo tambien']

Using filter:

>>> new = [' '.join(filter(None, row)) for row in rows]
>>> new
['concat me', 'too', 'yo tambien']

etc. You could do it in one line but I think it's clearer to separate it.

edited Aug 19, 2012 at 22:36

answered Aug 19, 2012 at 22:20

DSM

355k67 gold badges606 silver badges504 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

zach Over a year ago

Can you also eliminate None values using the list comprehension?

Wes McKinney Over a year ago

note that support for filter(None, iterable) ceased in Python 3, need to do filter(bool, iterable) there

Collectives™ on Stack Overflow

concatenate multiple columns based on index in pandas

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related