Python function partial string match

Question

I have a pandas dataframe like this:

a      b      c
foo    bar    baz
bar    foo    baz
foobar barfoo baz

I've defined the following function in python:

def somefunction (row):
    if row['a'] == 'foo' and row['b'] == 'bar':
        return 'yes'
    return 'no'

It works perfectly fine. But I need to make a small tweak to the if function to take into account partial string matches.

I've tried several combinations, but I can't seem to get it to work. I get the following error:

("'str' object has no attribute 'str'", 'occurred at index 0')

The function Iv'e tried is:

def somenewfunction (row):
    if row['a'].str.contains('foo')==True and row['b'] == 'bar':
        return 'yes'
    return 'no'

jezrael · Accepted Answer · 2017-11-02 12:23:19Z

Use contains for boolean mask and then numpy.where:

m = df['a'].str.contains('foo') & (df['b'] == 'bar')
print (m)
0     True
1    False
2    False
dtype: bool

df['new'] = np.where(m, 'yes', 'no')
print (df)
        a       b    c  new
0     foo     bar  baz  yes
1     bar     foo  baz   no
2  foobar  barfoo  baz   no

Or if need alo check column b for substrings:

m = df['a'].str.contains('foo') & df['b'].str.contains('bar')
df['new'] = np.where(m, 'yes', 'no')
print (df)
        a       b    c  new
0     foo     bar  baz  yes
1     bar     foo  baz   no
2  foobar  barfoo  baz  yes

If need custom function, what should be slowier in bigger DataFrame:

def somefunction (row):
    if 'foo' in row['a'] and row['b'] == 'bar':
        return 'yes'
    return 'no'

print (df.apply(somefunction, axis=1))
0    yes
1     no
2     no
dtype: object

def somefunction (row):
    if 'foo' in row['a']  and  'bar' in row['b']:
        return 'yes'
    return 'no'

print (df.apply(somefunction, axis=1))
0    yes
1     no
2    yes
dtype: object

Timings:

df = pd.concat([df]*1000).reset_index(drop=True)

def somefunction (row):
    if 'foo' in row['a'] and row['b'] == 'bar':
        return 'yes'
    return 'no'

In [269]: %timeit df['new'] = df.apply(somefunction, axis=1)
10 loops, best of 3: 60.7 ms per loop

In [270]: %timeit df['new1'] = np.where(df['a'].str.contains('foo') & (df['b'] == 'bar'), 'yes', 'no')
100 loops, best of 3: 3.25 ms per loop

df = pd.concat([df]*10000).reset_index(drop=True)

def somefunction (row):
    if 'foo' in row['a'] and row['b'] == 'bar':
        return 'yes'
    return 'no'

In [272]: %timeit df['new'] = df.apply(somefunction, axis=1)
1 loop, best of 3: 614 ms per loop

In [273]: %timeit df['new1'] = np.where(df['a'].str.contains('foo') & (df['b'] == 'bar'), 'yes', 'no')
10 loops, best of 3: 23.5 ms per loop

alkanen · Accepted Answer · 2017-11-02 12:29:58Z

1

Your exception is probably from the fact that you write

if row['a'].str.contains('foo')==True

Remove '.str':

if row['a'].contains('foo')==True

answered Nov 2, 2017 at 12:29

alkanen

6546 silver badges16 bronze badges

Collectives™ on Stack Overflow

Python function partial string match

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related