Python pandas loop value conditional on two columns

Question

In my dataframe 'data' I have two columns 'trend' & 'rtrend'

trend has values -1, 0 and 1.

def newfunc(a):

j = -1

for i in a:

    j = j+1
    x = (j-1)

    if data.iloc[j]['trend'] != 0:

        return data.iloc[j]['trend'] 

    if data.iloc[j]['trend'] == 0:

        return data.iloc[x]['rtrend']

If trend is equal to -1 or 1 then I'd like to set the rtrend column value equal to trend.

If trend equals 0, then set rtrend equal to the last value in that series which appears above in the dataframe.

data['rtrend'] = newfunc(data['trend'])

All it currently returns is 0 for the whole series.

Please could someone point me in the right direction? I'm sure there must be a better way to do this. (I've tried np.where() which doesn't seem to do what I'm after).

Zeugma · Accepted Answer · 2017-01-22 01:44:45Z

4

Don't do a procedural slow for loop. Do the vectorized approach. Just copy non zero data into your new rtrend column, then forward fill the data:

df['rtrend'] = df[df.trend!=0]['trend']

df
Out[21]: 
   trend    b    c  rtrend
a   -1.0  1.0 -1.0    -1.0
c    0.0 -1.0  1.0     NaN
e    1.0 -1.0 -1.0     1.0
f   -1.0  1.0 -1.0    -1.0
h   -1.0  1.0  1.0    -1.0

df['rtrend'].ffill()
Out[22]: 
a   -1.0
c   -1.0
e    1.0
f   -1.0
h   -1.0
Name: rtrend, dtype: float64

answered Jan 22, 2017 at 1:44

Zeugma

32.3k9 gold badges73 silver badges85 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Zeugma Over a year ago

@piRSquared :-) Thanks!

Ted Petrou Over a year ago

You should use .loc whenever possible to avoid the setingwithcopy warning. df.loc[df.trend != 0, 'trend']

Zeugma Over a year ago

@TedPetrou hmm no because in this case I'm not dealing with a left-assignment value, the dataframe view I'm doing is the right side of the assignment (the left side being the new column rtrend)

Matthewj28 Over a year ago

@Boud Thanks, this would fix the problem although for some reason I can't get the last part to work. 'data['rtrend'].ffill()' to work. It doesn't return any errors but 'NaN' still exists in the code and appears to not have done anything. Any suggestions please?

Zeugma Over a year ago

Amend your question with a sample dump of your df to illustrate your issue

|

Collectives™ on Stack Overflow

Python pandas loop value conditional on two columns

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related