Adding row shifting in pandas dataframe

Question

I have a pandas df, which I created by using shift() function iterating through the original df:

for i in range(2, 4):
    df["lag_{}".format(i)] = df.x.shift(i)

So there will be actual x column and lag2-lag10 columns with shifted x values. I have trained this dataset for the regression model to make one-step forward prediction. Would like to add the new row in the end of the dataframe with nan value for x and shifted values from the last position to be able to use these new lags for fitting the model to predict this new nan value. How this can be done in pandas? Thanks!

Upd: There is the pic for the df, unbolded-the df, bold-the desired row to get:

Can you add expected output if df = pd.DataFrame({'x':range(5}) ? — jezrael
– jezrael, Commented Apr 16, 2020 at 7:35

jezrael · Accepted Answer · 2020-04-16 07:45:41Z

Use DataFrame.append with dictionary with key x:

df = pd.DataFrame({'x':range(10)})

df1 = df.append({'x':np.nan}, ignore_index=True)
#alternative
#df1 = df.append(pd.Series([np.nan], index=['x']), ignore_index=True)

for i in range(2, 10):
    df1["lag_{}".format(i)] = df1.x.shift(i)
print (df1)
      x  lag_2  lag_3  lag_4  lag_5  lag_6  lag_7  lag_8  lag_9
0   0.0    NaN    NaN    NaN    NaN    NaN    NaN    NaN    NaN
1   1.0    NaN    NaN    NaN    NaN    NaN    NaN    NaN    NaN
2   2.0    0.0    NaN    NaN    NaN    NaN    NaN    NaN    NaN
3   3.0    1.0    0.0    NaN    NaN    NaN    NaN    NaN    NaN
4   4.0    2.0    1.0    0.0    NaN    NaN    NaN    NaN    NaN
5   5.0    3.0    2.0    1.0    0.0    NaN    NaN    NaN    NaN
6   6.0    4.0    3.0    2.0    1.0    0.0    NaN    NaN    NaN
7   7.0    5.0    4.0    3.0    2.0    1.0    0.0    NaN    NaN
8   8.0    6.0    5.0    4.0    3.0    2.0    1.0    0.0    NaN
9   9.0    7.0    6.0    5.0    4.0    3.0    2.0    1.0    0.0
10  NaN    8.0    7.0    6.0    5.0    4.0    3.0    2.0    1.0

Collectives™ on Stack Overflow

Adding row shifting in pandas dataframe

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related