Pandas (with def and np.where): error with values in a dataframe row conditioned on another dataframe row

Question

I have dataframes A of shape XxY with values and dataframes B of shape ZxY to be filled with statistics calculated from A.

As an example:

A = pd.DataFrame(np.array(range(9)).reshape((3,3)))
B = pd.DataFrame(np.array(range(6)).reshape((2,3)))

Now I need to fill row 1 of B with quantile(0.5) of A columns where row 0 of B > 1 (else: np.nan). I need to use a function of the kind:

def mydef(df0, df1):
    df1.loc[1] = np.where(df1.loc[0]>1,
                          df0.quantile(0.5),
                          np.nan)
    pass

mydef(A,B)

Now B is:

    0   1   2
0   0.0 1.0 2.0
1   NaN NaN 3.5

It works perfectly for these mock dataframes and all my real ones apart from one. For that one this error is raised:

ValueError: cannot set using a list-like indexer with a different length than the value

When I run the same code without calling a function, it doesn't raise any error. Since I need to use a function, any suggestion?

makpalan · Accepted Answer · 2021-10-07 17:07:28Z

1

I found the error. I erroneously had the same label twice in the index. Essentially my dataframe B was something like:

B = pd.DataFrame(np.array(range(9)).reshape((3,3)), index=[0,0,1])

so that calling the def:

def mydef(df0, df1):
df1.loc[1] = np.where(df1.loc[0]>1,
                      df0.quantile(0.5),
                      np.nan)
pass

would cause the condition and the if-false lines of np.where to not match their shapes, I guess.

Still not sure why working outside the def worked.

edited Oct 7, 2021 at 17:07

answered Oct 7, 2021 at 15:15

makpalan

1459 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Francisco Maria Calisto Over a year ago

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

makpalan Over a year ago

@FranciscoMariaCalisto I've corrected my question and expanded the answer to make it clearer, thanks

Francisco Maria Calisto Over a year ago

Great work, congratulations! With your edited answer, we can now understand the scope issue of the problem that you are trying to solve.

Collectives™ on Stack Overflow

Pandas (with def and np.where): error with values in a dataframe row conditioned on another dataframe row

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related