Find max value of a dataframe column containing numpy arrays

Question

I was trying to find the maximum value of a column in a dataframe that contains numpy arrays.

df = pd.DataFrame({'id': [1, 2, 33, 4],
                   'a': [1, 22, 23, 44],
                   'b': [1, 42, 23, 42]})
df['new'] = df.apply(lambda r: tuple(r), axis=1).apply(np.array)

This how the dataframe can look like:

    id  a   b   new
0   1   1   1   [1, 1, 1]
1   2   22  42  [2, 22, 42]
2   33  23  23  [33, 23, 23]
3   4   44  42  [4, 44, 42]

Now I want to find the maximum (single) value of column new. In this case it is 44. What about a quick and easy way?

Yes, alway the same dimension! In the real world it has up to 8000 entries. — kopsman
– kopsman, Commented Feb 3, 2023 at 22:26

SomeDude · Accepted Answer · 2023-02-03 22:37:16Z

1

Because your new column is actually constructed from the columns id, a, b. Before you create the new column you can do:

single_max = np.max(df.values)

OR if you insist on your dataframe to contain the new column and then get max you can do:

single_max = np.max(df.drop('new',axis=1).values)

answered Feb 3, 2023 at 22:37

SomeDude

14.3k5 gold badges26 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

tdelaney · Accepted Answer · 2023-02-03 22:23:40Z

0

You can apply a lambda to the values that calls the array's max method. This would result in a Series that also has a max method.

df['new'].apply(lambda arr: arr.max()).max()

Just guessing, but this should be faster than .apply(max) because you use the optimized array method instead of converting the numpy ints to python ints one by one.

answered Feb 3, 2023 at 22:23

tdelaney

77.9k6 gold badges91 silver badges129 bronze badges

Comments

PaulS · Accepted Answer · 2023-02-03 23:10:15Z

0

A possible solution:

df.new.explode().max()

Or a faster alternative:

np.max(np.vstack(df.new.values))

Returns 44.

edited Feb 3, 2023 at 23:10

answered Feb 3, 2023 at 22:19

PaulS

27.1k3 gold badges19 silver badges40 bronze badges

Comments

mozway · Accepted Answer · 2023-02-04 00:46:42Z

0

Assuming you only want to consider the columns "new":

import numpy as np
out = np.max(tuple(df['new'])) # or np.max(df['new'].tolist())

Output: 44

answered Feb 4, 2023 at 0:46

mozway

267k13 gold badges56 silver badges106 bronze badges

Comments

G.G · Accepted Answer · 2023-02-04 04:05:27Z

0

df1.new.map(pd.eval).explode().max()

Output: 44

answered Feb 4, 2023 at 4:05

G.G

7654 silver badges5 bronze badges

Comments

Laurent B. · Accepted Answer · 2023-02-04 13:15:18Z

0

1- Combination of max and explode()

df['new'].explode().max()
# 44

2- List comprehension

max([max(e) for e in df['new']])
# 44

edited Feb 4, 2023 at 13:15

answered Feb 4, 2023 at 13:07

Laurent B.

2,3032 gold badges10 silver badges18 bronze badges

Collectives™ on Stack Overflow

Find max value of a dataframe column containing numpy arrays

6 Answers 6

Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related