Divide rows of python pandas DataFrame

Question

I have a pandas DataFrame df like this

   mat  time
0  101   20
1  102    7
2  103   15

I need to divide the rows so the column of time doesn't have any values higher than t=10 to have something like this

   mat  time
0  101   10
2  101   10
3  102    7
4  103   10
5  103    5

the index doesn't matter

If I'd use groupby('mat')['time'].sum() on this df I would have the original df, but I need like an inverse of the groupby func.

Is there any way to get the ungrouped DataFrame with the condition of time <= t?

I'm trying to use a loop here but it's kind of 'unPythonic', any ideas?

piRSquared · Accepted Answer · 2016-06-01 00:15:09Z

1

Use an apply function that loops until all are less than 10.

def split_max_time(df):
    new_df = df.copy()
    while new_df.iloc[-1, -1] > 10:
        temp = new_df.iloc[-1, -1]
        new_df.iloc[-1, -1] = 10
        new_df = pd.concat([new_df, new_df])
        new_df.iloc[-1, -1] = temp - 10
    return new_df


print df.groupby('mat', group_keys=False).apply(split_max_time)

   mat  time
0  101    10
0  101    10
1  102     7
2  103    10
2  103     5

answered Jun 1, 2016 at 0:15

piRSquared

296k68 gold badges509 silver badges654 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Stefan · Accepted Answer · 2016-06-15 00:34:29Z

1

You could .groupby('mat') and .apply() a combination of integer division and modulo operation using the cutoff (10) to decompose each time value into the desired components:

cutoff = 10
def decompose(time):
    components = [cutoff for _ in range(int(time / cutoff))] + [time.iloc[0] % cutoff]
    return pd.Series([c for c in components if c > 0])

df.groupby('mat').time.apply(decompose).reset_index(-1, drop=True)

to get:

In case you care about performance:

%timeit df.groupby('mat', group_keys=False).apply(split_max_time)
100 loops, best of 3: 4.21 ms per loop

%timeit df.groupby('mat').time.apply(decompose).reset_index(-1, drop=True)
1000 loops, best of 3: 1.83 ms per loop

edited Jun 15, 2016 at 0:34

answered Jun 1, 2016 at 4:05

Stefan

43.1k13 gold badges80 silver badges84 bronze badges

Collectives™ on Stack Overflow

Divide rows of python pandas DataFrame

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related