Index duplicate error in pandas when there are no duplicates

Question

I have a dataframe with a datetime index and then I try to convert it to json using to_json() I get an error saying:

ValueError: DataFrame index must be unique for orient='columns'

However my index column should have unique values. I tried df.duplicates() and got this:

time
2018-01-28    False
2018-02-04    False
2018-02-11    False
2018-02-18    False
2018-02-25    False
              ...
2019-12-29    False
2020-01-05    False
2020-01-12    False
2020-01-19     True
2020-01-26    False
Length: 112, dtype: bool

You can clearly see that the second to last column is marked as a duplicate when it's not. I also tried drop_duplicates(keep='last') but I still got the same error.

Why is this happening?

ansev · Accepted Answer · 2020-01-26 12:03:25Z

2

the problem is that your index has duplicates, Series.drop_duplicates eliminates series with duplicate values (values, not in index).

Your error:

DataFrame index must be unique

Use groupby.first:

s.groupby(s.index).first()

for each unique index we select the first value

edited Jan 26, 2020 at 12:03

answered Jan 26, 2020 at 11:10

ansev

31k5 gold badges21 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Index duplicate error in pandas when there are no duplicates

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related