Dataframe ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() [duplicate]

Question

I have a dataframe (reportingDatesDf) which the head looks like this:

           unique_stock_id reporting_type
date                                     
2015-01-28  BBG.MTAA.STM.S         2014:A
2015-01-28  BBG.MTAA.STM.S        2014:S2
2015-01-28  BBG.MTAA.STM.S        2014:Q4
2014-10-29  BBG.MTAA.STM.S        2014:C3
2014-10-29  BBG.MTAA.STM.S        2014:Q3

I am trying to reduce the dataframe to include entries that are only between 2 dates with the following line:

reportingDatesDf = reportingDatesDf[(reportingDatesDf.index >= startDate) and (reportingDatesDf.index <= endDate)]

The dataframe is created from a CSV using the following code:

def getReportingDatesData(rawStaticDataPath,startDate,endDate):
    pattern = 'ReportingDates'+ '.csv'
    staticPath = rawStaticDataPath
    
    with open(staticPath+pattern,'rt') as f:
        
         reportingDatesDf = pd.read_csv(f, 
                 header=None,
                 usecols=[0,1,2],
                 parse_dates=[1],
                 dayfirst=True,
                 index_col=[1],
                 names=['unique_stock_id','date','reporting_type'])       
         #print(reportingDatesDf.head())
         print('reportingDatesDf.index',reportingDatesDf)      
         reportingDatesDf = reportingDatesDf[(reportingDatesDf.index >= startDate) and (reportingDatesDf.index <= endDate)]

I however get the error:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Why has this happened? I am using similar code elsewhere which works.

Even better than the answers below, you can just use slicing: reportingDatesDf.loc[startDate:endDate] (assuming the index is datetime dtype) — wjandrea
– wjandrea, Commented Dec 1, 2024 at 15:35

user2357112 · Accepted Answer · 2016-04-27 21:36:22Z

2

and doesn't broadcast. It can't, because it has to short-circuit, and there's no good answer for making the short-circuiting broadcast.

If you need to do an elementwise and, you should use & instead.

answered Apr 27, 2016 at 21:36

user2357112

286k32 gold badges490 silver badges569 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Tim · Accepted Answer · 2016-04-27 21:30:09Z

0

It looks to me like in the line

reportingDatesDf = reportingDatesDf[(reportingDatesDf.index >= startDate) and (reportingDatesDf.index <= endDate)]

the variable

reportingDatesDf.index

is an array. Thus, saying

(reportingDatesDf.index >= startDate)

is ambiguous. You need to specify whether you are checking if all of the values in the array are greater than startDate, or if it contains any values greater than startDate. Editing your code to the following

reportingDatesDf = reportingDatesDf[any(reportingDatesDf.index >= startDate) and any(reportingDatesDf.index <= endDate)]

or

reportingDatesDf = reportingDatesDf[all(reportingDatesDf.index >= startDate) and all(reportingDatesDf.index <= endDate)]

should fix the issue.

answered Apr 27, 2016 at 21:30

Tim

2,8531 gold badge18 silver badges33 bronze badges

1 Comment

Stacey Over a year ago

Thanks Tim, I get a KeyError: True error when I implement the code, Does this mean I'm possibly missing some data somewhere?

Thanos · Accepted Answer · 2016-04-27 21:49:49Z

0

Try changing this:

reportingDatesDf = reportingDatesDf[(reportingDatesDf.index >= startDate) and (reportingDatesDf.index <= endDate)]

to this:

reportingDatesDf = reportingDatesDf[(reportingDatesDf.index >= startDate) & (reportingDatesDf.index <= endDate)]

In other words, user the proper operator ;)

answered Apr 27, 2016 at 21:49

Thanos

2,5601 gold badge20 silver badges33 bronze badges

Collectives™ on Stack Overflow

Dataframe ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() [duplicate]

3 Answers 3

Comments

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Linked

Related