Spring Data MongoDB Aggregation framework, exception accessing computed value in group

Question

I have a collection of trips and every trip has a startDateTime and a completionDateTime property.

I'm trying to use the aggregation framework to find the average duration of a users' trips.

My aggregation seems simple enough and but it throws an exception.

There are 3 steps below, the first just matches the trips to a specific userId. In the second step of the pipeline I project the data I'm interested in along with the newly calculated duration of each trip.

I expected the duration from the projection to be available in the group step but I get the IllegalArgumentException below. Duration Is not a property on my Trip class but from the examples I've seen in the docs I don't think it needs to be. Duration is a property on my analytics class.

TypedAggregation<Trip> aggregation = newAggregation(Trip.class, 
    match(Criteria.where("userId").is(aUserId)),

    project("completionDateTime", "startDateTime", "userId")
    .and("completionDateTime").minus("startDateTime").as("duration"),

    group("userId").avg("duration").as("averageDuration")
);
AggregationResults<Analytics> result = mongoTemplate.aggregate(aggregation, Analytics.class);

I believe the duration is getting calculated correctly as if I leave out the group step I get back a collection of documents and they all have the duration set.

java.lang.IllegalArgumentException: Invalid reference 'duration'!
    at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:78)
    at org.springframework.data.mongodb.core.aggregation.GroupOperation$Operation.getValue(GroupOperation.java:367)
    at org.springframework.data.mongodb.core.aggregation.GroupOperation$Operation.toDBObject(GroupOperation.java:363)
    at org.springframework.data.mongodb.core.aggregation.GroupOperation.toDBObject(GroupOperation.java:308)
    at org.springframework.data.mongodb.core.aggregation.Aggregation.toDbObject(Aggregation.java:247)

I'm using 1.3.4.RELEASE

Any Suggestions?

could you update your question with exactly how you ran the attempt against 1.4 and whether you got the exact same exception (if not could you include the one you did get?) — Asya Kamsky
– Asya Kamsky, Commented Apr 1, 2014 at 13:08
@AsyaKamsky I used the exact same code and got the exact same exception with 1.4.1.RELEASE — DannyLane
– DannyLane, Commented Apr 2, 2014 at 7:32

DannyLane · Accepted Answer · 2014-04-02 07:16:48Z

3

After upgrading to 1.4.1 I was able to get the desired result using:

.andExpression("completionDateTime - startDateTime").as("duration")

edited Apr 2, 2014 at 7:16

answered Apr 1, 2014 at 11:36

DannyLane

2,0961 gold badge16 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Asya Kamsky Over a year ago

you can edit your question or add this as a comment, it shouldn't go down here unless it's an answer...

DannyLane Over a year ago

This is the answer, it takes 24 hours to be able to accept my own answer.

Artem Bilan Over a year ago

This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.

DannyLane Over a year ago

@ArtemBilan Are you looking for clarification? I wanted to know how to compute a value in a projection and access it in a group. This is a work around for what appears to be a bug in the spring data framework.

Artem Bilan Over a year ago

Got it, sorry. Voting you

|

Asya Kamsky · Accepted Answer · 2014-03-31 23:36:26Z

1

It looks like you are running into https://jira.spring.io/browse/DATAMONGO-838 so until it's fixed you may need to figure out a different way to do what you need.

In mongo shell using js you don't need to have a $project field, you can just substitute in the computed expression into the group directly like so:

db.collection.aggregate( {$match:{ your-match } },
     { $group : { 
         _id : "$userId",
         avgDuration : { $avg : {$subtract:["$completionDateTime","startDateTime"]}},
     } } );

It might be possible to return the expression to do it this way in Spring. If not, your other option is to pass the aggregation directly to MongoDB through the Java driver, bypassing Spring until the bug is fixed.

answered Mar 31, 2014 at 23:36

Asya Kamsky

42.4k5 gold badges113 silver badges133 bronze badges

7 Comments

Oliver Drotbohm Over a year ago

Great advice. Actually the bug fix is already available in a released version. Would you mind updating your answer accordingly? I don't really want to add another answer basically repeating what you said.

DannyLane Over a year ago

I prematurely accepted this answer I'm afraid. I just updated to the 1.4.1.RELEASE version and still get the same error. Is the fix in a different version? Or maybe it's a different issue?

Asya Kamsky Over a year ago

sure, I'll edit based on updated bug info - but I do see that it was marked as being fixed in 1.4GA so either it wasn't fixed, wasn't fixed in all cases or isn't the exact same bug but a different one that manifests the same way.

Asya Kamsky Over a year ago

@DannyLane did you try the workaround of not using project and putting the computed expression directly into $group stage?

DannyLane Over a year ago

@AsyaKamsky I didn't try putting the computed expression directly in the group stage as I wanted to use the projected value multiple times (min, max, avg). I really wanted to get it setup in the project step and my answer above using .andExpression works for this.

|

Collectives™ on Stack Overflow

Spring Data MongoDB Aggregation framework, exception accessing computed value in group

2 Answers 2

6 Comments

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related