Decimal in input JSON to Python 3 Lambda Function

Question

Consider the following input in a financial application, where precision matters:

{ "value": 3.8 }

And the following AWS Lambda function:

from decimal import Decimal

def lambda_handler(event, context):
    value = event['value']
    print(Decimal(value))

The output is: 3.79999999999999982236431605997495353221893310546875 because Python parsed the number in the JSON into a float, which can't precisely store 3.8.

I know that I can serialize event back to a string and then instruct the parser to use Decimal (this is from the DynamoDB Python docs):

import json
def lambda_handler(event, context):
    parsed = json.loads(json.dumps(event), parse_float=Decimal)
    print(Decimal(parsed['value']))

But that feels like a hack. Is there some way to control the deserialization in the first place so that event prefers Decimal to float?

Please provide complete (running) code. Try to print value and not Decimal(value), Decimal(value) may get casted to a float. — cgte
– cgte, Commented May 10, 2018 at 22:15
This is complete, running code. I was copy/pasting from the AWS Lambda console. I need a Decimal later in the code for other reasons, but the print demonstrates the problem. — Travis P
– Travis P, Commented May 10, 2018 at 22:18
When I say "other reasons," I mean, that rounding is not acceptable. E.g., Decimal(3.8)*Decimal(10) returns 37.99999999999999822364316060. No bueno in a financial application. — Travis P
– Travis P, Commented May 10, 2018 at 22:22
If you always wish to round to cents, it might be easier to store such values as integers in cents. This way, there will never be a partial-cent: 380 * 10 = 3800 — John Rotenstein
– John Rotenstein, Commented May 10, 2018 at 22:41
I meant code that one can copy paste and run. Maybe put json.loads(json.dumps(event), parse_float=Decimal) in a function so that the code looks at a bit better. Maybe you can change but you will have to look where the event object is created. Maybe Sympy can help you. Good luck. — cgte
– cgte, Commented May 10, 2018 at 22:43

jpp · Accepted Answer · 2018-05-14 13:59:49Z

3

Update: There is nothing wrong with your current solution.

There is no float to str to decimal.Decimal round-trip.

As the docs explain (my emphasis):

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default, this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

Initial answer below

Passing a float value to decimal.Decimal does not ensure the precision you require. This is because, by its nature, float is not stored as a decimal but in binary.

This can be alleviated if you are able to pass string inputs into decimal.Decimal:

from decimal import Decimal

res1 = Decimal(3.8)*Decimal(10)
res2 = Decimal('3.8')*Decimal('10')

print(res1)  # 37.99999999999999822364316060
print(res2)  # 38.0

So one solution would be to ensure you store / read in JSON numeric data as strings instead of floats.

Be careful, an implementation as below may work but relies on str doing a particular job, i.e. rounding a float correctly for decimal representation.

def lambda_handler(event):
    value = event['value']
    print(Decimal(str(value)))

lambda_handler({"value": 3.8})  # 3.8

edited May 14, 2018 at 13:59

answered May 10, 2018 at 22:48

jpp

166k37 gold badges301 silver badges362 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Travis P Over a year ago

Thanks for the reply. This is essentially what my json.dumps -> json.loads hack does. As you say, it relies on a black box to print strings from floats in a particular way. I think the problem here is the way Lambda parses the JSON into the event dict and decides that value is a float and not a Decimal.

jpp Over a year ago

@TravisPettijohn, Yep, I'm sorry I don't know a better solution. Hopefully someone can enlighten us. Or we offer a bounty to get some more attention in a couple of days.

jpp Over a year ago

@TravisPettijohn, We did a bit more research. It seems your solution is not a hack. I've added a reference to the documentation to explain. Hope that helps.

Travis P Over a year ago

Thanks so much for the additional digging. I'm not sure I agree. I interpret the documentation you reference as describing how json.loads() works: json.loads('{ "value": 3.8 }', parse_float=Decimal) uses the string value - that is, calls Decimal("3.8") to handle the number in JSON. There's still ambiguity on how Lambda handles floats (we know it uses float, not Decimal), and how json.dumps(event) converts those numbers to strings.

jpp Over a year ago

Decimal('3.8') is exactly what we need, though, right? This is recommended by the decimal docs and guarantees you have no rounding issues. What we don't want is Decimal(str(3.8)).

|

Collectives™ on Stack Overflow

Decimal in input JSON to Python 3 Lambda Function

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related