AWS Gateway API fails to convert DynamoDb JSON in array to regular JSON

Question

I am attempting to create a service which outputs json from a dynamo db database. However, after implementing body mapping templates in amazon gateway api, the service only converts pieces of the dynamodb json to regular json. There is no error visible in the logs. I have attached below the body mapping templates, and the response body for the GET.

Body Mapping Template:

    #set($inputRoot = $input.path('$'))
    #foreach($elem in $inputRoot.Items) {
        "accession" : "$elem.accession.S",
        "entryName" : "$elem.entryName.S",
        "sequence" : "$elem.sequence.S",
        "sequenceChecksum" : "$elem.sequenceChecksum.S",
        "taxid" : "$elem.taxid.N",
        "features" : "$elem.features.L"
    } #if($foreach.hasNext),#end
    #end

Response:

 {
   "accession" : "P05067",
   "entryName" : "A4_HUMAN",
   "sequence" : "MLPGLALLLLAAWTARALEVPTDGNAGLLAEPQIAMFCGRLNMHMNVQNGKWDSDPSGTKTCIDTKEGILQYCQEVYPELQITNVVEANQPVTIQNWCKRGRKQCKTHPHFVIPYRCLVGEFVSDALLVPDKCKFLHQERMDVCETHLHWHTVAKETCSEKSTNLHDYGMLLPCGIDKFRGVEFVCCPLAEESDNVDSADAEEDDSDVWWGGADTDYADGSEDKVVEVAEEEEVAEVEEEEADDDEDDEDGDEVEEEAEEPYEEATERTTSIATTTTTTTESVEEVVREVCSEQAETGPCRAMISRWYFDVTEGKCAPFFYGGCGGNRNNFDTEEYCMAVCGSAMSQSLLKTTQEPLARDPVKLPTTAASTPDAVDKYLETPGDENEHAHFQKAKERLEAKHRERMSQVMREWEEAERQAKNLPKADKKAVIQHFQEKVESLEQEAANERQQLVETHMARVEAMLNDRRRLALENYITALQAVPPRPRHVFNMLKKYVRAEQKDRQHTLKHFEHVRMVDPKKAAQIRSQVMTHLRVIYERMNQSLSLLYNVPAVAEEIQDEVDELLQKEQNYSDDVLANMISEPRISYGNDALMPSLTETKTTVELLPVNGEFSLDDLQPWHSFGADSVPANTENEVEPVDARPAADRGLTTRPGSGLTNIKTEEISEVKMDAEFRHDSGYEVHHQKLVFFAEDVGSNKGAIIGLMVGGVVIATVIVITLVMLKKKQYTSIHHGVVEVDAAVTPEERHLSKMQQNGYENPTYKFFEQMQN",
   "sequenceChecksum" : "A12EE761403740F5",
   "taxid" : "9606",
   "features" : "[{"M":{"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":" "},"type":"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"Amyloid beta A4 protein"},"type":{"S":"CHAIN"},"end":{"S":"770"},"begin":{"S":"18"}}},{"M":{"ftId":{"S":"PRO_0000000089"},"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"Soluble APP-alpha"},"type":{"S":"CHAIN"},"end":{"S":"687"},"begin":{"S":"18"}}},{"M":{"ftId":{"S":"PRO_0000000090"},"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"Soluble APP-beta"},"type":{"S":"CHAIN"},"end":{"S":"671"},"begin":{"S":"18"}}},{"M":{"ftId":{"S":"PRO_0000381966"},"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"N-APP"},"type":{"S":"CHAIN"},"end":{"S":"286"},"begin":{"S":"18"}}},{"M":{"ftId":{"S":"PRO_0000000091"},"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"C99"},"type":{"S":"CHAIN"},"end":{"S":"770"},"begin":{"S":"672"}}},{"M":{"ftId":{"S":"PRO_0000000092"},"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"Beta-amyloid protein 42"},"type":{"S":"CHAIN"},"end":{"S":"713"},"begin":{"S":"672"}}}]"
        }         
Response Headers
{"Content-Type":"application/json"}
Logs
Execution log for request test-request
Mon Aug 08 15:26:05 UTC 2016 : Starting execution for request: test-invoke-request
Mon Aug 08 15:26:05 UTC 2016 : HTTP Method: GET, Resource Path: /proteins/{accession}
Mon Aug 08 15:26:05 UTC 2016 : Method request path: {accession=P05067}
Mon Aug 08 15:26:05 UTC 2016 : Method request query string: {}
Mon Aug 08 15:26:05 UTC 2016 : Method request headers: {}
Mon Aug 08 15:26:05 UTC 2016 : Method request body before transformations: null
Mon Aug 08 15:26:05 UTC 2016 : Endpoint request URI: https://dynamodb.us-west-2.amazonaws.com/?Action=Query
Mon Aug 08 15:26:05 UTC 2016 : Endpoint request headers: {Authorization=****************************************************************************************************************************************************************************************************************************************************************************************b0b302, X-Amz-Date=20160808T152605Z, x-amzn-apigateway-api-id=9x56sueb85, Accept=application/json, User-Agent=AmazonAPIGateway_9x56sueb85, X-Amz-Security-Token=AgoGb3JpZ2luEJv//////////wEaCXVzLXdlc3QtMiKAAlXlB1cz9vo5Kf2llpupTpP1fTiHMBBbZhOmQW30/jCc5Q3RV+BM9k0LtqfJXRdRpzw5DEHg1dmlA1k8Ljha+og4RGYFdpj/9wdc4u1WKnZdy/lZFUAMey0YotNc+RniWyMq+ZiVhY94Sv/zKJ+dxSGkDZbz5A6Jbfj4EfVFuMLC3kHA4tJKWp6PCXpyHJqFqQ+UuI/q0coHNQv0euBD6hNUBOEBZes2TIQdTha8f4k+avX7o1f3LcpIjfvdPN4InOXZ7ZMHDgpLEuxurOZ7taZjoXftHxpRG2GAciTNj7gQASCsxRAQL/4gujC6yydGievEE6V5Zn5prIRnHPz0Lmcq8QII8f//////////ARAAGgw5MTUzMzI4Mzc1NDAiDD4AxveOBPRjUQ28ZirFAj5mJUN8gxUfXUQc1AzD08pLgpAtrz11K1Xgax/ATvptUj//Pcy+4fS90PqdZSqMSmS8KsD0X46m7GfhNNzuQypCdY3lyyN [TRUNCATED]
Mon Aug 08 15:26:05 UTC 2016 : Endpoint request body after transformations: {
    "TableName": "Protein_DB",
    "IndexName": "accession-index",
    "KeyConditionExpression": "accession = :v1",
        "ExpressionAttributeValues": {
            ":v1":  {
                "S": "P05067"
            }
        }
}
Mon Aug 08 15:26:05 UTC 2016 : Endpoint response body before transformations: {"Count":1,"Items":[{"accession":{"S":"P05067"},"features":{"L":[{"M":{"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":" "},"type":{"S":"SIGNAL"},"evidences":{"L":[{"M":{"source":{"M":{"id":{"S":"12665801"},"alternativeUrl":{"S":"http://europepmc.org/abstract/MED/12665801"},"name":{"S":"PubMed"},"url":{"S":"http://www.ncbi.nlm.nih.gov/pubmed/12665801"}}},"code":{"S":"ECO:0000269"}}},{"M":{"source":{"M":{"id":{"S":"2900137"},"alternativeUrl":{"S":"http://europepmc.org/abstract/MED/2900137"},"name":{"S":"PubMed"},"url":{"S":"http://www.ncbi.nlm.nih.gov/pubmed/2900137"}}},"code":{"S":"ECO:0000269"}}},{"M":{"source":{"M":{"id":{"S":"3597385"},"alternativeUrl":{"S":"http://europepmc.org/abstract/MED/3597385"},"name":{"S":"PubMed"},"url":{"S":"http://www.ncbi.nlm.nih.gov/pubmed/3597385"}}},"code":{"S":"ECO:0000269"}}}]},"end":{"S":"17"},"begin":{"S":"1"}}},{"M":{"ftId":{"S":"PRO_0000000088"},"category":{"S":"MOLECULE_PROCESSING"},"description":{"S":"Amyloid b [TRUNCATED]
Mon Aug 08 15:26:05 UTC 2016 : Endpoint response headers: {x-amzn-RequestId=J0J7K1RFDOGEK20V70T0HKJ14JVV4KQNSO5AEMVJF66Q9ASUAAJG, x-amz-crc32=273504943, Content-Length=75281, Date=Mon, 08 Aug 2016 15:26:05 GMT, Content-Type=application/x-amz-json-1.0}
Mon Aug 08 15:26:05 UTC 2016 : Method response body after transformations: 
         {
            "accession" : "P05067",
            "entryName" : "A4_HUMAN",
            "sequence" : "MLPGLALLLLAAWTARALEVPTDGNAGLLAEPQIAMFCGRLNMHMNVQNGKWDSDPSGTKTCIDTKEGILQYCQEVYPELQITNVVEANQPVTIQNWCKRGRKQCKTHPHFVIPYRCLVGEFVSDALLVPDKCKFLHQERMDVCETHLHWHTVAKETCSEKSTNLHDYGMLLPCGIDKFRGVEFVCCPLAEESDNVDSADAEEDDSDVWWGGADTDYADGSEDKVVEVAEEEEVAEVEEEEADDDEDDEDGDEVEEEAEEPYEEATERTTSIATTTTTTTESVEEVVREVCSEQAETGPCRAMISRWYFDVTEGKCAPFFYGGCGGNRNNFDTEEYCMAVCGSAMSQSLLKTTQEPLARDPVKLPTTAASTPDAVDKYLETPGDENEHAHFQKAKERLEAKHRERMSQVMREWEEAERQAKNLPKADKKAVIQHFQEKVESLEQEAANERQQLVETHMARVEAMLNDRRRLALENYITALQAVPPRPRHVFNMLKKYVRAEQKDRQHTLKHFEHVRMVDPKKAAQIRSQVMTHLRVIYERMNQSLSLLYNVPAVAEEIQDEVDELLQKEQNYSDDVLANMISEPRISYGNDALMPSLTETKTTVELLPVNGEFSLDDLQPWHSFGADSVPANTENEVEPVDARPAADRGLTTRPGSGLTNIKTEEISEVKMDAEFRHDSGYEVHHQKLVFFAEDVGSNKGAIIGLMVGGVVIATVIVITLVMLKKKQYTSIHHGVVEVDAAVTPEERHLSKMQQNGYENPTYKFFEQMQN",
            "sequenceChecksum" : "A12EE761403740F5",
            "taxid" : "9606",
             [TRUNCATED]
Mon Aug 08 15:26:05 UTC 2016 : Method response headers: {Content-Type=application/json}
Mon Aug 08 15:26:05 UTC 2016 : Successfully completed execution
Mon Aug 08 15:26:05 UTC 2016 : Method completed with status: 200

I've attempted to format the featuers field with little sucess.Here is my attempt:

#set($inputRoot = $input.path('$'))

        #foreach($elem in $inputRoot.Items) {
            "accession" : "$elem.accession.S",
            "entryName" : "$elem.entryName.S",
            "sequence" : "$elem.sequence.S",
            "sequenceChecksum" : "$elem.sequenceChecksum.S",
            "taxid" : "$elem.taxid.N",
            "features" : "$elem.features.L"
        }#if($foreach.hasNext),#end
#end

#foreach($elem in $inputRoot.Items.features)
      {
        "alternativeSequence": "$elem.alternativeSequence.S",
        "begin": "$elem.begin.S",
        "category": "$elem.category.S",
        "description": "$elem.description.S",
        "end": "$elem.end.S",
        "evidences": "$elem.evidences.L",
        "ftID": "$elem.ftId.S",
        "type": "$elem.type.S"
      }#if($foreach.hasNext),#end

#end``

jackko · Accepted Answer · 2016-08-08 16:55:35Z

1

Is the problem the 'features' field? I think you just added quotes when they aren't necessary.

#set($inputRoot = $input.path('$'))
#foreach($elem in $inputRoot.Items) {
    "accession" : "$elem.accession.S",
    "entryName" : "$elem.entryName.S",
    "sequence" : "$elem.sequence.S",
    "sequenceChecksum" : "$elem.sequenceChecksum.S",
    "taxid" : "$elem.taxid.N",
    "features" : $elem.features.L
} #if($foreach.hasNext),#end
#end

answered Aug 8, 2016 at 16:55

jackko

7,37431 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

ndeluca18 Over a year ago

Yes, the problem is the features field. It is a large list, and will format correctly in regular json, however, its contents remain in dynamodb json format.

ndeluca18 Over a year ago

This appears to only un-minimize the dynamodb json.

jackko Over a year ago

So what exactly are you thinking should happen? API Gateway doesn't know the schema of the DDB record, so you'll have to do any transformation to pretty JSON yourself, as you have done with the other fields.

ndeluca18 Over a year ago

I can't figure out how to modify the contents of the features array so that it becomes regular JSON.

ndeluca18 · Accepted Answer · 2016-08-12 19:07:48Z

The issue was that $elem is a variable not a function, and could not be reused in the foreach loop. A new name for the variable must be given such as $elem1. I have attached below the final working body template.

#set($inputRoot = $input.path('$'))

     #foreach($elem in $inputRoot.Items) {
        "accession" : "$elem.accession.S",
        "entryName" : "$elem.entryName.S",
        "sequence" : "$elem.sequence.S",
        "sequenceChecksum" : "$elem.sequenceChecksum.S",
        "taxid" : $elem.taxid.N,
        "features" : [
                        #foreach($elem1 in $elem.features.L){
                            "alternativeSequence": "$elem1.M.alternativeSequence.S",
                            "begin": "$elem1.M.begin.S",
                            "category": "$elem1.M.category.S",
                            "description": "$elem1.M.description.S",
                            "end": "$elem1.M.end.S",
                            "evidences": [
                                            #foreach($elem2 in $elem1.M.evidences.L){
                                                "code":"$elem2.M.code.S",
                                                "source" : #foreach($elem3 in $elem2.M.source){
                                                           "alternativeUrl" : "$elem3.alternativeUrl.S",
                                                           "id":"$elem3.id.S",
                                                           "name":"$elem3.name.S",
                                                           "url":"$elem3.url.S",
                                                       }#if($foreach.hasNext),#end
                                                       #end
                                            }#if($foreach.hasNext),#end
                                            #end
                                        ],
                            "ftID": "$elem1.M.ftId.S",
                            "type": "$elem1.M.type.S"
                        }#if($foreach.hasNext),#end
                        #end
                    ]
    }#if($foreach.hasNext),#end

Collectives™ on Stack Overflow

AWS Gateway API fails to convert DynamoDb JSON in array to regular JSON

2 Answers 2

4 Comments

end

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

end

Comments

Your Answer

Sign up or log in

Post as a guest

Related