AWS Step Functions: Combine task input with partial task output

Question

I'm working on AWS Step functions. I've read the documentation on InputPath, OutputPath, and ResultPath. My problem is that I want to combine values from the input to a Lambda Task with a partial output from the task.

The input to the task is similar to below. I want to continue passing these values on to subsequent tasks.

{
    "previous_task_result": 100,
    "next_task_input": "asdf"
}

What I would like, is to have the output of my Lambda Task look like this:

{
    "previous_task_result": 100,
    "next_task_input": "asdf",
    "current_task_result": {
        "val1": "ghjk",
        "val2": [0,2,13,100]
    }
}

The specific problem is that the raw output of the Lambda Task looks like this. I only care about the Payload node. The rest of the output is boilerplate I would rather not pass through the Step Function.

{
  "resourceType": "lambda",
  "resource": "invoke",
  "output": {
    "ExecutedVersion": "$LATEST",
    "Payload": {
        "val1": "ghjk",
        "val2": [0,2,13,100]
    },
    "SdkHttpMetadata": {
      "HttpHeaders": {
        "Connection": "keep-alive",
        "Content-Length": "42",
        "Content-Type": "application/json",
        "Date": "Tue, 25 Feb 2020 14:36:29 GMT",
        "X-Amz-Executed-Version": "$LATEST",
        "x-amzn-Remapped-Content-Length": "0"
      },
      "HttpStatusCode": 200
    },
    "SdkResponseMetadata": {
      "RequestId": "redacted"
    },
    "StatusCode": 200
  }
}

I understand how to use ResultPath and OutputPath to combine the input with the output, or assign the output to a specific node, but what I can't find is a way to merge just the Payload node from the result with the existing input.

Joe · Accepted Answer · 2020-06-30 22:21:56Z

7

When working with Lambda Task States there's 3 ways in which you can structure the your task state:

1. Request Response:

This method returns the full API response including the SdkHttpMetadata field. I suspect the reason for returning the full HTTP response is because this is the only way in which you can invoke a Lambda function asynchronously from Step Function (Asynchronous Lambda invocations only return a status code). Example:

"CallLambda": {
    "Type": "Task",
    "Resource": "arn:aws:states:::lambda:invoke",
    "Parameters": {
        "FunctionName": "MyFunction",
        "InvocationType": "Event|RequestResponse|DryRun",
        "Payload.$": "$"
    },
    "End": true
}

2. Lambda ARN as the Resource:

I believe this is what you are looking for. When specifying the ARN of the Lambda function as the Resource, Step Functions will invoke your Lambda function synchronously and return only the result from your Lambda function without the API response metadata. In this case your State Input will be passed as the payload to your Lambda function or you can use InputPath/Parameters to filter/modify the data sent as the Payload.

"CallLambda": {
    "Type": "Task",
    "Resource": "arn:aws:lambda:us-west-2:123456789012:function:my-function",
    "ResultPath": "$.current_task_result",
    "End": true
}

Using the input as in your example the above Task will give you the output as follows:

{
    "previous_task_result": 100,
    "next_task_input": "asdf",
    "current_task_result": {
        <Your Lambda Functions Result>
    }
}

3. .WaitForTaskToken:

Passes a token to your function, pauses the execution untill either a call to SendTaskSuccess or SendTaskFailed has been received (this method will only return the Result/Error from SendTaskSuccess or SendTaskFailed with no additional HTTP metadata).

"CallLambda": {
    "Type": "Task",
    "Resource":"arn:aws:states:::lambda:invoke.waitForTaskToken",
    "Parameters": {
        "FunctionName": "MyFunction",
        "Payload":{  
           "token.$":"$$.Task.Token"
        }
    },
    "End": true
}

edited Jun 30, 2020 at 22:21

answered Mar 3, 2020 at 20:27

Joe

1,1918 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

maddie Over a year ago

Isn't sync not supported with Lamda in AWS Step Functions? docs.aws.amazon.com/step-functions/latest/dg/… Were you able to achieve this? If so, did you use ServiceIntergation.sync?

Joe Over a year ago

The .sync integration pattern is NOT supported for Lambda, so you can not use arn:aws:states:::lambda:invoke.sync. However both option 1 (by specifying InvocationType as RequestResponse) and option 2 (InvocationType is RequestResponse by default) as stated above allow you to call a lambda function synchronously.

SouthShoreAK · Accepted Answer · 2020-08-13 22:20:50Z

5

For anyone who finds this now, the ResultSelector (new as of 8/2020) performs exactly the function I was looking for when this was posted.

InputPath, Parameters and ResultSelector

answered Aug 13, 2020 at 22:20

SouthShoreAK

4,3162 gold badges30 silver badges50 bronze badges

Collectives™ on Stack Overflow

AWS Step Functions: Combine task input with partial task output

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related