Terraform: AWS Codepipeline multiple Codecommit sources

Question

I am moving away from Github.com and to Codecommit, I have been leveraging terraforms modular approach to import GitHub repos as modules for years. That said Codecommit is very different in that nature. I have seen where people leverage SSH to clone the repos locally but I have also noticed codepipeline can leverage multiple sources. I need a way to add multiple repos to my pipeline so I can replicate the modular github approach offered by terraform. I want that code locally to execute it in a modular fashion.

I have googled looking for an example that shows me how to leverage multiple codecommmit resources in my pipeline and i can not find anything that clearly outlines how to leverage multiple resources in terraform. Has anyone figured this out or have examples they can point me to?

docs.aws.amazon.com/codebuild/latest/userguide/… - pretty easy to find on Google. CodePipeline is focused on deploying using AWS native constructs so it's not surprising to me there is not much information on using it with Terraform. Essentially you will be using Codebuild projects to call Terraform-so you would be best advised to focus on how to use CodeBuild projects with CodePipeline ( with multiple input sources as in the provided example ) with the understanding it's up to you as to what you do in those Codebuild projects — Ronan Cunningham
– Ronan Cunningham, Commented Sep 18, 2021 at 22:09
Pay attention to how multiple input working directories are used, they are not nested e.g $CODEBUILD_SRC_DIR and $CODEBUILD_SRC_DIR_source2 — Ronan Cunningham
– Ronan Cunningham, Commented Sep 18, 2021 at 22:18
@ronan thanks for the reply. I found that link earlier and working through the terraform resource to construct the second source. Really appreciate the heads up on the directories will look into that. — Maxamis4
– Maxamis4, Commented Sep 18, 2021 at 22:34
No worries. Maybe I misunderstood? Are you using Terraform to create the pipelines or pipelines to run Terrraform - or both? — Ronan Cunningham
– Ronan Cunningham, Commented Sep 18, 2021 at 22:54
Great comment, i did both, I created terraform to create the pipeline and the build projects. That part is working nicely if you want to see it let me know. The part i am struggling with is getting the terraform to create a second resource which in this case is another codecommit repo into the source. Let me know if that clears it up for you. — Maxamis4
– Maxamis4, Commented Sep 23, 2021 at 3:57

Daniel · Accepted Answer · 2025-03-13 20:23:17Z

Looking into this, I have found that it's not very well documented anywhere which is actually very frustrating. Leveraging hashicorp vague description of the service and AWS multi-input example I was finally able to come up with this for terraform:

 "aws_codepipeline" "foo" {
  name     = "tf-test-pipeline"
  role_arn = "codepipeline service role arn"

  artifact_store {
    location = "s3 bucket name, NOT THE ARN"
    type     = "S3"
  }

  stage {
    name = "Source"

    action {
      name             = "Source"
      category         = "Source"
      owner            = "AWS"
      provider         = "CodeCommit"
      version         = "1"
      output_artifacts = ["src"]

      configuration = {
        RepositoryName = "vpc" //MUST BE the name of the your codecommit repo
        BranchName = "master"
      }

      run_order = "1"
    }

    action {
      name             = "2ndSource" //you can make this any name
      category         = "Source"
      owner            = "AWS"
      provider         = "CodeCommit"
      version         = "1"
      output_artifacts = ["src2"]

      configuration = {
        RepositoryName = "ec2" 
        BranchName = "master"
      }
      run_order = "2"
    }


  }


  stage {
    name = "Build"

    action {
      name            = "Build"
      category        = "Build"
      owner           = "AWS"
      provider        = "CodeBuild"
      input_artifacts = ["src","src2"] //pass through both repositories
      version         = "1"

      configuration = {
        ProjectName = "codebuild_project_name"
        PrimarySource = "src"
      }
    }
  }
}

The trick here is to add additional sources into one stage, not separate ones. The reference below shows two of them but I have been able to add three with no problem. Note that the primary source identifier must match the one from the input_artifacts.

Reference Links:

Hashicorp CodePipeline https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline#run_order

AWS Multiple Inputs Json Example: https://docs.aws.amazon.com/codebuild/latest/userguide/sample-pipeline-multi-input-output.html

For those of you getting started for the first time, I recommend this link, it's pretty comprehensive and walks you through the entire build process which includes roles and policies: https://medium.com/swlh/intro-to-aws-codecommit-codepipeline-and-codebuild-with-terraform-179f4310fe07

Ronan Cunningham · Accepted Answer · 2021-09-23 18:05:29Z

#    _____  ____  _    _ _____   _____ ______ 
#   / ____|/ __ \| |  | |  __ \ / ____|  ____|
#  | (___ | |  | | |  | | |__) | |    | |__   
#   \___ \| |  | | |  | |  _  /| |    |  __|  
#   ____) | |__| | |__| | | \ \| |____| |____ 
#  |_____/ \____/ \____/|_|  \_\\_____|______|
                                            
                                            
      Stages:
        - Name: Source
          Actions:
            - ActionTypeId:
                Category: Source
                Owner: AWS
                Provider: CodeStarSourceConnection
                Version: "1"
              Configuration: 
                ConnectionArn: !Ref CodeStarConnectionArn
                FullRepositoryId: !Ref BitBucketRepo
                BranchName: !Ref BitBucketRepoReleaseBranch
                OutputArtifactFormat: "CODE_ZIP"
                DetectChanges: true
              Name: SourceCode
              OutputArtifacts:
                - Name: !Sub ${SourceArtifactName}
              Namespace: SourceVariables1
              RunOrder: 1
            - ActionTypeId:
                Category: Source
                Owner: AWS
                Provider: CodeStarSourceConnection
                Version: "1"
              Configuration: 
                ConnectionArn: !Ref CodeStarConnectionArn
                FullRepositoryId: !Ref PipelineBitBucketRepo
                BranchName: !Ref PipelineBitBucketRepoReleaseBranch
                OutputArtifactFormat: "CODE_ZIP"
                DetectChanges: true
              Name: PipelineDefinition
              OutputArtifacts:
                - Name: !Sub ${PipelineCodeArtifactName}
              Namespace: SourceVariables2
              RunOrder: 1



#    _____ ______ _      ______   __  __ _    _ _______    _______ ______ 
#   / ____|  ____| |    |  ____| |  \/  | |  | |__   __|/\|__   __|  ____|
#  | (___ | |__  | |    | |__    | \  / | |  | |  | |  /  \  | |  | |__   
#   \___ \|  __| | |    |  __|   | |\/| | |  | |  | | / /\ \ | |  |  __|  
#   ____) | |____| |____| |      | |  | | |__| |  | |/ ____ \| |  | |____ 
#  |_____/|______|______|_|      |_|  |_|\____/   |_/_/    \_\_|  |______|
                                                                        
                                                                        


        - !If
          - ShouldUpatePipelineStackOnChange
          - Name: UpdatePipeline

            Actions:
              - Name: CreateChangeSet
                ActionTypeId:
                  Category: Deploy
                  Owner: AWS
                  Provider: CloudFormation
                  Version: "1"
                Configuration:
                  ActionMode: CHANGE_SET_REPLACE
                  StackName: !Ref AWS::StackName
                  ChangeSetName: !Sub ${AWS::StackName}-ChangeSet
                  TemplatePath: !Sub ${PipelineCodeArtifactName}::${PipelineTemplateName}
                  Capabilities: CAPABILITY_NAMED_IAM
                  RoleArn: !GetAtt PipelineStackCloudFormationExecutionRole.Arn
                InputArtifacts:
                  - Name: !Sub ${PipelineCodeArtifactName}
                RunOrder: 1
              - Name: ExecuteChangeSet
                ActionTypeId:
                  Category: Deploy
                  Owner: AWS
                  Provider: CloudFormation
                  Version: "1"
                Configuration:
                  ActionMode: CHANGE_SET_EXECUTE
                  StackName: !Ref AWS::StackName
                  ChangeSetName: !Sub ${AWS::StackName}-ChangeSet
                  RoleArn: !GetAtt PipelineStackCloudFormationExecutionRole.Arn
                OutputArtifacts:
                  - Name: !Sub ${AWS::StackName}ChangeSet
                RunOrder: 2
          - !Ref AWS::NoValue

Ronan let me check this out and see how i can work it. Will get back to you with the progress. thank you for this.

Collectives™ on Stack Overflow

Terraform: AWS Codepipeline multiple Codecommit sources

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related