7

I want to mirror my Bitbucket repository to CodeCommit and build from it, without affecting developer's workflow. Migration is easy, now the goal is to sync CodeCommit with Bitbucket, ideally, triggered by pushing to Bitbucket, but time-based sync is also acceptable.

I found this post: https://aws.amazon.com/blogs/devops/replicating-and-automating-sync-ups-for-a-repository-with-aws-codecommit/ and unanswered question here Mirror a Git Repo directly to AWS CodeCommit

Please, share you ideas how to write aws lambda function to sync CodeCommit repository with Bitbucket.

Improve this question
2
  • 1
    Thank you for all comments. I'd like to share the solution I found and implemented. Commented Jul 13, 2017 at 8:57
  • 1
    marketplace.atlassian.com/plugins/… This plugin allows repo mirroring to any git compatible server. However, if your Bitbucket server is using git client compiled with gnutls lib (default for Ubuntu) you will probably get the error "gnutls_handshake() failed: Illegal parameter" in bitbucket log. To solve it, build your git client against openssl lib -> askubuntu.com/questions/186847/… Commented Jul 13, 2017 at 9:07

2 Answers 2

Reset to default
2

While I haven't tried this, I suspect the following workflow could work. The general idea would be to use the BitBucket webhooks feature to trigger the lambda function through the use of API gateway.

  1. Write a Lambda function which 'mirror' clones from your bitbucket repository url and pushes to your CodeCommit repository url. Your function would likely need to include its own standalone git client library.

  2. Create an API using API gateway which calls your lambda function. Potential challenges here might be verifying that POST requests going to your API are coming from Bitbucket, and not some other source.

  3. Create a new webhook for your Bitbucket repository with the URL being the url of your API you created in step 2. A 'Repository push' trigger would be sufficient to trigger a replication event after each push.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Interesting, thanks. What do you think about using SNS instead of Gateway API? Using: github.com/juhamust/bitbucket2sns ?
Looks to me like this project actually is doing something very similar. It looks like it's using [Serverless] (serverless.com/framework/docs/providers/aws) to create API Gateway and Lambda resources on your behalf, using [CloudFormation] (aws.amazon.com/cloudformation). However, instead of just stopping with lambda, it is notifying a SNS topic. To do replication from here, I think you would need a subscriber which listens for notifications from SNS to trigger it. This probably requires hardware, or another lambda function, so I don't see the benefit of going through SNS.
AWS provide code for a similar solution to copy code to s3 github.com/aws-quickstart/quickstart-git2s3
1

One option is to use CodePipeline. Unfortunately with CodePipeline there is currently a weird process in making a Lambda function work as the sole process for the pipeline. Basically it comes down to:

  1. Make a CodePipeline with the AWS CodeCommit as your source repo
  2. Create one of the required Build/Deploy stages with the settings it asks for. Note that you will remove these later (unless you actually plan to use CodePipeline stages as given) so create a new CodeBuild project or something just to get through the wizard.
  3. Create a Lambda function that talks to BitBucket to sync your changes. The role attached must have permissions to interface with CodePipeline and CodeCommit. The Lambda function must also call either one of PubJobSuccessResult or PutJobFailureResult so that CodePipeline knows the Lambda actually did something and to not sit and wait for the function to complete.
  4. Now go back to the CodePipeline and edit it. For whatever stage was generated remove the existing action. Add a new Invoke action that points to the Lambda function and set the Role which gives access to CodePipeline for setting the job result and CodeCommit for reading the repo.

Another option is to use CloudWatch scheduling to invoke the Lambda at X interval if you're okay with a more delayed sync. This will probably be easier to setup in the long run, with the possibility of having null Lambda runs if there's nothing to sync, counting against your allocation.

Improve this answer

3 Comments

Many thanks for sharing, I will check the CodePipeline now. One question. When Lambda starts, what do you mean by sync with Bitbucket? git-clone and push to CodeCommit? Some of my repos are pretty large. Is there any way to run git-pull on CodeCommit side, without intermediate cloning?
@C.B. CodePipline will pass Lambda some event variables as described here docs.aws.amazon.com/codepipeline/latest/APIReference/… . You'll want to check on revision for the commit hash that you can use to pull the individual commits and sync incremental instead of one giant sync.
Nice! I'll check it. Thanks again

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.