10

In an attempt to give developers the ability to write their lambda function locally, then commit and push to git, we used SAM and SAM Local.

We have about 50 lambda functions and anticipate this number to grow into the hundreds if not thousands.

I'm wondering how to group the lambda functions repositories i.e should we group all the lambda functions that get triggered by api gateway in their own repo, then all the ones that get triggered by s3 separately etc ...

The challenge is some lambda functions have their own dependencies and it just looks weird to try and have multiple lambdas with different dependencies within same repo. One lambda might have npm package dependencies, another might have python libraries dependencies, etc ... so do I git commit and git push those two lambda in their own separate repos or organize them in separate folders and push them to same repo? Or have separe repos per lambda function?

It doesn't seem feasible to commit each lambda to its own repo especially as the number of those functions grow overtime.

I'm new to aws tools and would appreciate someone's insights on this!

Maybe you've ran into this problem before?

Improve this question
2
  • What do these 50 lambda functions have in common? Commented Feb 24, 2018 at 12:37
  • Not much in common; most of them are doing one-off tasks; only the api gateway functions are grouped in same repo but the rest are not api dependent; they're triggered by SNS, others by kinesis, others by S3 buckets. I don't like the idea of having one git repo per function because overtime this won't be sustainable! Commented Feb 26, 2018 at 12:39

1 Answer 1

Reset to default
3

I would group them according to their purpose. For example,

  • REST API (HTTP endpoints)
  • GraphQL API (HTTP endpoints)
  • Auth API (HTTP endpoints)
  • Admin API (HTTP endpoints not part of our user-facing product but is essential for managing our customers such as billing systems and company dashboards)
  • Background workers (for processing background tasks like image resize, sending emails, aggregating data, etc.)
  • Operations (not directly part of the product but is essential in daily operations such as Slack WebHooks, CI/CD triggers)

I'd also recommend tools such as serverless-framework to help manage each project.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Thank you for your response; when you say you'd group them, you mean group them in the same git repository? So far, I grouped all API Gateway lambdas in same repo using SAM templates. But say you have two lambda functions: one for image resize with npm package dependencies and the other for sending emails with python dependencies. Will you check in both of these functions in same repo? Each function has its own SAM template file listening on the develop branch! when there's a checkin to that branch, which of the 2 SAM template will trigger the build?
Yes, I group them like one group = one repository = one build/deploy pipeline.
So, if I change only one function in a group of ten functions, all ten functions get deployed together even if others didn't change (though I think serverless is doing something about this). It's somewhat overkill to deploy all ten when you only changed one, but that's more sane for me than managing 100 different deploy pipelines for 100 functions.
Yeah thank you for this; that's why I was feeling a bit uncomfortable/unsure about the proper process. I thought about putting each function on its own separate branch within the same repo but it would defeat the purpose of git which is to eventually merge the branches to master and you wouldn't want to merge these branches since, for me, the point of creating branches is to try and separate my functions while reducing number of repos created. I'm still thinking of ideas, will update here as I go.I have a couple of days left in my sprint to come up with something solid for production purposes.
@pelican i also thought about the possibility of each function in its own branch - but yeah i agree this defeats git and is hacky. How did you ultimately go about this, if you don't mind me asking? I have an issue too that I want these multiple Lambdas to share a common API Gateway -- not really sure what the best CI flow would be
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.