I actually took the approach of storing my web.config files for each environment in an S3 bucket. As part of the CodeDeploy Deployment group process, it would download the config file from the S3 bucket in the After Install hook. This way you can build the application once and push the same application files for each environment. This also separates the configuration of the application from the actual code so that the development team doesn't need to know things like connection string values, etc...