Extending dotnet lambda package --function-layers

[gshively11]

Hi, I'd like to run a feature by the maintainers of this package. It's something we'd be open to contributing via PR, but I want to make sure the idea is sound before putting in the effort.

Quick Backstory

We are trying to deploy a dotnet core project as a collection of lambdas. Due to the restrictions involved in our AWS environment, we are only able to deploy AWS resources using Service Catalog. This prevents us from using the normal dotnet CLI commands to publish layers and deploy functions directly. Instead, we need to package everything up, push it to S3, and then run some CloudFormation templates to create the layers/functions.

We also want to use layer optimizations to reduce cold start time. Lastly, we will typically have two layers per project: one which exists within the project to store all of a project's external dependencies, and another that will be published separate from the project, to be shared across all of our projects. I'm mentioning both cases since they involve different build processes. The layer in each project would need to be built alongside the other project files, likely within a single CI pipeline, whereas the shared layer would be built and published in a separate CI process.

We've written tooling to support the special deploy process in our environment, and it works well with node and python lambdas, but we're hitting a blocker with the build process for dotnetcore lambdas.

The Problem

Since dotnet publish-layer is out, we're stuck using dotnet store directly. This does allow us to create a package suitable for publishing as a layer. However, it appears that dotnet lambda package does not support file paths for the artifact.xml in the --function-layers argument.

This means that we have no way of providing the store manifests created by dotnet store to the dotnet publish command called internally in Amazon.Lambda.Tools.

The Solution

This would be an advanced use case, but theoretically, if we were able to manage the artifact.xml's on our side, knowing which one mapped to which lambda layer, and we could pass the files to the dotnet lambda package command, we would be able to get a fully optimized package to deploy using our own tooling.

This request looks very similar to #79, except that we wouldn't be adding a new package-layer command, and instead of --function-layers accepting a zip containing the artifact.xml, we would just pass the artifact.xml directly. While our suggested implementation is different, it would still unblock #79.

I saw your point in the previous issue:

Without this description the layer is really only good for that single Lambda function deployment where your build system has the context in its pipeline state

Given that passing file paths to --function-layers would be an advanced use case, I think it's reasonable to allow it with a caveat detailing the need to maintain a mapping between artifact.xml files and their layers.

Without this feature, we will either have to use a forked version of Amazon.Lambda.Tools, which we typically avoid at all costs, or we'll have to do some janky scripting to replicate what Amazon.Lambda.Tools is doing (we sort of already did this with as a POC using dotnet store, dotnet publish --manifest, and then overwriting/removing stuff after dotnet lambda package, but it's inefficient and fragile).

If you're willing to accept this feature, we will gladly work on a PR! Thanks for your time.

[normj]

You can actually sneak parameters into the underlying call to the underlying dotnet publish call with the --msbuild-parameters switch for dotnet lambda package. For example I was able to force the publish to look at my manifest with the following command.

dotnet lambda package --msbuild-parameters "--manifest C:\Temp\ManualLayerTest\MyLayer\x64\netcoreapp2.1\artifact.xml"

You can also set the --msbuild-parameters via the aws-lambda-tools-defaults.json with a JSON key of msbuild-parameters.

[gshively11]

Oh nice! Let me give that a shot and if it works I'll come back and close this out.

[gshively11]

That seems to do the trick, thank you @normj!