Question on private python modules

[sjatkins]

I am familiar with serverless and it took quite some hacking to get such modules to be available to my python lambda functinos even with putting them in lambda layers. It seemed the build/deploy time and lambda run time both needed access (as far as I can tell).

I am wondering what terraform_aws_lambda_python does to make this easier. I sort of dream of just using an exsiting cleanly created virtualenv at build/deploy time to incorporate these private modules cleanly. Or perhaps the virtualenv you have has some hooks or access for doing such.

How would you personally go about it given this tooling of yours?

[ruzin]

Hey! Good question! Disclaimer: I wrote this module nearly a year ago around the the when lambda layers came out. I am not intimately familiar with lambda layers but I took a quick look and it is trying to solve a similar problem but in a different manner. It seems more appropriate for a scenario wherein you have several functions that share base modules. However, if you're looking for something that can spin up a virtualenv and cleanly install dependencies, then my module may be helpful as it does exactly that. However, in order for it to cater to private modules, there is a bit of work on your end. Here is a link that may be helpful https://dev.to/rf_schubert/how-to-create-a-pip-package-and-host-on-private-github-repo-58pa

[sjatkins]

Hey Ruzin, That may help using terraform but the ssh github references to a project is known to not work properly in serverless framework. Yes it works in normal pip install but for some reason than doesn't seem to be what serverless is actually doing.

…

On Thu, Sep 26, 2019 at 5:22 PM Ruzin ***@***.***> wrote: Hey! Good question! Disclaimer: I wrote this module nearly a year ago around the the when lambda layers came out. I am not intimately familiar with lambda layers but I took a quick look and it is trying to solve a similar problem but in a different manner. It seems more appropriate for a scenario wherein you have several functions that share base modules. However, if you're looking for something that can spin up a virtualenv and cleanly install dependencies, then my module may be helpful as it does exactly that. However, in order for it to cater to private modules, there is a bit of work on your end. Here is a link that may be helpful https://dev.to/rf_schubert/how-to-create-a-pip-package-and-host-on-private-github-repo-58pa — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1?email_source=notifications&email_token=AACWMNX2TRBQM4ST3MH5TULQLU747A5CNFSM4IZAG5AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7XHK7Y#issuecomment-535721343>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACWMNVW27TWUVKXAMZMS6TQLU747ANCNFSM4IZAG5AA> .

[sjatkins]

Oddly in serverless it appears in the requirements subdirectory under .serverless that is build but is not properly present in the .zip uploaded.

…

On Fri, Sep 27, 2019 at 12:47 PM Samantha Atkins ***@***.***> wrote: Hey Ruzin, That may help using terraform but the ssh github references to a project is known to not work properly in serverless framework. Yes it works in normal pip install but for some reason than doesn't seem to be what serverless is actually doing. On Thu, Sep 26, 2019 at 5:22 PM Ruzin ***@***.***> wrote: > Hey! Good question! Disclaimer: I wrote this module nearly a year ago > around the the when lambda layers came out. I am not intimately familiar > with lambda layers but I took a quick look and it is trying to solve a > similar problem but in a different manner. It seems more appropriate for a > scenario wherein you have several functions that share base modules. > However, if you're looking for something that can spin up a virtualenv and > cleanly install dependencies, then my module may be helpful as it does > exactly that. However, in order for it to cater to private modules, there > is a bit of work on your end. Here is a link that may be helpful > https://dev.to/rf_schubert/how-to-create-a-pip-package-and-host-on-private-github-repo-58pa > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#1?email_source=notifications&email_token=AACWMNX2TRBQM4ST3MH5TULQLU747A5CNFSM4IZAG5AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7XHK7Y#issuecomment-535721343>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AACWMNVW27TWUVKXAMZMS6TQLU747ANCNFSM4IZAG5AA> > . >

[sjatkins]

See this for part of what goes wrong if curious. serverless/serverless-python-requirements#240

…

On Fri, Sep 27, 2019 at 12:48 PM Samantha Atkins ***@***.***> wrote: Oddly in serverless it appears in the requirements subdirectory under .serverless that is build but is not properly present in the .zip uploaded. On Fri, Sep 27, 2019 at 12:47 PM Samantha Atkins ***@***.***> wrote: > Hey Ruzin, > That may help using terraform but the ssh github references to a project > is known to not work properly in serverless framework. Yes it works in > normal pip install but for some reason than doesn't seem to be > what serverless is actually doing. > > On Thu, Sep 26, 2019 at 5:22 PM Ruzin ***@***.***> wrote: > >> Hey! Good question! Disclaimer: I wrote this module nearly a year ago >> around the the when lambda layers came out. I am not intimately familiar >> with lambda layers but I took a quick look and it is trying to solve a >> similar problem but in a different manner. It seems more appropriate for a >> scenario wherein you have several functions that share base modules. >> However, if you're looking for something that can spin up a virtualenv and >> cleanly install dependencies, then my module may be helpful as it does >> exactly that. However, in order for it to cater to private modules, there >> is a bit of work on your end. Here is a link that may be helpful >> https://dev.to/rf_schubert/how-to-create-a-pip-package-and-host-on-private-github-repo-58pa >> >> — >> You are receiving this because you authored the thread. >> Reply to this email directly, view it on GitHub >> <#1?email_source=notifications&email_token=AACWMNX2TRBQM4ST3MH5TULQLU747A5CNFSM4IZAG5AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7XHK7Y#issuecomment-535721343>, >> or mute the thread >> <https://github.com/notifications/unsubscribe-auth/AACWMNVW27TWUVKXAMZMS6TQLU747ANCNFSM4IZAG5AA> >> . >> >