Getting error "perl: error while loading shared libraries: libcrypt.so.1:"

[nicu-chiciuc]

For the past couple of hours I've been trying to setup exiftool on an Lambda instance and have a Node handle invoke it, process the output and send it back. I've managed to set this up easily for ffprobe since I could just upload it as a static binary (although now I realise that I can use a layer for that).
Since exiftool is a script I was trying to install perl somehow and finally, after a lot of digging I found this nice solution.

I'm using the Serverless framework and my .yml is minimal:

provider:
  name: aws
  runtime: nodejs10.x
  versionFunctions: false
  region: eu-central-1

functions:
  hello:
    handler: handler.hello
    layers:
      - arn:aws:lambda:eu-central-1:652718333417:layer:perl-5_26-layer:1

The handler function is also minimal, it executes an command that is sent to it:

module.exports.hello = (event, context, callback) => {
  if (event.cmd) {
    const child = child_process.exec(event.cmd, (error, stdout, stderr) => {
      // Resolve with result of process
      callback(null, { error, stdout, stderr });
    });
  }
}

And I'm sending the follwing json:

{
  "cmd": "perl -ver"
}

It deploys without problems but when I try to invoke it, I get the following error:

{
    "error": {
        "killed": false,
        "code": 127,
        "signal": null,
        "cmd": "perl -ver"
    },
    "stdout": "",
    "stderr": "perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory\n"

I'm still new to the whole serverless architecture stuff so there may be a simple solution but I can't seem to find it. Any help would be appreciated.

[Hideman85]

I recently have the same issue maybe due lambda environment changes (because works before with Node.js 8.x and now on Node.js 10.x).

Can you please have a look for Node.js 10.x?

Edit: I revert back my lambda to Node.js 8.x I don't have any issue so it's definitely come with the lambda environment Node.js 10.x

[davehodg]

On my mac:
$ node -v
v12.6.0

Is that far ahead in time?

[Hideman85]

Why you ask this? I mean AWS lambda environment is not related to your computer node version.

Amazon Linux AMI (https://aws.amazon.com/amazon-linux-ami/) this is the os used by lambda environment.

I think public layer that was built by the owner only support Node.js 8.10 and not Node.js 10.x, I'm not a expert of the Amazon Linux AMI but the only things that I can say is it work for Node.js 8.10 and not Node.js 10.x.

[davehodg]

It was a question from ignorance. I have a personal (Catalyst for now) app which will probably be better as a few Lambda functions. I'll get round to playing with it as soon as I've finished re-skinning it.

[patrickd-]

Probably a super dirty hack but copying the library from my local machine into the /lib directory of the layer zip file worked

$ whereis libcrypt.so.1
libcrypt.so: /lib/x86_64-linux-gnu/libcrypt.so.1

Related: https://stackoverflow.com/a/59816483/3950298

[moznion]

Sorry for my late reply. I'll take a look at this.

[jasonmacdonald]

I got bit by this today as well. Even trying to simply call /opt/bin/perl --version from inside a Node 10 lambda using Perl fails with the Libcrypt message.

/opt/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or director

[nicu-chiciuc]

Previously I resorted to using the nodejs8 runtime which had multiple libraries. Now that it's getting deprecated and I can't even update my existing lambda, I have to update to nodejs10/12. Not only is libcrypt.so absent but also libcurl.so which creates the same problems for mediainfo.
The most annoying is that an ec2 instance with the same runtime as aws lambdas (Amazon Linux 2) ami-0df0e7600ad0913a9 has both libcurl and libcrypt.
At this point I'm thinking about dropping aws lambdas altogether and start leveraging docker containers. Lambdas were supposed to make things simpler but I have to make a lot of hacks to make things work (or maybe I don't have enough knowledge/experience to understand my mistakes).

[moznion]

I’m sorry to have kept you waiting!

I've just published the new layers that include libcrypt and libcurl, which referred to the comment from @patrickd-. Thanks a lot!

related commits:

• 833ded0
• 84f3ee3

The new layers are the following:

• arn:aws:lambda:${REGION}:652718333417:layer:perl-5_26-layer:4
• arn:aws:lambda:${REGION}:652718333417:layer:perl-5_28-layer:2
• arn:aws:lambda:${REGION}:652718333417:layer:perl-5_30-layer:1

I've confirmed that it works well on those layers, could you please try it? @nicu-chiciuc

(and this might not be a bug of this layer, so I remove the label)

[moznion]

Note:
I gave up to include entire shared libraries in the layers because it touches the AWS quota of the maximum limit of layer size.

[nicu-chiciuc]

@moznion, Thanks for your updates.

This is definitely not a bug of the current layer. There are multiple issues scatered around github about libraries removed from Amazon Linux 2 and the problems they cause. I've resorted to using https://github.com/lambci/yumda and add curl so that I can make at least mediainfo work in my function.
I was using exiftool (and your layer) when I needed more specific information about files but I guess I'll add it back (and check your changes) after we pass the deadline.

[moznion]

I close this issue. Please feel free to reopen if you're still facing the same or similar problem 👍