Question:
I am using a docker container image on lambda to run my ML model. My lambda function has a S3 trigger to fetch images. I am trying to run my lambda function but I am getting this error. Can someone please help me?
PS – now i am aware /tmp is the only writable directory in lambda but how to solve this with that?
Answer:
As others have mentioned, /tmp is the only writable directory in any AWS Lambda environments, either using containers or not.
Having said that, you should move your entire library (during the lambda runtime — during container image build time doesn’t work) to that directory — such that everything remains connected within the library — and then reference your new library directory in the library path environment for Lambda: LD_LIBRARY_PATH
Referencing your new library directory in the library path environment for Lambda should be done because Lambda looks at the /opt/ directory by default; and since you just moved your library to /tmp, you should also update LD_LIBRARY_PATH to contain that location. This can be done in the Dockerfile:
|
1 2 3 |
# Set the LD_LIBRARY_PATH ENV LD_LIBRARY_PATH="/opt/my-lib-folder/:$LD_LIBRARY_PATH" |
or during Lambda runtime:
|
1 2 3 4 5 |
os.environ['LD_LIBRARY_PATH'] = '/tmp/my-lib-folder:' + os.environ['LD_LIBRARY_PATH'] def lambda_handler(event, context): # your code ... |
If there are still problems, it may be related to linking problems of your library, or that you didn’t update your LD_LIBRARY_PATH correctly.
EDIT: As pointed out by @rok, you cannot move your libraries during the container image build time, because the /tmp folder will be erased by AWS automatically.