EPHEMERAL STORAGE ZIP
Similarly, applications that generate zip files also benefit from more /tmp space. These can now unzip to the local file system without the need for in-memory processing. Zip processing: Some workloads use large zip files from data providers to initialize local databases. Workloads that create PDFs, use headless Chromium, or process media also benefit from more ephemeral storage. To learn more about using Lambda for ML inference, read Building deep learning inference with AWS Lambda and Amazon EFS and Pay as you go machine learning inference with AWS Lambda.ĭata processing: For workloads that download objects from S3 in response to S3 events, the larger /tmp space makes it possible to handle larger objects without using in-memory processing. More ephemeral storage allows you to download larger models from Amazon S3 to /tmp and use these in your processing. Machine learning (ML) inference: Many inference tasks rely on large reference data files, including libraries and models. More temporary space enables more complex ETL jobs to run in Lambda functions.
EPHEMERAL STORAGE CODE
There are several common customer use cases that can benefit from the expanded ephemeral storage.Įxtract-transform-load (ETL) jobs: Your code may perform intermediate computation or download other resources to complete processing. All data stored in /tmp is encrypted at rest with a key managed by AWS. It’s also available for Provisioned Concurrency.
The behavior is identical whether you use zip or container images to deploy your functions. With this new configurable setting, ephemeral storage works in the same way. The contents are deleted when the Lambda service eventually terminates the execution environment. This is a common use case that can help reduce function duration for subsequent invocations. This means that you can use the same execution environment to cache static assets in /tmp between invocations. This storage is not shared between instances of Lambda functions and the space is guaranteed to be empty when a new instance starts. This provides a fast file system-based scratch area that is scoped to a specific instance of a Lambda function. How ephemeral storage works in LambdaĪll functions have ephemeral storage available at the fixed file system location /tmp.
EPHEMERAL STORAGE HOW TO
This blog post explains how this works and how to use this new setting in your Lambda functions. You can set this in the AWS Management Console, AWS CLI, or AWS SDK, AWS Serverless Application Model (AWS SAM), AWS Cloud Development Kit (AWS CDK), AWS Lambda API, and AWS CloudFormation. With this new feature, you can now configure ephemeral storage for up to 10 GB per function instance. This was set at 512 MB for every function, regardless of runtime or memory configuration. AWS Lambda functions have always had ephemeral storage available at /tmp in the file system.
0 kommentar(er)
