AWS Lambda functions can now mount Amazon S3 buckets as file systems

AWS Lambda has introduced support for Amazon S3 Files, allowing Lambda functions to mount Amazon S3 buckets as file systems. This new capability enables users to perform standard file operations without the need to download data for processing. Built on Amazon EFS, S3 Files combines the performance and simplicity of a file system with the scalability, durability, and cost-effectiveness of S3.

Multiple Lambda functions can now connect to the same S3 Files file system simultaneously, facilitating data sharing through a common workspace without requiring custom synchronization logic. This integration significantly simplifies stateful workloads in Lambda by removing the need to download objects, upload results, and manage ephemeral storage limits.

This feature is particularly beneficial for AI and machine learning workloads, where agents need to maintain memory and share state across different pipeline steps. Lambda’s durable functions make these multi-step AI workflows achievable by orchestrating parallel execution with automatic checkpointing. For instance, an orchestrator function can clone a repository to a shared workspace, allowing multiple agent functions to analyze the code in parallel. The durable function manages the checkpointing of execution state, while S3 Files ensures seamless data sharing across all steps.

To utilize S3 Files with Lambda, users need to configure their function to mount an S3 bucket through the Lambda console, AWS CLI, AWS SDKs, AWS CloudFormation, or the AWS Serverless Application Model (SAM). Further information on using S3 Files with Lambda functions can be found in the Lambda developer guide.

S3 Files is available for Lambda functions not configured with a capacity provider in all AWS Regions where both Lambda and S3 Files are accessible. There is no additional charge for using S3 Files beyond the standard Lambda and S3 pricing.