[serverless] Connection Reuse is broken in a Lambda environment
Problem
Issue type: [ ] question [x] bug report [ ] feature request [ ] documentation issue Database system/driver: [ ] `cordova` [ ] `mongodb` [ ] `mssql` [ ] `mysql` / `mariadb` [ ] `oracle` [x] `postgres` [ ] `sqlite` [ ] `sqljs` [ ] `react-native` [ ] `expo` TypeORM version: [x] `latest` [ ] `@next` [ ] `0.x.x` (or put your version here) Steps to reproduce or a small repository showing the problem: I'm primarily expanding further on https://github.com/typeorm/typeorm/issues/2598#issue-345445322, the fixes described there are catastrophic in nature, because: 1. They either involve closing connections without understanding that the lambda may not just be serving that one request. 2. Full bootstrap of the connection a second time anyway (defeating the purpose of caching it in the first place)) and I believe this is something that should be addressed in core. 3. Even the somewhat saner metadata rebuild causes problems (since it cannot be awaited and runs in parallel). This results in metadata lookups while queries are running (for another controller, perhaps) randomly start failing. The issue is exactly as described, if we attempt to reuse connections in a lambda environment, the entity manager no longer seems to know anything about our entities. The first request made completes successfully (and primes our connection manager to reuse the same connection). Subsequent requests quit with `RepositoryNotFoundError: No repository for "TData" was found. Looks like this entity is
Error Output
Error: No repository for "TData" was found. Looks like this entity is not registered in current "default" connection?`
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Implement Connection Pooling for TypeORM in Lambda
In a Lambda environment, connections to the PostgreSQL database are not reused effectively due to the stateless nature of Lambda functions. When a new request is made, the existing connection may not be recognized, leading to the 'RepositoryNotFoundError'. This occurs because the TypeORM connection manager does not maintain the state of entities across invocations, causing the entity metadata to be lost.
Awaiting Verification
Be the first to verify this fix
- 1
Install pg and typeorm
Ensure that you have the required PostgreSQL driver and TypeORM installed in your Lambda environment.
bashnpm install pg typeorm - 2
Create a connection pool
Modify your database connection setup to use a singleton pattern that maintains a single connection instance across Lambda invocations. This will allow connection reuse and prevent the loss of entity metadata.
typescriptimport { createConnection, getConnection } from 'typeorm'; let connection; export const connectToDatabase = async () => { if (!connection) { connection = await createConnection({ type: 'postgres', host: 'your-host', port: 5432, username: 'your-username', password: 'your-password', database: 'your-database', entities: [YourEntity], synchronize: true, }); } return connection; }; - 3
Use the connection in your handler
In your Lambda function handler, call the `connectToDatabase` function to ensure you are using the same connection instance across requests.
typescriptimport { connectToDatabase } from './db'; export const handler = async (event) => { const connection = await connectToDatabase(); const repository = connection.getRepository(YourEntity); // Perform your database operations here }; - 4
Handle connection errors gracefully
Implement error handling to manage connection issues. This ensures that if the connection fails, it can be retried or logged appropriately.
typescripttry { const connection = await connectToDatabase(); } catch (error) { console.error('Database connection error:', error); throw new Error('Database connection failed'); } - 5
Test the implementation
Deploy the changes and test the Lambda function with multiple requests to ensure that the connection pooling works and the 'RepositoryNotFoundError' no longer occurs.
Validation
To confirm the fix worked, invoke the Lambda function multiple times with different requests and verify that no 'RepositoryNotFoundError' is thrown. Additionally, monitor the database connections to ensure they are being reused correctly.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep