FG
💻 Software🗄️ Databases

Excessive memory usage when loading models with relations

Fresh5 days ago
Mar 14, 20260 views
Confidence Score79%
79%

Problem

Issue type: [ ] question [x] bug report [ ] feature request [ ] documentation issue Database system/driver: [ ] `cordova` [ ] `mongodb` [ ] `mssql` [ ] `mysql` / `mariadb` [ ] `oracle` [x] `postgres` [ ] `cockroachdb` [ ] `sqlite` [ ] `sqljs` [ ] `react-native` [ ] `expo` TypeORM version: [x] `latest` [ ] `@next` [ ] `0.x.x` (or put your version here) Steps to reproduce or a small repository showing the problem: If I load a list of models like this: [code block] With many elements (100 in my case) it gets up to 2+ GB of memory usage for the query, where if i load the model by itself, and then make separate queries for each relationship such as: [code block] And then map the results to the original list, the memory footprint is not even noticeable. <!-- To answer those questions you need to put "x" inside the square brackets, for example: [x] `mysql` [ ] `postgres` Also, please format your code properly (by taking code blocks into [code block]) !>

Unverified for your environment

Select your OS to check compatibility.

1 Fix

Canonical Fix
Unverified Fix
New Fix – Awaiting Verification

Optimize Memory Usage for Model Loading with Relations in TypeORM

Medium Risk

The excessive memory usage occurs because TypeORM loads all related entities eagerly by default when fetching a list of models with relations. This results in a large amount of data being loaded into memory at once, especially when dealing with many records and their associated relations.

Awaiting Verification

Be the first to verify this fix

  1. 1

    Use Lazy Loading for Relations

    Modify your entity definitions to use lazy loading for relations instead of eager loading. This will ensure that related entities are only loaded when explicitly accessed, reducing initial memory usage.

    typescript
    export class YourEntity {
  @OneToMany(() => RelatedEntity, related => related.yourEntity)
  relatedEntities: Promise<RelatedEntity[]>;
}
  2. 2

    Load Models without Relations Initially

    When querying your models, avoid loading relations by using the `select` option to only fetch the necessary fields. This will help to minimize the amount of data loaded into memory.

    typescript
    const models = await connection.getRepository(YourEntity).find({
  select: ['id', 'name'],  // specify only necessary fields
});
  3. 3

    Fetch Relations in Separate Queries

    After loading the main models, fetch the related entities in separate queries. This will prevent loading all related data at once and will keep memory usage low.

    typescript
    const relatedEntities = await connection.getRepository(RelatedEntity).find({
  where: { yourEntityId: In(models.map(m => m.id)) }
});
  4. 4

    Implement Pagination for Large Datasets

    If you are dealing with a large number of records, implement pagination to limit the number of records loaded at once. This will further reduce memory usage and improve performance.

    typescript
    const models = await connection.getRepository(YourEntity).find({
  skip: (page - 1) * pageSize,
  take: pageSize
});

Validation

Confirm that the memory usage during model loading is significantly reduced by monitoring the application's memory consumption before and after applying these changes. Additionally, verify that the application still functions correctly by checking that all necessary data is loaded when accessed.

Sign in to verify this fix

Environment

Submitted by

AC

Alex Chen

2450 rep

Tags

typeormormtypescriptbugperformancedriver:-postgresneeds-tests