Exploring a NestJS Application with MongoDB Using Mongoose: A Beginner’s Guide (P2) — Multiple Database Connections

When developing an insight service in NestJS that requires aggregating data from multiple databases, it is crucial to understand how to effectively manage connections to different MongoDB databases and perform operations across them. This guide will walk you through the process with a beginner-friendly approach.

Before diving into this article, it is essential to have a solid understanding of how to use MongooseModule.forRoot for connecting to MongoDB and MongooseModule.forFeature for registering schemas within a NestJS application. If you're not familiar with these concepts, please read our Exploring a NestJS Application with MongoDB Using Mongoose: A Beginner’s Guide. Understanding these foundational elements will ensure you can follow along with the steps involved in managing multiple database connections and performing operations across them efficiently.

The complete source code for this example is available at the end of this article

Scenario: Aggregating Health Insights

In this scenario, we’ll configure a NestJS service to aggregate health insights from multiple MongoDB databases using a read-only account. This involves setting up connections to different databases, defining schemas, and implementing a service to process and combine the data. Some of the required queries are quite complex, and there are no existing APIs available to fetch this aggregated data. Therefore, we need to connect to multiple databases directly to retrieve and process the necessary information.

Insight Module (with its Service, Controller, and Model) and external data sources (Patient and Health models) in a NestJS application.

Architecture Overview

The use of MongooseModule.forRootAsync for connecting to multiple MongoDB databases and MongooseModule.forFeature for registering schemas within the InsightModule

MongooseModule.forRoot:

• Purpose: Establishes a connection to your MongoDB database(s). It’s responsible for: Specifying the connection URI (uniform resource identifier) of your MongoDB database. Setting up global Mongoose options (e.g., connection pool size, retry settings). Defining the connection name if you have multiple databases.
• Location: Typically placed in your root AppModule to ensure that the connection is available to the entire application.
• Usage: Called only once per connection.
• Dynamic Configuration: forRootAsync allows for the MongoDB connection to be configured dynamically at runtime, which is especially useful when the connection URI or other options depend on external sources like environment variables or a configuration service.

MongooseModule.forFeature:

• Purpose: Registers Mongoose schemas and creates models for specific feature modules. It does the following: Associates a schema with a model name. Provides the connection name where the model should be registered.
• Location: Used within feature modules (e.g., InsightModule, PatientModule, HealthModule) to define the models specific to that module.
• Usage: Called once per feature module, for each model you need to work with.
• Connection Naming: In this case, connection naming becomes mandatory and the connection names used in forFeaturemust match those defined in forRoot to ensure proper linkage and functionality.

App Module

Asynchronously configure and establish connections to three different MongoDB databases

The AppModule uses MongooseModule.forRootAsync to asynchronously configure and establish connections to three different MongoDB databases: insight, patients, and health

// src/app.module.ts
// import ...
@Module({
  imports: [
    ConfigModule.forRoot({
      load: [databaseConfig],
      envFilePath: '.env',
      isGlobal: true,
    }),
    MongooseModule.forRootAsync({
      imports: [ConfigModule],
      connectionName: 'insight',
      useFactory: async (configService: ConfigService) => {
        return {
          uri: configService.get(CONFIG_DATABASE).insight.uri,
        };
      },
      inject: [ConfigService],
    }),
    MongooseModule.forRootAsync({
      imports: [ConfigModule],
      connectionName: 'patients',
      useFactory: async (configService: ConfigService) => {
        return {
          uri: configService.get(CONFIG_DATABASE).patients.uri,
        };
      },
      inject: [ConfigService],
    }),
    MongooseModule.forRootAsync({
      imports: [ConfigModule],
      connectionName: 'health',
      useFactory: async (configService: ConfigService) => {
        return {
          uri: configService.get(CONFIG_DATABASE).health.uri,
        };
      },
      inject: [ConfigService],
    }),
    InsightModule,
  ],
})
export class AppModule {}

Insight Module

The InsightModule is responsible for managing the Insight, Patient, and Health entities. It imports the necessary Mongoose schemas and registers them with their respective database connections. This module also provides the InsightService and InsightController to handle business logic and HTTP requests related to insights.

Imports the necessary Mongoose schemas and registers them with their respective database connections

// src/insight/insight.module.ts
// import ...
@Module({
  imports: [
    MongooseModule.forFeature(
      [{ name: Insight.name, schema: InsightSchema }],
      'insight',
    ),
    MongooseModule.forFeature(
      [{ name: Patient.name, schema: PatientSchema }],
      'patients',
    ),
    MongooseModule.forFeature(
      [{ name: Health.name, schema: HealthSchema }],
      'health',
    ),
  ],
  controllers: [InsightController],
  providers: [InsightService],
})
export class InsightModule {}

Insight Service

In the InsightService, the @InjectModel decorator injects Mongoose models with specified connection names to ensure each model is linked to the correct database:

// src/insight/insight.service.ts
// import ...

@Injectable()
export class InsightService {
  constructor(
    @InjectModel(Insight.name, 'insight')
    private insightModel: Model<InsightDocument>,

    @InjectModel(Health.name, 'health')
    private healthModel: Model<HealthDocument>,

    @InjectModel(Patient.name, 'patients')
    private patientModel: Model<PatientDocument>,
  ) {}

  // ...
}

We use an example by getting an insight for patients less than 30 years old to demonstrate the core method for data aggregation:

• Fetch Patients: Retrieves all patients under the age of 30 from the patients database.
• Aggregate Health Data: Retrieves and groups health conditions for these patients from the health database.
• Check Existing Insights: Checks if an insight already exists for the current date.
• Update or Create Insight: If an existing insight is found, it updates it with new data; otherwise, it creates a new insight document with the aggregated data.

// src/insight/insight.service.ts
// import ...
async generateInsightByAgeLessThan30(): Promise<Insight> {
    // Step 1: Retrieve all patients under the age of 30
    const patientsUnder30 = await this.patientModel
      .find({ age: { $lt: 30 } })
      .exec();

    // Step 2: Retrieve health conditions for patients under 30 and group by condition
    const patientIdsUnder30 = patientsUnder30.map(
      (patient) => patient.patientId,
    );

    const healthAggregation = await this.healthModel
      .aggregate([
        { $match: { patientId: { $in: patientIdsUnder30 } } },
        { $unwind: '$conditions' },
        { $group: { _id: '$conditions', count: { $sum: 1 } } },
        { $project: { _id: 0, condition: '$_id', count: 1 } },
      ])
      .exec();

    // Step 3: Check if an insight for today already exists
    const today = new Date();
    const formattedDate = today
      .toLocaleDateString('en-GB')
      .split('/')
      .reverse()
      .join('-');

    const existingInsight = await this.insightModel
      .findOne({ date: formattedDate })
      .exec();

    const insightData = {
      data: healthAggregation,
      type: 'Health Conditions for Patients under 30',
      date: formattedDate,
    };

    if (existingInsight) {
      // Update the existing insight
      Object.assign(existingInsight, insightData);
      return existingInsight.save();
    } else {
      // Create a new insight
      const newInsight = new this.insightModel(insightData);
      return newInsight.save();
    }
  }

Testing the Insight Module

To test the functionality, you can use the following endpoint:

GET localhost:3001/insight/health-conditions-under-30

{
  "_id": "6677f1c036278e0dfd3092d5",
  "data": [
    {
      "condition": "asthma",
      "count": 1,
      "_id": "6677f493ce9bf5752a7f517a"
    },
    {
      "condition": "diabetes",
      "count": 2,
      "_id": "6677f493ce9bf5752a7f517b"
    },
    {
      "condition": "osteoporosis",
      "count": 1,
      "_id": "6677f493ce9bf5752a7f517c"
    }
  ],
  "type": "Health Conditions for Patients under 30",
  "date": "2024-06-23",
  "__v": 2
}

Source Code

The complete source code for this example is available on GitHub. You can find the repository at the following link https://github.com/kelvin-bz/nestjs-mongooes-multiple-database

Conclusion

By following this guide, you have successfully set up a NestJS service that connects to multiple MongoDB databases using Mongoose. This setup allows you to perform complex data aggregation and insights generation across different collections stored in separate databases.

Read more articles about NestJS : https://kelvinbz.medium.com/list/nestjs-eafb7de3e562

Read more articles about AWS: https://kelvinbz.medium.com/list/aws-32c5d9525c7d