Mongodb Use Cases For Aggregation Pipelines Complete Guide

MongoDB Use Cases for Aggregation Pipelines

1. Data Transformation and Filtering

Use Case: Data cleaning and preprocessing are crucial stages in data analysis. Aggregation pipelines can perform these tasks efficiently by filtering out unwanted data and transforming the existing data into a suitable format.

Important Information:

• $match: Filters documents that pass the specified condition.
• $project: Reshapes each document in the stream, such as by adding new fields or removing existing fields.
• $redact: Restricts the documents to the set of fields that match the given condition.
• $replaceRoot: Replaces the input document with the specified document.

Example:

db.sales.aggregate([
  { $match: { status: "A" } },
  { $project: { _id: 0, item: 1, price: 1, quantity: 1 } }
])

This pipeline filters documents with a status of "A" and projects only the item, price, and quantity fields.

2. Data Aggregation and Grouping

Use Case: Summarizing and aggregating large datasets is essential for generating insights and driving business decisions. Aggregation pipelines can perform complex grouping operations.

Important Information:

• $group: Groups input documents by a specified expression and outputs to the next stage a document for each distinct grouping.
• $accumulator: Adds user-defined accumulators to the $group stage.
• $bucket: Categorizes incoming documents into groups based on specified boundaries.

Example:

db.sales.aggregate([
  { $group: { _id: "$item", totalQuantity: { $sum: "$quantity" } } }
])

This pipeline groups documents by the item field and sums up the quantity for each item.

3. Data Sorting and Limiting

Use Case: Sorting and limiting results are common requirements in data analysis to focus on top-performing or bottom-performing items.

Important Information:

• $sort: Sorts all input documents and returns them in sorted order.
• $limit: Limits the number of documents passed to the next stage in the pipeline.
• $skip: Skips a specified number of documents and passes the remaining documents to the next stage.

Example:

db.sales.aggregate([
  { $sort: { totalQuantity: -1 } },
  { $limit: 5 }
])

This pipeline sorts the documents by totalQuantity in descending order and limits the results to the top 5.

4. Joining Data from Different Collections

Use Case: Data often resides in multiple collections or databases. Aggregation pipelines can perform lookups to combine data from different collections.

Important Information:

• $lookup: Performs a left outer join to another collection in the same database to filter in documents from the “joined” collection for processing.
• $graphLookup: Recursively joins subdocuments to the current document from the same collection.

Example:

db.orders.aggregate([
  {
    $lookup:
      {
        from: "customers",
        localField: "cust_id",
        foreignField: "customer_id",
        as: "customer_info"
      }
   }
])

This pipeline joins the orders collection with the customers collection based on the cust_id and customer_id fields.

5. Calculating Running Totals and Moving Averages

Use Case: Running totals and moving averages are useful for analyzing trends over time, such as stock price movements or sales performance.

Important Information:

• $addFields: Adds new fields to documents. $addFields outputs documents that contain all existing fields from the input documents and newly added fields.
• $window: Performs operations on a set of documents in a defined window and outputs the result for each document individually. $window can be used to calculate running totals and moving averages.

Example:

db.sales.aggregate([
  {
    $window:
      {
        output:
          {
            runningTotal: { $sum: "$value" },
            movingAvg: { $avg: "$value", window: { documents: [-2, 2] } }
          }
      }
  }
])

This pipeline calculates a running total and a moving average with a window of 5 documents centered on the current document.

6. Geospatial Aggregation

Use Case: Geospatial data is critical in location-based services, logistics, and urban planning. Aggregation pipelines can perform geospatial operations directly within MongoDB.

Important Information:

• $geoNear: Outputs documents in order of nearest to farthest from a specified point.
• $geoWithin: Selects documents with geospatial data within a specified shape.
• $geoIntersects: Selects documents with geospatial data that intersect with a specified shape.

Example:

db.places.aggregate([
  {
    $geoNear:
      {
        near: { type: "Point", coordinates: [ -73.99279 , 40.719296 ] },
        distanceField: "dist.calculated",
        maxDistance: 2000,
        spherical: true
      }
  }
])

This pipeline retrieves locations within a maximum distance of 2000 meters from the specified point.

7. Data Analysis and Reporting

Use Case: Generating business reports and performing data analysis are vital for making informed decisions. Aggregation pipelines can provide the required data in real-time.

Important Information:

• $facet: Performs multiple aggregation operations in parallel.
• $bucketAuto: Categorizes incoming documents into a specific number of groups based on a specified expression.
• $out: Writes the result of the aggregation pipeline to a specified collection.

Example:

db.orders.aggregate([
  {
    $facet:
      {
        byCategory:
          [
            { $group: { _id: "$category", totalSales: { $sum: "$amount" } } },
            { $sort: { totalSales: -1 } },
            { $limit: 5 }
          ],
        salesByDay:
          [
            { $group: { _id: { $dayOfYear: "$orderDate" }, totalSales: { $sum: "$amount" } } },
            { $sort: { _id: 1 } },
            { $limit: 12 }
          ]
      }
  }
])

This pipeline generates two reports: the top 5 categories by total sales and the total sales by day for the last 12 days.

Conclusion