Mongodb Querying And Aggregating With Python Complete Guide

Prerequisites:

1. You have MongoDB installed and running.
2. You have installed the MongoDB Python Driver (pymongo) which can be done using pip install pymongo.

We'll use a simple example dataset of a bookstore containing books.

Step 1: Connect to MongoDB

First, let's connect to MongoDB using Python. Make sure your MongoDB server is running (default port 27017).

from pymongo import MongoClient

# Connect to MongoDB
client = MongoClient('localhost', 27017)

# Select the database
db = client['bookstore']

# Select the collection
books_collection = db['books']

Step 2: Insert Sample Data

Let's insert some sample data into our books collection.

sample_books = [
    {"title": "The Great Gatsby", "author": "F. Scott Fitzgerald", "genre": "Novel", "year": 1925, "price": 9.99},
    {"title": "To Kill a Mockingbird", "author": "Harper Lee", "genre": "Novel", "year": 1960, "price": 8.59},
    {"title": "1984", "author": "George Orwell", "genre": "Dystopian", "year": 1949, "price": 8.99},
    {"title": "Brave New World", "author": "Aldous Huxley", "genre": "Dystopian", "year": 1932, "price": 10.50},
    {"title": "Animal Farm", "author": "George Orwell", "genre": "Satire", "year": 1945, "price": 4.99},
    {"title": "Pride and Prejudice", "author": "Jane Austen", "genre": "Romance", "year": 1813, "price": 12.99}
]

# Insert multiple documents into the collection
books_collection.insert_many(sample_books)

Step 3: Simple Queries

Querying All Documents

To retrieve all documents from the books collection:

# Querying all documents
all_books = books_collection.find({})

for book in all_books:
    print(book)

Querying Documents with Specific Criteria

Let's find books written by George Orwell:

# Querying books by George Orwell
george_orwell_books = books_collection.find({"author": "George Orwell"})

for book in george_orwell_books:
    print(book)

Querying with Multiple Conditions

Find books authored by George Orwell that were published after 1944:

# Querying books by George Orwell published after 1944
george_orwell_after_1944 = books_collection.find({"author": "George Orwell", "year": {"$gt": 1944}})

for book in george_orwell_after_1944:
    print(book)

Step 4: Filtering Fields to Display

Sometimes we don't need to display all fields. We can filter them:

# Querying books by George Orwell and displaying only title and year
filtered_books = books_collection.find({"author": "George Orwell"}, {"title": 1, "year": 1, "_id": 0})

for book in filtered_books:
    print(book)

Step 5: Sorting the Results

Let's sort the books by their price in ascending order:

# Querying all books and sorting by price
sorted_books_by_price = books_collection.find().sort("price", 1)

for book in sorted_books_by_price:
    print(book)

Step 6: Aggregation Framework

Now let's use MongoDB's aggregation framework. We'll do some examples such as grouping by genre, counting the number of books per genre, and calculating average prices per genre.

Grouping Books by Genre and Counting Number of Books

# Aggregation: Grouping by genre and counting books
pipeline = [
    {"$group": {"_id": "$genre", "count": {"$sum": 1}}}
]

genres_count = books_collection.aggregate(pipeline)

for genre in genres_count:
    print(genre)

Average Price per Genre

# Aggregation: Grouping by genre and calculating average price
pipeline = [
    {"$group": {"_id": "$genre", "average_price": {"$avg": "$price"}}}
]

average_price = books_collection.aggregate(pipeline)

for item in average_price:
    print(item)

Adding More Stages: Grouping, Filtering, and Projecting

Let's aggregate data in stages: group by genre, add a condition to filter out genres with less than 2 books, calculate the average price per genre and format the results nicely.

# Complex Aggregation: Grouping by genre, filtering, and calculating average price
pipeline = [
    {"$group": {"_id": "$genre", "count": {"$sum": 1}, "average_price": {"$avg": "$price"}}},
    {"$match": {"count": {"$gte": 2}}},
    {"$project": {"_id": 0, "genre": "$_id", "number_of_books": "$count", "average_price": 1}},
    {"$sort": {"average_price": 1}}
]

formatted_aggregated_data = books_collection.aggregate(pipeline)

for item in formatted_aggregated_data:
    print(item)

Step 7: Close Connection

Finally, close the connection when done.

client.close()

Final Code

Combining everything into one script: