Mongodb Architecture Overview Complete Guide

Step 1: Understand the Basics

What is MongoDB?

• MongoDB is a popular NoSQL database. It stores data in flexible, JSON-like documents, which makes it highly suitable for applications requiring agile development and scalability.

NoSQL vs SQL:

• NoSQL:
  • Stores data in non-relational formats like JSON, XML, key-value pairs, etc.
  • Designed for unstructured data, large-scale data distribution, high performance, and availability.
• SQL:
  • Stores data in tables with fixed schemas.
  • Uses structured query language (SQL) for data manipulation.

Example:

// SQL example:
CREATE TABLE users (
    id INT PRIMARY KEY,
    username VARCHAR(50),
    email VARCHAR(100)
);

// MongoDB example (document):
{
    "_id": ObjectId("60a9f8e4c3b2f12345678901"),
    "username": "johndoe",
    "email": "john.doe@example.com"
}

Step 2: Learn About the Core Components of MongoDB Architecture

1. Database Server (mongod):

   • The mongod process runs as a daemon service and handles data requests from clients.

2. Database Shell (mongo):

   • An interactive JavaScript shell used to access MongoDB and perform administrative tasks.

3. Clients:

   • Application servers (like Node.js, Python, Java, etc.) that connect to the MongoDB server via drivers (official MongoDB libraries).

4. Configuration Files:

   • YAML/JSON files that define settings such as storage configuration, security options, and networking details.

Step 3: Explore MongoDB Data Models

• **Database:``

  • A container for all collections. In MongoDB, databases are created when we first store data in them.

• Collection:

  • A group of documents. It's conceptually similar to a table in relational databases but more flexible.

• Document:

  • The basic unit of data in MongoDB, stored as BSON (similar to JSON).

Example:

Let’s create a simple library database, containing a books collection with several documents.

Creating Database through mongo shell:

$ mongo

Inside the MongoDB shell:

> use library
switched to db library

Insert Documents into Collection:

> db.books.insertMany([
    { title: "1984", author: "George Orwell", year: 1949 },
    { title: "To Kill a Mockingbird", author: "Harper Lee", year: 1960 },
    { title: "Pride and Prejudice", author: "Jane Austen", year: 1813 }
]);

Step 4: Understand Storage Engine

• MongoDB stores documents in collections within databases on disk using a storage engine. As of version 4.4, MongoDB uses WiredTiger as its default storage engine, which offers features like document-level concurrency, compression, and efficient storage of index structures.

Key Concepts:

• Journaling: Ensures data durability by recording modifications to the database and replaying them after a restart if necessary.
• Snapshot Views: Allows querying a consistent view of the database at a specific point in time without locking.

Example:

Check the current storage engine with the following command inside the mongo shell:

> db.serverStatus().storageEngine.name
"WiredTiger"

Step 5: Learn about Indexing in MongoDB

• Indexes speed up the query process by allowing MongoDB to skip over large amounts of data.

Types of Indexes:

• Single Field: Created on an individual field of the documents.
• Compound Index: Created on multiple fields. Sorting matters; indexes can be ordered in either ascending/descending order.
• Multikey Index: Automatically created when a document’s indexed field is an array.
• Geospatial Index: Enables querying geographic data efficiently using operators like $near, $within.
• Text Index: Useful for searching textual content.

Example:

Create an index for the author field in our books collection:

> db.books.createIndex({ author: 1 }) // 1 for ascending
{
    "createdCollectionAutomatically" : false,
    "numIndexesBefore" : 1,
    "numIndexesAfter" : 2,
    "ok" : 1
}

Step 6: Understand Sharding

• Sharding: Horizontally scales out operations across multiple physical servers, distributing the dataset and the workload.

Key Concepts:

• Shard: A MongoDB instance that holds a part of the sharded data.
• Config Servers: Hold the sharding metadata.
• Mongos Routes: Direct requests to the appropriate shards or config servers.

Basic Steps to Set Up a Sharded Cluster:

1. Start Config Servers:
2. Start Shard Servers (mongod):
3. Start the Routing Process (mongos):
4. Add Shard Servers to the Cluster:
5. Enable Sharding on a Database:
6. Create the Shard Key to Define the Data Distribution:

Example:

Imagine you have a users database that you want to shard based on the state field. Here are some simplified steps:

1. Start config servers:

$ mongod --configsvr --dbpath /data/configdb --port 27019

2. Start shard servers:

$ mongod --shardsvr --dbpath /data/shard0 --port 27020
$ mongod --shardsvr --dbpath /data/shard1 --port 27021

3. Start the routing process:

$ mongos --configdb localhost:27019 --port 27017

4. Add shard servers to the cluster:

$ mongo

Inside the MongoDB shell connected to mongos:

> sh.addShard("localhost:27020")
> sh.addShard("localhost:27021")

5. Enable sharding on a database:

> sh.enableSharding("users")

6. Create the shard key to distribute data based on state:

> sh.shardCollection("users.users", { state: "hashed" })

Step 7: Explore Replication

• Replication: Provides redundancy and fault tolerance through multiple copies of data distributed across different physical servers.

Core Concepts:

• Primary: Accepts all write operations.
• Secondaries: Receive copies of primary data and can take over as primary if the primary goes down.
• Replica Set: Consists of one primary and several secondaries, usually 3-5 nodes.

Basic Steps to Set Up a Replica Set:

1. Configure each member of the replica set with the same replica set name and unique identifier (_id).
2. Start MongoDB instances on each member machine.
3. Connect to any one member of the replica set using the mongo shell.
4. Initialize the replica set by connecting to any node and running rs.initiate().
5. Add more secondaries to the replica set using rs.add() command.
6. Ensure read queries are routed appropriately using readPreference.

Example:

Set up a replica set with three members:

1. Configure instances in three separate configurations:

# config for member 0 (replica0.conf):
replication:
  replSetName: "myReplSet"
systemLog:
  destination: file
  path: "/var/log/mongodb/repl0.log"
  logAppend: true
storage:
  dbPath: "/var/lib/mongo/repl0"
net:
  bindIp: "127.0.0.1"
  port: 27017

# config for member 1 (replica1.conf):
replication:
  replSetName: "myReplSet"
systemLog:
  destination: file
  path: "/var/log/mongodb/repl1.log"
  logAppend: true
storage:
  dbPath: "/var/lib/mongo/repl1"
net:
  bindIp: "127.0.0.1"
  port: 27018

# config for member 2 (replica2.conf):
replication:
  replSetName: "myReplSet"
systemLog:
  destination: file
  path: "/var/log/mongodb/repl2.log"
  logAppend: true
storage:
  dbPath: "/var/lib/mongo/repl2"
net:
  bindIp: "127.0.0.1"
  port: 27019

2. Start mongod instances:

$ mongod -f replica0.conf
$ mongod -f replica1.conf
$ mongod -f replica2.conf

3. Connect to any one member and initialize the replica set:

$ mongo --port 27017

Inside the shell:

> rs.initiate({
    _id: "myReplSet",
    members: [
        { _id: 0, host: "localhost:27017" },
        { _id: 1, host: "localhost:27018" },
        { _id: 2, host: "localhost:27019" }
    ]
})

Step 8: Dive Into MongoDB Security

• Authentication: Ensures that only legitimate users can access the database.
• Authorization: Defines what each user can do within the database.

Key Concepts:

• Access Control Lists (ACL): Define roles and permissions.
• Users: Stored in the admin or system.users collection.
• Built-in Roles: Provide pre-defined authorization levels (e.g., read, readWrite, clusterAdmin).

Example:

Setting up a simple admin user for authentication:

Enable Authentication: Edit the configuration file mongod.conf:

security:
  authorization: enabled

Restart the MongoDB server.

Create an Admin User:

$ mongo --port 27017

Inside the mongo shell:

use admin
db.createUser(
  {
    user: "adminUser",
    pwd: "securePassword",
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)

Now you need to login:

$ mongo --port 27017 -u "adminUser" -p "securePassword" --authenticationDatabase "admin"

Step 9: Familiarize Yourself with Backup and Restore Processes

• Backup: Ensures data safety by creating a copy of the entire or part of the MongoDB environment.
• Restore: Reconstructs the database from a backup in case of data loss, corruption, etc.

Key Methods:

• mongodump: Exports data into a binary format.
• mongorestore: Imports data from a dump back into MongoDB.

Example:

Perform a backup of library database:

$ mongodump --db library

This command creates a directory named dump/library with BSON files of the collections.

Restoring the library database:

$ mongorestore --db library dump/library/

Step 10: Monitor and Manage MongoDB Instances

• Monitoring: Ensures that the operations are running smoothly.
• Management Tools: Include the MongoDB Management Service (MMS), which provides automated monitoring, alerting, and backup.

Key Metrics:

• CPU Usage: Should remain low to medium. High usage indicates CPU-bound queries or operations.
• Disk I/O: Should not be maxed out. MongoDB is sensitive to slow disks.
• Memory Usage: MongoDB tries to keep data in RAM, so monitor memory usage carefully.

Example:

Using mongostat to monitor database performance:

$ mongostat
insert query update delete locked flushV % mem vsize res qrw arw net_in net_out conn      set repl       time
    *0    *0     *0     *0  0.0%        1 2.2G 1.0G   0   0|0 1.1G 1.1G 11.8G 2.90G   2 myReplSet PRI Jun 01 08:17:18.670

This tool shows various runtime statistics, such as the number of queries, inserts, updates, and the system’s response to these queries.

Conclusion

MongoDB’s architecture allows it to handle complex scenarios with flexibility and efficiency. Starting with databases, collections, and documents, and moving towards advanced features like sharding, replication, security, and monitoring will give you a comprehensive understanding of how MongoDB works and how to deploy it effectively in your projects.