Atlas Search Deployment Options

Node Architecture

For testing and prototyping environments, we recommend a node architecture in which MongoDB processes and Atlas Search processes run on the same node. In the following diagram of this deployment model, the Atlas Search mongot process runs alongside mongod on each node in the Atlas cluster and they share the same resources.

By default, Atlas enables the Atlas Search mongot process on the same node that runs the mongod process when you create your first Atlas Search index.

When you run a query, Atlas Search uses the configured read preference to identify the node on which to run the query. The query first goes to the MongoDB process, which is mongod for a replica set cluster or mongos for a sharded cluster.

For a replica set cluster, the mongod process routes the query to the mongot on the same node. For sharded clusters, your cluster data is partitioned across mongod instances (shards) and each mongot process can only access the data on the mongod instance on the same node. Therefore, you can't run Atlas Search queries that target a particular shard. mongos routes the query to all shards, making these scatter gather queries. If you use zones to distribute a sharded collection over a subset of the shards in the cluster, Atlas Search routes the query to the zone that contains the shards for the collection that you are querying and runs your $search queries on just the shards where the collection is located.

After the query is routed to an Atlas Search mongot process, the mongot process performs the search and scoring and returns the document IDs and other search metadata for the matching results to its corresponding mongod process. The mongod process then performs a full document lookup implicitly for the matching results and returns the results to the client. If you use the $search concurrent option in your query, Atlas Search enables intra-query parallelism. To learn more, see Parallelize Query Execution Across Segments.

To learn more about the mongot process, see Query Processing.

You can define stored source fields in your Atlas Search index so that the mongot process can store the specified fields on mongot. You can then use the returnStoredSource Option in your Atlas Search query to retrieve the stored fields for matching documents directly from mongot instead of doing a full document lookup on the database.

Benefits

When you enable Atlas Search, you can easily build search on top of your data with an integrated, fully managed search engine that automatically syncs to your database. Atlas Search provides a rich query language that uses Atlas Search aggregation pipeline stages like $search and $searchMeta for full-text search and $vectorSearch for semantic search in conjunction with other MongoDB aggregation pipeline stages, and score-based results ranking.

Depending on the resources provisioned for your cluster, deploying both processes on the same node might be more cost-efficient than running the search process on a separate, dedicated node.

Limitations

You might experience resource contention between the database mongod and the search mongot processes. This could negatively impact the performance of your index and latency of your queries. To support production-ready applications and their search workloads migrate to dedicated Search Nodes.

Cost

There are no additional fees or charges when you enable Atlas Search on your Atlas cluster. However, you might observe an increase in resource utilization on the cluster for large indexed collections or index definitions.

Considerations

Since both the mongod and mongot processes run on the same node, mongot might become unavailable under certain circumstances. The following table describes potential causes:

Node Architecture

For production environments, we recommend a node architecture in which MongoDB processes and Atlas Search processes run on separate nodes. To deploy separate Search Nodes, see Migrate to Dedicated Search Nodes.

In the following diagram of this deployment model, the Atlas Search mongot process runs on dedicated Search Nodes, which are separate from the cluster nodes on which the mongod process runs.

Atlas deploys Search Nodes with each cluster or with each shard on the cluster. For example, if you deploy two Search Nodes for a cluster with three shards, Atlas deploys six Search Nodes (two per shard). You can also configure the number of Search Nodes and the amount of resources provisioned for each search node.

When you deploy separate Search Nodes, Atlas automatically assigns a mongod for each mongot for indexing. The mongot communicates with the mongod to listen for and sync index changes for the indexes that it stores. Atlas Search indexes and processes your queries similar to a deployment where both the mongod and mongot processes run on the same node. To learn more, see Manage Atlas Search Indexes and Queries and Indexes. To learn more about deploying Search Nodes separately, see Search Nodes for Workload Isolation.

When you migrate to Search Nodes, Atlas deploys the Search Nodes, but doesn't serve queries on the nodes until it successfully builds all the indexes on the cluster on the Search Nodes. While Atlas builds the indexes on the new nodes, it continues to serve queries using the indexes on the cluster nodes. Atlas starts serving queries from the Search Nodes only after it successfully builds the indexes on the Search Nodes and removes the indexes on the cluster nodes.

When you run a query, the query routes to the mongod based on the configured read preference. The mongod process routes the search query through a load balancer on the same node, which distributes the requests across all of the mongot processes.

The Atlas Search mongot process performs the search and scoring and returns the document IDs and metadata for the matching results to mongod. The mongod then performs a full document lookup for the matching results and returns the results to the client. If you use the $search concurrent option in your query, Atlas Search enables intra-query parallelism. To learn more, see Parallelize Query Execution Across Segments.

If you delete all the Search Nodes on your cluster, there will be an interruption in processing your search query results. To learn more, see Modify a Cluster. If you delete your Atlas cluster, Atlas pauses and then deletes all associated Atlas Search deployments (mongot processes).

You can define stored source fields in your Atlas Search index so that the mongot process can store the specified fields on mongot. You can then use the returnStoredSource Option in your Atlas Search query to retrieve the stored fields for matching documents directly from mongot instead of doing a full document lookup on the database.

Benefits

Deploying separate Search Nodes provides the following benefits:

High Availability

When you deploy separate Search Nodes, Atlas enforces a minimum of two Search Nodes to ensure your workload remains operational, with minimal downtime, in the event of a failure or disruption.

Scalability

When you deploy separate Search Nodes, you can scale the storage and compute independently of your MongoDB cluster. This allows you to also scale the query load independently of MongoDB.

To scale Search Nodes horizontally, increase or reduce the number of Search Nodes. You can provision from a minimum of 2 up to a maximum of 32 Search Nodes. To balance the query load, Atlas Search distributes search queries across all available Search Nodes.

To scale Search Nodes vertically, select different search tiers, CPU, RAM and storage configurations that support your full-text workloads.

Performance

When you deploy dedicated Search Nodes, you improve the performance and resource utilization for both the mongod and mongot processes and eliminate resource contention between these processes.

Dedicated Search Nodes support concurrent segment search, which allows Atlas Search to search multiple index segments at the same time. Using concurrent segment search improves query response time in some cases.

Tips for Sizing and Scaling Search Nodes

To determine the memory requirements for Search Nodes, use the following Atlas metrics:

• Size of the Search Index

• Total RAM on the Search Node

Consider an application that has a 10GB Search Index and 4GB total RAM on the Search Node. In this case, if 1GB of RAM is used by other processes and only 3GB are available for the index data, the remaining 7GB of the index data (10GB - 3GB = 7GB) is paged in, as needed, from the disk. Frequent paging from disk causes increased page faults, disk I/O, and CPU IOWait, resulting in performance degradation.

If you use a higher search cluster tier with more RAM, such as 8GB or more, this allows Atlas to serve most of the data for the Search Index from memory, minimizing disk reads and page faults, thereby improving performance.

Search Nodes Cost

MongoDB supports separate Search Nodes on dedicated (M10 or higher) clusters. Search Nodes are deployed on compute-intensive NVMe instances. You must deploy a minimum of two nodes. You will be billed daily for hourly resource usage per node. To learn more, see Search Node Costs.

Enable Encryption at Rest

You can enable Encryption at Rest with Customer Key Management for all data on Search Nodes to secure your Atlas Search workloads with customer-managed encryption keys. To learn more, see Enable Customer Key Management for Search Nodes.

This feature is currently available for AWS KMS only.

Add Dedicated Search Nodes

Adding dedicated Search Nodes to a new cluster allows you to:

• Change the size and scale of your search deployment independently from your cluster.

• Eliminate resource contention that you might experience on a cluster that runs both the MongoDB database and search processes on the same node.

To add dedicated Search Nodes:

1. Create your cluster as an M10 or higher tier in a cloud provider and region that supports node isolation. To learn more, see Create a Cluster.

   Dedicated Search Nodes are supported only for M10 and higher cluster tiers and in cloud provider regions that support node isolation.

2. Enable Search Nodes for workload isolation and Configure Search Nodes.

Migrate to Dedicated Search Nodes

To migrate from staging to production and add dedicated Search Nodes, make the following changes to your existing staging and prototyping deployment:

1. If your deployment uses a Flex cluster or shared tier cluster, change the cluster tier to a higher tier. Dedicated Search Nodes are supported only for M10 and higher cluster tiers.

2. Deploy your cluster in regions where Search Nodes are also available. Dedicated Search Nodes are available on a subset of the AWS and Azure regions and in all supported Google Cloud regions. If your existing cluster is hosted in regions where Search Nodes aren't available, migrate your cluster to regions where Search Nodes are available. To learn more, see Cloud Provider Regions that Support Node Isolation.

3. Enable Search Nodes for workload isolation and configure Search Nodes. To learn more, see Add Search Nodes.

   When you deploy dedicated Search Nodes, the following sequence of actions takes place:

   • Atlas builds the search indexes on the Search Nodes and removes the indexes from the cluster nodes.

   • Atlas routes search queries to the Search Nodes.

   • Atlas Search uses the search indexes to serve queries on the Atlas cluster.

Troubleshoot Deployment