#TECH

Step by Step Guide to Deploy MongoDB Replica Sets to CentOS VMs Running on Azure

MongoDB is an open-source document database, and the leading NoSQL database. Using the Azure Management Portal, we can create a virtual machine running CentOS Linux from the image library, than we can install and configure a database MongoDB data on the virtual machine.

We will learn the following:

  • Select and install 3 CentOS Linux virtual machine from the library using the management portal making two vms as the replica vms.
  • Connect to the virtual machines using SSH or PuTTY;
  • Assign a  virtual hard disk (VHD)  to all the vms .
  • Install MongoDB  on the virtual machine.
  • Connect to mongodb and its replica sets.

Before we start  lets clear our concept on why we need Mongodb Replication on the first place.

A replica set in MongoDB is a group of MongoDB processes that maintain the same data set. Replica sets provide redundancy and high availability, and are the basis for all production deployments.

Replica Set Primary – The primary is the only member of a replica set that accepts write operations.

Replica Set Secondary Members- Secondary members replicate the primary’s data set and accept read operations. If the set has no primary, a secondary can become primary.

Creating a virtual machine running CentOS Linux

Step 1 

Set Up Affinity Group:  Affinity groups are a way you can group your cloud services by proximity to each other in the Azure data center in order to achieve optimal performance. When you create an affinity group, it lets Azure know to keep all of the services that belong to your affinity group as physically close to each other as possible. 

To create affinity group login to https://manage.windowsazure.com than go to settings  and add new affinity group.

Azure affinity group

Step 2 

Create and Set Up the Virtual Machines:

Instance 1

In the Management Portal, in the lower left corner of the Web page, click + New on Virtual Machine, and then click From Gallery.
Select a virtual machine image from CentOS Platform Images and click the next arrow in the lower right corner of the page.

Centos image azure

On the VM Configuration page, specify values for the parameters

  • Virtual Machine Name – “mongodbrs1”
  • New User Name – “mongouser.”
  • New Password box – type a strong password that conforms to the Azure specifications
  • Retype the password to confirm
  • Size – Choose appropriate size from drop down list. For anything but small test instances, choose a size larger than Medium
  • Leave “Upload SSH key for Authentication” unchecked

Mongo azure step1

Click the next arrow to continue

  • On the VM Mode page, specify values for the required parameters
  • Select “Standalone Virtual Machine”
  • DNS – a valid DNS prefix e.g., “mongodbrs”
  • Storage Account box – choose “Use Automatically Generated Storage Account”
  • In the “Region/Affinity Group/Virtual Network” box, select the affinity group created previously “mongoaffinitygroup”

Click next arrow to continue
On the VM Options page select “None for Availability Set”

Instance 2

A similar process to creating instance 1.
Set the Virtual Machine Name – “mongodbrs2”

Instance 3

A similar process for creating instance 2. Choose “mongodbrs3” to be the Virtual Machine Name in step 3.

Configure Endpoints

Once the virtual machines are connected you need to configure the endpoints to:

  • Allow remote connection
  • Allow MongoDB traffic

Instance 1

In the management portal, click virtual machines and click the name of instance 1, “mongodbrs1”
Now click on endpoints
The ssh endpoint should be automatically created. For this endpoint ensure the following are set

  • Protocol – tcp
  • Public Port – 22
  • Private Port – 22
  • Load Balanced – No

Create a new endpoint for MongoDB by clicking on “Add Endpoint” at the bottom of the screen
Ensure “Add endpoint” is selected and click the right arrow to go to the next screen.
Specify the endpoint details as below:

  • Name : MongoDB-Port
  • Protocol : TCP
  • Public Port : 27018
  • Private Port : 27018
  • Click on check mark to create endpoint.The instance now should have 2 endpoints, 1 for SSH and 1 for MongoDB

Mongo azure Endpoints

Instance 2

We need to configure the endpoints for instance 2 similar to instance 1:
In the management portal, click “virtual machines” and click the name of instance 2
Now click on “endpoints”
The ssh endpoint should be automatically created. Ensure that:

  • Name – SSH
  • Protocol – TCP
  • Private Port – 22
  • Load Balanced – No

Now click on Edit Endpoint at the bottom of the screen and set “Public Port” to 23. Click on the “check mark” to update
Create a new endpoint for MongoDB by clicking on “Add Endpoint” at the bottom of the screen
Ensure “Add Endpoint” is selected and click the right arrow to go to the next screen
Specify the endpoint details as below:

  • Name : MongoDB-Port
  • Protocol : TCP
  • Public Port : 27019
  • Private Port : 27019
  • Click on check mark to create endpoint.The instance now should have 2 endpoints, 1 for SSH and 1 for MongoDB

Mongo azure instance 2 endpoints

Instance 3

Create endpoints for instance 3 similar to instance 2 with the following changes:
In step 4, set public port to 24
In step 7, set public and private ports to be 27020
The instance now should have 2 endpoints, 1 for SSH and 1 for MongoDB.

Mongo Azure instance 3 Endpoints

Connecting to the virtual machine 

Once the virtual machine deployed and endpoints configured, you can connect to it using SSH or PuTTY.

Login with SSH
If you use Linux, log on to the virtual machine using SSH. At the command prompt, run:
$ Ssh newuser@testlinuxvm.cloudapp.net ServerAliveInterval = 180
Enter the password of the user.

Login with PuTTY

If you use a Windows computer, connect to the virtual machine using PuTTY, which you can download from the download page of PuTTY.

Download and save putty.exe on your computer. Open a command prompt, navigate to that folder and run putty.exe.

Enter the Host Name >> << testlinuxvm.cloudapp.net and port number << 22>>

Azure mongo login putty

 

Update the virtual machine
Once connected to the virtual machine, you can optionally install updates. run:

$ Sudo yum update 

Re-enter the password. Wait for the installation of updates.

Imp – When connecting to instances 2 and 3, remember to use ports 23 and 24 and not the default ssh port of 22.

Set up Disks 

Initialize the data disk by following the steps described at How to: Initialize a new data disk in Linux.
After this Connecting to the virtual machine using SSH or PuTTY and completion of installation
Once added, the data disk that you just attach to the virtual machine is offline and is not initialized. You must log in to the machine, and set the disc to use for data storage.

Once the virtual machine deployed, log on as newuser (as described in the previous steps) using SSH or PuTTY.

In the PuTTY SSH or window, enter the following command and the password of the account:

$ Sudo grep SCSI / var / log / messages 

azure mongo data disk step 1

The identifier of the last record added data is available in the messages displayed (ndc, in this example).
Image data disk -1
In the PuTTY SSH or window, enter the following command to partition the disk / dev / sdc:

$ Sudo fdisk / dev / sdc
Enter n to create a partition.

Azure mongo data disk step 2

Type p to set the partition as a primary partition, type 1 to set it as the first partition, and then press Enter to accept the default (1) of the cylinder.

Mongodb azure set it as the first partition

Type p to display the details of the disk covered by the partition.

moggodb display the details of the disk covered by the partition

Type w to write the disk parameters.

Mongodb5

Format the new disk using the mkfs command:

$ sudo mkfs -t ext4 /dev/sdc1

You should then have a directory available to mount the new file system. For example, type the following command to create a directory to mount the drive, then enter the password for the account:

sudo mkdir /datadrive

Type the following command to mount the drive:

sudo mount /dev/sdc1 /datadrive

The data disk is now ready to be used as / datadrive.

Add the new drive to / etc / fstab:

To make sure the reader is raised automatically after a reboot, you must add it to / etc / fstab. In addition, it is strongly recommended to use the UUID (GUID) in / etc / fstab as a reference to the reader, rather than the device only (for example, / dev / sdc1) name. To find the UUID of the new drive, you can use the blkid utility:

sudo -i blkid

The result looks like the following:

`/dev/sda1: UUID=”11111111-1b1b-1c1c-1d1d-1e1e1e1e1e1e” TYPE=”ext4″`
`/dev/sdb1: UUID=”22222222-2b2b-2c2c-2d2d-2e2e2e2e2e2e” TYPE=”ext4″`
`/dev/sdc1: UUID=”33333333-3b3b-3c3c-3d3d-3e3e3e3e3e3e” TYPE=”ext4″`

Caution: If you incorrectly modify the / etc / fstab file, it may be that the system can not boot. If in doubt, refer to the documentation of the distribution for information on appropriate changes to this file. It is also strongly recommended to create a backup of the file / etc / fstab before editing.

In a text editor, enter the information for the new file system at the end of the file / etc / fstab. In this example, we use the UUID for the new device / dev / sdc1 created in the previous steps and the mount point / datadrive:

`UUID=33333333-3b3b-3c3c-3d3d-3e3e3e3e3e3e /datadrive ext4 defaults 1 2`

If data drives or additional partitions are created, you must also be added separately to the / etc / fstab.

You can now check if the file system is mounted correctly by removing and then the back, for example, using the example of the mount point / datadrive created in the previous steps:

sudo umount /datadrive sudo mount /datadrive

Installing and Running MongoDB on the virtual machine

Once mounted, create a data directory by:

  1. Log on onto the instance
  2. Then run sudo chown `id -u` /mnt/datadrive/ to make the linux user the owner of the data directory
  3. Run the following command: mkdir -p /mnt/datadrive/data
  1. Log onto the instance

  2. Create an /etc/yum.repos.d/mongodb.repo file to hold information about your repository using your favorite editor as sudo. Place the following configuration in/etc/yum.repos.d/mongodb.repo file:

[mongodb]

name=MongoDB Repository

baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64

gpgcheck=0

enabled=1

After saving the new .repo file, issue the following command to update the local package database:

sudo yum update

Issue the following command (as root or with sudo) to install the latest stable version of MongoDB and the associated tools:

sudo yum install mongo-10gen mongo-10gen-server

When this command completes, you have successfully installed MongoDB.

Configure Mongodb

Instance 1
Connect to the instance using ssh or PuTTY
As sudo, edit /etc/mongod.conf to set the following parameters:

port = 27018
dbpath = /mnt/datadrive/data
logpath = /mnt/datadrive/mongod.log
replSet = mongors

Save the config file

Instance 2
Connect to the instance using ssh or PuTTY, remembering the SSH port for this node has been changed from the default of 22
As sudo, edit /etc/mongod.conf to set the following parameter:

port = 27019
dbpath = /mnt/datadrive/data
logpath = /mnt/datadrive/mongod.log
replSet = mongors

Save the config file

Instance 3
Connect to the instance using ssh or PuTTY, remembering the SSH port for this node has been changed from the default of 22
As sudo, edit /etc/mongod.conf to set the following parameter:

port = 27020
dbpath = /mnt/datadrive/data
logpath = /mnt/datadrive/mongod.log
replSet = mongors

Save the config file

Start MongoDB
Once the configuration files have been edited, start the database process mongod on each instance by:

Log on onto the instance
Run the following command to start the process:
mongod –config /etc/mongod.conf

This should start the mongod process

Verify that mongod start by tailing the log file using the command
tail –f /mnt/datadrive/mongod.log

Configure the Replica Set

At this point in time, you should have mongod running on all 3 of your instances. You can now configure the 3 instances as a replica set by connecting to 1 of the 3 instances from within Azure or from outside.

Connect to the running mongod process using the mongo shell:

If connected to the VM instance type the following command where port is 27018 for instance 1, 27019 for instance 2 and 27020 for instance 3:

mongo –port <port number>

If connecting to the mongod process in Azure from your local machine use the following command:

mongo –host mongodbrs.cloudapp.net –port <port number>

In the mongo shell type the following:

> conf = {
_id : “mongors”,
members : [
{_id:0, host:”mongodbrs.cloudapp.net:27018″},
{_id:1, host:”mongodbrs.cloudapp.net:27019″},
{_id:2, host:”mongodbrs.cloudapp.net:27020″}]}
> rs.initiate(conf)

This will start the initialization of the MongoDB replica set
Type the command rs.status() to check the status of the replica set. Upon successful initialization, you should see 1 of the 3 instances being the “Primary” of the set and the other 2 being the “Secondaries”

 

Troubleshooting

After Installation you may encounter some errors most common errors are –

your mongodb installation may run into pid access issue at mongodb startup. To fix this grant acces to var /var/run/mongodb

Fix – Run the following command

Sudo mkdir -p /var/run/mongodb
sudo chown `id -u` /var/run/mongodb

Another common issue is Connection refused to MongoDB replica sets

Fix –  To fix this error edit the /etc/mongos.conf file and set your bind_ip = 0.0.0.0 in order to make connections externally from other servers.

Better to comment out bind_ip.