Linux DRBD Setup

This tutorial will explain how to create a two node DRBD cluster (block level clustered storage), additional nodes can be added easily.

For the purpose of this tutorial (and because I believe it to be the easiest distribution) I will be using Ubuntu Server 12.04 LTS. This will work on other Linux distributions but this guide is written specifically for Ubuntu. This tutorial will assume you are using the root user.

For the purposes of this tutorial I will use the below IP config:

Server 1 (drbd01) = 192.168.0.111
Server 2 (drbd02) = 192.168.0.112

1) Install drbd and ocfs2 packages from respository using command:
apt-get install drbd8-utils ocfs2-tools

2) Edit hosts file (use command: vim /etc/hosts), add the hostnames & ip addresses of all drbd nodes

3) Create a {resource name}.res file in “/etc/drbd.d/” directory (use command: vim /etc/drbd.d/{resource name}.res) and populate with the below (edit as required). Please note anything between a set of *** are comments E.G. ***blah blah***

resource {resource name} { ***name the resource what you want, use the same as the filename***
  protocol C;
  startup {
    wfc-timeout  15;
    degr-wfc-timeout 60;
    #become-primary-on both; ***allows drbd to go primary/primary on startup, un-comment if you want to use but it is recommended to get drbd setup and working fully first***
  }

  net {
    cram-hmac-alg sha1;
    shared-secret "secret"; ***set a password***
    after-sb-0pri discard-zero-changes;
    after-sb-1pri discard-secondary;
    after-sb-2pri disconnect;
    allow-two-primaries; ***set this if you want to allow a primary/primary setup***
  }

  on drbd01 {
    device /dev/drbd0; ***if using multiple resources change this as required***
    disk /dev/sdb1; ***set the physical disk/partition to use for drbd***
    address 192.168.0.111:7788;
    meta-disk internal;
  }

  on drbd02 {
    device /dev/drbd0;  ***if using multiple resources change this as required***
    disk /dev/sdb1; ***set the physical disk/partition to use for drbd***
    address 192.168.0.112:7788;
    meta-disk internal;
  }
}

4) Copy the {resource name}.res file to other node(s) (use command: scp /etc/drbd.d/{resource name}.res drbd02:/etc/drbd.d/)

5) Run the command: drbdadm create-md {resource name} to initialize the meta data storage. This command should be ran on all nodes.

6) Create a cluster.conf file in “/etc/ocfs2/” directory (use command: vim /etc/ocfs2/cluster.conf) and populate with the below (edit as required). Please note anything between a set of *** are comments E.G. ***blah blah***

cluster:
  node_count = 2 ***set number of nodes***
  name = {cluster name} ***set required cluster name***

node:
  ip_port = 7777
  ip_address = 192.168.0.111
  number = 1
  name = drbd01
  cluster = {cluster name}

node:
  ip_port = 7777
  ip_address = 192.168.0.112
  number = 2
  name = drbd02
  cluster = {cluster name}

7) Start DRBD (use command: service drbd start)

8) Copy the cluster.conf file to other node(s) (use command: scp /etc/ocfs2/cluster.conf drbd02:/etc/ocfs2/)

9) Run the command: dpkg-reconfigure ocfs2-tools and follow the screen prompts to configure OCFS2 and cluster

10) Run the command: drbdadm -- --overwrite-data-of-peer primary {resource name} to start the data sync. This command should be ran from drbd01 only. You can watch progress by running the command: watch drbd-overview. To stop watching the output press Ctrl+C

11) Run the command: mkfs.ocfs2 /dev/drbd0 on drbd01 to create the file system on the DRBD device

12) If required you can now un-comment the “become-primary-on both” line in the resource file to allow primary/primary on startup. Restarting DRBD will put both nodes into primary mode (service drbd restart)

13) You are now free to mount the DRBD device to wherever you want
E.G. mount /dev/drbd0 /srv

14) If you want the DRBD device mounted automatically at boot then run echo '/dev/drbd0 /srv ocfs2 _netdev,defaults 0 0' >> /etc/fstab to populate fstab (remember to change the mount point from “/srv” to whatever you want)

Thanks goes out to the below websites that I used to collect information:

Ubuntu Documentation for DRBD
Linbit DRBD User’s Guide