Tom Cannaerts

Setting up a replicated filesystem using GlusterFS

When scaling out an application, chances are that you will need to have some sort of distributed and/or replicated filesystem. If a dedicated storage solution is not within the options, you can easely set-up a posix compliant filesystem cluster with GlusterFS. Compare GlusterFS with a RAID setup that spans across hosts. You can do things comparable to JBOD (sharding), mirroring (RAID 1), striping (RAID 0) or combinations across multiple hosts. And the best part, it isn’t even hard to do.

In this post, we’ll set-up a 2 host replicated filesystem on Debian. This means that each host will have a copy of all the files. If one host fails, you’ll still have all of your files. If you already have you’re Debian systems up and running, you can have GlusterFS running in a matter of minutes.

The first thing to do, is make sure DNS resolution is working properly on both hosts. Make sure you can resolve both hostname for both of the servers (eg. use ping). If you can’t properly resolve the names, try adding them to the /etc/hosts file.

Next, it’s time to install the GlusterFS server package. This needs to be installed on each server that will host the files. In our case, both servers.

# apt-get update
# apt-get install glusterfs-server

Once installed, you can join the 2 hosts together. This is done by doing a probe from one server to the other. You only need to do this on one server.

# gluster peer probe host2
peer probe: success

Now that you have the two hosts joined together, it is time to create a volume. To do this, you will need to specify the storage location on each of the hosts, as well as the number of hosts. You can add additional hosts to your cluster as your needs change.

# gluster volume create volume1 replica 2 transport tcp host1:/var/gluster/volume1 host2:/var/gluster/volume1

In the above line, volume1 is the name of your volume, and /var/gluster/volume1 is the path on the server where the GlusterFS filesystem will reside. Note that this will contain the GlusterFS structure, not the POSIX filesystem. This needs to be mounted first before you can use it. GlusterFS can be mounted using the GlusterFS native client, or through the NFSv3 built-in server. Performance wise, the native client is the best choice if you need to access the filesystem on the same host.

# apt-get install glusterfs-client
# mount -t glusterfs host1:/volume1 /mnt/volume1

At this point, you can start creating files and folders under /mnt/volume1. Any changes you make one either host will be replicated to the other host.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.