A Beginner Guide to Redis Clustering

When developing applications sometimes you might need to save data for a short time and do some transactions in very quick time. In this case if you use a database transaction it may time consuming.So here comes redis to make things quick and simple on this case.So in this article let’s learn about redis, redis clustering and the features of it and finally let’s install and setup redis on local machine.

What is Redis?

• An open source in-memory key value store used for data caching
• Extremely fast because data stored in-memory,also note that using redis data can be written to hard disk as well.
• A NoSQL database -No structures like tables, rows,columns. Stores data as key value pairs
• Supports data structures like — Strings,Hashes,Lists,Sets,Sorted sets,Bitmaps and HyperLogLogs

Why Redis?

• Performance — It’s very quick comparing to database transactions,since data stored in cache.
• Simple and flexible — Data stored as key value pairs and read/write simple and straight forward
• Multi language support — supports for number of languages, check here
• Platform support — Linux based OS, no official support for Widows bulids
• Compatibility — Can be used as a second database to make transactions faster
• Scalability — Master slave feature,Different instances of redis as a master and slaves to perform different operations which makes performance faster
• Applications using Redis — Flickr,stackof,github,pintrest,twitter

What is Redis Clustering?

• Redis cluster is a way to share data automatically across multiple redis nodes.
• A redis cluster can continue operations when some nodes fail or not able to communicate.However the cluster stops in the occurrence of large failures

Redis Cluster Topology

• A redis cluster requires minimum of 3 master nodes and 3 slaves(1 slave for each master) to work properly.

Redis Cluster Data Sharding

• Data splited into multiple redis instances, so that every instance will contain a subset the keys
• Redis Cluster does not use consistent hashing, but a different form of sharding where every key is conceptually part of what we call an hash slot.
• There are 16384 hash slots in a redis cluster, to compute the hash slot of a key we can use CRC16 of the key modulo 16384.

slot = CRC16(“key”) % 16384

• We know a cluster contains multiple nodes, so every node contains a subset of the hashslot.

For example if we have 3 nodes,

node A : 0 - 6000

node B : 6001 -10000

node C : 10001 - 16384

if we add another node D then a set of hash slots will be moved from the three nodes to the newly created node.Like that if we remove a node from the 3 nodes, then the hash slots will be served to the available two nodes.Because moving hash slots from a node to another does not require to stop operations, adding and removing nodes, or changing the percentage of hash slots hold by nodes, does not require any downtime.

TCP Connections

• Two TCP connections required to be opened for every node in a redis cluster.

1. The client communication port
2. The cluster bus port

The client communication port should be opened to communicate with clients and the cluster bus port to communicate with other nodes in the cluster.The cluster bus uses a different, binary protocol, for node to node data exchange, which is more suited to exchange information between nodes using little bandwidth and processing time.The Cluster bus is used by nodes for failure detection, configuration update, fail over authorization and so forth.

Inter communication between nodes - PING/PONG

Key Features of Redis Cluster

• Availability - Redis cluster provides some degree of availability during partitions, where it can be able to continue the operations even if some of the nodes are failed.Redis Cluster is able to survive partitions where the majority of the master nodes are reachable and there is at least one reachable slave for every master node that is no longer reachable. Moreover using replicas migration, masters no longer replicated by any slave will receive one from a master which is covered by multiple slaves.
• Scalability - A redis cluster can scale upto 1000 nodes horizontally
• Failure detection - A node has information like- Node Id(unique),IP and Port,set of flags,master port(if the node is slave),last time PINGED and PONGED.

There are two flags used to detect whether the node is failed.

• PFAIL - A non-acknowledged failure type
• FAIL- Acknowledged by majority of the masters

Steps to setup redis cluster in local machine

• First install redis in the local machine

$ wget http://download.redis.io/releases/redis-6.0.3.tar.gz
$ tar xzf redis-6.0.3.tar.gz
$ cd redis-6.0.3
$ make

• Next start the redis server

So in the above we looked how to start a single redis instance in the local machine.Now let’s see how to deploy a redis cluster

• There are two ways

1. By creating empty redis instances in cluster mode

Create a directory and copy the redis-server from the above redis download,also create six sub diretories for each node

add redis configuration file(redis.conf) in every node directories by specyfying the respective port to run the redis instance.

A minimal redis config file

Then in the redis-cli type the following command

redis-cli — cluster create 127.0.0.1:6000 127.0.0.1:6001 \
127.0.0.1:6002 127.0.0.1:6003 127.0.0.1:6004 127.0.0.1:6005 \
— cluster-replicas 1

So now you can start the instances in their respective ports and work.

2.Using the create cluster script

Go to the create-cluster directory in the downloaded redis distribution path and type the following commands

Next you can test the cluster by caching values.

So that’s it, I hope you got a good knowledge about redis clustering and enjoyed reading my article.Cheers!!!!