This article will introduce you to Redis. Redis is an open source data structure store that you can use as a database, cache, and message broker. It is great to use when tying microservices together, and in data streams.

Key Redis characteristics:

• Fast
• NoSQL database
• Works with many environments

(This tutorial is part of our Redis Guide. Use the right-hand menu to navigate.)

What is Redis?

Redis is an in-memory database. That means all its storage is held in memory—not on a hard drive. Memory enables quick access, and quick searches, but is limited by space.

In the database architecture, Redis is usually positioned between the client and the database to relieve the load off your NoSQL database or application.

Redis is open-source software that is released under a BSD 3-cause license, a specific type of permissive free software licenses. Redis began when developer Salvatore Sanfilippo needed to improve scalability on his website. He soon open-sourced the platform. Today a core project team develops and maintains Redis, which has been sponsored by Redis Labs since 2015.

Redis use cases

Top use cases for Redis are:

• Queues
• Publish/subscribe (Pub/sub)
• Real-time analytics
• Machine learning
• Geospatial processing
• Leaderboards/counting
• Session cache
• Full page cache

Redis is desirable because its speed and ease complement the increasing use of data streams, signaled from IoT devices, and the business desire for real-time analytics. Its speed and NoSQL structure make it a good choice for both real-time processing and increasing the amounts of data and types that need to pass through it.

Redis can:

• Decrease the load from the database or application
• Decrease data access latency when caching
• Quickly process a high volume, variety, and velocity of data (the 3 V’s) for machine learning
• Pair easily with streaming solutions like Amazon Kinesis, Apache Kafka, and Google Storage

Redis works with many languages

Redis will work with many different languages, making it easily adoptable by whatever the developer is familiar with using or the language the company has adopted.

• Julia
• Ruby
• Rust
• Python
• R
• Java
• C
• Go
• Perl
• And so many more……

Who uses Redis?

Many companies have adopted Redis, including these big, global organizations. This list offers a look at many more companies who use Redis.

Common Redis commands

Redis has commands available to work with its data types: Strings, hashes, lists, sets, sorted sets, bitmaps, and hyperloglogs.

These are sort of standard commands available in most languages that deal with data structures.

Strings

Lists

Sets

Explore the rest of our Redis Guide for more detail, or check out this cheat sheet at GitHub.

Installing Redis

Ready to get started with Redis? We show you how to install Redis in our next article. Or, you can follow these links to a web page where you can look at the different download options:

• Official Redis install documentation
• Redis for Windows

Related reading

• BMC Machine Learning & Big Data Blog
• MongoDB Guide, a 30-part article and tutorial series
• Docker Commands: A Cheat Sheet
• CAP Theorem for Databases: Consistency, Availability & Partition Tolerance
• ACID: Atomic, Consistent, Isolated & Durable

Redis has these data types:

• Binary-safe strings
• Lists
• Sets
• Sorted sets
• Hashes
• Bit arrays
• HyperLogLogs

Here, we briefly explain each of these.

(This tutorial is part of our Redis Guide. Use the right-hand menu to navigate.)

Redis keys

The first element of any Redis data structure is a key. One notable difference with Redis than other systems is that they you can set keys to expire after a certain amount of time.

The keys are binary safe, meaning you can use any kind of pattern of characters, even those that cannot be rendered on the screen, like a JPEG image.

The Redis style guide recommends to use colons (:) to give some meaning to keys like sports:balls:tennisball.

Strings

Strings are just strings. Strings are numbers too, like floats, meaning there is no separate float or integer data type.

set mystrng strivalue
OK
localhost:7001> get mystrng
"strivalue"

Sets

A set in any programming language is a list that does not allow duplicate values.

Notice below when we try to add the number 3 twice to the same set (use sadd) it ignores the second request.

localhost:7001> sadd integers 1
(integer) 1localhost:7001> sadd integers 2
(integer) 1localhost:7001> sadd integers 3
(integer) 1localhost:7001> sadd integers 3
(integer) 0

And you can make the intersection, unions, and difference of sets.

Sorted Sets

Redis also has sorted sets. You can control the order in which items are sorted by add a score when you add a value to a set. If you give them all the same score then the order is alphabetical order (to be specific lexicographical order).

Below we change the order of the alphabet by putting c between a and b.

localhost:7001> zadd alphabet 1 a
(integer) 1
localhost:7001> zadd alphabet 2 b
(integer) 1
localhost:7001> zadd alphabet 1.5 c
(integer) 1

Now list them.

zrange alphabet 0 -1
1) "a"
2) "c"
3) "b"

Lists

Lists in Redis are different than other languages, like Python. They are more like C or C++ linked lists. This does not mean much for the programmer in terms of using them except with regards to performance. In a linked list, each element stores the address of the next element. To insert a value between two elements means to change what both of those point to. That operation is more efficient that shifting the whole list in memory.

You use left push (lpush) and right push (rpush) to add items to the end or beginning of a list. To initialize the list you can use lpush to let Redis know you are adding more that one item at a time. Otherwise it would think it was a string:

lpush mylist 1 2 3
(integer) 3localhost:7001> rpush mylist 4
(integer) 4

Use right pop (rpop) to pop (i.e. use and remove) the right-most element:

rpop mylist
"4"

Use the range command to list part of a list. We popped 4 off the end above so the relative address -1 (i.e., the end) is the last element in the list. 0 is the first element. So to list all elements:

lrange mylist 0 -1
1) "3"
2) "2"
3) "1"

You can use ltrim to change the range of a list, meaning move the pointer to a new offset and ignore the rest of the list. And use linsert to add values at a particular offset (i.e., position) in the list.

Lists are thread safe, meaning one process or programming operating on a list will make another process wait until it finishes. You can tell Redis to timeout when that situation occurs using the block list operations: brpop and blpop.

Hashes

A hash is a key value pair. It is called hash because the key is hashed for faster retrieval.

Write the set in multiple steps:

localhost:7001> hmset myset mykey myvalue
OKlocalhost:7001> hmset myset my2ndkey my2ndvalue
OK

Retrieve all the keys and values.

hgetall myset1) "mykey"
2) "myvalue"
3) "my2ndkey"
4) "my2ndvalue"

As with strings you can increment them when they are numbers:

hmset counter val 1
OKlocalhost:7001> hget counter val
"1"localhost:7001> hincrby counter val 1
(integer) 2localhost:7001> hget counter val
"2"

Bit Maps

A bitmap is not very useful unless you want to use the bit data structures to store some data as efficiently as possible.

To illustrate how bitmaps work in Redis we can change the number 6 to 7 using bitmap operation.

The number 6 is 00110110 in binary. We can change this to 7 by changing the last bit (offset 7) to 1.

set a 6get a
"6"setbit a 7 1get a
"7"

HyperLogLogs

This is a data structure used to count items without having to hold all of them in memory. Normally in order to could the number of elements in the set {1,2,3,4,5} you would have to keep all those in memory. The HyperLogLogs just makes that operation super efficient by taking a different approach.

This tutorial introduces some basic commands for Redis.

(This tutorial is part of our Redis Guide. Use the right-hand menu to navigate.)

Redis architecture

Redis is a distributed in-memory database. It stores data in key, value pairs. There are no tables, schema, or collections.

Redis processes data in memory but stores it on disk. Processing it in memory means it is very fast, because there are no mechanical moving parts as there is no disk I/O.

Of course in memory processes that spill over to other servers in the cluster would be limited by the speed of the ethernet connection between the servers (which is usually 1 GBPS). But that is true with any distributed architecture.

Redis makes the analogy between their distributed memory system and memcached, which its authors say “Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.” So, memcached is temporary storage and Redis is permanent.

Installing Redis

You can try Redis online from here or install it.

Redis does not promote installing their server from packages. Instead make sure you
have installed build-essential, so that you can compile code, then download from source then issue this command to compile the code:

make

if you have any problems look here.

Now, start it:

src/redis-server

open the command line interface (cli):

src/redis-cli

Add key->Value pairs

Add a key value pair then retrieve it. Here the key is walker and the value is walker.

set walker "walker"
OK
127.0.0.1:6379> get walker
"walker"

Redis will let you add to numeric values using the incr function. The function is atomic meaning it will work even when two people are adding to the same key at the same time.

set age 10
OK
127.0.0.1:6379> incr age
(integer) 11

Lists

Lists are a data structure that contains lists of items. The items are stored in the order you add them.

rpush students "walker" "stephen" "ellen"
(integer) 3

You can remove the right-most element by popping it from the list:

rpop students
"ellen"

Now look at the list. First we need to get the length so that we can list then as the lrange function requires that you tell it where to stop. List commands start with l while set commands start with s. The first element is 0.

llen students
(integer) 2lrange students 0 1
1) "walker"
2) "stephen"

You can also list elements starting from the end of the list using negative numbers. This lists the last element:

lrange students -1 1
1) "stephen"

Sets

Sets are lists that do not allow duplicate values. Use SADD to add elements.

See below. Here we show you cannot add the same element twice to a set as the second command returns 0 elements added:

sadd numbers 1
(integer) 1
127.0.0.1:6379> sadd numbers 1
(integer) 0

Here is the union of two sets. We use odd and even numbers as the set names.

sadd odd 1sadd odd 3sadd odd 5sadd even 2sadd even 4sadd even 6sunion odd even
1) "1"
2) "2"
3) "3"
4) "4"
5) "5"
6) "6"

And the intersection should be empty since no number can be both even and odd:

sinter odd even
(empty list or set)

Redis also supports sorted sets and many other data structures.

Scripting

Redis scripting uses the Lua programming language. Lua’s authors say:

“Lua is a powerful and fast programming language that is easy to learn and use and to embed into your application.”

In the next blog post we will explore how to use Lua with Redis. But if you don’t want to use Lua you can use any of the many Redis clients listed here.

This tutorial introduces you to clustering and partitioning in Redis.

(This tutorial is part of our Redis Guide. Use the right-hand menu to navigate.)

Clustering & partitioning in Redis

One way to boost the performance of Redis is to put all records with the same keys into the same node. In that case only one node needs to be read when looking for values with that key. Here we explain the principles behind that.

Suppose you want to separate customers, employees, and vendors into separate nodes. The way to do that would be to:

• set up a Redis cluster.
• assign cluster slots to specific nodes in the cluster.

Then write data for each customer like this, where the hset function sets the key using the hash slot function function:

hset {customer}.dodgedealer car "dodge" year 2016
hset {customer}.forddealer car "ford" year 2016

In this example, both car dealers Dodge and Ford will be assigned to the same hash slot, and thus the same node, since the curly braces {} determines what part of the key is used to calculate the hash slot.

Redis assigns keys to nodes by calculating the hash slot like this:

CRC16(key) mod 16384

CRC16 is a hashing function (i.e., converts a string to a number) and mod 16384 means modulus 16384. Modulus is the integer remainder when dividing an integer by the modulus. For example, 4 mod 2 = 0 because 2 divides 4 evenly. 3 mod 2 = 1 since 3 = (2 * 1 ) + 1.

CRC16(key) mod n will always produce the same result. So it yields a predictable value suitable for this assignment. To see that open a Redis shell and type what is shown below.

The first command adds the hash slot to the node to which you have attached:

CLUSTER ADDSLOTS 10528
OK

Then we show on the screen what the hash slot value of the key walker would be:

CLUSTER KEYSLOT walker
(integer) 10528

Now assign a key and write it to this node:

hset walker "walker"
(error) CLUSTERDOWN The cluster is down

We get an error CLUSTERDOWN because I have not yet set up the cluster, only one node in the cluster.

Below we show how to set up nodes in a cluster. If you want to automate the whole process install Ruby and then run this Ruby program following these instructions.

Create Redis Instance

Install Redis following these instructions and then:

Make 3 directories for 3 instances:

mkdir 7001 7002 7003

Into each of these folders copy this redis.conf, changing the port number as you do.

port 7001
cluster-enabled yes
cluster-config-file 7001.nodes.conf
cluster-node-timeout 5000
appendonly yes

Now start each instance:

nohup src/redis-server 7001/redis.conf >7001.log 2>&1&
nohup src/redis-server 7002/redis.conf >7002.log 2>&1&
nohup src/redis-server 7003/redis.conf >7003.log 2>&1&

Check the logs to see each started:

tail 7001.log
19908:M 18 Jan 01:10:27.927 * Ready to accept connections

Now pick one node and connect to it. Notice that since we have not run the Ruby program the nodes are not aware of each other. So this is not a complete cluster install. But it is enough to show you how to set up key partitioning. The Ruby program updates these node files to put in master/slave/replica/nodes and other info, which makes the cluster complete.

redis-cli -h localhost -p 7001

responds:

localhost:7001>

Ask the node what other nodes there are and it shows the configuration only for itself.

cluster nodes
>eeba92cc34ef032da1f5dbcf7450b93aca9bf62f :7001@17001 myself,master - 0 0 0 connected
localhost:7001>

See below we have not assigned any hash_slot to this node yet:

cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0

Alternate techniques

There are other ways to do this including using the Redis proxy written by Twitter.

Twitter says theses companies are using it:

• Twitter
• Wikimedia
• Pinterest
• Snapchat
• Flickr
• Yahoo!
• Tumblr