Redis Sharding

Partitioning is the process of splitting data into multiple Redis instances, so each instance only saves a subset of keys.

Advantages of partitioning

• By utilizing the sum of the memory of multiple computers, we can construct larger databases.

• By using multiple cores and computers, we can expand computational power; by using multiple computers and network adapters, we can expand network bandwidth.

Shortcomings of partitioning

Some features of Redis are not very good in terms of partitioning:

• Operations involving multiple keys are usually not supported. For example, when two sets are mapped to different Redis instances, you cannot perform intersection operations on these two sets.

• Redis transactions involving multiple keys cannot be used.

• When using partitioning, data processing is more complex, such as when you need to handle multiple rdb files./aof file, and from multiple examples and host backup persistence files.

• Adding or deleting capacity is also complex. Most Redis clusters support the ability to add or remove nodes transparently during runtime, but other systems like client partitioning or proxy do not support this feature. However, a technology called presharding is helpful for this.

Partitioning Types

Redis has two types of partitioning. Suppose there are4Redis instances R0, R1, R2, R3, and similar user:1, user:2This representation of multiple keys of users, there are various ways to choose which instance this key is stored in for a given key. That is, there are different systems to map a key to a Redis service.

Range Partitioning

The simplest partitioning method is range partitioning, which is to map a certain range of objects to a specific Redis instance.

For example, ID from 0 to10000 users will be saved to the example R0, ID from10001to 20000 users will be saved to R1and so on.

This method is feasible and used in practice, but the drawback is that there needs to be a mapping table from a range of examples. This table needs to be managed, and it also requires mapping tables for various objects, which is usually not a good method for Redis.

Hash Partitioning

Another method of partitioning is hash partitioning. This is applicable to any key and does not need to be in the form of object_name: as described below:

• Use a hash function to convert the key to a number, such as using crc32 hash function. Execute crc on the key 'foobar' to32(foobar) will output something like93024922integer.

• Take the modulus of this integer, converting it to 0-3numbers, you can map this integer to4of the Redis examples.93024922 % 4 = 2That is, the key 'foobar' should be stored in R2In the examples. Note: The modulus operation is the remainder of the division, usually implemented using the % operator in various programming languages.