Redis

REmote DIctionary Server

September 17, 2015

This session covers -

  • What is Redis
  • Background
  • Install Redis
  • Redis Architecture
  • Redis Users and common use cases
  • Redis Clients
  • Redis in Developer Community
  • Benchmarking
  • Data Structures In Redis
  • Replication And Failover
  • Url Shortener using Redis with Spring Boot

What is Redis

  1. Distributed Cache
  2. Distributed In memory Database

NoSQL

Salvatore Sanfilippo

VMWare(Since March 15 2010)

Pivotal

Redis Lab (Since  June 2010)

Written In C

Redis

Current Version 3.0.3

Background

Initial release April 10, 2009

 

Installing Redis

Latest Stable Version: http://download.redis.io/redis-stable.tar.gz 

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make
make test

Unstable : https://github.com/antirez/redis/archive/unstable.tar.gz

For windows: https://github.com/MSOpenTech/redis/releases

  • redis-server redis server itself
  • redis-sentinel Redis Sentinel(monitoring and failover).
  • redis-cli command line interface utility
  • redis-benchmark Redis performances.
  • redis-check-aof and redis-check-dump are useful in the rare event of corrupted data files.

Basic Components

redis-stable/src/

Starting Server

 redis-server --test-memory
./src/redis-server

Higher Level Architecture

Redis: Server

Request-Response By Event Loop

Redis Server :  modules

Redis Data Structure

  • String
  • List
  • Set
  • Sorted Set
  • Hashes
  • Bit arrays

Redis Data Structure : String

redis>  SET mykey "Hello"
OK
redis>  GET mykey
"Hello"
redis>  
> set mykey newval nx
(nil)
> set mykey newval xx
OK
> set counter 100
OK
> incr counter
(integer) 101
> incr counter
(integer) 102
> incrby counter 50
(integer) 152

EX-- Set the specified expire time
PX-- Set the specified expire time
NX-- Only set the key if it does not 
already exist.
XX-- Only set the key if it already exist.


> exists mykey
(integer) 1
> del mykey
(integer) 1
> exists mykey
(integer) 0
+------------+------------------------+-----------+---------------\
| Len | Free | H E L L O W O R L D \n | Null term |  Free space   \
+------------+------------------------+-----------+---------------\

https://github.com/antirez/sds

struct sdshdr {
    long len;
    long free;
    char buf[];
};

Redis Data Structure : List

> rpush mylist A
(integer) 1
> rpush mylist B
(integer) 2
> lpush mylist first
(integer) 3
> lrange mylist 0 -1
1) "first"
2) "A"
3) "B"


> rpush mylist a b c
(integer) 3
> rpop mylist
"c"
> rpop mylist
"b"
> rpop mylist
"a"
> rpush mylist 1 2 3 4 5 "foo bar"
(integer) 9
> lrange mylist 0 -1
1) "first"
2) "A"
3) "B"
4) "1"
5) "2"
6) "3"
7) "4"
8) "5"
9) "foo bar"

> rpush ids 1 2 3 4 5
(integer) 5
> ltrim ids 0 2
OK
> lrange ids 0 -1
1) "1"
2) "2"
3) "3"

Redis Data Structure : Sorted  Set

> zadd hackers 1940 "Alan Kay"
(integer) 1
> zadd hackers 1957 "Sophie Wilson"
(integer 1)
> zadd hackers 1953 "Richard Stallman"
(integer) 1
> zadd hackers 1949 "Anita Borg"
(integer) 1
> zadd hackers 1965 "Yukihiro Matsumoto"
(integer) 1
> zadd hackers 1914 "Hedy Lamarr"
(integer) 1
> zadd hackers 1916 "Claude Shannon"
(integer) 1
> zadd hackers 1969 "Linus Torvalds"
(integer) 1
> zadd hackers 1912 "Alan Turing"
(integer) 1
> zrange hackers 0 -1
1) "Alan Turing"
2) "Hedy Lamarr"
3) "Claude Shannon"
4) "Alan Kay"
5) "Anita Borg"
6) "Richard Stallman"
7) "Sophie Wilson"
8) "Yukihiro Matsumoto"
9) "Linus Torvalds"

> zrevrange hackers 0 -1
1) "Linus Torvalds"
2) "Yukihiro Matsumoto"
3) "Sophie Wilson"
4) "Richard Stallman"
5) "Anita Borg"
6) "Alan Kay"
7) "Claude Shannon"
8) "Hedy Lamarr"
9) "Alan Turing"

Redis Data Structure : Sorted Set

> zrangebyscore hackers -inf 1950
1) "Alan Turing"
2) "Hedy Lamarr"
3) "Claude Shannon"
4) "Alan Kay"
5) "Anita Borg"

zremrangebyscore hackers 1940 1960
(integer) 4

> zrank hackers "Anita Borg"
(integer) 4

Skiplist

Redis Data Structure : Hashes

> hmset user:1000 username antirez birthyear 1977 verified 1
OK
> hget user:1000 username
"antirez"
> hget user:1000 birthyear
"1977"
> hgetall user:1000
1) "username"
2) "antirez"
3) "birthyear"
4) "1977"
5) "verified"
6) "1"

> hincrby user:1000 birthyear 10
(integer) 1987
> hincrby user:1000 birthyear 10
(integer) 1997
127.0.0.1:6379> COMMAND COUNT
(integer) 163

Replication

  • non-blocking on both the master and slave side
  1. Disk-backed
  2. Diskless
#redis-conf
repl-diskless-sync no
#redis-conf
repl-backlog-size 1mb
#redis-conf
repl-backlog-ttl 3600

Text

#redis-conf
slave-serve-stale-data yes

Eviction Policy

  1. noeviction

  2. allkeys-lru

  3. allkeys-random

  4. volatile-ttl

  5. volatile-lru

  6. volatile-random

Single Point Of Failure

  1. Manually reconfigure a slave as master
  2. Update all other slaves to talk to new master

Failover : Redis Sentinel

  1. Failure detection when multiple Sentinels agree
  2. Works even if not all the Sentinel processes are working

Failover : Redis Sentinel

Lies, Damned Lies and Benchmarks

redis-benchmark

 -h <hostname>      Server hostname (default 127.0.0.1)
 -p <port>          Server port (default 6379)
 -s <socket>        Server socket (overrides host and port)
 -a <password>      Password for Redis Auth
 -c <clients>       Number of parallel connections (default 50)
 -n <requests>      Total number of requests (default 100000)
 -d <size>          Data size of SET/GET value in bytes (default 2)
 -q                 Quiet. Just show query/sec values
 --csv              Output in CSV format
 -l                 Loop. Run the tests forever
 -t <tests>         Only run the comma separated list of tests. The test
                    names are the same as the ones produced as output.
 -I                 Idle mode. Just open N idle connections and wait.

Redis Clients

•JRedis

•Jedis

•Spring Data Redis

JAVA

All....

Redis With Spring Boot

Depedency

dependencies {
    compile 'org.springframework.boot:spring-boot-starter-web'
    compile 'org.springframework.boot:spring-boot-starter-actuator'
    compile 'org.springframework.boot:spring-boot-starter-redis:1.1.5.RELEASE'
    compile 'com.google.guava:guava:17.0'
    compile 'org.apache.commons:commons-lang3:3.3.2'
    compile 'commons-validator:commons-validator:1.4.0'
}
shakhawat.hossain@shakhawat:/usr/local/redis-server/redis-stable$ redis-server
31254:C 25 Aug 12:49:52.803 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
31254:M 25 Aug 12:49:52.803 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
31254:M 25 Aug 12:49:52.803 # Redis can't set maximum open files to 10032 because of OS error: Operation not permitted.
31254:M 25 Aug 12:49:52.803 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.0.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 31254
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

Start Server

Redis With Spring Boot

@EnableAutoConfiguration
@Controller
public class UrlShortenerController {

    private static final String DOMAIN_URL = "http://rds.com/";
    private static final String[] VALID_SCHEMES = {"http", "https"};

    @Autowired
    private StringRedisTemplate redis;

    @RequestMapping(value = "/{urlCode}", method = RequestMethod.GET)
    public void redirect(@PathVariable String urlCode,
                         HttpServletResponse resp) throws Exception {
        String url = redis.opsForValue().get(urlCode);

        if (url != null) {
            resp.sendRedirect(url);
        } else {
            resp.sendError(HttpServletResponse.SC_NOT_FOUND);
        }
    }

    @RequestMapping(method = RequestMethod.POST)
    public ResponseEntity<String> save(HttpServletRequest req) {
        String queryParams = req.getQueryString() != null ? "?" + req.getQueryString() : "";
        String url = (req.getRequestURI() + queryParams).substring(1);

        UrlValidator urlValidator = new UrlValidator(VALID_SCHEMES);

        if (urlValidator.isValid(url)) {
            String urlCode = Hashing.murmur3_32().hashString(url, StandardCharsets.UTF_8).toString();
            redis.opsForValue().set(urlCode, url);
            return new ResponseEntity<>(DOMAIN_URL + urlCode, HttpStatus.OK);
        }

        return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
    }
}

Redis Most Common Usage

High Traffic News Portals

Gaming Data : Leader Board

Timeline

Follower Model

Redis Users

Twitter

Instagram

Udemy

Uber

Pinterest

stack-exchange

BitBucket

ask-fm

Trello

stack-overflow

The Guardian

.... and more

Codecademy

http://stackshare.io/redis

Popularity In Developer Community

db-engine ranking

Repo

Popularity In Developer Community

Thank You All

Redis In Action

By MUHAMMAD SHAKHAWAT HOSSAIN SAFAT