Rock IT

Monitoring Redis

Today I'll show how you can monitor Redis activity. It's very useful for low-level debugging.

Few words about Redis

In a single sentence, Redis is an in-memory key-value store. But it's also much more than that - it supports many complex data types and is blazingly fast (can perform milions operations per second).

It's mostly used via language wrappers, like redis-py for Python, but sometimes you want to take a look what exactly is happening underneath. There's a very useful redis command for that, and I'll show you how to use it.

Monitoring redis

Redis-cli is an entrypoint for most of redis tools. One of them is Monitor, that logs all executed commands. It adds some overhead (about 50% to each command), but you have full knowledge what's happening. How to use it? It's quite easy, run following command:

# use optional flags if you are not on a host with redis
redis-cli [-h hostname] [-p port] [-a password] monitor

# command for a redis Docker container
docker exec redis-container-name redis-cli monitor

What will you see? It depends on your redis usage. In my case, I couldn't read anything - messages were appearing way too fast. So, let's dump it into a file:

redis-cli monitor > redis.log
# wait a bit, then press Ctrl + C to stop

Now we're talking! Analyzing file is way easier. But still, in my case, result of a 10s dump has 12MB and more than 100k lines. Let's see how it looks like (redis is used in Django-channels application):

1495374947.551309 [0 lua] "expire" "asgi:9c17319ac1144e5d967e387ca4c9a159" "60"
1495374947.551324 [0 lua] "rpush" "asgi:http.request" "asgi:9c17319ac1144e5d967e387ca4c9a159"
1495374947.551376 [0 lua] "expire" "asgi:http.request" "61"
1495374947.552120 [0 167.114.254.56:58364] "GET" "asgi:9c17319ac1144e5d967e387ca4c9a159"
1495374947.552721 [0 167.114.254.63:39430] "EVALSHA" "3640886a0c8901ca9188f5f7a5f7a346145b9c5f" "14" "asgi:websocket.send!CaVgMqZFcpUk" "asgi:websocket.send!CNmEQsDcYUxm" "asgi:websocket.send!RvlvHsHxqYLa" "asgi:websocket.send!AEtVpVIjfvSf" "asgi:websocket.send!aWIQkBnAdTSJ" "asgi:websocket.send!eCmjCHjgYKtC" "asgi:websocket.send!tJwTIGtDuGbd" "asgi:websocket.send!iTCZRlrsXexr" "asgi:websocket.send!LEjMkRlBHmpf" "asgi:websocket.send!ozosqIsbEYLW" "asgi:websocket.send!frEaeJKHEHQG" "asgi:websocket.send!bhuzagrVicUj" "asgi:websocket.send!OGHHZBCFUIoJ" "asgi:websocket.send!HeDLryxfaqdI"
1495374947.552872 [0 lua] "LPOP" "asgi:websocket.send!CaVgMqZFcpUk"
1495374947.552885 [0 lua] "LPOP" "asgi:websocket.send!CNmEQsDcYUxm"

Searching in a Redis log file

Now we have a big file with many lines. How can we find anything there? There are couple options, like always:

  • Open file in text editor and just search for relevant lines
  • Use commands to show and aggregate only interesting lines

Personally I make some initial filtering of my file, and open result in editor. Here are the most common commands that I'm using:

# Show all ip addresses that are using redis (with count)
cat redis.log | grep -v 'lua' | cut -d' ' -f3 | cut -d':' -f1 | sort | uniq
# sample output:
# 719  192.168.1.111
# 738  192.168.1.114
# 796  192.168.1.154
# 57   192.168.1.17

# Show all commands with specified type 
# and save it to file for later use
cat redis.log | grep 'SET' > redis-set.log

That's all. I've used this technique to find some nasty bugs in my code and profile it. Share if you found this post useful!

Author image
Warsaw, Poland
Full Stack geek. Likes Docker, Python, and JavaScript, always interested in trying new stuff.