Need to store/access your data as fast as possible? Try Redis!
"40% of people will leave your site if it takes longer than 3 seconds to load..."
https://econsultancy.com/blog/10936-site-speed-case-studies-tips-and-tools-for-improving-your-conversion-rate
Never make people wait; Learn how to cache your content/app with Redis.
Storing data is a necessity for all (but the simplest) web applications.
Often accessing that data can be the biggest bottleneck
in an app causing it to be slow and painful experience.
Redis is designed to be the fastest way to store and retrieve data.
Which means less waiting for the people using your app!
Redis is the closest you can get to direct access (to your computer's) memory.
Written in C
the entire server is less than a Megabyte
(smaller is better when it comes to executable size),
Redis is built for speed.
### Use Case
- sessions in your app (which require a DB/io read on every request but don't contain content) should be checked/set as quickly as possible.
- list online people in a chat application the list of people/devices that are connected changes frequently (as people join/leave). Wouldn't it be useful to be able to subscribe to this list and be notified when it changes?
- content cache given that Redis can store any size of string, it can store any content you need to serve in your app; from simple components to entire pages.
First thing you need to do - if you haven't already - is go through the online tutorial: http://try.redis.io/ (30mins to learn and take notes)
#### Make Sure you Understand
You can learn/practice all these commands online before installing Redis:
- SET - SET a the value for a given key.
- GET - GET the value of a key.
- INCR increment a counter (integer)
- DECR decrement a counter (decremented key needs to already exist)
redis> SET count 10
redis> GET count
"10"
redis> INCR count
(integer) 11
redis> DECR count
(integer) 10Easy, Right? lets keep moving.
If you don't already have Redis installed on your machine - why would you if you're reading an introductory tutorial, right? - there are two ways to install Redis:
The guys who make Redis suggest compiling it from source (this requires a working GCC compiler and libc) see: Quick Start: http://redis.io/topics/quickstart
If you are running Linux/Mac or prefer not to use Vagrant, try the download & compile described in the quickstart.
For people stuck on Windows (if you don't already have a C Compiler) and people who prefer to "contain" their , we recommend using Vagrant to run Redis (and your app) in a (Linux) Virtual Machine (VM). We have included a Vagrantfile in this repo which you can use to get Redis & Node.js working on any machine. simply:
git clone https://github.com/nelsonic/learn-redis.git && cd learn-redis
vagrant up
vagrant sshThese three commands will
- Clone the repo and move into the new directory
- Download and install the Ubuntu VM and latest Redis and Node.js (and start Redis server on TCP Port 6379)
- SSH into the Vagrant (Ubuntu) VM so you can use the vagrant CLI
To boot the VM with Redis open a new terminal window/tab (in the learn-redis directory) and issue the command: vagrant up
After a few minutes of downloading & installing you will see that Redis is running:
on Port 6379 and PID (process ID) 5786
To use Redis from the command line (open a new terminal window/tab and) log into the VM using vagrant ssh:
Then confirm that the CLI is working by issuing the command redis-cli ping: http://stackoverflow.com/questions/10137857/is-redis-just-a-cache
To exit the VM you just logged into, simply type exit in the terminal and hit the [return] key.
To close the Redis server runing on the VM, press the [ctrl] + [C] keys on your keyboard (twice).
And to shut down the Vagrant VM (to free up the memory on your computer),
issue the vagrant halt command in your terminal:
Note: your VM is preserved (still saved on your hard drive) so you can use it again for further redis-development simply by issuing the vagrant up command (from within a working directory that has the Vagrantfile and .vagrant)
If you are short on disk space, you can always delete the VM completely by issuing the vagrant destroy command.
Note: If you are running Linux on your laptop/dev machine
and prefer not to have the overhead of running Vagrant
you can simply copy the installation commands from the Vagrantfile
and paste them into your terminal.
A search for "redis" on NPM: https://www.npmjs.com/search?q=redis returns many results!
Don't be overwhelmed by the quantity of modules, focus on quality.
- redis https://www.npmjs.com/package/redis written in "Pure JavaScript"
this is by far the most popular node module for Redis,
and... if you install it with:
npm install redis hiredisthen it will use the C binding for speed. - hiredis-node https://github.com/redis/hiredis-node is a JS wrapper
around the hiredis C Library.
This means its much faster than "Pure JavaScript" code. Its faster, so what's the catch?
If you (or anyone on your team) using Microsoft Windows (stop hitting yourself!) will need to install a compiler
to get Redis working and that can sometimes take hours!
Our recommendation is to use the redis module because the documentation is better, but install hiredis so that it uses the C binding.
npm install redis hiredis --saveSee: examples/basic.js
Paste hand-type (or copy-paste) this code into a file called basic.js
var redis = require("redis");
var client = redis.createClient();
client.set("Hello", "World", redis.print);
client.get("Hello", function(err, reply) {
// reply is null when the key is missing
console.log('Hello ' + reply);
});Run the script in your terminal by issuing the command: node basic.js
You should expect to see:
$ node basic.js
Reply: OK
Hello WorldRedis has many fantastic features not limited to just SET/GET/DELETE key:value pairs. You can also atomically
- append lets us add data to record without having to read it first. ()
redis> EXISTS mykey
(integer) 0
redis> APPEND mykey "Hello"
(integer) 5
redis> APPEND mykey " World"
(integer) 11
redis> GET mykey
"Hello World"- incrment a value (e.g a counter) no delay having to read the current value before you can increase it by one.
redis> SET mykey "10"
OK
redis> INCR mykey
(integer) 11
redis> GET mykey
"11"If you don't get overwhelmed by lots of new words and want a sneak peek
at the all the cool commands you can use in Redis check: http://redis.io/commands
But we're getting ahead of ourselves, lets focus on the basics first.
- EXPIRE means you can delete data after a specified amount of time. e.g: expire a key after 10 seconds
redis> SET mykey "Hello"
OK
redis> EXPIRE mykey 10
(integer) 1
redis> TTL mykey
(integer) 10
# wait for 10 seconds then
redis> TTL mykey
(integer) -2-
TTL (time to live) lets you check how long it will be before a key (and its associated value) will expire from the cache. (see expire example above)
-
Lists - A list is a series of ordered values (comparable to an Array in JS)
RPUSH fruits "Apple"
RPUSH fruits "Banana"
LPUSH fruits "Mango"
LRANGE fruits 0 -1 => 1) "Mango", 2) "Apple", 3) "Banana"
LRANGE fruits 0 1 => 1) "Mango", 2) "Apple"
LRANGE fruits 1 2 => 1) "Apple", 2) "Banana"
LLEN fruits => 3-
Sets - Redis Sets are an unordered collection of Strings. It is possible to add, remove, and test for existence of members in O(1) (constant time regardless of the number of elements contained inside the Set). SISMEMBER tests if the given value is in the set. It returns 1 if the value is there and 0 if it is not.
-
Data Types: http://redis.io/topics/data-types
-
Transactions allow the execution of a group of commands in a single step, with two important guarantees:
- All the commands in a transaction are serialized and executed sequentially. It can never happen that a request issued by another client is served in the middle of the execution of a Redis transaction. This guarantees that the commands are executed as a single isolated operation.
- Either all of the commands or none are processed, so a Redis transaction is also atomic.
- PubSub lets you "listen" for changes in the value of a key. Useful if you ant to know when something has been changed by someone. e.g: you have two devices (Desktop & Mobile) logged into an app, something gets updated on one of them, how do we reflect this change on the other? Answer: all devices "subscribe" to the change event and receive the latest values as a result.
- What is Redis and what do I use it for? (good list of usecases)
http://stackoverflow.com/questions/7888880/what-is-redis-and-what-do-i-use-it-for - Redis Wikipedia: http://en.wikipedia.org/wiki/Redis (history and background)
- Good comparison of NoSQL datastores: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
- Why Redis beats Memcached for caching:
http://www.infoworld.com/article/2825890/application-development/why-redis-beats-memcached-for-caching.html - Why Redis is Awesome: http://www.robupcraft.com/why-redis-is-awesome
- Memcached vs. Redis? (good list feature/comparison) http://stackoverflow.com/questions/10558465/memcached-vs-redis
- Is Redis "just" a cache? http://stackoverflow.com/questions/10137857/is-redis-just-a-cache
- What are the differences between MongoDB and Redis? (v. basic) http://stackoverflow.com/questions/6445899/what-are-the-differences-between-mongodb-and-redis
- How to Pipeline in node.js to Redis: http://stackoverflow.com/questions/21416529/how-to-pipeline-in-node-js-to-redis
- List of Redis Features: https://redis-docs.readthedocs.org/en/latest/Features.html
- 11 Common Web Use Cases Solved In Redis:
http://highscalability.com/blog/2011/7/6/11-common-web-use-cases-solved-in-redis.html - How to Install and use Redis on Ubuntu https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis
There are a few options for hosted Redis on Heroku: https://addons.heroku.com/?q=redis
- https://addons.heroku.com/rediscloud free plan is 25mb
- https://addons.heroku.com/redistogo free plan is 5mb (paid plans are quite expensive per MB, but they have great service - hence being the #1 provider...)
- https://addons.heroku.com/redisgreen no free plan. #fail (also quite expensive per MB and no distinguishing features)
- https://addons.heroku.com/openredis no free plan. #fail (commitment to use the latest version of redis is good. notifications when reaching 90% of memory.
Conclusion: rediscloud!
- Redis Tips (Mozilla): https://developer.mozilla.org/en-US/docs/Mozilla/Redis_Tips
- Using Redis as an LRU cache: http://redis.io/topics/lru-cache
- Travis-CI gives you access to Redis: http://docs.travis-ci.com/user/database-setup/#Redis (see: .travis.yml this repo for an example)
- How fast is Redis? http://redis.io/topics/benchmarks
- Storing hundreds of millions of simple key-value pairs in Redis INSTAGRAM ENGINEERING Blog:
http://instagram-engineering.tumblr.com/post/12202313862/storing-hundreds-of-millions-of-simple-key-value - Redis Labs Achieves Over 1.2 Million Transactions per Second on a Single Amazon Web Services (AWS) Instance:
http://www.marketwired.com/press-release/redis-labs-achieves-over-12-million-transactions-per-second-on-single-amazon-web-services-1965269.htm
Because Redis is an in-memory datastore, persistence is not immediate. if you want to be sure that your data is saved, (incase your server/instance is re-booted) read about using the SAVE command.
- Redis persistence demystified: http://oldblog.antirez.com/post/redis-persistence-demystified.html
- Redis Commander https://github.com/joeferner/redis-commander is a node module (express app) you can install and run, then open a web browser to View/Set your Data
- Redis Desktop Manager https://github.com/uglide/RedisDesktopManager (an installed desktop app)
- List of others: https://redislabs.com/blog/so-youre-looking-for-the-redis-gui
There are many KV Stores to chose from: http://en.wikipedia.org/wiki/NoSQL#Key-value_stores our favourites are:
- Riak - great for fault tolerance and data replication
- LevelDB - Bundled with node.js so always available, but limited on heroku.
- RethinkDB - Probably the coolest real-time database on the planet! 😎
- and of course Redis - fast, free and fun!
We're here to focus on Redis because it has the right balance of simplicity (great for beginners and pros alike) and speed.
### Q: What does Redis mean/stand for?
The name Redis is an acronym for: REmote DIctionary Server
A Dictionary (see: http://xlinux.nist.gov/dads/HTML/dictionary.html) is another
way of saying "Key-Value Store";
In other programming languages a dictionary is known as a "map"
or "associative array"
and would look like this:
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"children": [],
"spouse": null
}The observant reader will notice that this looks exactly like a JSON Object; that's because it is! This is how we represent a set of key:value pairs in JS.
It can be easy to think of the real-world
dictionary, words are the keys and their definitions are the values.
Imagine you are the person writing the definitions in a dictionary,
you would specify the key (e.g: 'everything') for your item in the dictionary
and "SET" the value (e.g: 'awesome').
Then, when you read (or "GET") record for the 'everything' key
the value would be 'awesome'.
// basic key:value lookup in JavaScript. Try it: http://repl.it/jYk
var dictionary = {
'everything':'awesome'
}
for (var k in dictionary) {
console.log(k + ' is ' + dictionary[k]); // everything is awesome
} // whaaaat? >> https://youtu.be/StTqXEQ2l-YOr, sticking with the dictionary metaphor, here's a better example:
// basic key:value dictionary lookup in JavaScript. Try it: http://repl.it/jYk/4
var dictionary = {
'bycycle':'A bicycle, often called a bike or cycle, is a human-powered, \
pedal-driven, single-track vehicle, having two wheels attached to a frame, \
one behind the other.'
}
for (var k in dictionary) {
console.log(k +': ' +dictionary[k] );
}output:
"We are already using MongoDB to store our records, can't we just put everything in one place?"
MongoDB is a good tool. We like to think of it as
the Hammer of Databases
... a general purpose Datastore that works well for most NoSQL situations.
But if you need to save/access Millions of Records Per Second (yes, you read that right!)
then there's only one place to store your data; Redis.
Yes! Do it! Discover the power of data structures and the speed of an in-memory datastore!