redis: The Sicilian Dictionary

The Fastest Fork in Open Source History

This is episode nine of What Did I Just Install.

On March twentieth, twenty twenty-four, a company called Redis Inc. posted a blog announcement that changed the license of one of the most widely deployed pieces of software on earth. Redis, the in-memory data store running inside virtually every major web application you have ever used, was no longer open source. Not the modules. Not the add-ons. The core itself. The thing that Twitter built its timeline on, that GitHub built its job queue on, that Instagram ran its entire early infrastructure on. All of it, behind a new license that the Open Source Initiative refused to recognize.

Eight days later, on March twenty-eighth, the Linux Foundation announced a fork called Valkey, backed by Amazon, Google, Oracle, Ericsson, and Tencent. The woman leading it was Madelyn Olson, a former Redis maintainer who had personally added TLS encryption to the project. She knew the codebase as well as almost anyone alive. And she was furious.

I have gotten together with various former Redis contributors and we have started working on a fork. We are all unhappy with the license change and are looking to build a new truly open community to fill the void left by Redis.

Eight days. That is how long it took for the most widely used in-memory data store in the world to split in two.

But to understand how we got here, you need to go back twenty years, to a small town in southern Sicily, and a young man who wanted to be an artist.

A Modem, by Accident

Salvatore Sanfilippo was born on March seventh, nineteen seventy-seven, in Campobello di Licata, a town of maybe ten thousand people near the southern coast of Sicily. His father was an electrician at a petrochemical company who worked with programmable logic controllers, the kind of industrial hardware that bridges the physical and digital worlds. When the family acquired a TI ninety-nine slash four A home computer, young Salvatore was copying BASIC programs by first grade.

He dropped programming around fourteen, the way teenagers drop everything that is not music or friends or whatever absorbs them that year. Picked it back up at eighteen through three-D modeling and game hacking. At seventeen he moved to Palermo to study architecture at the university. He never finished the degree. A modem he bought by accident in the mid-nineties, misunderstanding what it was, pulled him into the world of Linux and network security instead. It was one of those wrong turns that defines an entire life.

By April nineteen ninety-eight, at twenty-one, Sanfilippo had found a vulnerability in a common ping implementation and posted his findings to the Bugtraq mailing list, one of the most important security disclosure forums of the era. This caught the attention of SECLAB, a cybersecurity firm in Milan. The owner's response was unusual. He told Salvatore to go home, read ten specific books about networking and security, and come back when he was done. Sanfilippo spent that summer on the beaches of Sicily, reading about TCP/IP and packet analysis while the rest of the island swam.

He returned to Milan in September, worked at SECLAB for six months, and built two things that would make his name in the security world. The first was hping, a command-line tool for crafting and sending custom TCP/IP packets, still used today for firewall testing and network analysis. The second was a technique called idle scan, a way to probe ports on a remote machine without revealing your own address. It was implemented in Nmap and remains a standard topic in cybersecurity textbooks. Sanfilippo was twenty-one years old. He left SECLAB because Milan was too much city for a twenty-one-year-old Sicilian. Late nights led to late mornings, late mornings led to problems at work, and he went home.

What came next was the dot-com era. He worked remotely for Linuxcare Italia until the bubble burst. Then in two thousand five, he and his friend Fabio Pitrola co-founded a company called Merzia during the Web two point zero wave. They built a social news site and a bookmarking site, both of which were acquired by Telecom Italia. But the project that mattered was a real-time web analytics tool they called LLOOGG.

The MySQL Wall

LLOOGG did something that sounds ordinary now but was genuinely novel in two thousand seven. It showed website visitors in real time, as they arrived, what pages they clicked, where they came from, how long they stayed. This was four years before Google Analytics added its real-time dashboard. For bloggers and small site owners, it was like suddenly being able to see through the walls of your house.

The problem was MySQL. LLOOGG needed to write constantly, every page view from every tracked site hitting the database as it happened, and MySQL's disk-based storage engine could not keep up with the write throughput. Sanfilippo tried every optimization he knew. It was not enough.

He wrote a prototype in about three hundred lines of Tcl, a scripting language he was fond of. He called it LMDB, for LLOOGG Memory Database. It could do SET, GET, LPUSH, RPUSH. It held everything in memory instead of writing to disk. It was fast. And it planted an idea that would grow into something much larger.

The intellectual spark came from another project Sanfilippo was working on at the time, an event loop extension for the Tcl language called Jim Tcl. The combination of building a scripting interpreter and working with databases all day triggered a conceptual leap.

Writing Jim Tcl and having to work for long hours with database systems is what really pushed me into writing a database that looked like an interpreter for commands.

That is the sentence that explains Redis. Not a query processor. Not a SQL engine. A command interpreter for data structures. You do not ask Redis questions. You give it orders. SET this. GET that. LPUSH onto this list. ZADD to this sorted set. The interface is a vocabulary of verbs, not a query language. This made it astonishingly simple to learn and astonishingly fast to execute.

Two Thousand Lines of C

On February tenth, two thousand nine, Sanfilippo released Redis version zero point one. It was approximately two thousand lines of C code. By February twenty-fifth, version zero point two had grown to thirty-seven hundred lines. The remarkable thing about that first release was its TODO list. It already outlined fork-based persistence, lists, sets, sorted sets, and key expiration. Every major data structure that would make Redis famous was in the blueprint from day one. This was not a cache that evolved features by accident. It was a data structure server designed as one from the beginning.

By June, Redis was stable enough to replace MySQL in LLOOGG entirely. The analytics service went on to process two billion page views running on a hundred-and-fifty-dollar-a-month virtual machine, handling three hundred and fifty to four hundred commands per second. LLOOGG continued for several years before quietly fading from view.

The name Redis stands for REmote DIctionary Server. The default port, six three seven nine, has a more playful origin. On a phone keypad, those digits spell MERZ, a reference to Italian showgirl Alessia Merz, an inside joke from the Merzia company days. This is the kind of detail that tells you something about the person who built it. Infrastructure software with a sense of humor. A default port chosen as a tribute to a celebrity crush. Salvatore Sanfilippo, who posted to security forums under the handle antirez and would become one of the most respected open source developers in the world, was never the kind of programmer who took himself too seriously.

The Ruby on Rails Rocket

Sanfilippo shared Redis on Hacker News through his friend David Welton. The response was modest. About four people engaged. But one of them was Ezra Zygmuntowicz, who created the first Ruby client library for Redis. This mattered enormously, because Ruby on Rails was the hottest web framework on earth in two thousand nine, and having a good Ruby client meant Redis was immediately accessible to the community with the most enthusiasm and the loudest megaphone.

The adoption curve went vertical. In two thousand nine, Chris Wanstrath, co-founder of GitHub, built Resque, a background job processing library, entirely on Redis. Resque became the template for how to build job queues, and it put Redis at the center of one of the most important developer platforms in the world.

In two thousand ten, Instagram co-founder Mike Krieger contacted Sanfilippo directly. Instagram, he said, was entirely built on Redis during its early period. Hundreds of Redis instances powering the feed data for what would become one of the largest social networks on earth.

That same year, Twitter began working with Sanfilippo on Redis-based solutions for its timeline service. When you tweeted, your message needed to appear in every follower's timeline. That fan-out operation, writing one tweet to potentially millions of individual timelines, was exactly the kind of high-speed, in-memory workload Redis was born for.

And then VMware called. They offered Sanfilippo a job to work on Redis full-time. No strings attached beyond a logo on the website. No feature requirements, no product roadmap, no corporate oversight. Just a salary to do what he was already doing. For an independent developer working from Sicily on a side project that had accidentally become critical infrastructure, this was the dream arrangement. He took it.

More Than a Cache

Here is the thing about Redis that made it dangerous. It was not a cache. Everybody called it a cache, everybody installed it as a cache, but its actual superpower was its data structures. Strings, lists, sets, sorted sets, hashes, and later streams, bitmaps, hyperloglogs, and geospatial indexes. Each one solved a different class of problem, and each one was absurdly fast because everything lived in memory.

The sorted set is the best example. You add members with scores, and Redis keeps them ranked automatically. This is, by itself, a complete real-time leaderboard. The gaming industry adopted it overnight. Riot Games, Electronic Arts, studios everywhere discovered that the leaderboard problem, which had required complex database queries and caching layers, was just ZADD and ZRANK in Redis. Two commands.

The pattern repeated across every use case. Need a job queue? LPUSH items onto a list, BRPOP to wait for the next one. Need pub/sub messaging? SUBSCRIBE and PUBLISH. Need rate limiting? INCR a counter with EXPIRE. Need session storage? SET with a time-to-live. Need geospatial queries for finding nearby drivers? GEOADD and GEORADIUS.

The joke in the industry became that you install Redis as a cache and six months later it is doing forty-seven different things in your architecture. The "we need a cache" conversation was really the "we need Redis" conversation, because once it was in your stack, its versatility made it irresistible for every problem that needed speed.

Each major version added another dimension. Pub/sub arrived in two thousand ten with Redis two point zero. Lua scripting in twenty twelve turned it into a programmable data store where you could execute atomic operations server-side. Redis Cluster in twenty fifteen gave it native distributed mode across up to a thousand nodes, after years of development. The module system in twenty seventeen opened the floodgates for third-party extensions. And Streams in twenty eighteen gave it a Kafka-like append-only log with consumer groups, the last major feature Sanfilippo personally designed.

Code is Like a Poem

Through all of this, Sanfilippo maintained Redis essentially alone for eleven years. He was the benevolent dictator for life, making every final design decision. And he approached the work with a philosophy that was genuinely unusual in infrastructure software.

He wrote a document called the Redis Manifesto. Most software projects have manifestos about scalability or performance. His was about aesthetics.

Code is like a poem. It is not just something we write to reach some practical result. We are against complexity. We believe designing systems is a fight against complexity. We optimize for joy. We believe writing code is a lot about the experience of the developer.

This was not marketing copy. You could see it in the code. Redis had essentially no external dependencies beyond the C standard library. It bundled its own memory allocator, its own event loop library, its own networking layer. Sanfilippo believed in self-contained software the way a watchmaker believes in self-contained movements. No supply chain. No dependency tree. Just C and libc.

He also embedded an Easter egg called LOLWUT in the Redis command set. Type LOLWUT into the Redis command-line interface and you get a piece of generative ASCII art that changes with every version. Fractal patterns, geometric designs, different every time Redis ships a new release. It serves no practical purpose. It is whimsy embedded in infrastructure that processes millions of operations per second. It is, in its own small way, art.

But by twenty twenty, the dream arrangement was over. VMware had become Pivotal had become Redis Labs, and the project had become a product. Sanfilippo was no longer an artist working on his creation. He was a maintainer working on a company's core asset.

On June thirtieth, twenty twenty, he published a blog post titled "The end of the Redis adventure."

I write code in order to express myself, and I consider what I code an artifact, rather than just something useful to get things done. I never wanted to be a software maintainer. Now I am asked more and more, by the circumstances created by a project that became so important, to express myself less and to maintain the project more. Redis was the most stressful thing I did in my career, and probably also the most important.

He stepped away. No dramatic exit, no angry manifesto. Just a quiet Sicilian admitting that maintenance was killing the thing he loved about the work. He went home to write a science fiction novel, make YouTube videos in Italian, and build tiny programs for the joy of building them. A text editor in a thousand lines of C called Kilo. A minimal chat server called Smallchat. Things with no commercial value and no corporate sponsors. Art for its own sake.

Three Hundred and Forty-Seven Million Dollars

While Sanfilippo was writing novels and building toy programs, the company bearing his project's name was raising money. A lot of money.

Redis Labs had been founded in twenty eleven by Ofer Bengal and Yiftach Shoolman, two Israeli entrepreneurs who saw the commercial potential of managed Redis services. They renamed from Garantia Data in twenty fourteen, raised round after round of venture capital, and by twenty twenty-one had accumulated approximately three hundred and forty-seven million dollars in funding. The Series F in twenty twenty valued them above a billion dollars. The Series G in twenty twenty-one, led by Tiger Global, pushed the valuation past two billion.

In August twenty twenty-one, Redis Labs rebranded to simply Redis Inc. They reportedly considered an IPO at a four to five billion dollar valuation.

But there was a problem that no amount of venture capital could solve. The three largest cloud providers in the world, Amazon Web Services, Google Cloud, and Microsoft Azure, all offered managed Redis services. AWS had ElastiCache. Google had Memorystore. Azure had Azure Cache for Redis. These services generated enormous revenue. And Redis Inc., the company employing the maintainers writing the actual code, saw almost none of it.

This is the cloud provider dilemma, and it had already driven several open source companies to drastic action. MongoDB switched to the Server Side Public License in October twenty eighteen, twelve months after going public, in direct response to AWS launching a compatible service called DocumentDB. Elastic changed its Elasticsearch license in January twenty twenty-one after AWS launched OpenSearch. HashiCorp put Terraform behind a Business Source License in August twenty twenty-three, spawning the OpenTofu fork.

Redis had been testing the waters since twenty eighteen, when they added a Commons Clause to some of their optional modules. The community backlash was immediate. In twenty nineteen they replaced it with their own Redis Source Available License, which was cleaner but still not recognized as open source by the OSI. Both times, core Redis itself stayed BSD-licensed. The modules were the canary.

The Nuclear Option

On March twentieth, twenty twenty-four, Redis CEO Rowan Trollope dropped the pretense and changed the license on everything.

Under the new license, cloud service providers hosting Redis offerings will no longer be permitted to use the source code of Redis free of charge. The majority of Redis' commercial sales are channeled through the largest cloud service providers, who commoditize Redis' investments and its open source community.

Core Redis, starting with version seven point four, would be dual-licensed under the Redis Source Available License version two and the Server Side Public License version one. Neither was recognized by the Open Source Initiative as open source. The BSD license that had governed Redis since the first two thousand lines of C code in February two thousand nine was gone.

The reaction was swift and coordinated. Linux distributions began planning to remove Redis from their repositories. The Open Source Initiative issued a statement clarifying that the SSPL lacked the requisites to be an approved license. And Madelyn Olson, who had been a Redis maintainer since twenty eighteen, who had personally driven the TLS implementation into Redis six point zero after, as she put it, pestering Sanfilippo at RedisConf twenty eighteen about why he was wrong about TLS, began assembling a team.

Eight days later, the Linux Foundation announced Valkey. The name came from a working title of "Placeholder KV" that evolved into a portmanteau of "value" and "key." It was forked from Redis seven point two point four, the last BSD-licensed version. Behind it stood not just Olson but Ping Xie from Google Cloud, Viktor Soderqvist from Ericsson, and Binbin Zhu from Tencent, who had contributed nearly a quarter of all Redis open source commits.

Valkey is an impressive effort by longstanding contributors in the Redis community to uphold open source principles. Having this project under a foundation rather than a single company means community-driven development without surprise license changes.

The numbers told the story of what happened next. Before the license change, twelve non-employee contributors had produced fifty-four percent of Redis commits. After, zero non-employees contributed more than five commits. The community did not just disagree with the license change. It left.

The Prodigal Creator

In November twenty twenty-four, four and a half years after walking away to write novels and build tiny programs, Salvatore Sanfilippo came back. He contacted Redis Inc., he said, not the other way around. Normal salary. Retained his existing stock options. His title was Redis evangelist, but his actual agenda was clear almost immediately.

I do not believe that openness and licensing are only what the OSI tells us they are.

In May twenty twenty-five, Redis eight point zero shipped with a third license option alongside the RSAL and SSPL. The Affero General Public License version three. AGPL is copyleft, meaning anyone who modifies Redis and offers it as a service must release their changes. But it is recognized by the OSI as genuinely open source. It was a partial correction, a concession that the March twenty twenty-four move had gone too far.

Whether it was enough remained an open question. Valkey had been growing fast. Within six months of forking, it shipped version eight point zero. By early twenty twenty-five, it had over a hundred and fifty unique contributors and nearly twenty thousand GitHub stars, the fastest adoption of any comparable fork in open source history. Independent benchmarks showed Valkey eight point one outperforming Redis eight point zero by thirty-seven percent on write throughput. AWS priced Valkey twenty to thirty-three percent below Redis on its managed services.

The pattern had played out exactly as it had for MongoDB, Elastic, and HashiCorp. A company builds on open source, cloud providers commoditize the service, the company changes the license, the community forks, the cloud providers back the fork, and the company ends up competing against a community-governed version of its own software. Elastic was the only one to reverse course, moving back to AGPL in August twenty twenty-four. Redis followed with the AGPL option nine months later. But by then, Valkey existed, and it was not going away.

The Dictionary on Your Shelf

Redis runs inside projects you have probably built yourself. A real-time dashboard uses it for pub-sub. A web application uses it with RQ, the Redis Queue library for Python, to process background jobs. Both are the exact pattern the industry discovered fifteen years ago. You install Redis as a cache or a queue, and it quietly becomes part of the load-bearing infrastructure.

The redis-py library in the requirements file connects to a Redis server running somewhere on your stack. That server has been there long enough that it feels like part of the operating system. You do not think about it the way you do not think about electricity. It is just there, holding sessions, queueing jobs, caching responses. The way it is there inside Twitter's timeline service and Instagram's feed infrastructure and GitHub's background processing and the leaderboards of every multiplayer game your kids play.

The irony of the Redis story is that the licensing war was fought over a piece of software whose creator never wanted to be in a war at all. Sanfilippo wanted to write beautiful code from Sicily and have people use it. He wrote a manifesto about joy and simplicity and treating code like poetry. He embedded generative art Easter eggs in database software. He named his default port after a television personality. And then three hundred and forty-seven million dollars of venture capital arrived, and cloud providers built billion-dollar services on his work, and the poetry became a product, and the product became a legal dispute.

He came back, and he pushed for the AGPL, and Redis eight shipped with vector sets he personally designed, the first new core data type in years. But the community had already forked. The contributors had already left. The Linux distributions had already switched. Whether the artist can rebuild what the corporation broke is the question the Redis story asks and has not yet answered.

Meanwhile, on servers all over the world, Redis instances hum along at six three seven nine. Not caring about licenses. Not caring about forks. Just holding keys and values in memory, the way they have since a young man in Sicily decided that databases should be more like poetry than plumbing.

That was episode nine.

Open a terminal and type brew install redis, then redis-server. Leave that running and open a second terminal. Type redis-cli. You are now talking directly to a Redis instance. Type SET hello world, then GET hello. That is the entire mental model. A key, a value, in memory, instant. Now try LPUSH mylist one, LPUSH mylist two, LRANGE mylist zero negative one. You just used a list, one of the data structures that made Redis more than a cache. And before you quit, type LOLWUT. That is Sanfilippo's generative ASCII art Easter egg, the whimsy embedded in infrastructure. Poetry, not plumbing.