ffmpeg: The Invisible Empire

The Thing You Cannot See

On February eighteenth, twenty twenty one, a six-wheeled robot named Perseverance touched down in Jezero Crater on Mars. Over the following months, it began sending back images and video of an alien landscape, rust-colored plains under a butterscotch sky. Every frame of that footage was reportedly compressed before transmission by a piece of software called ffmpeg.

This is episode eight of What Did I Just Install.

That same software is running right now, in your pocket. It is inside your web browser. It processed the last YouTube video you watched, the last Netflix show you streamed, the last voice message you sent on WhatsApp. It is in your smart television, your dashcam, your baby monitor, your doorbell camera. It is running on servers at Google, at Amazon, at every video platform you have ever used. Conservative estimates put it on more than twenty billion devices worldwide.

And the episode you are listening to right now, this very audio, was assembled by ffmpeg. The text-to-speech engine produced raw audio segments, and ffmpeg stitched them together into the MP3 file playing in your ears at this moment. You are inside ffmpeg's output. You have been since you pressed play.

The name stands for Fast Forward MPEG. It is a command-line tool and a set of libraries for reading, writing, converting, and streaming virtually any audio or video format ever invented. It is the universal translator for digital media. It is, by most measures, the single most important piece of multimedia software ever written.

It is maintained, in large part, by one man. It is funded by one hundred and fifty-seven thousand euros per year from a German government grant. And the story of how it came to be involves a reclusive French genius, a civil war that split the project in two, a decade-long patent battle over who owns the right to compress moving pictures, and a Hall of Shame listing sixty-five companies caught stealing its code.

This is the story of ffmpeg.

Before the Knife

To understand what ffmpeg does, you need to understand the problem it solved. In the late nineteen nineties, digital video was a Tower of Babel. Every media player, every video editor, every streaming service had to write its own code to understand each format. If you wanted your player to handle AVI files, you wrote an AVI parser. If you wanted it to handle QuickTime, you wrote a separate QuickTime parser. If you wanted it to decode the MPEG-2 video inside that AVI container, you wrote an MPEG-2 decoder. If you wanted it to also decode DivX, you wrote another decoder. And another for Real Video. And another for Windows Media. And each of those had subtly different audio formats nested inside them, each requiring its own code.

It was as if every email application had to independently implement the entire postal system from scratch. Hundreds of developers around the world were writing the same decoders over and over, in isolation, most of them poorly. Each implementation had its own bugs, its own edge cases it did not handle, its own security vulnerabilities that nobody was auditing because nobody had time to audit when they were busy reimplementing the same codec for the fifth time.

The open-source media players of the late nineties were particularly affected. MPlayer, xine, VLC, each had to maintain their own codec implementations. The Windows world had its own fragmented ecosystem of codec packs that users installed like they were patching a leaky roof, one codec at a time, hoping nothing conflicted. Anyone who lived through the era of "please install the K-Lite Codec Pack to play this video" knows the pain. It was a world where playing a video file was not a guaranteed outcome but a negotiation between your software, your codecs, and whatever format the person who encoded the video had chosen.

What ffmpeg did was end that chaos by centralizing everything into one open-source toolkit. One place where every codec, every container format, every streaming protocol could live. Write your media application, link against ffmpeg's libraries, and suddenly you could read and write everything. Every format. Every codec. Past, present, and the ones that had not been invented yet but would be added in the next release.

The first major adoption was MPlayer, which switched from its own codec implementations to ffmpeg's libavcodec. The improvement was immediate and visible. Better compatibility, fewer bugs, faster decoding. Other projects followed. VLC adopted ffmpeg's libraries. Then media converters. Then streaming servers. Then web browsers. The pattern was always the same: why maintain your own codecs when ffmpeg already has them, already tests them, and already fixes the security vulnerabilities?

The architecture is a set of interlocking libraries, each handling one layer of the problem. Libavformat reads and writes container formats, the outer shell of a media file. Libavcodec handles the actual compression and decompression, hundreds of audio and video codecs. Libavfilter processes the media, scaling, cropping, overlaying, adjusting color. Libswscale converts between pixel formats. Libswresample converts between audio formats. And sitting on top of all of them, the ffmpeg command-line tool ties it all together.

The result is a single program that can convert virtually any media file to any other media file. The command-line interface is notoriously complex, a running joke in developer culture. The meme goes like this: "I just need to convert a video," followed by a forty-seven-flag command cobbled together from three Stack Overflow answers, a blog post from twenty fourteen, and blind faith. Developers have spent more collective hours assembling ffmpeg commands than some languages have spent being designed.

But that complexity is a reflection of the problem it solves. Media is genuinely complicated. Containers within containers, codecs within codecs, metadata in fourteen different standards. The fact that one tool handles all of it is not bloat. It is a miracle of engineering.

And it started with one person.

The Man Who Builds Everything

Fabrice Bellard was born in nineteen seventy-two in Grenoble, France, in the shadow of the Alps. He started programming at the age of nine, not on a computer but on a TI-59, a programmable calculator from Texas Instruments. The TI-59 had a magnetic card reader and could store program steps on tiny strips, and a nine-year-old in Grenoble was writing programs for it.

By the time he was seventeen, still a student at Lycée Joffre in Montpellier, he had written LZEXE. This was not a school project. LZEXE was an executable compressor for MS-DOS, a tool that could take a program, compress it, and wrap it with a tiny decompression stub so it would decompress itself when run. It was the first widely used tool of its kind, and it became a standard utility across the entire DOS ecosystem. Millions of people used a tool written by a teenager.

In nineteen ninety, he entered the Ecole Polytechnique, France's most elite engineering school, a military-status institution whose graduates include three presidents, a Fields Medal winner, and the founders of half the French technology sector. He specialized at Telecom Paris in nineteen ninety-six. During this period he published Bellard's formula, a mathematical expression for computing individual hexadecimal digits of pi without computing any of the preceding digits. This was not a software tool. It was a contribution to mathematics itself.

And then his career became something that defies easy description.

In nineteen ninety-nine, two thousand, and two thousand and one, he won the International Obfuscated C Code Contest three times. His two thousand entry is legendary. It was a C program that computed the largest known Mersenne prime number. The entire program was four hundred and thirty-eight bytes of source code. Not four hundred and thirty-eight kilobytes. Four hundred and thirty-eight bytes. Roughly the length of this paragraph. And it found the largest prime number known to humanity at the time.

In two thousand, while working at Netgem, a French set-top box company that is one of the only known employers on his resume, he created ffmpeg. We will come back to that.

In two thousand and one, he created TCC, the Tiny C Compiler. This was a fully self-compiling C compiler, meaning it could compile its own source code. But the point was not just that it worked. The point was how small it was. TCC was tiny enough that you could use C as a scripting language, compiling and running programs instantly. Where other C compilers took seconds or minutes, TCC compiled in milliseconds. It was a joke about the entire compiler industry, delivered as a working product.

In two thousand and three, he created QEMU. This deserves its own paragraph because of what it became. QEMU is a generic machine emulator and virtualizer. It can emulate an entire computer, a different processor architecture, a different operating system, all running inside software on your existing machine. On its own, that would be a significant piece of software. But QEMU became the foundation of something much larger. When the Linux kernel gained its KVM virtualization module, QEMU became the standard userspace component. Today, when you spin up a virtual machine on Amazon Web Services, Google Cloud, or almost any cloud provider, there is a good chance QEMU is underneath it. A significant fraction of the world's cloud computing infrastructure runs on software written by one person.

Every one of these projects would be a career-defining achievement for an ordinary programmer. Bellard produced them in consecutive years like someone working through a to-do list. And he was not done.

In two thousand and five, he built a DVB-T transmitter using a standard personal computer. He literally broadcast a digital television signal from desktop hardware. In two thousand and nine, on the last day of the year, December thirty-first, he announced that he had computed two point seven trillion decimal digits of pi.

This was a world record. But the remarkable thing was not the number. It was the hardware. Every previous pi computation record had been set on supercomputers, machines costing millions of dollars, filling rooms, cooled by industrial air conditioning. Bellard did it on a single desktop PC. The total hardware cost was less than two thousand euros. A Core i7 processor running at two point nine three gigahertz. Six gigabytes of RAM. And five hard drives, each one and a half terabytes, for a total of seven and a half terabytes of storage. That was it. That was the supercomputer.

The binary computation took a hundred and three days. Verification across nine computers took thirty-four hours. The base-ten conversion took twelve days. Final verification took three more days. He published the results on his website in the same flat, matter-of-fact tone he uses for everything. No celebration. No press release. Just the numbers, the method, and the hardware specifications. A world record in computational mathematics, announced like a README update.

The record was broken in August twenty ten by Alexander Yee and Shigeru Kondo, who computed five trillion digits. But they needed significantly more expensive hardware to do it. Bellard had proven his point. You did not need a supercomputer. You needed the right algorithm and the discipline to implement it correctly.

In two thousand and eleven, he built JSLinux. This was the first PC emulator written entirely in JavaScript, running a full Linux kernel in a web browser tab. The year was twenty eleven. WebAssembly did not exist. Asm.js did not exist. He did it in pure JavaScript, the language that was still the butt of performance jokes, and he made it run a complete operating system. When the demo went live, the programming world treated it as a magic trick. It was Bellard treating it as a Tuesday afternoon.

In twenty nineteen, he released QuickJS, a compact embeddable JavaScript engine with full ES twenty-twenty specification compliance. It was small enough to run on microcontrollers. More recently, he has moved into neural network compression and large language model inference, building tools like TSAC for ultra-low-bitrate audio compression and TextSynth Server for LLM inference, bringing his characteristic efficiency to the AI era. He is in his fifties now and still producing foundational software at a pace that would exhaust a team of engineers.

Here is the thing about Fabrice Bellard that makes him difficult to write about. He is, by any reasonable measure, one of the most productive programmers who has ever lived. And he has done almost everything in his power to make sure nobody knows who he is.

He does not give conference talks. There are essentially no videos of him on the internet. He does not maintain a social media presence. His website, bellard.org, is a plain HTML page listing his projects with minimal prose. When the journalist Dan Dascalescu tried to write a comprehensive profile of him for SmartBear, Bellard politely declined to be interviewed.

And yet the people who have interacted with him describe someone surprising. Not the stereotype of the antisocial genius. When a developer emailed him to say thanks for QEMU, Bellard replied warmly, expressing how happy he was that they found it useful. He even gave them his phone number. Another developer who managed to optimize one of Bellard's programs received a congratulatory note.

Sent me some compliments after I was able to optimize it to run at a decent speed. A true master.

He is not reclusive in the sense of being unfriendly. He simply does not perform for an audience. He builds things, releases them, and moves on to the next thing. The Dascalescu profile concluded that the secret to his productivity was not some comic-book mutation but rather discipline, confidence, rigor, and many years of practice.

One detail reveals the environment in which ffmpeg was created. When Bellard released ffmpeg in two thousand, he did not use his real name. He used a pseudonym, Gerard Lantau. The reason was simple and chilling. Ffmpeg implemented decoders for patent-encumbered video codecs. The legal landscape for multimedia software in two thousand was a minefield of patents held by companies who were not shy about suing. The creator of the most important multimedia tool in history was literally hiding behind a fake name because he was afraid of getting sued for writing code that decoded video.

Think about that. The person who gave the world free, universal access to digital media had to do it anonymously because the corporations who held the patents on those media formats would have come after him.

Bellard was the principal developer of ffmpeg until October two thousand and one, less than a year of active leadership. By two thousand and three, he had mostly moved on. He made a few commits in two thousand and five, adding DVD subtitle support, and a single random patch in two thousand and seven. After that, nothing. He had built the foundation and walked away.

He did not hand-pick a successor. He did not establish a governance structure. He did not write a transition plan. He just left, the way he always does, moving on to the next impossible thing. The people remaining had to figure out how to run a project that the entire internet was beginning to depend on. And the person who stepped into that vacuum would spend the next decade paying for the lack of a plan.

The Heir Nobody Chose

Michael Niedermayer came from the MPlayer community. MPlayer was one of the first media players to adopt ffmpeg's libraries, and Niedermayer had been contributing critical infrastructure: video postprocessing filters, libswscale for image conversion and rescaling, x86 optimizations that made ffmpeg fast on Intel hardware. When Bellard drifted away, Niedermayer was the one still writing code every day. By two thousand and four, he was the de facto leader, not because anyone had appointed him but because he was the last person standing.

The thing that set Niedermayer apart from almost every other open-source maintainer was total commitment. Most maintainers have day jobs. They contribute to their projects on evenings and weekends, fitting code reviews between meetings and school runs. Niedermayer was different. He lived on minimal income specifically so he could dedicate himself entirely to ffmpeg. The project was not his side project. It was his life.

His technical contributions were enormous. He worked on x86 optimizations, MPEG-4 and H.263 codec support, substantial parts of the H.264 decoder, and created entirely new formats like FFV1, a lossless video codec designed for archival preservation, and Snow, an experimental wavelet codec. He replaced rudimentary image conversion routines with libswscale, a proper scaling library. He was relentless about testing and security fixes.

But his code had a reputation. A colleague named Kostya Shishkov, who would later become one of his sharpest critics, described it bluntly.

Despite his implementation was fast-working, it was not very nice to debug or refactor, especially if you are not Fabrice.

The code worked. It was fast. But it was impenetrable to anyone who was not Niedermayer. His infamous MpegEncContext structure grew into a monolith, adapted across codecs where it was poorly suited, accumulating hacks for edge cases rather than clean abstractions. And his governance style reflected the same philosophy. He made decisions. He applied rules selectively. There was an unwritten understanding in the project that the rules applied to everyone except the person doing most of the work.

For years, this arrangement held. The project was successful. The code was good, if difficult to read. The output was extraordinary. But underneath the surface, frustration was building among developers who wanted cleaner processes, mandatory code review, and a leader who listened more than he decided.

That frustration detonated in January two thousand and eleven.

The Coup

On a winter day in January two thousand and eleven, roughly eighteen ffmpeg developers published a joint announcement. They had, they said, appointed new project maintainers. Michael Niedermayer was no longer in charge.

It was not a fork. Not yet. It was a hostile takeover, an attempt to seize control of the project from within. The developers behind it included some of the most respected names in the community.

Diego Biurrun had spent years reforming ffmpeg's code formatting conventions and project organization. He was the one who had tried to bring order to the codebase through consistent style, and his frustration was fundamentally about governance, not code quality. The project needed structure, and Niedermayer was the obstacle to structure.

Kostya Shishkov was a multimedia codec expert who had written some of the most detailed and clear-eyed documentation of ffmpeg's internal history. He would later chronicle the fork from the libav side with a level of specificity that makes his accounts essential primary sources, even accounting for their perspective.

Jason Garrett-Glaser was the developer behind x264, the most widely used H.264 encoder in the world. He signed the coup announcement, though reportedly with reluctance. He would later express private regret about how the situation unfolded.

And then there was Mans Rullgard.

Rullgard was born in rural Sweden and studied at KTH Royal Institute of Technology in Stockholm. His colleagues believed he knew the C language specification by heart, not as a figure of speech but as a literal description. He was the architect of ffmpeg's FATE continuous testing infrastructure, the automated system that catches regressions across platforms. He wrote the critical ARM optimizations that made ffmpeg viable on mobile devices, the work that eventually put ffmpeg inside every smartphone. He worked at ARM's own Swedish subsidiary. And he had designed the original ffmpeg logo.

Rullgard was brilliant, essential, and already disillusioned with the project before the split. He had been drifting away. The coup brought him briefly back, but on the side of the dissidents. When the fork happened, he went with libav. When the dust settled, he had one more card to play. He owned the copyright on the ffmpeg logo, and he refused to grant the Niedermayer faction permission to keep using it. This triggered what was reportedly the only formal legal consultation in ffmpeg's entire history, as the project sought counsel on whether they needed to comply. They designed a new logo. And Mans Rullgard left open source entirely, walking away from a community that he had helped build into something global.

Their grievances were real. Niedermayer's governance style, the selective application of rules, the impenetrable code, the resistance to mandatory peer review. In his own history of the project, Kostya catalogued the frustrations with precision. Niedermayer's code was difficult to maintain. His leadership was autocratic without the diplomatic skills to make autocracy sustainable. There was an unwritten but widely understood rule in the project that Niedermayer could do whatever he wanted, and the same freedoms did not extend to others.

But the execution of the takeover had a problem. Before the coup, there had been a vote of confidence in Niedermayer's leadership. He had won it, fifteen votes in favor of him staying versus five for a compromise where he would step down. And roughly fifteen of the eighteen people who signed the coup announcement had not participated in that earlier vote. The faction trying to take over did not even have the numbers to win a democratic process, so they skipped the democratic process.

And then they hit the wall that ended everything. They did not control the ffmpeg.org domain.

Fabrice Bellard, who had not contributed a line of code to the project in years, still owned the DNS records. When the dust settled, Bellard sided with Niedermayer. It was the original creator's one remaining act of stewardship, choosing a side from a distance. With Bellard's support, Niedermayer's faction restored the repository, the mailing lists, and the project identity. The developers who had attempted the takeover were locked out.

They had no choice but to fork. In March two thousand and eleven, they created a new project called Libav, hosted at libav.org.

What followed was, in the words of one observer, a civil war.

A civil war started, with inflamed private exchanges and various threats. Developers who had worked together for years turned on each other.

The Long War

The libav fork was not just a split between two repositories. It became a split across the entire Linux ecosystem. And the side that won the first battle was not the side you might expect.

Debian is the distribution that underpins Ubuntu, which in turn underpins the majority of Linux servers on the internet. The decision of what Debian ships matters not because Debian is the most popular desktop distribution, but because its choices cascade through Ubuntu into millions of servers, containers, and cloud instances. When Debian makes a packaging decision, the ripples reach every major cloud provider.

Several libav developers were also Debian developers. They had influence over which packages Debian shipped. And libav's release process was considered more conservative than ffmpeg's, fitting Debian's ethos of stability over features. Starting with Debian Wheezy, the distribution replaced ffmpeg with libav. Ubuntu followed, as it does for nearly all Debian packaging decisions. Overnight, the most popular Linux distribution family in the world was shipping the fork instead of the original.

The switch created one of the most confusing episodes in open-source history.

Libav deprecated the ffmpeg command name. They replaced it with avconv. This was not just a rename. The command-line options were different. The behavior on edge cases was different. And when users on Debian or Ubuntu typed ffmpeg, the name that every tutorial, every blog post, and every Stack Overflow answer used, they got a warning message. The message claimed that the program was deprecated and they should use avconv instead.

This was a very destructive lie. FFmpeg was not deprecated. Only libav had deprecated the name.

Imagine the experience from a user's perspective. You are a developer trying to convert a video file. You search the internet and find a tutorial that says to run ffmpeg with specific flags. You type the command. Your system tells you the program is deprecated and you should use something called avconv instead. You try avconv with the same flags. It does not work because the flags are different. You search for avconv tutorials and find almost nothing because the rest of the world is still using ffmpeg. You spend an hour trying to figure out what is happening before discovering that your operating system has replaced a widely used tool with a fork that renamed everything, and the deprecation warning was issued by the fork, not by the original project.

This went on for years. From Debian Wheezy in twenty thirteen through to Debian Jessie. Every tutorial said ffmpeg. Every Debian system said avconv. Developers who needed to support both had to write compatibility layers or detect which fork was installed at runtime. Open-source projects that used ffmpeg as a dependency had to document different installation procedures for Debian-family systems versus everything else.

But here is where the story turns. And it turns on a single decision that Michael Niedermayer made in the aftermath of the fork.

He decided to merge.

Every day, Niedermayer reviewed the commits that libav developers were making to their fork. Every improvement, every bug fix, every new feature. And he merged them into ffmpeg. He pulled their work into his codebase systematically, making ffmpeg a strict superset of libav. Everything libav had, ffmpeg also had. Plus everything that ffmpeg's own developers were building independently.

Libav did not reciprocate. They rarely, if ever, cherry-picked changes from ffmpeg. Their philosophy was to build independently, to prove that they could do it better. And in theory, their codebase was cleaner. They had mandatory code review. They had stricter standards. But in practice, they had fewer developers, and those developers were doing duplicate work, reimplementing features that already existed in ffmpeg rather than merging them.

The numbers told the story. In a twelve-month comparison period, ffmpeg had three hundred and two developers contributing eight thousand seven hundred and five commits. Libav had one hundred and twenty-one developers and one thousand seven hundred and twenty-one commits. Ffmpeg was producing major releases more frequently and maintaining support for older branches longer.

But the decisive blow was not about features. It was about security.

Google's security team, which had been actively fuzzing both projects to find vulnerabilities, published a comparison. Their conclusion was unambiguous.

Every single issue we have found, even the least severe ones, has been fixed in a timely manner in FFmpeg. Libav remains affected by hundreds of such bugs.

One specific example made the case in miniature. A security vulnerability discovered in twenty fourteen was fixed in ffmpeg promptly. In libav, the same vulnerability sat unpatched for more than two months. When your software runs on billions of devices and processes untrusted media files from the internet, a two-month window on a known vulnerability is not a policy disagreement. It is a liability.

In July two thousand and fifteen, the Debian Multimedia Maintainers team voted to return to ffmpeg. The technical case was overwhelming. Fedora, Arch Linux, and openSUSE had never left. The switch back to ffmpeg for Debian Stretch was the end of the war, even if libav's servers stayed online for a few more years. Its last release, version twelve point three, came in two thousand and eighteen. By twenty twenty, it was effectively abandoned.

Some of its developers eventually returned to ffmpeg, older and presumably wiser about the difference between winning a governance argument and winning the long game.

The Man Left Standing

On July thirty-first, two thousand and fifteen, just weeks after Debian's decision validated his years of stubborn merge work, Michael Niedermayer posted a message to the ffmpeg development mailing list.

I have been in FFmpeg since fourteen years and been the leader since eleven years and I feel that I am not the best person for the leader position. The work and pressure caused by the merges is a main reason for my resignation.

He had won. The fork was dying. Debian was coming back. And he was done.

The resignation letter is a remarkable document. It is short, unpolished, and heartbreaking in the way that honest exhaustion always is. He described the role of "leader" not as a position of authority but as a description of thankless labor.

Leader was interpreted as the guy who does all work no one else does, and takes all responsibility no one else wants to take.

He said he would consider returning only if there was a nice and friendly environment and no hostile forks. The conditions read like a wish list from someone who had spent four years in a trench.

After the fork, his attitude had changed. Multiple observers noted that he became more collaborative, more open to code review, more willing to listen. The very things the libav faction had wanted. But the change came too late to prevent the split, and the years of daily merge work had ground him down. His contributions shifted from pioneering development to what one observer called janitorial work, primarily fixing bugs found by automated fuzzers. The creative energy that had produced FFV1 and Snow was spent on keeping the lights on.

The Register ran a headline that captured the dark comedy of the situation perfectly: "Fork off! FFmpeg leader quits after mass exodus."

But the story does not end with the resignation. Niedermayer kept contributing. He is still, in twenty twenty-six, one of the most active contributors to ffmpeg. When Germany's Sovereign Tech Fund provided the project's first significant funding in twenty twenty-four, one hundred and fifty-seven thousand five hundred and eighty euros for maintenance work, three of the four funded projects were assigned to Niedermayer. Not because he wanted all the money. Because nobody else was willing to do the work.

In November twenty twenty-four, nearly a decade after his resignation, Niedermayer posted to the mailing list about the rise of bitter hatred in the community discussions. After three weeks of sustained attacks and one administrator's offer to resign, he wrote something that stripped away any pretense of professional distance.

It starts affecting my physical health and the health of the project. No matter on which side of some of these discussions you are, they cost everyone a lot of time.

This is the maintainer paradox. The person who keeps the project alive is also the person the project is slowly killing. Michael Niedermayer has given more than two decades of his life to a piece of software that runs on twenty billion devices, and his reward is a German government grant paying him to fix fuzzer bugs while the mailing list debates whether he should still be involved.

The Patent Wars

To understand why Fabrice Bellard hid behind a pseudonym, you need to understand the economics of moving pictures. This is the story of who owns the right to compress video, and how an industry built on patent royalties accidentally created its own worst enemy.

Video compression is one of the most valuable technologies on earth. Raw, uncompressed video from a modern camera generates roughly three gigabytes per minute. Without compression, streaming would be impossible, video calls would not exist, and your phone's storage would fill up in under an hour. The algorithms that shrink those three gigabytes down to a few megabytes are worth billions of dollars.

And for decades, the organizations that developed those algorithms made sure everyone paid for them.

The pattern was established in nineteen eighty-eight with H.261, the first practical video codec, designed by the International Telecommunication Union for videoconferencing over ISDN telephone lines. It worked. It was standardized. And the patents required to implement it were pooled into a licensing arrangement that required anyone building a product with H.261 to pay a royalty.

H.263 followed in nineteen ninety-six, improving compression for low-bandwidth connections. Then, in two thousand and three, came the codec that conquered the world.

H.264, also known as Advanced Video Coding, was the result of a collaboration between the ITU and the Moving Picture Experts Group, the same organization whose name Bellard had borrowed for ffmpeg. H.264 was a quantum leap in compression efficiency. It made high-definition video on the internet practical for the first time.

The licensing model was pragmatic. A single organization, MPEG-LA, operated one patent pool. You negotiated with one entity and got a license. The first hundred thousand units were free. Internet broadcasting was royalty-free as long as the content was free to end users. This was good enough. YouTube adopted it. Blu-ray adopted it. Every smartphone, every streaming service, every video call application adopted it. H.264 became the codec of the internet, and MPEG-LA collected its toll.

For a decade, this arrangement held. The licensing was expensive for large companies but manageable. The technology was genuinely excellent. The world standardized on H.264 and moved on.

But the web was not entirely comfortable with a patent-encumbered standard. Mozilla, in particular, resisted shipping H.264 in Firefox for years, arguing that the open web should not depend on patented technology. They were losing the argument, users were leaving Firefox for Chrome because videos would not play, when Cisco did something remarkable in twenty thirteen.

Cisco was already paying millions of dollars per year in H.264 patent royalties for their video conferencing products. Webex, Telepresence, the entire enterprise video stack, all licensed through MPEG-LA. Cisco realized that the marginal cost of licensing H.264 for a free, open-source codec module was essentially zero for them. They were already paying the full freight. So they open-sourced their H.264 implementation as OpenH264 and agreed to cover the MPEG-LA royalties for anyone who used it. If you downloaded OpenH264 from Cisco's servers, the patent license came included, paid by Cisco.

Mozilla immediately integrated OpenH264 into Firefox. The browser that had refused to ship H.264 on principle could now ship it without compromising that principle, because the patent cost was being absorbed by a corporation that had already paid it. It was an elegant hack, not of technology but of licensing. Cisco got goodwill. Mozilla got compatibility. And the patent holders still got their money. Everyone won, which is how you know it was an unusual outcome in the codec wars.

And then the successor arrived, and everything fell apart.

The Disaster of HEVC

H.265, officially called High Efficiency Video Coding, was finalized in two thousand and thirteen. It offered roughly fifty percent better compression than H.264. In theory, this should have been an easy upgrade for the industry. In practice, it triggered a crisis that is still unresolved more than a decade later.

The problem was the patents. Instead of one licensing pool, H.265 had three. MPEG-LA operated one pool. A new entity called HEVC Advance operated a second. A third entity called Velos Media operated a third. Each charged different royalties. You had to negotiate with all three. And beyond those pools, there were patents held by companies that had not joined any pool, patents that could surface at any time with their own licensing demands.

The total cost was dramatically higher and more uncertain than H.264. Companies doing due diligence for a product launch could not even determine the total patent liability because the patent landscape was incomplete. You could negotiate with all three pools, sign all three licenses, and still get sued by a fourth party you had never heard of.

On July twenty-first, two thousand and fifteen, HEVC Advance announced its initial licensing terms. The industry had been hoping for a decrease in royalties compared to H.264, reflecting the maturity of the technology. Instead, HEVC Advance announced an increase.

The reaction was something between horror and fury. Companies that had been planning their transition from H.264 to H.265 froze. Product roadmaps were thrown into chaos. The calculation was simple and devastating: better compression was not worth the legal liability.

Exactly forty-two days later, on September first, two thousand and fifteen, seven companies announced the formation of the Alliance for Open Media. The founding members were Amazon, Cisco, Google, Intel, Microsoft, Mozilla, and Netflix. Their mission was explicit. They were going to build a royalty-free video codec and they were going to make the patent problem go away forever.

Forty-two days. That is how fast the largest technology companies on earth moved from "this licensing is unacceptable" to "we will build our own codec." The HEVC patent holders had priced themselves out of the market so aggressively that their customers decided it would be cheaper to invent new physics.

Google had already been working on alternatives. They had acquired a company called On2 Technologies in twenty ten and released its VP8 codec as royalty-free, followed by VP9 in twenty thirteen. VP9 became YouTube's primary codec for several years. But VP8 and VP9 were Google-controlled projects, not industry standards, which limited their adoption beyond Google's own platforms.

The Alliance for Open Media took a different approach. They combined research from Google's VP10, Mozilla's Daala, and Cisco's Thor into a single new codec called AV1. The patent licensing used rules derived from the World Wide Web Consortium. Every contributor had to license their AV1-related patents to every other participant. And the arrangement included a nuclear deterrent: if you sued anyone over AV1 patents, you automatically lost the right to use all other contributors' patents. Mutually assured destruction, applied to intellectual property.

AV1 was finalized on March twenty-eighth, twenty eighteen. It has won the web. YouTube supports it. Netflix supports it. Hardware decoding chips are now standard in most new processors. As of November twenty twenty-five, Dell and Hewlett-Packard announced they would disable HEVC hardware decoding in BIOS on entry and mid-range laptops due to further HEVC license fee increases. The patent holders are still raising prices on a codec the industry has already decided to abandon.

The lesson of the codec wars is simple. Make the tax too high and the taxpayers will build their own country. It just takes a decade.

And through all of it, from H.261 in nineteen eighty-eight to AV1 in twenty eighteen, ffmpeg has implemented every single one. The patent-encumbered ones and the royalty-free ones. The winners and the losers. The ones with three licensing pools and the ones with none. Every codec that has ever mattered lives inside ffmpeg's libavcodec library, and the legal status of each one is someone else's problem. Ffmpeg just does the math.

The Shame Wall

Ffmpeg's own license is a carefully balanced act. By default, the project is released under the GNU Lesser General Public License, LGPL version two point one. This means you can use ffmpeg's libraries in a proprietary application, as long as you dynamically link to them and provide the ffmpeg source code you used. If you compile with the enable-gpl flag, the entire binary becomes GPL, meaning your entire application must also be open source. And the enable-nonfree flag adds components that cannot be redistributed at all.

In practice, a staggering number of companies ignore these terms entirely. They statically link ffmpeg into their products, strip the copyright notices, and ship it as if they wrote the code themselves.

To fight back, the ffmpeg project maintained a public page known as the Hall of Shame. It listed, by name, every company and product caught violating ffmpeg's license. The stated purpose was blunt: to get them to comply with the licensing terms by shaming them in public.

The list contained more than sixty-five entries. DVDFab. GOM Player. VirtualDJ. Wondershare. Xilisoft Video Converter. ImTOO. Netgem. Baofeng Storm. PowerPoint DVD Converter. DownloadHelper ConvertHelper. Dozens of video converters and media players that had taken ffmpeg's code, the work of hundreds of volunteer developers over more than a decade, and claimed it as their own.

The project has no legal team. There is no budget for lawyers. Enforcement is done by individual developers who notice violations, sometimes because users report finding ffmpeg version strings or build artifacts hidden inside closed-source products. For most of ffmpeg's history, public shaming was the only enforcement mechanism available.

But in twenty twenty-four, one case pushed past shaming into formal legal action, and it involved a company worth billions.

Rockchip is a Chinese semiconductor manufacturer that designs system-on-chip processors. Their chips power tablets, streaming boxes, single-board computers, and embedded devices sold around the world. They are not a small company. They are a publicly traded semiconductor firm with annual revenue in the billions of yuan and their processors are in millions of consumer devices.

In February twenty twenty-four, ffmpeg developers discovered something in Rockchip's Media Process Platform, a key component of their chip software that handles video decoding in hardware. The repository, hosted openly on GitHub, contained thousands of lines of code copied directly from ffmpeg's libavcodec library. The copied code included decoders for H.265, AV1, and VP9, three of the most important modern video codecs. These are the same codecs that the codec wars had been fought over, the codecs that companies pay millions in patent royalties to use.

The copying was not subtle. Rockchip had taken the code, stripped every original copyright notice, replaced them with their own company attribution in the file headers, and redistributed the code under the Apache license. Apache is a permissive license that imposes far fewer obligations than ffmpeg's LGPL. What Rockchip had done was take volunteer-written copyleft code, the work of developers who had chosen a specific license to ensure that their code remained open, erased those developers' names, wrote their own names in place, and changed the license to one that removed the obligations the original authors had deliberately imposed. It was not borrowing. It was not an oversight. It was erasure.

Ffmpeg publicly confronted Rockchip. The project posted about the violation, named the company, and demanded compliance with the LGPL terms: either release the code under the correct license with proper attribution, or stop distributing it. Rockchip did not respond meaningfully. Months passed. The ffmpeg community posted updates. Still no compliance. A year passed. Then nearly two years.

On December eighteenth, twenty twenty-five, an ffmpeg developer filed a DMCA takedown notice with GitHub. This was the nuclear option for a project that has historically relied on public shaming. GitHub disabled Rockchip's Media Process Platform repository shortly after.

A billion-dollar chipmaker, stripping attribution from code written by volunteers who are funded by a hundred and fifty-seven thousand euro government grant. A company whose products contain ffmpeg's code in their silicon, running on millions of devices, whose response to being caught was two years of silence. That is the ffmpeg licensing story.

And Rockchip is just the most dramatic recent case. A study of one point six million free applications on the Google Play Store found more than four thousand three hundred apps violating the GPL, many of them using ffmpeg without attribution or source code release. The actual number is certainly higher. Most violations are never detected.

One Hundred and Fifty-Seven Thousand Euros

Let us talk about money. Specifically, let us talk about how much money the world invests in a piece of software that runs on twenty billion devices and processes virtually every video on the internet.

For most of its history, ffmpeg operated on donations to Software in the Public Interest, a nonprofit that serves as the project's fiscal sponsor. The amounts were small. Individual donations. Occasional grants. Nothing approaching the scale of the project's importance.

In April twenty twenty-four, something unprecedented happened. Germany's Sovereign Tech Fund, a government program dedicated to sustaining critical open-source infrastructure, provided ffmpeg with one hundred and fifty-seven thousand five hundred and eighty euros. It was the project's first significant governmental funding in more than two decades of existence.

The money was split across four maintenance projects. Fifty thousand euros for classifying and fixing Coverity static analysis issues. Sixty thousand for modernizing the libswscale library. Twenty-five thousand for maintaining FFV1, the archival video codec. Fifteen thousand for separating the libpostproc library into its own repository.

Three of the four projects were assigned to Michael Niedermayer. Not because the project wanted to concentrate funding in one person. Because nobody else was willing to do the work.

One hundred and fifty-seven thousand euros. For context, a single senior software engineer at Netflix, one of ffmpeg's heaviest users, earns several times that amount per year. The total budget for maintaining ffmpeg, the multimedia infrastructure of the entire internet, would not cover one engineer at one of the companies that depends on it.

This is the open-source funding crisis distilled to its essence. We have seen it in every episode of this series. Daniel Stenberg maintaining curl on one salary from wolfSSL for twenty billion devices. Andrey Petrov receiving five dollars in total early donations for urllib3. The pattern is the same every time. But ffmpeg is the most extreme example because the gap between the project's importance and its funding is the widest.

The Sovereign Tech Fund grant, significant as it was, is a rounding error compared to the value extracted. YouTube alone processes millions of hours of video per day using ffmpeg's libraries. Discord uses it for voice channels serving a hundred and fifty million monthly active users. Twitch uses it for live streaming. Spotify uses it for audio processing. Every one of these companies generates billions in revenue. The total annual investment in the software they all depend on is less than the cost of a modest apartment in San Francisco.

The Community Problem

The ffmpeg community has a reputation. It is not a good one.

The project's mailing list, the primary venue for development discussion, is notoriously harsh. Newcomers who ask basic questions or submit imperfect patches have historically been met with terse dismissals or outright hostility. The culture rewards deep technical expertise and punishes everything else. Several observers have described it as a community where the most brilliant minds eventually just leave and stop contributing to open source entirely.

After the trauma of the libav fork, ffmpeg formalized its governance. There is now a General Assembly of active members who make decisions through voting. A five-member Technical Committee arbitrates technical conflicts. A five-member Community Committee handles interpersonal conflicts and enforces a Code of Conduct. Both committees serve one-year terms.

This is the structure that the fork should have produced a decade earlier. If ffmpeg had had formal governance in two thousand and ten, the group of eighteen developers might have had a legitimate path to their complaints instead of resorting to a coup. But governance structures are like smoke detectors. Nobody installs them before the fire.

The development itself continues at a remarkable pace despite the community challenges. Ffmpeg seven point zero, released in April twenty twenty-four, was described as the biggest code refactoring in many years. A developer named Anton led a multi-year project to make the command-line tool multi-threaded, enabling decoding, demuxing, muxing, and encoding to run in separate threads. This required extensive cleanup of thread-safety issues across a codebase that had been single-threaded since its creation in two thousand.

Ffmpeg eight point zero, codenamed Huffman, arrived in August twenty twenty-five with a fully Vulkan-based decode-filter-encode pipeline and a stable VVC decoder, the first software implementation of the next-generation video codec. Google Summer of Code students contributed significantly to the VVC work. The project, for all its dysfunction, keeps producing extraordinary software.

The Command Line

There is a genre of developer humor built entirely around ffmpeg's command-line interface. The joke always follows the same arc. Someone has a simple task. Convert a video from one format to another. Trim the first thirty seconds. Extract the audio track. A task that should take one command. And then they discover what that command actually looks like.

Ffmpeg's command-line interface is powerful in the way that a nuclear reactor is powerful. It can do virtually anything, and the path from "I want to trim a video" to the correct incantation of flags passes through a thicket of codec specifications, container format options, bitrate controls, filter chains, stream mappings, and metadata injection directives. A working command might have twenty flags. A complex one might have fifty. The average developer constructs their ffmpeg commands through a process that has been described, only half-jokingly, as archaeology. You find a Stack Overflow answer from twenty fourteen. You find another from twenty nineteen that contradicts the first. You find a blog post from someone who seems confident. You combine elements from all three, run the result, and pray.

The complexity is not gratuitous. It reflects the genuine complexity of multimedia. A video file is not one thing. It is a container format holding multiple streams, each encoded with different codecs, at different bitrates, with different metadata, potentially with multiple audio tracks in different languages, subtitle tracks in different formats, and chapter markers. Manipulating any of these independently, or all of them at once, requires a correspondingly complex interface. The alternative is not a simpler tool. The alternative is a dozen specialized tools, each handling one piece, none of them talking to each other.

But the result is that ffmpeg has become a kind of gatekeeping ritual in developer culture. If you can write an ffmpeg command from memory, you have been through something. If you can write a filter chain that overlays text, adjusts color, scales to a target resolution, and encodes with two-pass variable bitrate in a single command, you are either deeply experienced or deeply disturbed. Possibly both.

In twenty twenty-four, someone on social media criticized ffmpeg's codebase as messy. The response from the ffmpeg community was four words that went viral, reaching two hundred thousand views.

Talk is cheap. Send patches.

It is the distilled essence of open-source culture: criticism without contribution is noise. And it is the distilled essence of ffmpeg's culture specifically: abrasive, direct, and frustratingly correct.

The Dependency Tree Down

Ffmpeg itself depends on an ecosystem of libraries, each one its own story. The core builds with minimal dependencies, but a full-featured installation pulls in dozens.

For video encoding, there is x264 for H.264, x265 for H.265, libvpx for VP8 and VP9, libaom and SVT-AV1 for AV1, and dav1d, a purpose-built AV1 decoder optimized for speed. For audio, there is LAME for MP3 encoding, the library that matters most to this podcast because it is the specific dependency that converts raw audio into the MP3 file you are listening to. There is also libopus, libvorbis, and libfdk-aac for other audio formats.

For hardware acceleration, ffmpeg can hook into VAAPI on Linux, VideoToolbox on macOS, CUDA and NVENC on Nvidia hardware, Quick Sync Video on Intel, and Vulkan for cross-platform GPU compute. On the Mac where this podcast is assembled, VideoToolbox provides hardware-accelerated encoding, and the LAME library provides MP3 encoding.

The build system is a custom configure script, not autotools, not cmake, but a hand-written shell script that probes for available libraries and enables features accordingly. It is one of the most complex configure scripts in open source, which is saying something. The script reflects ffmpeg's design philosophy: support everything, make everything optional, let the user decide what they need.

When a podcast pipeline depends on ffmpeg, it transitively depends on whatever ffmpeg was compiled with. On macOS via Homebrew, brew install ffmpeg pulls in a substantial tree. But the critical transitive dependency for any audio project is just one library. LAME. Every podcast episode that goes through an ffmpeg pipeline uses LAME's MP3 encoder for the final compression. The audio processing tools produce raw segments, ffmpeg concatenates them and encodes the result, and LAME does the compression into the MP3 that arrives in your podcast app.

The Empire

Let us count the kingdom.

Ffmpeg runs on Mars, compressing images for transmission to Earth from a robot in Jezero Crater. It runs on YouTube, processing millions of hours of video uploads every day. It runs inside Google Chrome and Mozilla Firefox, decoding the video you watch in your browser. It runs inside VLC, the media player that promises to play anything and delivers on that promise because of ffmpeg's libraries underneath. It runs inside OBS Studio, powering every Twitch stream and every recorded presentation. It runs inside Handbrake, converting your home movies. It runs inside Blender, powering video editing in a three-dimensional modeling suite. It runs inside Netflix's encoding pipeline, optimizing the streams that arrive at your television.

It runs inside Discord, compressing the voice channels where a hundred and fifty million people talk every month. It runs inside WhatsApp, encoding the voice messages that cross the globe. It runs in your smart television, your security camera, your dashcam, your baby monitor, your doorbell. It runs on every operating system: Linux, macOS, Windows, Android, iOS, FreeBSD, and more. It runs on ARM processors, x86, MIPS, PowerPC, and RISC-V.

Every few years, someone tries to estimate the total number of devices running ffmpeg. The current consensus is north of twenty billion. That is more than two devices for every person on earth. And the code running on all those devices is maintained by a community funded at a level that, as we noted, would not cover one engineer at one of the companies that depends on it.

The developer culture around ffmpeg has its own folklore. The command-line complexity alone spawned a genre of memes. One popular one shows the progression from "I just need to trim a video" to a terminal command spanning multiple lines with flags for codec selection, bitrate targeting, audio channel remapping, filter chains, and metadata injection. The codebase is abrasive, the community is direct, and the software keeps getting better despite everything working against it.

Where It Connects

Every episode of this podcast is an ffmpeg artifact. Let me be specific about how.

A typical podcast audio pipeline works like this. Text-to-speech engines generate speech as raw WAV files at twenty-four kilohertz. Sound effects are generated by signal processing libraries at the same sample rate. Music jingles come from audio generation models, also at twenty-four kilohertz. All of these segments need to be assembled into a single MP3 file.

That is where ffmpeg enters. The audio module calls ffmpeg to concatenate WAV segments using stream copy mode, which works because everything is at the same sample rate and format. Then ffmpeg encodes the result to MP3 via the LAME encoder, with proper bitrate and metadata. Ffprobe, ffmpeg's companion analysis tool, reads the duration of each segment so the pipeline can calculate chapter marker positions.

If ffmpeg disappeared tomorrow, thousands of podcast pipelines would stop working. There is no practical replacement. Converting a sequence of WAV files to a properly encoded MP3 with chapter markers, metadata, and consistent audio levels is a task that ffmpeg makes trivial and every alternative makes painful.

Ffmpeg also appears in the broader stack in ways that are easy to miss. Homebrew installs ffmpeg as a dependency for other tools. Image and video processing utilities on the server depend on it. The deployment pipeline uses it indirectly through tools that themselves depend on ffmpeg.

But the direct dependency is the one that matters here. This podcast exists because a twenty-eight-year-old French programmer, hiding behind a fake name to avoid patent lawsuits, wrote a multimedia toolkit in his spare time. A decade later, a German developer nearly destroyed his health maintaining it through a civil war he did not start. A German government grant now pays for the bug fixes. And here we are, listener and narrator, connected by a piece of code that has touched every video and every audio file that has mattered in the twenty-first century.

We have told eight stories now in this series. A Turkish developer who deleted everything on principle. A photographer with a mental illness who gave Python its most beloved library. A Canadian machine who built the backbone of the web and walked away to a farm. A college student who forked a dead imaging library because nobody else would. A Swede in a Stockholm suburb who has maintained one tool for twenty-eight years. A man in Charlotte, North Carolina who competes against fopen. A journalist who decided frameworks were doing it wrong.

And now a ghost. A man who builds world-changing software and disappears. Who creates the universal translator for digital media and hands it off without a succession plan. Whose code runs on twenty billion devices while he quietly moves on to neural networks and compression algorithms in a country where he barely exists in public.

The pattern underneath all eight stories is the same. The most critical infrastructure in the software world is built by individuals, maintained by stubbornness, funded by scraps, and used by everyone. The ratio between what these people give and what they receive in return is not just unfair. It is structurally absurd. And ffmpeg is the most extreme case because the gap is the widest. No other project touches as many devices while being maintained by so few people with so little money.

The next time you run a command that starts with ffmpeg, maybe you will pause for a moment. Think about Fabrice Bellard at twenty-eight, using a pseudonym because the patent holders might come after him. Think about Michael Niedermayer, merging code every morning from people who tried to overthrow him. Think about sixty-five companies on a wall of shame, and a billion-dollar chipmaker who stripped the names off volunteer code. Think about a hundred and fifty-seven thousand euros to maintain the multimedia infrastructure of the entire internet.

Ffmpeg asks nothing of you. It runs on your phone and in your browser and on a robot on Mars. It processes every video you watch and every podcast you hear. And the people who built it are mostly invisible, mostly unpaid, and mostly tired.

You just pressed play. Ffmpeg did the rest.

Open a terminal and type ffmpeg dash version. If something comes back, you already have it. If not, brew install ffmpeg on macOS or check your package manager. Now try this. Find any video file on your machine and type ffmpeg dash i input dot mp4 output dot gif. That is it. One command, and you have a GIF. Or try extracting the audio: ffmpeg dash i video dot mp4 dash q colon a zero audio dot mp3. The command line looks intimidating, but the basic pattern is always the same: ffmpeg, dash i for input, then the output filename. The file extension tells ffmpeg what format you want. Everything Bellard built, every codec Niedermayer maintained, all of it hiding behind two flags and a filename.

That was episode eight.