Scaleway, Part Two: The Plumbing That Makes It A System

When A Server Stops Being A System

A single virtual machine running a few Python processes is not a system. It's an island. The thing that turns an island into infrastructure is everything around the virtual machine. Private networking that keeps internal traffic off the public internet. Load balancers that route traffic to the right place. Caching layers that absorb traffic before it ever reaches the application. Container orchestration that handles deployment across multiple boxes. Managed databases that survive a server crash. Message queues that decouple producers from consumers.

For Pärception today, most of this is missing, or handled in the simplest possible way. One virtual machine. One Postgres on that virtual machine. Files served directly. Deployment through rsync aliases. Which is fine. The whole point of being a one-person operation is keeping things small enough to fit in one head.

But the rest of the Scaleway catalog exists, and some of it would meaningfully simplify things rather than add complication. This episode is the tour through that middle layer. Not what you adopt today, but what you reach for the first time a system grows past one machine. The shape of when each piece becomes relevant.

The Virtual Private Cloud

Start with the Virtual Private Cloud, usually abbreviated as V P C. The name is a Scaleway adoption of the Amazon Web Services term for the same concept, and the concept is this. Your resources live on an internal network that the public internet cannot reach directly. Communication between your own services travels over private routes. The internet only sees the parts you explicitly expose.

Today, on a single Scaleway virtual machine, this is not very relevant. The machine has a public address and that's it. There's nothing else to talk to it privately. The machine is the network.

The moment a second machine joins the picture, the calculus changes. If Pärkit grew from one Postgres on the application server to a separate Postgres machine, the question becomes how the application reaches the database. The naïve answer is over the public internet, which means anyone on the public internet can reach the database too. The better answer is a private network, where only your other resources can reach the database, and the database has no public address at all.

That's what the Virtual Private Cloud provides. You define the network. You attach resources to it. The traffic between them is private. The public internet only sees what you explicitly route through a Public Gateway.

V P C Peering is the extension when you have multiple Virtual Private Clouds, perhaps in different regions, and you want them to be able to talk to each other privately without going through the public internet. For most one-person operations this is academic. For an operation that runs distinct production and development networks and wants them to share certain services, it's the answer.

I P A M, which stands for Internet Protocol Address Manager, is Scaleway's name for the tool that keeps track of which addresses are used by which resources. It exists because once you have private networks, reserved public addresses, and floating addresses that can be attached and detached, the question of which address is what becomes hard to answer in your head. The Internet Protocol Address Manager is the answer to that question. You don't think about it until you need it.

Public Gateways And Site-to-Site V P N

The Public Gateway is the doorway between a private network and the public internet. It does what its name suggests. Resources on the private side can reach the internet through it for downloading updates, calling external application programming interfaces, fetching dependencies. The internet cannot reach the private side except through ports you explicitly open.

If Pärkit ever ran across multiple machines on a private network, this is the piece that lets those machines pull updates without exposing each individual machine to the internet. You expose the gateway. The gateway holds the rules. Everything behind it inherits those rules. One place to check, one place to update.

Site-to-Site V P N is the next layer. This is for connecting a Scaleway network to a network somewhere else. The classic use case is connecting cloud infrastructure to a company's office network. The traffic between the two travels through encrypted tunnels over the public internet.

For Pär specifically, the relevant version of this would be connecting pinkserver, the Raspberry Pi sitting at home in Kall handling home automation and the RUTM fifty-one cellular router, to a private network at Scaleway. Right now pinkserver and the Scaleway virtual machine talk to each other over the public internet using whatever security each side bothers to put in place. A Site-to-Site V P N would mean they talk to each other over a private encrypted channel that doesn't touch the public internet at all.

Whether that's worth doing today is a separate question. The answer is probably no. The point is that the tool exists. If pinkserver ever became more central, if it started handling actual sensitive workloads rather than home automation, the option to put it behind a proper V P N tunnel back to Paris is sitting there in the catalog.

Scaleway InterLink is the heavier version of this. Where Site-to-Site V P N tunnels through the public internet, InterLink establishes a hosted connection through partner networks. For an actual office with real bandwidth needs, this is the answer. For a Raspberry Pi in Kall, this is overkill. But the option exists, and knowing what's there is part of why this episode exists.

Edge Services And The Web Application Firewall

The next layer up is Edge Services. This is Scaleway's name for the package of capabilities you put in front of a web application to make it faster and safer. Caching for static content. Secure Sockets Layer certificates managed for you. Custom domains. The Web Application Firewall, which is a security layer that inspects incoming traffic and blocks patterns that match known attacks.

Pärception runs several web faces. Årebladet's website. The Pärception site. The PärCel shops at derbyshop dot s e, shop dot parception dot s e, and butik dot arebladet dot s e. Each one is a public-facing FastAPI application served from the Scaleway virtual machine.

Today, the caching is probably whatever the browser does on its own, which is to say not much. The certificate management is probably Let's Encrypt and a renewal script that runs occasionally. The Web Application Firewall is probably the philosophy of, if a bot finds an exploit, we deal with it. Which is fine for a small site. The minute one of these sites gets actual traffic, especially during a summer edition of Årebladet when the local press attention picks up, that calculus shifts.

Edge Services would absorb that traffic. The traffic hits Scaleway's edge servers, gets served from cache when possible, and only what cannot be cached touches the actual application server. The Web Application Firewall would inspect everything that passes through, blocking the most obvious bot patterns before the application ever sees them.

The cost is small. The value is mostly insurance against the day something goes briefly viral. For something like Årebladet, where the summer edition is the year's anchor revenue event, small insurance against unexpected traffic is meaningfully more valuable than small monthly cost.

The Web Application Firewall as a standalone product is also offered. Same principle. Inspect, classify, block. Worth knowing it's there even if Edge Services is the more obvious entry point.

Load Balancers

The Load Balancer is the piece that sits in front of multiple application instances and routes traffic between them. The basic shape. You have three machines all running the same application. The Load Balancer accepts incoming requests and distributes them to whichever machine is least busy. If one machine dies, the Load Balancer notices and routes around it. The application appears to the user as if it never went down.

For Pärception today, with one virtual machine, this is mostly academic. The moment Pärception runs anything across two or more machines, the Load Balancer is what makes them feel like one service.

The interesting Scaleway-specific note is that their Load Balancer supports multi-cloud configurations. You can put a Scaleway Load Balancer in front of machines running on Scaleway, on Amazon Web Services, on Google Cloud, on Hetzner. The Load Balancer itself doesn't care where the destinations live. This matters for the day you want hybrid deployments, or the day you're testing a migration from one provider to another and want to send only some traffic to the new infrastructure.

The pricing is per hour and modest. The complexity it removes from your head is significant. You stop thinking about which machine to point traffic at. You point traffic at the Load Balancer. The Load Balancer figures out the rest.

The Kubernetes Branch

Now containers, and specifically Kubernetes. The container orchestration system that has become the default answer for how you run multiple containers across multiple machines.

Scaleway runs Kubernetes Kapsule. This is their managed Kubernetes product. You ask the application programming interface for a cluster. You get a cluster. The control plane, which is the brain of Kubernetes, runs on Scaleway's infrastructure and is free. The worker nodes, which are where your containers actually run, are billed as ordinary virtual machines.

The free control plane is important. Most cloud providers charge for the control plane, and the bill creeps up. Scaleway's policy of free control plane means a small cluster of two or three nodes is genuinely affordable. The cost of running Kapsule is approximately the cost of the underlying nodes, which is much lower than the equivalent setup on Amazon Web Services with their Elastic Kubernetes Service control plane fees layered on top.

For Pärception today, Kubernetes is probably overkill. The whole point of a one-person operation is to avoid running infrastructure that needs maintenance. Kubernetes needs maintenance. Updates, certificate rotations, occasional pod debugging. None of that is hard. All of that is time.

But there's a version of Pärception five years from now where there are enough side projects, enough small services, enough random demos, that running each one as a separate virtual machine becomes silly. That's when Kubernetes starts paying off. You have one cluster. You deploy new services as new container manifests. The cluster handles where they run.

Kubernetes Kosmos is the Scaleway product for multi-cloud Kubernetes. The thing that makes Kosmos interesting is that the worker nodes don't have to live on Scaleway. They can be Scaleway virtual machines, but they can also be your existing Hetzner servers, or your machines at Online dot net, or anywhere else you can run a small agent. The Kubernetes control plane lives at Scaleway. The compute can live anywhere. For a setup that gradually accumulated machines across providers over the years, Kosmos is the cleanest way to bring them under one orchestration umbrella without migrating any of them.

The Three Flavors Of Serverless

The trinity. Functions, Containers, Jobs.

Serverless Functions are the smallest unit. You write a function. You upload it. Scaleway runs it whenever it's triggered. You don't manage the server. You don't manage anything except the function code. You're billed for the time the function actually runs, which for most functions is milliseconds at a time.

For Pär, the use cases here would be small webhook handlers. The thing that receives a payload from Stripe when a PärCel order goes through. The thing that gets called when a new Director ingest is recorded. The thing that processes a single image upload to PärImage. Each of these is a discrete operation. None of them needs a permanently running server. Functions are the right shape for each.

Serverless Containers are the next step up. Instead of writing a single function in a supported language, you write a full container with whatever's inside it, and Scaleway runs that container on demand, scaling from zero to many instances based on incoming requests. This is the right shape for slightly bigger things. A small FastAPI service that handles a specific kind of request. A worker that processes uploads. A small website that gets traffic in bursts.

The benefit of Serverless Containers over a virtual machine is that you pay nothing when the container is idle. No requests, no charge. A small project that gets ten visitors a day costs almost nothing. A small project that suddenly gets ten thousand visitors a day still works, because Scaleway spins up more instances of the container automatically. You don't change anything. The bill goes up because traffic went up, then comes back down when traffic comes back down.

Serverless Jobs are for batch processing. The shape is different from Functions and Containers. A Job is something that runs once, does work, and finishes. You define what it does. You schedule it or trigger it. Scaleway runs it. When it's done, it stops.

This is the right shape for nightly aggregations. The script that recalculates Pärkit's summary tables overnight. The script that generates the daily LifeLab face recognition batch. The script that pushes new Årebladet article drafts through a quality-check pipeline. Each of these is a job. Each of them currently probably runs as a cron task on the virtual machine. Serverless Jobs would move them off the virtual machine, run them on dedicated capacity, and not consume the application server's resources while they run.

The pattern across the three serverless products is the same. Pay for what you actually use. Don't manage servers. Let Scaleway handle the scaling. The tradeoff is that you give up some control. You don't choose exactly when your function runs. You don't get a stable hostname for your container. For batch and event-driven workloads, this is fine. For workloads where you need consistent control, you go back to virtual machines.

Managed Databases

Now databases. The first thing the Scaleway catalog has that Pärkit could plausibly migrate to today is Managed Database for PostgreSQL. Worth a careful look.

Pärkit right now is a single Postgres on the application server. It works. The downside of that arrangement is that backups, replication, security patches, and tuning are all things Pär has to remember to do. Managed Database for PostgreSQL is the same Postgres, except Scaleway handles the operational pieces.

You provision a database. Scaleway gives you a connection string. You point your application at it. Scaleway takes backups on a schedule. Scaleway applies security patches in windows you can configure. Scaleway can provision a read replica with one application programming interface call. Scaleway can promote that read replica to primary if the original dies. The Postgres version stays current. The replication stays healthy. The on-call shift sits on someone else's calendar.

The cost is somewhat higher than running Postgres yourself on a virtual machine. Whether the cost difference is worth the operational simplification depends on how much Pär enjoys the operational work. For some operators, running their own Postgres is a point of pride and a useful exercise. For others, paying the small premium to never think about it is the right answer. For an attention-deficit-aware solo operator who has summer edition deadlines and would rather not have a database emergency in mid-July, leaning toward managed is probably the right answer.

Managed Database for MySQL is the same product for the other major relational database. Less relevant since Pärkit is Postgres.

Managed Database for Redis is the cached-data option. Redis is the in-memory key-value store that almost every web application eventually adds to handle session storage, rate limiting, real-time leaderboards, cached query results. If Pärception ever needed any of those, Managed Redis is the answer. Right now probably not. Eventually likely yes.

Managed MongoDB is the third managed database option. NoSQL document store. Mostly useful for very specific workloads. Probably not in Pär's path.

The interesting new arrival in the database corner is Serverless SQL Database. This is Postgres without provisioning. You don't choose an instance size. You don't choose a region. You ask for a database and you get one, and you pay for the queries you actually run rather than for capacity that sits idle.

For a side project that occasionally needs a database but doesn't justify even a small managed instance, this is the right shape. The tiny demo. The one-time pipeline that needs persistent storage for a week. The minimum viable product of an idea where you don't know yet whether it'll grow. Serverless SQL is the bridge between "I need a database for an hour" and "I need a database forever."

Data Warehouse For ClickHouse

This one deserves its own section because it just went general availability in the first quarter of two thousand and twenty-six, and Scaleway is currently leading their marketing with it.

ClickHouse is a column-oriented database originally built at the Russian search company Yandex, now open source and widely adopted. It is famously fast at analytical queries over very large datasets. The thing it does well is the kind of query that asks, for every row in this hundred-million-row table, count the ones that match a complex condition and aggregate them by some dimension. Postgres can do that. ClickHouse does it many times faster on the same hardware. The architectural reason is that ClickHouse stores data by column rather than by row, which means an analytical query that touches three columns out of fifty only reads three columns of data from disk. The savings are dramatic.

The thing Scaleway has built is a managed Data Warehouse for ClickHouse. You don't operate the cluster. You don't tune it. You ask Scaleway for a warehouse. You load data into it. You run analytical queries against it. It returns answers fast.

For Pär specifically, this is genuinely interesting in two directions.

First, Pärkit. Right now Pärkit stores BMW telemetry, Apple Health data, zone transitions, location traces, ingest events, all of it in Postgres. Postgres is fine for transactional queries. Postgres struggles when you ask it to compute aggregations across years of data. How many hours did I spend in Kall in two thousand and twenty-three versus two thousand and twenty-four, broken down by day of week. That is the kind of query that takes Postgres several minutes and ClickHouse several seconds.

A Data Warehouse for ClickHouse could sit alongside Pärkit, with a regular sync that copies the analytical-shaped data from Postgres into ClickHouse for fast querying. Pärkit's transactional side stays in Postgres. The analytical face moves to ClickHouse. Best of both. The cost of running a small ClickHouse warehouse is meaningful but not prohibitive, and the gain in query speed is the kind of thing that turns a slow exploration into a fast one. Slow exploration is where curiosity dies.

Second, LifeLab. The optical character recognition pipeline, the face recognition results, the spatial enrichment, all of this generates analytical data over hundreds of thousands of photos. DuckDB is the current local answer for this. DuckDB is excellent for local analytical workloads. It is not designed for shared multi-user analytical workloads. The day LifeLab needs to support a query interface for someone other than Pär, the day a journalist asks to see everywhere Pär was photographed in March two thousand and twelve, ClickHouse is the shape that fits.

Whether to adopt it today is the question. The answer is probably, not until there's a concrete query that Postgres or DuckDB cannot answer in reasonable time. The Data Warehouse exists. The shape of when to reach for it is clearer now.

Apache Spark, Apache Kafka, NATS

A briefer pass through three more data services.

Clusters for Apache Spark are the Scaleway managed offering of the Spark distributed processing framework. Spark is what you reach for when you have data too large to fit on one machine and you want to run computations across many machines. It's the standard answer for, I have terabytes of logs and I want to extract patterns from them. For Pärception, the use case would arise if PärImage ever needed to process several terabytes of image metadata in parallel. Not today. Possibly someday.

Clusters for Apache Kafka are the Scaleway managed offering of the Kafka distributed message broker. Kafka is the standard answer for high-throughput event streams where many different consumers need to process the same stream. Web analytics, click streams, sensor data, all classic Kafka workloads. For Pär, the interesting use would be Pärkit. BMW telemetry, Apple Health pulls, location pings, all of these could be modeled as Kafka topics with Pärkit subscribing to all of them. Today it's all direct writes to Postgres. Kafka would buffer between source and destination, decoupling them. The benefit of decoupling is that the source doesn't break when the destination is down, and the destination can be replaced without the source noticing.

NATS is the lighter-weight messaging system that Scaleway also offers as a managed service. NATS is to Kafka what Redis is to Postgres. Faster, simpler, less durable. The right answer for low-latency message passing where you don't need every message to be replayable forever. For Pärception's internal services talking to each other, NATS would fit better than Kafka. Lower overhead. Simpler operations.

Topics and Events is Scaleway's managed publish-subscribe service for sending messages to multiple consumers through a single application programming interface. Queues is the managed message queue product for the producer-to-consumer pattern. Both are useful. Both fit the pattern of small services that need to talk to each other without being tightly coupled.

For Pär, the practical adoption path is, don't reach for any of this until two distinct services need to communicate and you don't want them to talk to each other directly. The day that happens, the queue or the topic is the answer.

The Data Orchestrator

A new arrival worth noting. The Data Orchestrator is Scaleway's managed product for scheduling and managing data workflows across various systems. The shape is what tools like Apache Airflow do in the open-source world, with the operational pieces taken care of by Scaleway.

The use case is straightforward. You have multiple data sources. You have transformations that need to happen between them. You have downstream destinations. The Data Orchestrator is where you define the pipeline. Pull data from object storage. Transform it through a serverless function. Load it into the data warehouse. Notify a Slack channel when it's done. All of these steps coordinated, scheduled, retried on failure, monitored from one dashboard.

For Pärception today this is a service waiting for a workload. The day LifeLab's pipeline grows to the point where the steps need real coordination rather than a sequence of shell scripts, the Data Orchestrator is the answer.

Web Hosting And Domain Names

A short note on the simplest layer at the other end of the catalog. Scaleway sells Web Hosting that is essentially WordPress and similar simple hosting. They sell Domain Names that you can register and manage in the same console as the rest of your infrastructure.

For Pärception, neither of these is exciting on its own. The Pärception sites are FastAPI applications, not WordPress. The Årebladet site likely has its own domain registrar relationship.

The reason these matter is consolidation. If everything lived in the Scaleway console, including domains and hosting, the cognitive overhead of asking where a particular thing lives drops to zero. Whether that's worth migrating existing relationships is a separate question. Worth knowing the option is there. The next domain you register, the next simple site you set up for a small project, can land in Scaleway and reduce the count of vendor relationships by one.

What This Episode Was For

This was the plumbing. The networking layer that turns a virtual machine into part of a network. The container orchestration that handles multiple services. The serverless products that handle the small or bursty workloads. The managed databases that take operational work off your plate. The analytical data layer that just got general availability this quarter. The message brokers that decouple producers from consumers. The data orchestrator that ties the steps of a pipeline together.

For Pärception today, most of this is not adopted. That's fine. The point of this episode is to know what's there. The day Pärkit grows enough to want a separate database server, Managed Database for PostgreSQL is sitting there. The day Pärception needs Edge caching because the Årebladet website got hugged by a regional news pickup, Edge Services is sitting there. The day a side project needs to scale from zero to a hundred instances and back to zero in a single afternoon, Serverless Containers are sitting there. The day Pärkit's analytical queries get slow, ClickHouse is sitting there.

Next episode, the wild end. Generative artificial intelligence services, with models hosted entirely in Europe. The graphics processing unit instances, including bleeding-edge Blackwell hardware. The custom-built clusters that train foundation models. The security primitives. The Internet of Things hub. Cost management. Environmental footprint. Transactional email. And a return visit to quantum, which has expanded considerably since the quick experiment ten minutes ago.

What's there is the Scaleway map. One episode left.