Building a Serverless Platform From Scratch

Preview https://scale.searchhq.org

When we use modern cloud platforms, the experience can feel like magic. You push your code, and seconds later it's cleanly packaged, actively monitored, routed, and secured behind an edge network.

But as an engineer who fundamentally enjoys tickering and building, I found myself having this idea behind Scale. I wanted to understand exactly what happened between git push and a live, auto-scaling deployment and to do that, I decided to build a serverless-style platform inspired by the Vercel experience amoungst others.

Why Build This?

Like i've briefly mentioned above, it is my own belied that the easiest way to get to know and understand a piece of technology it to use it somewhere and build with it.

The goal for this project was to get out of the application code sandbox and tackle the mechanical problems of orchestration that platform engineering teams handle daily:

1. Dynamic Routing: How do you map wildcard subdomains to ephemeral, shifting IP addresses without dropping traffic during a deploy?
2. Auto-Scaling: How do you listen to system resource metrics to seamlessly spin up container replicas under load, and immediately terminate them to zero when idle?
3. The Developer Experience (DX): How do you make deploying a multi-component microservice architecture as simple as dropping in a single configuration file?
4. Serverless: How do you build a setup that manages servers on behalf of the user?

The Architecture Stack

The "magic" of serverless boils down to an exceedingly well-choreographed stack of open-source technologies. Here are some of the used technologies:

• Orchestration & Compute: The platform uses K3s, a lightweight Kubernetes distribution. It treats all deployments as standard Kubernetes primitives (Deployments, Services, HorizontalPodAutoscalers). I use a private local Docker registry running alongside K3s to host the built application images before they are pulled down by the nodes.
• Routing: The ingress layer relies on a proxy that handles firewalling, rate-limiting, and inspecting incoming packets, sitting behind Caddy for rapid, on-demand TLS provisioning.
• Observability: Running workloads blind is a recipe for disaster. I integrated the Promtail, Loki, and Grafana stack directly into the platform. Promtail tails the logs from the Kubernetes pods, pushes them to Loki, and they are indexed and visualized in Grafana.
• Asynchronous Pipelines: Pushing code triggers a worker queue managed by RabbitMQ. The system dynamically clones the Git repository, creates a Dockerfile if absent, builds the OCI image locally, pushes it to the private registry, and executes a rolling deployment in Kubernetes.

Serverless Deploys via Docker Compose

One idea i had that get me going was, If want has a log of components on the project (cache, ui, backend, db), that person can quickly spin all it up on their local machie using docker compose up -d. How can we take this and apply it on a serverless setup? Instead of deploying multiple components independantly, why not use a one deployment, which under the hood, will automatically deploy all components, and make it easier to manage and debug. Of course this mean we have to eliminate this single point of failure, which means each component should be entirely treat as a single separate unit.

I did this by treating the entire stack as a single deployment entity mapped from a standard docker-compose.yaml file. The platform parses the Compose file, extracts the relationships, volumes, and ports, and translates them into raw Kubernetes objects.

This means that developers can simply bring their standard Compose file, hit deploy, and the platform natively provisions the necessary K3s infrastructure, configures the proxy routing, and wires up everything else.. It reduces the steps to production significantly.

Agents Managing Infrastructure

Given that infrastructure state and metrics are entirely exposed via APIs, this makes it easy to integrate agents and explore capabilities they have on thes platform's architecture.

This gives agents secure, scoped access to the user's deployments. AI is getting more capable, and it's interesting to what we can do, to try and integrate it within managing and troubleshooting infrastructure. A log of logs get generated, traces etc, and having an agent that has the ability to parse those and offer fixes makes it easier going forward:

• "Check the logs for the backend pod to see why it crashed."
• "Scale the auth service Deployment up to 5 replicas."
• "My deployments are failing with a mismatch in exposed port, make changes, commit to github and trigger a new deployment."

This is just a start. I believe agents can help teams manage a log of infrastructure, and make it easier for them.By forcing myself to build this natively, I had to deeply engage with network primitives, resource scheduling, and observability pipelines.

Conclusion

Scale is very much in the early stages of tinkering with now. There's a lof of work that has to be done on it, and making it work well. i look forward to improving it contantly.