Building an Enterprise Anycast CDN at the Network Edge

This series is a theory — my theory. It is not presented as a standard, a prescription, or a finished product, but as a deliberate exploration of an idea that emerges from operating large networks over time. Some parts are well‑understood practices; others are hypotheses tested through reasoning, experience, and constraint. Like any good theory, it is meant to be examined, challenged, adapted, and occasionally rejected. What follows is an attempt to think clearly and honestly about what might be possible, not to declare what must be done.

Section 4 — Forming the Overlay Without Trusting the Inside

By this point in the series, two things should be clear:

• Anycast gets traffic to a reachable edge
• An overlay is required to decide what happens next

What has not yet been explained is how that overlay comes into existence without undermining the safety properties established so far.

This is where many designs quietly fail. It is tempting to assume that because systems live "inside" the enterprise, they can simply trust one another. That assumption does not hold at scale.

Discovery Is Not Trust

The first mistake to avoid is conflating discovery with trust.

Nodes must be able to discover that other nodes exist. This does not mean they should automatically trust them, exchange routes, or forward traffic.

In this architecture:

• Discovery answers the question: Who might be out there?
• Trust answers the question: Who am I willing to talk to?

These are deliberately separate steps.

Why Bootstrap Happens Over the Internet

Overlay formation begins over the same Internet connectivity used for anycast ingress.

This is not accidental.

The Internet-facing interface is:

• Universally reachable
• Already required for operation
• Treated as untrusted by default
• Protected through explicit authentication

By contrast, private transport (such as MPLS) is powerful but dangerous if misused. Allowing it to act as a bootstrap mechanism expands the blast radius of mistakes.

For that reason, initial contact between nodes always occurs over the Internet underlay, using a simple control-plane protocol with strong authentication.

A Minimal Control Plane

Each edge node exposes a small control-plane endpoint on its Internet-facing interface.

This endpoint:

• Exists solely to establish identity
• Does not carry user traffic
• Does not install routes
• Does not forward packets

Its only job is to answer a single question:

Is this peer who it claims to be?

Only after both sides can answer "yes" does anything else happen.

From Authentication to Adjacency

Once two nodes have mutually authenticated:

• They may establish an overlay adjacency
• They may exchange node identity information
• They may bind routing protocols to that adjacency

Nothing is implied. Every step is explicit.

If authentication fails, the process stops cleanly. No routes are installed. No partial state remains.

Private Transport as an Optimization

Only after an Internet-based overlay adjacency exists may nodes choose to make use of private transport.

Even then:

• Routes are narrow and peer-specific
• No defaults are introduced
• No transitive reachability is assumed

Private transport accelerates known-good relationships. It never creates them.

Safety Through Ordering

The ordering here is intentional:

1. Discover peers
2. Authenticate over an untrusted medium
3. Form explicit adjacencies
4. Exchange narrowly scoped routing information
5. Optionally optimize transport

Reversing this order creates implicit trust and unpredictable failure modes.

In the next section, we will look more closely at how nodes signal service truth into the overlay — and why explicit withdrawal is more important than clever selection.