Direct
/api/usersLesson 6 · Scale the entry layer
No single API Gateway.
An Edge/L4 Load Balancer accepts the public connection and selects one of two identical API Gateway instances. The selected gateway then reads the HTTP path and continues the Lesson 5 routing design.
Interactive architecture lab
Distribute connections across the gateway tier.
Send one request or open six concurrent connections. Observe which gateway receives each connection before any HTTP path is inspected.
Choose an APIEdge cannot see this path
Only public URL
http://localhost:7612/api/users
You can run the selected request in Postman.
Start the lesson, then execute a request.
Execute a request to inspect the original response.
Execute a request to inspect the response headers.
One public address
L4
Infrastructure statusSTOPPED
Current journey
Edge L4 → waitingStateless gateway tierTwo identical L7 routers
Internal routingService destinations
Scaled
/api/products
L7
Direct
/api/ordersObserve the gateway tier
Which API Gateway accepted the connection?
The Edge Load Balancer makes its decision before the HTTP request is visible. A reused TCP connection can therefore keep reaching the same gateway.
Latest request journey
WAITING
Execute a request to reveal every selected component.
Selected gatewayWaiting
Final serverWaiting
Gateway historyNo connections yet
- 1Start the lessonThe trace will separate connection distribution from API and service routing.
Why concurrent connections?
Layer 4 balances TCP connections. Sequential browser requests may reuse one existing connection and remain on one gateway. The experiment sends requests concurrently so the browser is more likely to open multiple connections.
Edge/L4 logconnection → gateway
Start Lesson 6.
API Gateway 1 logpath → service
Start Lesson 6.
API Gateway 2 logpath → service
Start Lesson 6.
Catalog L7 logProducts only
Only Product requests enter this node.
What this teaches
Remove the single API Gateway SPOF.
Once the API Gateway becomes the main entry point, it can become a Single Point of Failure (SPOF). To avoid this, we run multiple API Gateway instances and place another Load Balancer in front of them.
Client→Edge/L4 Load Balancer→API Gateway Instances→Services
The outer Load Balancer distributes incoming connections across available API Gateway instances. It can operate at Layer 4 because its main responsibility is to use transport information such as IP addresses and ports, rather than inspect application-level paths.
It is also called an Edge Load Balancer because it sits near the outermost entry point of the infrastructure.
High availability
More than one API Gateway can accept traffic.
No gateway-instance SPOF
Another healthy gateway can receive new connections if one fails.
Horizontal scaling
More gateway instances can be added as traffic grows.
Higher entry capacity
Incoming connections are spread across the gateway tier.
Edge Load Balancer
Distributes connections across API Gateway instances without understanding API paths.
API Gateway
Understands each HTTP request and routes it to the appropriate service type.
Internal Catalog Load Balancer
Distributes Product requests across instances of one particular service.
What this gives us
- No single API Gateway instance SPOF
- Horizontally scalable gateway tier
- Higher public-entry capacity
- Clear separation of L4 and L7 decisions
- Selective Catalog scaling remains intact
What still remains
- The single Edge Load Balancer is now the outermost SPOF
- Gateway configurations must remain consistent
- The internal Catalog Load Balancer is one instance
- Every cacheable request still reaches the backend