Once you’ve built all of your microservices and are ready to deploy them to AWS ECS (other container orchestration tools are available… but not in this post), you need a way to connect them together. Historically you could use an elastic or application load balancer that would act as a front door to all of your applications – every time one service needed to talk to another it would do so through the load balancer.
This is known as centralised load balancing. While effective, you can very quickly hit limitations such as exhausting the number of target groups (which compounds as you start to manage multiple app versions, blue/green deploys etc.) and relying on a single link in your network, resulting in a bottleneck or single point of failure.
As companies with increasingly large and complex microservice systems began hitting these limitations, they started moving towards client-side load balancing. Netflix built Ribbon, and the Spring Framework popular in Java introduced it in their Spring Cloud capabilities. These act as a service catalogue that maintain and track direct connections between service instances instead of using a centralised load balancer As and when instances go down or get created, this catalogue is updated and all services are notified so they can adapt accordingly. There are many competing solutions that offer these capabilities – this is the one I was most familiar with back in 2018.
AWS and other cloud vendors started incorporating client side load balancing (and its bigger brother, service meshes) into their offerings. If you want to connect various compute types (EC2, ECS, EKS, Fargate etc.) in AWS you have the option of AppMesh, which is a full-fat service mesh. If you want something lightweight you could use ECS Service Discovery, which is a DNS based service catalogue. Instead of going through an ALB, services make DNS requests and ECS Service Discovery uses CloudMap to synchronise the Route53 entries with service instances as they are registered/deregistered within ECS.
While a step in the right direction from ALBs, it suffers from the problems associated with DNS propagation. Failed instances can take a while to be updated in Route53, and for their IP to be purged from DNS caches operating at various levels.
ECS Service Connect (launched in 2022) takes this a step further but instead of using DNS as the protocol for disseminating the catalogue, it uses a combination of sidecar proxies and APIs. Every request that applications make to services being managed by Service Connect goes through a sidecar (powered by a proxy called Envoy), which goes straight to the CloudMap API to determine the available endpoints for that service.
DNS propagation is mitigated in this approach, as the Envoy sidecar performs a combination of active health checks against target endpoints, and regularly polls CloudMap for the latest set of healthy instance endpoints.
ECS Service Connect is a great fit for situations where you’d benefit from a self-managed basic service discovery mechanism, where all of your services are running within ECS. It can help when you’re new to service discovery and service meshes, or simply when you want a common problem like connecting services to be managed by the platform so you can focus on developing and delivering.
To enable Service Connect is relatively straightforward, and with a redeployment can be performed on existing applications. From the “Update Service” interface you can enable the “Service Connect” checkbox, then fill out the details as shown below. Select “Client side only” when your application purely consumes others, and “Client and server” when it not only consumes services but provides services to other upstream applications.
Be wary that when you do this, Service Connect may attempt to perform additional tasks such as creating log groups. Ensure you update your execution role with any newly required policies otherwise your application will fail to deploy and likely get stuck in a crash loop.
If your applications are already managed by AWS CDK, then you can add the following lines to enable Service Connect and have CDK automatically update your execution policies (if you don’t manually manage them):
const yelbAppServerService = new ecs.FargateService(this, "yelb-appserver-service", {
...
serviceConnectConfiguration: {
services: [{
portMappingName: "yelb- appserver",
dnsName: "yelb-appserver",
port: 4567
}],
namespace: dnsNamespace.namespaceName,
});If you would like to see a working demo of ECS Service Connect, then you can check out my GitHub repository here:
https://github.com/foyst/ecs-service-connect-demo
It extends a popular repository called Yelb, which demonstrates various architectural approaches and deployment patterns for building a webapp and accompanying backend.
ECS Service Connect allows you to get up and running with a service mesh pretty quickly, but it does have some drawbacks. With the simplicity and unobtrusive approach, you lose a lot of the control and extensibility you get with other platforms such as AppMesh, and full-fat container orchestration frameworks such as Kubernetes.
For example, the Envoy sidecars are not updated when new applications are deployed into the cluster. For the most part this typically isn’t a problem, as most applications would need a code update to make use of new services (and the deployment of the updated ECS Service would update the sidecar). However, it can make for awkward race conditions when deploying multiple services at the same time that rely on each other (or when deploying a whole environment from scratch).
Also, as mentioned Service Connect only works within ECS. Although it can connect services across ECS Clusters, if you want to communicate with services outside of ECS in a consistent manner then you need to look at something like AppMesh.
I would definitely recommend trying out Service Connect to see if it’s a good fit for you. It can definitely shift the arduous job of managing a service mesh, but as with all simple managed services evaluate not only if it’s a good fit for you now but if it’ll handle your future needs too.
