In Amazon Elastic Container Service (ECS), HealthCheck is a mechanism for monitoring the health status of containerized applications running in tasks. It helps ensure that only healthy containers (with health check passing) serve traffic and unhealthy containers are replaced automatically. Configuring the ECS Managed Healthcheck is crucial for maintaining the availability, reliability, and scalability of your containerized applications running on ECS Fargate.
So that we’re on the same page, Amazon ECS is a fully managed container orchestration service that helps development teams deploy, manage, and scale containerized applications. AWS Fargate is a technology that can be used with Amazon ECS to run containers without managing servers or clusters of Amazon EC2 instances. AWS Fargate supports the Amazon ECS Managed HealthCheck, a built-in health check provided by Amazon ECS. With this health check, ECS periodically sends a ping (HTTP, HTTPS, or gRPC) to a specific endpoint into the container, and if the container responds successfully, it is considered healthy.
In this note, I explain how to (a) add a health check to the AWS Fargate task and (b) monitor the health check logs with Amazon CloudWatch.
I built on the concepts discussed in deploying an Amazon ECS service using Terraform. Hence, it will be helpful if you read that note first. If you are interested and want to follow along, here is a link to my code repository on GitHub: kunduso/add-aws-ecr-ecs-fargate.
Step 1: Add HealthCheck to the AWS Fargate task
To enable HealthCheck, I first added a file inside the Docker image at a specific location (/route/healthcheck.js), and then updated the healthCheck configuration in the Amazon ECS task definition to run the curl command.
The application is hosted on port# 8080, and a health check file exists in the folder. Also, if you examine, the contents of the CURL command are written into the docker logs folder (/proc/1/fd/1). The logs that are written into the path /proc/1/fd/1 are redirected to Amazon CloudWatch because of the log configuration settings in the Amazon ECS task definition.
Step 2: Monitor logs with Amazon CloudWatch
Once the AWS Fargate task is deployed and after the service comes up, you’ll see the container Health status as “Healthy.”
On the AWS Console, I searched for Amazon CloudWatch, navigated to Log Groups (under Logs), and chose /amazon-ecs/app-6/log. That is the name of the Amazon CloudWatch log group created as part of infrastructure deployment, which was discussed in detail at deploy-ecs-infrastructure. Under the log group, there were two log streams, one for each container, with log messages such as the one below:
The container’s health check command posts a new message to the log stream every 30 seconds. And that is how to enable and monitor health checks in AWS Fargate tasks.
There are some necessary features of the ECS Managed HealthCheck in AWS Fargate tasks that you must understand before using it:
1. Configuration: The health check is configured in the healthCheck section of the AWS Fargate task definition. You can specify the protocol (HTTP, HTTPS, or gRPC), port, path, interval, timeout, and other settings.
2. Load Balancing: When you use an Elastic Load Balancing (ELB) service with AWS Fargate tasks, the load balancer routes traffic only to healthy containers based on the health check status. Unhealthy containers are automatically removed from the load balancer’s target group.
3. Container Replacement: If a container becomes unhealthy (based on the health check), Amazon ECS automatically stops the unhealthy task and starts a new task to replace it. This feature ensures that your application remains available and responsive.
4. Auto Scaling: When using Auto Scaling with AWS Fargate service, health checks help determine when to scale out (add more tasks) or scale in (remove tasks) based on the number of healthy and unhealthy containers.
5. Monitoring and Alerting: You can monitor the health check status of your containers using Amazon CloudWatch metrics or other monitoring tools, which can trigger alerts if containers become unhealthy.
Please note that as of May 2024, ECS Managed Healthcheck is the only health check mechanism available for AWS Fargate tasks. AWS Fargate does not support Docker container health checks defined in the Dockerfile. Also, please note that not all Docker images come with the CURL utility, and you might have to add an installation step to the Dockerfile.
I hope this note will be helpful when you configure a health check for your Amazon ECS Fargate task the next time. Let me know if you have any questions or suggestions.
My Devops Journal 