Build an Amazon ECS Anywhere cluster that can connect to your Amazon VPC

This article is a translation of Building an Amazon ECS Anywhere home lab with Amazon VPC network connectivity.

Since 2014, Amazon Elastic Container Service (Amazon ECS) has helped his AWS customers orchestrate the deployment of containerized applications across a variety of compute environments. Previously, Amazon ECS could only be used with AWS-managed compute hardware such as Amazon Elastic Compute Cloud (Amazon EC2) instances, AWS Fargate, AWS Wavelength, and AWS Outposts. With the general availability of Amazon ECS Anywhere (ECS Anywhere), you can now use your own compute hardware as the capacity of your Amazon ECS cluster.

This blog will walk you through the process of building a home lab to perform his ECS tasks on your ECS Anywhere cluster. Home Labs can use the Amazon ECS API to launch tasks on their compute hardware. You can also use AWS Site-to-Site VPN to access remote Amazon Virtual Private Cloud (Amazon VPC) from your local network, or access your local network from a remote Amazon VPC. Site-to-Site VPN allows local tasks performed in a local Amazon ECS Anywhere cluster to communicate with Amazon Relational Database Service (Amazon RDS) within Amazon VPC, Amazon ElastiCache, or other AWS fully managed services. It makes it possible. In addition, local machines can receive inbound connections from services hosted on Amazon VPC, such as Application Load Balancer (ALB) and Network Load Balancer (NLB).

architecture

To understand how ECS Anywhere works, we need to look at the components that work with ECS Anywhere. Some components are required for the hardware and virtual machines used for ECS Anywhere to work as part of an Amazon ECS cluster.

The first component is the agent that connects to AWS Systems Manager. When you install the AWS Systems Manager Agent (SSM Agent), you specify a secret activation code for the agent to register itself with her AWS Systems Manager. The agent uses the activation code to register the hardware device as a managed instance and downloads the secret key for that managed instance. From this point on, you can assign an AWS Identity and Access Management (IAM) role to your managed instance and automatically receive her IAM credentials for that her IAM role. This her IAM role is essential for the instance to make the necessary communications with other AWS services such as Amazon ECS.

The following required component launches a container on a managed host with Docker. One of the containers that Docker launches is the Amazon ECS Agent. This agent uses her managed instance's IAM role to connect to the Amazon ECS control plane in the AWS Region. Once connected, you can be instructed by the Amazon ECS control plane which task or container to launch. The Amazon ECS agent can also send telemetry information for tasks such as container lifecycle and health to the control plane.

The next thing to understand is how the network works between Amazon VPCs in the AWS Region and the local network running the ECS Anywhere cluster.

On the Amazon VPC side, you can use AWS Site-to-Site VPN to provide his fully managed VPN gateway. This gateway is configured to add routes to your Amazon VPC route table. This route sends all traffic destined for the CIDR range of your on-premises network through the VPN gateway. On-premises, you set up your own VPN gateway and routing that is paired with her VPN gateway on the cloud side, which causes all traffic generated on each node to the CIDR range of the Amazon VPC to be on the on-premises side. Let it flow to her VPN gateway.

In this configuration, any resource in your on-premises network can interact with resources in your VPC using the private IP address of the resource hosted by Amazon VPC. For example, an on-premises Raspberry Pi can send traffic to an Amazon RDS instance running in an Amazon VPC. In addition, resources hosted in Amazon VPC can use private IP addresses to communicate with on-premises resources. In the figure above, the Amazon VPC-hosted NLB uses the Raspberry Pi's private IP address to communicate with the Raspberry Pi.

The important thing to keep in mind here is that if your on-premises device can connect to the Internet, you can communicate with many AWS services without a VPN configuration like the one above. It includes Amazon ECS, Amazon DynamoDB, Amazon Simple Storage Service (S3), and many AWS services that are globally accessible through public service endpoints. Additional network settings are only required for AWS services associated with a particular Amazon VPC.

Home lab hardware construction

Since ECS Anywhere is designed to work with a variety of devices and operating systems, there are many hardware choices for building a home lab. You may already have the hardware you want to use for your Amazon ECS cluster, but maybe you're looking for a reason to buy a new one !? This section uses a Raspberry Pi device to ECS Anywhere. Introducing the parts list for building a home lab. Other alternatives are available for these parts, but this list will be a good starting point for building a home lab.

The Raspberry Pi has several important advantages. Raspberry Pi can build large clusters on a relatively low budget. In addition, the Raspberry Pi is an ARM-based device, so it's perfect for testing his ARM builds at home if all your development devices are Intel-based. Finally, the Raspberry Pi is a low-power device that can operate with natural cooling, making it ideal if you don't want to have a noisy fan in your office.

You can use the following components for your compute environment:

To get these individual devices to work as a neat, self-managed cluster, also consider the following components:

Software settings

After assembling all the hardware, you need to make a few software settings. Specifically, for Raspberry Pi, use the Raspberry Pi imager to install the OS on his SD card. This home lab example uses Ubuntu 20. Add the public key as an SSH accessible user. Then use SSH to run the command inside the Raspberry Pi. You may need to make some adjustments to your firmware settings.

First, memory cgroups must be enabled in the boot config for the device to perform Docker and Amazon ECS tasks. This may not be enabled by default, but it is required for Docker to work properly when you set hard or soft memory limits in the Amazon ECS task definition. This can be done by adding cgroup_enable=memory to the /boot/firmware/cmdline.txt file.

In addition, for Raspberry Pi, you may want to reduce noise from the cluster. Genuine Power over Ethernet Hats have a cooling fan that attempts to maintain temperatures well below exactly what is required. If you have a heatsink installed, under typical conditions the device will naturally cool to a temperature well below the maximum operating temperature. You can prevent the fan from turning on until the device reaches 68 ° C by setting it in /boot/firmware/usercfg.txt .

dtoverlay=rpi-poe dtparam=poe_fan_temp0=68000 dtparam=poe_fan_temp1=72000 dtparam=poe_fan_temp2=76000 dtparam=poe_fan_temp3=80000

You can monitor the temperature of your Raspberry Pi by running cat /sys/class/thermal/thermal_zone0/temp to keep it at the right temperature even under load. Unless the ambient temperature is too high, the heatsink will naturally cool the Raspberry Pi for extended periods of high load. If the temperature exceeds 68 ° C, the fan will operate for forced cooling.

After these initial adjustments, you can use the AWS Management Console to get activation commands to run on each device. This command registers the device as the capacity of the Amazon ECS cluster.

This script automatically installs and configures the SSM Agent, Docker, and Amazon ECS Agent, so no further input is required. You can see the device appear in AWS Systems Manager Fleet Manager when the script finishes running. It also displays some details such as the local IP address in your home network.

One of the useful features of Fleet Manager is connecting to a managed instance. This also works for devices that have only a private IP address behind NAT. This is because his SSM agent on the host opens a control channel to his SSM. You can use this to monitor a managed instance or start a session in AWS SSM Session Manager. Select “Start session” to open the shell in your browser. When you start htop , the process tree is displayed, and you can see that the SSM agent is starting the worker that runs the shell.

AWS Site-to-Site VPN Settings

There are several types of network approaches that you can use for your Amazon ECS cluster. The easiest approach to inbound traffic is to set up port forwarding on your home router. This allows traffic to be sent to your home IP address and forwarded to one of the devices on your network.

What if I want to connect to a resource in an Amazon VPC? In large on-premises environments, you can use AWS Direct Connect to connect directly to AWS. But for home labs, this is not ideal. You can use AWS Site-to-Site VPN as an alternative to lower costs. Then you can assign one of your home lab's Raspberry Pis to run strongSwan for IPsec VPN and act as a VPN gateway. You can access the Amazon VPC console to create a VPN gateway on the AWS side and download instructions for configuring your on-premises VPN gateway.

You can follow the downloaded instructions to set up an IPsec VPN tunnel between your home network and your Amazon VPC. Run ipsec status to verify that the connection is established. In this case you can see the output of the VPN connection configured between the 192.168.1.0/24 network and the Amazon VPC 10.0.0.0/16 network.

Next, you need to set up a VPN gateway route on the other Raspberry Pi device. This route configuration tells you that other Raspberry Pi devices must use the VPN Raspberry Pi's local IP address as the gateway to deliver all traffic destined for the Amazon VPC's IP range to the VPC. In the example below, the local IP address of the VPN Raspberry Pi is 192.168.1.196 .

sudo route add -net 10.0.0.0/16 gw 192.168.1.196

Use Fleet Manager again to check network connectivity. Open a session in one of your Raspberry Pis and ping your private IP address if you have an Amazon EC2 instance running inside your Amazon VPC.

In the screenshot above you can see the result of ping from a Raspberry Pi (on my desk) connected to my home network in New York. The destination address for ping is an Amazon EC2 instance running inside a VPC in the eastern United States (Northern Virginia). Looking at the ping results, it takes less than 11ms to make a round trip from New York to the eastern United States (Northern Virginia). This result may vary depending on your network environment and the distance to the AWS Region where you provisioned your Amazon VPC.

Launching load balancing workloads on your home cluster

Once you have all the necessary hardware and software setups, launch a test workload on your cluster to make sure everything works fine. You can use the new launch type, EXTERNAL , to launch Amazon ECS services in your home lab cluster. Both the task definition and the Amazon ECS service must be created with the EXTERNAL launch type.

Now let's look at an example of a redis service that you can also see in the screenshot below. Redis is a stateful service that requires information to be persisted to disk. Note that stateful services need to run in the same place where the data is stored. ECS Anywhere can resolve this using the Task Placement Restrictions feature. Task placement restrictions can be used to pin a workload to a particular device. In this case, the Amazon ECS instance attribute redis=true and the "task placement restrictions" attribute:redis=true are used to pin the redis task to a particular Raspberry Pi.

When a task is launched, you can retrieve the task's metrics and logs as if it were running on an Amazon EC2 instance in his VPC.

If you are hosting a service that needs to receive traffic from the Internet, you may need a load balancer. The advantage of hosting your load balancer in his AWS region is that you don't have to configure DNS to point to your home network address. The load balancer can use her IP address to handle traffic and her home network is protected by her VPN connection.

To achieve this configuration, he needs to create an NLB or ALB in his AWS Site-to-Site VPN-configured VPC. If your load balancer is in this VPC, you can send traffic through the VPN gateway to the private IP address of the device in your home network. At this time, you need to manually add the private IP address and port combination of your home lab device to your load balancer. Fortunately, devices on your home network often have static IP addresses, so this configuration should be stable.

Once each load balancer target is Healthy, let's send traffic to her DNS name on the load balancer. In this case the response is a simple HTML page returned by his small Node.js application, showing the hit counters stored in Redis.

summary

With ECS Anywhere, you can orchestrate experiments using containers in your home lab from the cloud side. You don't have to run the control plane on your device. Instead, you can use your device purely as application capacity. ECS Anywhere allows you to define the desired state of software on your device and leave the placement of tasks to the host to the Amazon ECS control plane. Amazon ECS monitors the tasks and restarts them if necessary. In addition, Fleet Manager allows you to connect and control managed devices in your private network from anywhere over the Internet. This is true even if those devices are behind her NAT.

This article alone does not convey the appeal of ECS Anywhere, so be sure to check out the documentation and release announcements. He will also showcase his ECS Anywhere in a live stream of Containers from the Couch in June. We will be happy to answer any questions you may have, so please join us.

The translation was done by Solutions Architect Kaji. The original text is here.