In this tutorial you'll use devx to generate Terraform IaC with security best-practices to run workloads on AWS.
Things you get out of the box:
- secrets management with SSM parameter store or secrets manager
- least privilege ECS task IAM policies
- buttoned up security groups
- TLS configuration for application loadbalancer
- task logging with cloudwatch
- application loadbalancer healthchecks
Sections:
Prerequisites
devx is installed on your machine Use homebrew
brew tap stakpak/stakpak
brew install devxOr
download the latest binary
AWS cli configured on your machine
You have to make sure your Terraform IAM role/user have enough permissions to manage ECS clusters, ALBs, Security Groups, SSM Parameters, VPCs, Subnets, IAM Roles, IAM Policies, etc...
If you're just trying this out, make sure you're using an empty account and give Terraform Administrator Access, but in production you have to reduce the permission to just what you need.
AWS Route53 is managing your domain
Terraform and Git are installed on your machine
Building your infrastructure (Platform Engineers)
This project will house the definitions of common infrastructure components like your VPC, and any configurations that you want to be re-usable by your developers.
1) Create a new project
mkdir platform
cd platform
devx project init stakpak.dev/demo-ecs
update your module name
// Add your own module name
{
module: "stakpak.dev/demo-ecs"
cue: lang: "v0.6.0-alpha.1"
deps: "github.com/stakpak/devx-catalog": {
// You can optionally pin dependencies to a specific git tag or commit
v: "v2.0.0-alpha.117"
}
}
2) Create a VPC, Cluster, and ALB
You first have to create a stack
touch stack.cue
package main
import (
"stakpak.dev/devx/v1"
"stakpak.dev/devx/v1/traits"
)
stack: v1.#Stack & {
components: {
// Virtual private network
network: {
traits.#VPC
vpc: {
name: "tutorial"
cidr: "10.0.0.0/16"
subnets: {
private: ["10.0.1.0/24", "10.0.2.0/24"]
public: ["10.0.101.0/24", "10.0.102.0/24"]
}
}
}
// ECS cluster
cluster: {
traits.#ECS
ecs: {
name: "tutorial"
capacityProviders: fargateSpot: enabled: true
vpc: network.vpc
}
}
// Application load balancer
gateway: {
traits.#Gateway
gateway: {
name: "tutorial"
public: true
// Replace this with you own domains
// You have to move your domain name servers to Route53 for this to work
addresses: [
"tutorial-aws-ecs.stakpak.dev",
]
// We want our gateway to listen on port 80 and 443
listeners: {
http: {
port: 80
protocol: "HTTP"
}
https: {
port: 443
protocol: "HTTPS"
}
}
}
}
}
}
then tell devx how to build this stack for the prod environment
touch builder.cue
package main
import (
"stakpak.dev/devx/v2alpha1"
tf "stakpak.dev/devx/v1/transformers/terraform"
"stakpak.dev/devx/v1/transformers/terraform/aws"
)
builders: v2alpha1.#Environments & {
prod: {
// Set terraform output directory to "./deploy"
drivers: terraform: output: dir: ["deploy"]
components: {
// Add this component to customize the generated Terraform AWS providers
tfprovider: {
// This reserved field contains all the low-level resources
$resources: {
// The name of the resource doesn't matter
// We use the tf.#Terraform schema to make sure we define valid Terraform configurations
terraform: tf.#Terraform & {
provider: aws: region: "us-west-1"
}
}
}
// The aws.#AddGateway transformer requires vpc data as input
// We set the vpc here because we cannot reference external components from within a transformer
network: _
gateway: aws: vpc: network
}
flows: {
// This flow expects a VPC trait and generates Terraform code to add an AWS VPC
"terraform/add-vpc": pipeline: [aws.#AddVPC]
// This flow expects an ECS trait and generates Terraform code to add an AWS ECS cluster
"terraform/add-ec-cluster": pipeline: [aws.#AddECS & {
aws: {
// Add your AWS region
region: "us-west-1"
// Add your AWS account id
account: "000000000000"
}
}]
// This flow expects an Gateway trait and generates Terraform code to add an AWS ALB
"terraform/add-gateway": pipeline: [aws.#AddGateway & {
// Allow the transformer to create Route53 DNS records
createDNS: true
// Allow the transformer to create TLS certificates
createTLS: true
}]
}
}
}
Make sure to set your AWS account id and region.
3) Generate configurations
devx build prod
Terraform resources generated for the network:
Terraform resources generated for the cluster:
Terraform resources generated for the gateway:
4) Apply configurations
cd deploy
terraform init
terraform plan
if all looks good
terraform apply --auto-approve
> Apply complete! Resources: 28 added, 0 changed, 0 destroyed.
5) Environment builders
Define the builders that your developers will use to build their configurations.
cd ..
mkdir platform
touch platform/platform.cue
package platform
import (
"stakpak.dev/devx/v2alpha1"
"stakpak.dev/devx/v2alpha1/environments"
"stakpak.dev/devx/v1/transformers/terraform/aws"
infra "stakpak.dev/demo-ecs:main"
)
Builders: v2alpha1.#Environments & {
prod: environments.#ECS & {
drivers: terraform: output: dir: ["deploy"]
config: {
aws: {
// Add your AWS region
region: "us-west-1"
// Add your AWS account id
account: "000000000000"
}
vpc: name: infra.stack.components.network.vpc.name
ecs: {
name: infra.stack.components.cluster.ecs.name
launchType: "FARGATE"
}
secrets: service: "ParameterStore"
gateway: infra.stack.components.gateway.gateway
}
// Add this component to configure the generated Terraform AWS providers
// We re-use the provider definition of our infra stack
components: tfprovider: infra.builders.prod.components.tfprovider
// Add an extra flow to auto-generate secrets in SSM parameter store
// The aws.#AddSSMSecretParameter transformer use random_password to generate a 32-character string
flows: "terraform/autogenerate-secrets": pipeline: [aws.#AddSSMSecretParameter]
}
}
Make sure to set your AWS account id and region.
6) Commit and push your code
git init
cat > .gitignore <<EOL
cue.mod/pkg
deploy/.terraform
deploy/terraform.tfstate
EOL
git add .
git commit -m "init devx"
git remote add origin https://github.com/stakpak/devxtut-aws-ecs.git
git branch -M main
git push -u origin main
Deploying apps (Developers)
1) Create a new project
mkdir myapp
cd myapp
devx project init stakpak.dev/demo-ecs/apps/api
update your module name and add your platform repository as a dependency
// Add your own module name
{
module: "stakpak.dev/demo-ecs/apps/api"
cue: lang: "v0.6.0-alpha.1"
deps: {
"github.com/stakpak/devx-catalog": {
// You can optionally pin dependencies to a specific git tag or commit
v: "v2.0.0-alpha.117"
}
// Your platform repository
"github.com/stakpak/devxtut-aws-ecs": {}
}
}
2) Define what your service needs
We create a stack for our service
touch stack.cue
package main
import (
"stakpak.dev/devx/v1"
"stakpak.dev/devx/v1/traits"
"stakpak.dev/demo-ecs/platform"
)
stack: v1.#Stack & {
// Components can be workloads, resources, CI pipeline etc...
components: {
// Add secrets
commonSecrets: {
// The Secret trait let's you add references to your secrets
traits.#Secret
secrets: apisecret: name: "API_SECRET"
}
// Define your workload
api: {
// The Workload trait runs some containers
traits.#Workload
containers: default: {
image: "hashicorp/http-echo:alpine"
command: ["-listen=:8000", "-text=hello"]
resources: requests: {
cpu: "256m"
memory: "512M"
}
env: {
API_SECRET: commonSecrets.secrets.apisecret
NAME: "Devopzilla API"
}
}
// The Exposable trait exposes ports on your containers
traits.#Exposable
endpoints: default: ports: [
{
port: 8000
},
]
}
// Expose your workloads through a public HTTPS ingress
route: {
// The HTTPRoute trait exposes some endpoints through your ingress gateway
traits.#HTTPRoute
http: {
listener: "https"
hostnames: [
"tutorial-aws-ecs.stakpak.dev",
]
rules: [{
match: path: "/*"
backends: [
{
name: api.appName
endpoint: api.endpoints.default
containers: api.containers
port: 8000
},
]
}]
}
}
// Redirect HTTP traffic to HTTPS
redirect: {
traits.#HTTPRoute
http: {
listener: "http"
hostnames: route.http.hostnames
rules: [{
match: path: "/*"
redirect: {
scheme: "https"
port: 443
}
}]
}
}
}
}
// Use the central platform builders
builders: platform.Builders
3) Generate configurations
devx build prod
Terraform resources generated for the stack:
4) Apply configurations
cd deploy
terraform init
terraform plan
if all looks good
terraform apply --auto-approve
> Apply complete! Resources: 10 added, 0 changed, 0 destroyed.
5) Test the workload
curl "https://tutorial-aws-ecs.stakpak.dev"
> hello
Clean up
Destroy the workload
cd myapp/deploy
terraform destroy --auto-approve
> Destroy complete! Resources: 10 destroyed.
Destroy the infrastructure
cd platform/deploy
terraform destroy --auto-approve
> Destroy complete! Resources: 28 destroyed.
Why not just use Terraform modules?
1) Stack definitions are technology-agnostic. You don't have to know much about ECS to deploy a workload, and if you swap transformers you will be able to deploy with Docker Compose, or even Kubernetes
2) You can package and re-use other types of configurations in devx modules e.g. CI pipelines, policies, workflows, shared variables, validation rules, and application configurations
3) Nested configurations like the task definition are validated much earlier at generation