AWS Application Deployment

PackyTrace has an AWS demonstration deployment in eu-central-1, last documented as verified on 2026-08-01. The application runs on one EC2 instance: Terraform creates the infrastructure, cloud-init prepares the host, Docker Compose runs the containers, and Caddy exposes the application over HTTPS. A single box keeps the MVP inexpensive and operationally simple while preserving service boundaries inside the container network.

Technology Role
Terraform Provisions the VPC, subnet, security group, EC2 instance, IAM, Elastic IP, EBS storage, and SSM parameters
Amazon EC2 and Docker Compose Run the application services and supporting containers
Caddy Provides automatic HTTPS and handles static hosting and API proxying with a small configuration
Amazon EBS and Elastic IP Preserve data and provide a stable public address
AWS Systems Manager Provides shell access and encrypted configuration without opening SSH
GitHub Actions and GHCR Build and store versioned container images
Postgres, Redpanda, and Keycloak Provide persistence, Kafka-compatible messaging, and authentication
Prometheus and Grafana Collect metrics and display SLO dashboards

Only HTTP and HTTPS are open to the internet. All services, Postgres, Redpanda, and Keycloak remain private inside Docker Compose. This page covers deployment and routine operations; incident diagnosis belongs in the separate Runbook.

Examples use a profile named packytrace-deployer and region eu-central-1. Configure that profile with only the permissions needed by the Terraform and operations in this guide; do not use an account-wide administrator identity for routine deployment. Override the Makefile's legacy default with AWS_PROFILE=packytrace-deployer. Run Terraform from deployment/aws/terraform only for first-time infrastructure work. Day-to-day deploys run from the repository root through the Makefile.

Prerequisites (one-time)

  • AWS CLI configured with a least-privilege packytrace-deployer profile.
  • terraform (>= 1.10) and the gh CLI installed.
  • deployment/aws/terraform/terraform.tfvars filled in from the example (domain + GHCR token). Never commit it.

First deploy

  1. Publish the images once:

    gh workflow run deploy-images.yml

  2. Apply the infrastructure:

    cd deployment/aws/terraform
terraform init
terraform apply

  3. Add the DNS A records at your registrar (Namecheap; DNS is manual, not Route53). The apex/app record is printed by Terraform; the two observability subdomains point at the same Elastic IP:

    terraform output dns_record       # the app record (apex or your chosen host)
terraform output -raw public_ip   # the IP the grafana/prometheus records also use
Host Type Value
(app, e.g. apex) A the box Elastic IP
grafana A the same Elastic IP
prometheus A the same Elastic IP

Without the grafana/prometheus records, Caddy cannot issue their TLS certificates and those dashboards stay unreachable even though the containers are running.

  1. Watch the first boot until containers are healthy:

    $(terraform output -raw ssm_session_command)
sudo tail -f /var/log/packytrace-bootstrap.log
sudo docker compose -f /opt/packytrace/docker-compose.yml ps

  2. Open the app:

    terraform output -raw app_url

Deploy a new version

This is the routine path. Merging does not change the box; it only publishes images. The box updates only when you bump the tag and redeploy.

  1. Merge to main. Semantic-release creates Git tag vX.Y.Z; the image workflow publishes GHCR tags X.Y.Z and latest. Note the new Git version with git describe --tags --abbrev=0, then omit its leading v when selecting the image.

  2. Deploy the new version from the repository root:

    make aws-deploy TAG=X.Y.Z

Use the image tag without a leading v or colon. Correct: latest or 1.11.0. Incorrect: v1.11.0 or :latest.

To redeploy whatever tag is already stored in SSM:

    make aws-deploy
  1. Verify before calling it done:
    make aws-status

Then open the app URL, hard refresh the browser, and run one physical barcode scan.

Roll back

Use the same mechanism with an actual previously published immutable tag. First list recent tags, select the version that was verified before the failed deployment, and then deploy it:

git tag --sort=-version:refname | head -n 5
AWS_PROFILE=packytrace-deployer make aws-deploy TAG=<version-without-leading-v>

For example, Git tag v1.10.2 corresponds to image tag 1.10.2. Confirm that the selected image tag exists in GHCR before deployment.

Caveat: this rolls back code, not the database. Services apply migrations forward at startup with no automatic down step, so a rollback is only clean if the migration was backward-compatible. Keep migrations additive (nullable/defaulted columns, deprecate before delete) so a rollback never strands the schema.

Runtime configuration

Secret flow, recipe-provider rollout and producer-catalog configuration are maintained in AWS Runtime Configuration.

Observability dashboards (Grafana + Prometheus)

The AWS demonstration box runs the full observability stack, fronted by Caddy on two subdomains. See the Observability page for metric and dashboard semantics; this section covers AWS access.

Dashboard URL Auth
Grafana 'https://grafana.' login admin / SSM grafana_admin_password; opens straight on the PackyTrace SLOs & QAS dashboard
Prometheus 'https://prometheus.' HTTP basic auth, admin / SSM prometheus_password (Prometheus has no native auth, so Caddy gates it)

Both passwords are generated by Terraform and stored as SSM SecureStrings: never put them in terraform.tfvars. Read them with:

aws ssm get-parameter --name /packytrace/secret/grafana_admin_password \
  --with-decryption --query Parameter.Value --output text --profile packytrace-deployer --region eu-central-1
aws ssm get-parameter --name /packytrace/secret/prometheus_password \
  --with-decryption --query Parameter.Value --output text --profile packytrace-deployer --region eu-central-1

Notes:

  • A terraform apply is required to add or change the observability stack. Inspect the plan: depending on the affected resource and lifecycle configuration it may update in place or replace the instance. The everyday make aws-deploy image-pull path does not pick up these changes.
  • Grafana's anonymous viewing is off in the AWS demonstration (it is on only in the local development stack).
  • Prometheus publishes no host port; it is reachable only through Caddy. Its basic-auth bcrypt hash is rendered into the Caddyfile on the box by render-config.sh, so the $ characters never round-trip through docker-compose interpolation.

Everyday operations

Task Command
Deploy a tag make aws-deploy TAG=X.Y.Z
Redeploy current tag make aws-deploy
Check deployed state make aws-status
Seed demo analytics make aws-seed
Open shell $(terraform output -raw ssm_session_command)
Check containers sudo docker compose -f /opt/packytrace/docker-compose.yml ps
Stream logs sudo docker compose -f /opt/packytrace/docker-compose.yml logs -f
Re-render config sudo /opt/packytrace/render-config.sh
Grafana password make grafana-password
Prometheus password make prometheus-password
Stop spending, keep data make aws-stop
Resume after a stop make aws-start
Plan infrastructure changes terraform plan from deployment/aws/terraform

make aws-stop halts the instance-hour charge; the /data volume and Elastic IP persist, so make aws-start brings the box back on the same IP with no DNS change and containers restart on their own (give it a minute, then make aws-status). The EBS volumes and the public IPv4 continue to incur charges while the instance is stopped. The two '*-password' helpers read the generated dashboard credentials from SSM (see Observability dashboards).

make aws-seed runs the seed-profile one-shot on the box (over SSM), publishing demo facts for the four demo brands through the deployed pipeline so the brand dashboard has data to show. The aggregates appear after the next flush. Run it once on a fresh demonstration environment; re-running will not re-emit an already-published window (the same caveat as the local make seed).

If something breaks

  • Boot looks stuck: sudo tail -f /var/log/packytrace-bootstrap.log.
  • A service is unhealthy: ... compose ps to see which, then '... compose logs -f '.
  • Config/secret looks wrong: re-run sudo /opt/packytrace/render-config.sh, then ... compose up -d.
  • UI did not change: make aws-deploy forces the one-shot web-app copy step and restarts Caddy. If it still looks old, hard refresh or use a private window.
  • Data persists across ordinary container restarts: state lives on the /data EBS volume. Instance replacement, volume operations, manual deletion and terraform destroy can still lose data. Verify backups and inspect every Terraform plan before an infrastructure change.

Known limits

  • No high availability, one box, single point of failure.
  • No automated backups or tested restore procedure. Until recovery objectives, scheduled snapshots and restore tests exist, this environment is not production-ready for irreplaceable data.
  • Keycloak still ships demo setup that must be replaced before production users.