Cloud Cost Optimiser

Overview

This project demonstrates the transformation of a local Python/React application into a fully production-ready cloud architecture on AWS. It solves a critical financial problem, untracked cloud spend, by identifying idle EC2 instances, oversized databases, and orphaned storage volumes.

Beyond the utility itself, this project serves as a comprehensive example of the DevOps lifecycle, featuring Infrastructure as Code (Terraform), Serverless Compute (ECS Fargate), and a robust CI/CD pipeline (GitHub Actions).

The Problem

In many AWS environments, costs spiral out of control due to forgotten resources.

• Idle Compute: Dev servers left running 24/7.
• Oversized Instances: Using t3.large when t3.micro would suffice.
• Orphaned Storage: EBS volumes left behind after instances are terminated.
• Stale Backups: Snapshots retained for years unnecessarily.

This tool automates the discovery of this waste, calculating potential savings and presenting them on a dashboard.

Architecture & Cloud Transformation

I moved the application from a monolithic local setup to a decoupled 3-tier production architecture.

┌─────────────────────────────────────────────────────────────────┐
│                        USER INTERFACE                           │
│  React Dashboard (Nginx) - Shows costs & recommendations        │
└────────────────────────────┬────────────────────────────────────┘
                             │ HTTPS/REST (via ALB)
┌────────────────────────────▼────────────────────────────────────┐
│                         API LAYER                               │
│  FastAPI (ECS Fargate) - Business logic & AWS Scanning          │
└────────────────────────────┬────────────────────────────────────┘
                             │
          ┌──────────────────┼──────────────────┐
          │                  │                  │
┌─────────▼─────────┐ ┌─────▼──────┐ ┌────────▼────────┐
│   EC2 Scanner     │ │ EBS Scanner│ │  RDS Scanner    │
│ (CloudWatch)      │ │ (Volumes)  │ │  (Databases)    │
└─────────┬─────────┘ └─────┬──────┘ └────────┬────────┘
                            │
┌────────────────────────────▼────────────────────────────────────┐
│                    DATABASE (RDS PostgreSQL)                    │
│  • Scan history & Recommendations (Private Subnet)              │
└─────────────────────────────────────────────────────────────────┘

Key Technical Decisions

1. Compute: ECS Fargate (Serverless)

I migrated the compute layer to AWS Fargate to remove the operational overhead of managing EC2 instances. Containers are deployed across multiple Availability Zones (AZs) for high availability.

• Why: No OS patching, automatic scaling, and pay-per-use pricing.

2. Networking: Application Load Balancer

An ALB sits at the edge, handling all incoming traffic on Port 80. It performs path-based routing:

• /api/* → Routes to the Backend container (FastAPI).
• /* → Routes to the Frontend container (Nginx/React).
• Benefit: Decouples the frontend and backend while exposing a single unified endpoint to the user.

3. Storage: Private RDS

The PostgreSQL database runs in a Private Subnet, completely isolated from the public internet. It allows traffic only from the Backend container via strict Security Group rules.

4. Infrastructure as Code (Terraform)

The entire environment (VPC, Subnets, ALB, ECS, RDS) is provisioned via Terraform (/terraform folder). This ensures the infrastructure is reproducible, versioned, and can be destroyed/recreated in minutes.

Security Strategy ("Zero Keys")

I followed a strict security posture to prevent credential leakage.

• IAM Task Roles: The application uses IAM roles attached to the ECS Task to interact with AWS services (Cost Explorer, EC2 Read-Only). No long-lived access keys are ever stored in the container or code.
• Secrets Management: Database credentials are auto-generated by Terraform and stored in AWS Secrets Manager. They are injected into the containers only at runtime as environment variables.
• Local Development: Locally, the app falls back to the standard boto3 credential chain (reading from your .env file in Docker or ~/.aws/credentials on your host), ensuring that dev secrets stay on the developer's machine.

CI/CD Pipeline

I implemented a robust automation pipeline using GitHub Actions.

1. Build: Creates multi-arch Docker images (AMD64) for frontend and backend.
   • Frontend Optimization: Uses a multi-stage build to compile the React app and serve it via Nginx, keeping the image lightweight.
2. Push: Uploads versioned images to Amazon ECR.
3. Deploy: Forces a rolling update on the ECS Cluster. This replaces old tasks with new ones only after health checks pass, ensuring zero downtime.

How to Run

Option 1: Local Development

For a detailed guide on running the app locally with Docker Compose (including Python env setup), see README-local.md.

git clone https://github.com/wegoagain-dev/cloud-cost-optimiser.git
cp .env.example .env # Add your AWS Read-Only credentials
docker-compose up --build

Option 2: Production Deployment (Terraform)

To deploy the full stack to your own AWS account:

1. Provision Infrastructure:

   cd terraform
   terraform init
   terraform apply -var="my_ip=$(curl -s ifconfig.me)/32"

2. Configure GitHub Secrets: Add the following to your repository secrets to enable the CI/CD pipeline:

   • AWS_ACCESS_KEY_ID & AWS_SECRET_ACCESS_KEY (For an IAM user with permissions to push to ECR/ECS).
   • AWS_REGION (e.g., eu-west-2).
   • ECR_BACKEND_URI & ECR_FRONTEND_URI (Output from Terraform).

3. Deploy: Simply push to main: git push origin main.

---

Teardown: To avoid unexpected costs, always destroy the infrastructure when finished:

cd terraform
terraform destroy