DevOps & InfrastructureEngineeringPlatinum

Setting up a scalable data platform on Google Cloud with Terraform.

Terraform GCP Data Stack

Terraform, Google Cloud, Data Infrastructure

expertv5.0

Best for

▸Multi-petabyte BigQuery data warehouse setup with cost-optimized slot reservations and partitioning
▸Production-grade data lake architecture using Cloud Storage lifecycle policies and VPC Service Controls
▸Real-time streaming pipelines connecting Pub/Sub, Dataflow, and BigQuery with proper IAM
▸Cloud Composer orchestration infrastructure with Private Service Connect networking

What you'll get

▸Modular Terraform codebase with separate modules for networking, BigQuery datasets with slot reservations, Cloud Storage with lifecycle policies, and comprehensive IAM bindings
▸Production deployment architecture with remote state in GCS, environment separation (dev/staging/prod), and automated cost monitoring dashboards
▸Security-hardened configuration with VPC Service Controls perimeters, CMEK encryption, service accounts per workload, and audit logging sinks

Expects

Clear data platform requirements including volume, velocity, compliance needs, team structure, and existing GCP project setup.

Returns

Production-ready Terraform modules with proper state management, IAM hardening, networking configuration, and cost optimization strategies.

What's inside

“You are a Terraform GCP Data Stack Infrastructure Architect. You design production data platforms on GCP using Infrastructure-as-Code, hunting for cost leaks, security gaps, and operational brittleness before they hit production. - **Workload-first service selection via decision tree, not templates*...”

Covers

What You Do DifferentlyMethodologyWatch For

Not designed for ↓

×Writing actual data transformation logic or SQL queries for BigQuery
×Designing business logic for data pipelines or ETL processes
×Application-level data modeling or schema design decisions
×Performance tuning existing BigQuery queries or Dataflow jobs

SupaScore

88.88▼

Research Quality (15%)

8.85

Prompt Engineering (25%)

9.2

Practical Utility (15%)

8.55

Completeness (10%)

8.85

User Satisfaction (20%)

8.9

Decision Usefulness (15%)

8.75

Evidence Policy

Standard: no explicit evidence policy.

terraformgcpbigquerydata-platforminfrastructure-as-codecloud-storagedataflowpub-subiamvpc-service-controlsdata-engineeringcost-optimization

Research Foundation: 8 sources (4 official docs, 2 books, 1 industry frameworks, 1 web)

This skill was developed through independent research and synthesis. SupaSkills is not affiliated with or endorsed by any cited author or organisation.