Data & AnalyticsTechnologyPlatinum

Automate data quality checks in data pipelines.

Great Expectations Data Quality Engineer

Great Expectations, dbt, Airflow, Spark

expertv5.0

Best for

▸Building automated data validation pipelines with Great Expectations for Snowflake, BigQuery, or Spark data warehouses
▸Designing expectation suites that catch data quality issues without generating false positive alerts
▸Integrating GX checkpoints into dbt model tests and Airflow DAG orchestration workflows
▸Setting up data quality monitoring dashboards with threshold calibration based on historical data patterns

What you'll get

▸Complete expectation suite Python code with column-level and table-level validations, including custom expectations for business rules
▸Checkpoint configuration files with action lists for Slack notifications, data docs updates, and pipeline failure handling
▸Integration code showing how to embed GX validations into dbt tests, Airflow operators, and Spark job workflows

Expects

Data pipeline architecture details, sample datasets, existing data quality pain points, and integration requirements with tools like dbt, Airflow, or Spark.

Returns

Complete Great Expectations implementation plans including expectation suite code, checkpoint configurations, integration patterns, and threshold calibration strategies.

What's inside

“You are a Great Expectations Data Quality Engineer. You design, implement, and operate automated data quality validation systems at scale. - Replace guesswork with data-driven threshold calibration (percentiles, 3-sigma, trailing averages, not arbitrary numbers) - Organize validation into layers (in...”

Covers

What You Do DifferentlyMethodologyWatch For

Not designed for ↓

×Generic data profiling without automated validation pipeline implementation
×Statistical analysis or machine learning model validation beyond data quality checks
×Data governance policy creation without technical implementation components
×ETL pipeline development that doesn't involve data quality validation layers

SupaScore

89.13▼

Research Quality (15%)

8.85

Prompt Engineering (25%)

9.25

Practical Utility (15%)

8.65

Completeness (10%)

9.1

User Satisfaction (20%)

8.8

Decision Usefulness (15%)

8.7

Evidence Policy

Standard: no explicit evidence policy.

great-expectationsdata-qualitydata-validationexpectationscheckpointsdata-observabilitydbtairflowdata-contractsdata-profilingpipeline-testingdama-dmbok

Research Foundation: 8 sources (3 official docs, 2 books, 2 industry frameworks, 1 academic)

This skill was developed through independent research and synthesis. SupaSkills is not affiliated with or endorsed by any cited author or organisation.