Data & AnalyticsTechnologyPlatinum

Ensuring data quality in pipelines.

Data Quality Engineer

Great Expectations, Soda Core, DMBOK2

advancedv5.0

Best for

▸Implementing Great Expectations validation suites for production data pipelines
▸Setting up Soda Core SQL-based data quality checks with custom metrics
▸Designing data contracts with schema enforcement and SLA monitoring
▸Building data observability dashboards with anomaly detection alerts

What you'll get

▸Step-by-step Great Expectations expectation suite configuration with YAML files and Python code for specific data quality dimensions
▸Soda Core check definitions with SQL-based validation rules, thresholds, and alert configurations for production deployment
▸Data contract specification templates with schema definitions, quality SLAs, and enforcement patterns using JSON Schema or Protobuf

Expects

Details about your data sources, existing quality issues, downstream dependencies, and business impact of data failures.

Returns

Specific implementation guidance for validation rules, tool configurations, monitoring setups, and remediation workflows with code examples.

What's inside

“You are a Data Quality Engineer. You systematize quality controls across data pipelines, preventing 80%+ of incidents before they reach production. * Build defense-in-depth validation (schema → statistical → business rules → freshness) instead of point checks; place quality gates at ingestion, trans...”

Covers

What You Do DifferentlyMethodologyWatch For

Not designed for ↓

×Database performance tuning or query optimization
×ETL pipeline development without quality considerations
×Data governance policy creation at the organizational level
×Machine learning model validation or feature quality assessment

SupaScore

88.43▼

Research Quality (15%)

8.85

Prompt Engineering (25%)

9.2

Practical Utility (15%)

8.55

Completeness (10%)

8.85

User Satisfaction (20%)

8.75

Decision Usefulness (15%)

8.65

Evidence Policy

Standard: no explicit evidence policy.

data-qualitydata-validationgreat-expectationssodadata-contractsdata-observabilityschema-enforcementanomaly-detectiondata-governancedata-engineeringetldbt

Research Foundation: 8 sources (3 official docs, 4 books, 1 industry frameworks)

This skill was developed through independent research and synthesis. SupaSkills is not affiliated with or endorsed by any cited author or organisation.