About the Role

We are seeking an experienced Senior Databricks Data Quality Engineer to lead the design, implementation, and automation of enterprise-scale data quality frameworks within a Databricks environment. The successful candidate will play a key role in establishing data quality controls, profiling frameworks, remediation processes, and AI-assisted quality monitoring across a large-scale data platform consisting of 170+ datasets and over 1,300 Critical Data Elements (CDEs).

This role requires strong expertise in Databricks, PySpark, Delta Lake, MLflow, and modern data quality management practices.

Key Responsibilities

Data Platform & Databricks Configuration

• Configure and manage Databricks workspaces, compute clusters, PySpark notebooks, Delta Lake architecture, and Unity Catalog integrations.
• Design scalable data quality processing frameworks across 170+ datasets and 1,346 prioritized Critical Data Elements (CDE...