Senior Data Engineer - NBA

Become a part of our caring community

=========================================

The Senior Data Engineer builds and evolves the data foundation that powers the NBA platform on the Databricks Lakehouse. This role is responsible for developing scalable data pipelines, feature engineering workflows, and real-time ingestion patterns that transform raw healthcare, member, behavioral, engagement, and socioeconomic data into trusted assets for decision intelligence, machine learning, reinforcement learning, and AI workloads. This is a hands-on engineering role focused on delivering reliable, well-governed, and high-quality data products while partnering closely with Data Science, AI Engineering, and Platform Engineering teams.Key Responsibilities

• Data pipeline development — Design, build, and maintain Bronze, Silver, and Gold lakehouse pipelines that power member profiles, clinical signals, engagement data, and decision intelligence use cases.
• Feature engineering — Develop and maintain Gold-layer feature tables and reusable data products that support model training, model scoring, reinforcement learning, and real-time decisioning.
• Batch and streaming ingestion — Build and support batch and real-time ingestion pipelines using Databricks, Spark, Delta Lake, and event-driven data architectures.
• Data quality engineering — Implement data quality validation, reconciliation, monitoring, alerting, and testing controls to ensure trusted and production-ready data assets.
• Data modeling — Design scalable data models that balance usability, governance, performance, and long-term maintainability.
• Performance optimization — Tune Spark workloads, storage strategies, partitioning schemes, and query performance to improve efficiency, scalability, and cost management.
• Data governance — Follow established standards for data lineage, security, PHI handling, auditability, and regulatory compliance.
• **Platform integration