SearchEuropeanJobs.com

Sr Data Engineer, Python + Spark (Data Federation skillset - Data Lakehouse - Eg: Starburst) - New York

Company

Photon

Location

United States, United States

Type

Full-time

Senior Data Engineer (Data Federation & Lakehouse)

As a Senior Data Engineer, you will be responsible for breaking down data silos. This role focuses on building a unified, high-performance data layer using Data Federation techniques. You won't just move data; you will architect a Data Lakehouse environment where disparate sources feel like a single, cohesive database for our analytics and AI teams.

### Core Responsibilities

  • Data Federation Architecture: Design and implement federated query layers (., Starburst/Trino) to allow high-speed analytics across distributed data sources without unnecessary data movement.
  • ETL/ELT Pipeline Development: Build scalable, distributed data processing pipelines using Python and Apache Spark (PySpark).
  • Lakehouse Implementation: Manage and optimize modern table formats like Delta Lake, Apache Iceberg, or Hudi to bring ACID trans...
  • ★ Ready to Start Your European Career?

    Take the next step and apply for this exciting opportunity

    Apply Now