DevJobs

Senior Data Engineer

Overview
Skills
  • Python Python
  • SQL SQL
  • Spark Spark
  • ClickHouse
  • Delta Lake
  • Iceberg
  • S3
  • Temporal Workflow

We are seeking an experienced Senior Data Engineer.

The ideal candidate is a self-motivated, multi-tasking team player with a proven track record of collaboration.

You will be responsible for designing, developing, managing, and maintaining our open-source data platform, including our Data Lakehouse (S3, Delta Lake, and ClickHouse), ETL processes, and orchestration tool (Temporal Workflow).


Responsibilities

  • Develop a scalable data platform that integrates multiple sources for easy access.
  • Design and enhance data tools, including orchestration, governance, Data Lakehouse, BI, and more.
  • Ensure the smooth operation of data systems for analysts, data scientists, and engineers.
  • Optimize data pipelines—ingestion, processing, and output—within a microservices environment.



Qualifications

  • 5+ years of experience in a data engineering-related position
  • SQL expertise, including working with various databases, data warehouses, third-party data sources, and AWS cloud services
  • Proficient in Python (OOP + Processing packages like Polars)
  • Experience in building, designing, and optimizing data pipelines
  • Self-driven, can-do attitude

Nice to Have:

  • Experience with Spark
  • Experience with Open Table Format (Delta Lake / Iceberg)
  • Experience with ClickHouse
  • Experience with Temporal Workflow
  • Familiarity with ERP systems (big advantage)
  • Familiarity with supply chain systems (advantage)
  • Passion for open-source tools


Onebeat