Senior Data Engineer

We are looking for an experienced Senior Data Engineer to join our team and build a state-of-the-art data platform using Spark Structured Streaming and the Lakehouse architecture with Apache Iceberg.

Responsibilities:

• Design and maintain scalable data pipelines with Spark Structured Streaming.
• Implement Lakehouse architecture with Apache Iceberg.
• Develop ETL processes for data transformation.
• Ensure data integrity, quality, and governance.
• Collaborate with stakeholders and IT teams for seamless solution integration.
• Optimize data processing workflows and performance.

Requirements:

• 5+ years in data engineering.
• Expertise in Apache Spark and Spark Structured Streaming.
• Hands-on experience with Apache Iceberg or similar data lakes.
• Proficiency in Scala, Java, or Python.
• Knowledge of big data technologies (Hadoop, Hive, Presto).
• Experience with cloud platforms (AWS, Azure, GCP) and SQL.
• Strong problem-solving, communication, and collaboration skills.

Preferred:

• Real-time data processing and streaming analytics.
• Familiarity with Docker, Kubernetes, CI/CD, and Git.

Join us and contribute to innovative data solutions!