Senior Data Engineer

Key Responsibilities

• Architect and implement robust, scalable lakehouse data platforms that support both batch and streaming pipelines.
• Develop and maintain high-performance data pipelines using Spark, Trino and DBT.
• Optimize query performance and cost-efficiency across AWS S3, Athena, and Druid.
• Own and drive data modeling, schema evolution, and partitioning strategies to support BI, ML, and real-time use cases.
• Champion engineering excellence through clean code, code reviews, documentation, and knowledge sharing.
• Collaborate with data analysts, data scientists, and product managers to design and deliver impactful data solutions.
• Monitor and troubleshoot mission-critical data workflows in a production-grade environment.

Required Skills & Experience

• 5+ years of hands-on data engineering experience, including lakehouse architecture principles.
• Expert SQL developer – able to write optimized, readable, and complex queries for large datasets.
• Experience with DBT, building production-grade DBT projects.
• Apache Airflow - must have development experience
• Advanced development experience with Spark (batch & streaming), Trino, Druid, and Athena.
• Proven skills with Python, especially for data processing and orchestration.
• Deep familiarity with AWS stack, including S3, Lambda, Glue, EC2, and CloudWatch.
• Strong understanding of data modeling, partitioning, performance tuning, and columnar formats (Parquet, ORC).
• Working knowledge of Git and CI/CD processes for data pipelines.

Preferred Qualifications

• Development experience with Apache Iceberg or Delta Lake.
• Familiarity with Apache Flink, Kafka, or other real-time stream processing frameworks.
• Development experience with deploying scalable systems using Docker, Kubernetes, and Terraform.
• Development experience of NoSQL databases (e.g., ScyllaDB, Cassandra, Couchbase).
• Exposure to BI/reporting tools and embedded analytics platforms.