Senior Data Engineer

Position Overview

We are looking for a Senior Data Engineer to join our engineering team. This is a high-impact role at the core of Peer39’s data architecture. You will help drive our evolution to modern lakehouse-style architectures and cutting-edge big data solutions - enabling high-throughput, low-latency analytics and insights across massive datasets.

●       Architect and implement robust, scalable lakehouse data platforms that support both batch and streaming pipelines.

●       Develop and maintain high-performance data pipelines using Spark, Trino and DBT.

●       Optimize query performance and cost-efficiency across AWS S3, Athena, and Druid.

●       Own and drive data modeling, schema evolution, and partitioning strategies to support BI, ML, and real-time use cases.

●       Champion engineering excellence through clean code, code reviews, documentation, and knowledge sharing.

●       Collaborate with data analysts, data scientists, and product managers to design and deliver impactful data solutions.

●       Monitor and troubleshoot mission-critical data workflows in a production-grade environment.

●       5+ years of hands-on data engineering experience, including lakehouse architecture principles.

●       Expert SQL developer – able to write optimized, readable, and complex queries for large datasets.

●       Experience with DBT, building production-grade DBT projects.

●       Apache Airflow - must have development experience

●       Advanced development experience with Spark (batch & streaming), Trino, Druid, and Athena.

●       Proven skills with Python, especially for data processing and orchestration.

●       Deep familiarity with AWS stack, including S3, Lambda, Glue, EC2, and CloudWatch.

●       Strong understanding of data modeling, partitioning, performance tuning, and columnar formats (Parquet, ORC).

●       Working knowledge with Git and CI/CD processes for data pipelines.

●       Development experience  with Apache Iceberg or Delta Lake.

●       Familiarity with Apache Flink, Kafka, or other real-time stream processing frameworks.

●       Development experience with deploying scalable systems using Docker, Kubernetes, and Terraform.

●       Development experience of NoSQL databases (e.g., ScyllaDB, Cassandra, Couchbase).

●       Exposure to BI/reporting tools and embedded analytics platforms.