Senior Data Infrastructure Engineer

About Us

At Milestone, we’re building a robust, scalable, and intelligent data platform to power advanced analytics and machine learning across our business. Our infrastructure supports diverse workflows, from real-time decision-making to large-scale data transformations, all running across AWS and GCP environments. We’re looking for a Senior Data Infrastructure Engineer to join our growing team and help take our platform to the next level.

• Design, build, and maintain a scalable data infrastructure across AWS and GCP.
• Lead the development and orchestration of robust data pipelines using Argo Workflows and Dockerized components.
• Build and optimize data transformations using Databricks and PySpark.
• Manage and monitor data pipelines that write to PostgreSQL databases for downstream applications.
• Collaborate with DevOps and software engineers to ensure data is accurate, timely, and accessible.
• Drive infrastructure decisions with a focus on reliability, observability, and maintainability.
• Participate in code reviews, set best practices, and mentor junior engineers.
• Evaluate and integrate new technologies that enhance our data platform’s capabilities.

• Workflow Orchestration: Argo Workflows, K8s, Docker
• Big Data Processing: Databricks, PySpark
• Programming Languages: Python (primary), Bash
• Storage & Databases: PostgreSQL, S3, GCS, Delta Lake
• Cloud Platforms: AWS, GCP

• 6+ years of experience in data engineering or data infrastructure roles.
• Strong proficiency in Python and Spark (PySpark) for distributed data processing.
• Strong knowledge of working with Databricks for data processing, analytics and ML.
• Deep understanding of modern data architectures, including batch/streaming pipelines and lakehouse paradigms.
• Hands-on experience managing cloud-native infrastructure on AWS and/or GCP.
• Proficiency with PostgreSQL and understanding of how to optimize I/O from big data pipelines to relational databases.
• Excellent communication skills and the ability to work cross-functionally.
• Experience with Argo Workflows, Docker, and orchestration of distributed data jobs (Optional).

• High proficiency with Delta Lake or Kubernetes.
• Familiarity with CI/CD pipelines and infrastructure as code.
• Previous experience in a startup or fast-paced product environment.

• Opportunity to shape core infrastructure powering real-world analytics products.
• Ownership to make architecture and tooling decisions.
• Collaborate with a sharp, mission-driven team that values engineering excellence.
• Flexible work environment and competitive compensation package.