Python Data Engineer

Fetcherr is an AI-driven company specializing in deep learning, algorithmic trading, and large-scale data solutions. Our core technology, the Large Market Model (LMM), enables accurate demand forecasting and real-time, data-driven decision-making. Originally focused on the airline industry, Fetcherr is expanding its AI solutions across additional industries.

Fetcherr is looking for a highly skilled Senior Python Engineer to join the Platform team. You will build our "Data Platform as a Service" on Google Cloud Platform (GCP), creating the infrastructure that enables data scientists and engineers across the organization to develop, deploy, and manage data applications at scale.

In this role, you will design and implement scalable systems that power our data ecosystem, working with modern technologies and solving challenging distributed systems problems.

Why join this team?

• Build the foundation that powers pricing decisions for major airlines and expanding industries
• Work with a modern, cloud-native stack on GCP: Apache Iceberg, Spark, ClickHouse, Dagster, and Ray.
• Shape the architecture of a platform serving multiple business verticals
• Collaborate with a team of experienced engineers tackling hard data engineering challenges
  
  

• Internal Developer Platform (IDP): Build self-service tools and APIs that abstract complex infrastructure, prioritizing Developer Experience (DevEx) to accelerate data science workflows.
• Orchestration Framework: Develop and manage asset-based orchestration pipelines using Dagster. Create reusable components, libraries, and patterns to standardize data workflows across the organization.
• Data Architecture: Implement and optimize our Medallion architecture using Apache Iceberg, Spark, and ClickHouse.
• Cloud Infrastructure: Leverage GCP to build and manage infrastructure, including GKE, Cloud Storage, BigQuery, and networking resources.
• Engineering Excellence: Establish and maintain best practices for Python development, including coding standards, testing (unit, integration, E2E), environment reproducibility (e.g. dependency locking), and version control.
• Production Engineering: Manage production life cycles, from CI/CD to observability and incident response.
  
  

• Python Expertise: 5+ years of backend development with Python, including deep understanding of Python internals (GIL, memory management), OOP, and modern design patterns.
• Data Engineering: Proven experience with Big Data systems, including analytical databases, partition pruning, horizontal scalability, and batch processing pipelines.
• Orchestration Platforms: Solid hands-on experience with orchestration tools such as Dagster or Airflow, including production workflows.
• Cloud Engineering: Strong experience building scalable, reliable cloud-native systems, preferably on GCP (Compute Engine, GKE, GCS, IAM).
• Infrastructure & Operations: Experience with Docker, Kubernetes, CI/CD pipelines, and infrastructure-as-code tools (Terraform, CloudFormation, or similar).
• System Design: Ability to design and reason about complex distributed data systems, including CAP trade-offs.
• Dagster Expertise: Deep experience with Dagster, especially asset-based orchestration, reusable components, and production-grade data workflows.
• Open Table Formats: Strong understanding of Apache Iceberg or similar technologies such as Delta Lake or Hudi.
• Apache Spark: Hands-on experience with Spark for large-scale data processing, including Iceberg integration, partition management, and performance optimization in the cloud.
• Python Typing & Configurations: Deep familiarity with Pydantic, @dataclass, and configuration frameworks like Dynaconf for building robust, type-safe systems.
• Analytical Databases: Production experience with ClickHouse or other columnar analytical databases.
• GitOps & Declarative Infra: Hands-on experience with GitOps practices and declarative infrastructure patterns.
  
  

• Hands-on experience with data engineering platforms and frameworks (e.g., Apache Spark, Beam, Airflow).
• Experience building and maintaining large-scale data pipelines and ETL/ELT workflows.
• Experience with data streaming technologies (e.g., Kafka, Pub/Sub).
• Understanding of data reliability, data quality, and observability practices.
• Familiarity with infrastructure-as-code tools (e.g., Terraform, CloudFormation).
• Knowledge of security best practices for cloud-based data platforms and applications.