Data Engineer

Cust2Mate is a global leader in smart-cart platforms, transforming in-store shopping through digitalization and personalization. Our award-winning Smart Carts, trusted by leading grocery chains, elevate the customer experience, optimize store operations, and bridge online and physical retail.

We’re looking for a Data Engineer to build and scale reliable data pipelines and analytics infrastructure. You’ll own ingestion, transformation, modeling, and orchestration with a stack centered on Python, Apache Spark, BigQuery, and dbt (today) We are in a process of redesigning our infrastructure.

Responsibilities:

• Design, build, and maintain batch and streaming pipelines from multiple sources into BigQuery
• Develop modular dbt models (staging, marts) with tests, documentation, and clear ownership
• Optimize Spark jobs for performance and cost; tune partitions, caching, and I/O.
• Implement data quality checks, observability, and alerting (freshness, volume, schema).
• Collaborate with Analytics/BI and Product teams to model data for self-serve reporting.
• Design, implement, and maintain CI/CD pipelines and manage orchestration
• Manage and optimize Azure & GCP cloud infrastructure.

Requirements:

• 7+ years building production data pipelines and models.
• 5+ years Databricks data engineer – must.
• Strong Python (coding, packaging, unit tests, dependency mgmt).
• Hands-on Apache Spark (PySpark preferred): optimizing joins, shuffles, and partitions.
• Expert BigQuery: SQL, storage design, performance tuning, and cost controls.
• Production dbt experience (modeling conventions, tests, docs, exposures).
• Experience with orchestration (Airflow/Cloud Composer or similar) and Git-based CI/CD.
• Data quality/observability mindset; familiarity with testing frameworks.
• Clear communication and stakeholder collaboration.
• Streaming (Kafka/Pub/Sub), incremental change data capture (CDC) (Nice to have)
• Terraform/IaC, GCP networking & security (Nice to have)