Data Engineer

We're Hiring: Data Engineer to Join Our AI Team!

📍 Lod | Hybrid - 1 day WFH

Join a team building the data infrastructure behind cutting-edge AI projects! You'll be responsible for ingesting large volumes of new data, gaining deep understanding and insight into it in close collaboration with Data Scientists, and designing and developing critical, large-scale data processes - both in cloud and on-prem environments.

Requirements:

✅ At least 3 years of experience as a Data Engineer - required

✅ 3 years of experience with Object-Oriented Programming (OOP) languages - required

✅ 3 years of experience with Python - required

✅ Hands-on experience with Spark for large-scale data - required

✅ BSc in Computer Science, Engineering, Mathematics, or Statistics - required

Nice to have:

⭐ At least 2 years of hands-on experience with AWS services such as Athena/Glue/Step Functions/EMR/Redshift/RDS - significant advantage

⭐ Deep understanding of designing, developing, and optimizing complex solutions for transferring/processing large data volumes

⭐ Understanding of optimization techniques and working with partitions across data formats such as Parquet, Avro, HDF5, Delta Lake

⭐ Experience with Docker, Linux, CI/CD tools, and Kubernetes

⭐ Experience with data pipeline tools such as Airflow, Kubeflow

⭐ Understanding of machine learning concepts and processes

⭐ Familiarity with GenAI solutions / prompt engineering - advantage

Team notes:

⭐ AWS experience with core services - Glue, Step Functions, Lambda, SQS/SNS; familiarity with Redshift and Kafka, and working with APIs is a significant advantage. Candidates without any cloud environment experience won't be relevant.

⭐ Extensive experience writing functional Python code and complex ETL processes, including building generic frameworks/templates for managing ETL workflows (controls, generic process management, parameterization, etc.)

⭐ Experience deploying services - significant advantage; experience deploying models on SageMaker - especially significant advantage

⭐ Experience writing microservices - significant advantage