Senior Data Engineer

About the job

Join an Israeli cybersecurity company, specializing in cloud-based, AI-powered solutions for connected vehicles, smart mobility, and IoT ecosystems.

What You'll Be Doing

You’ll join the core platform engineering team, developing the AI-powered automotive data management platform.

At the core of the Engineering Team, you will build and operate scalable, production-grade customer-facing data and ML platform components, focusing on reliability and performance.

• Design, build, and maintain scalable data pipelines to ingest and transform batch data on our data lake, enabling analytics and ML with strong data quality, governance, observability, and CI/CD.
• Build and expand our foundational data infrastructure, including our data lake, analytics engine, and batch processing frameworks.
• Create robust infrastructure to enable automated pipelines that will ingest and process data into our analytical platforms, leveraging open-source, cloud-agnostic frameworks and toolsets.
• Develop and maintain our data lake layouts and architectures for efficient data access and advanced analytics.
• Build our ML platform and automate the ML lifecycle.
• Drive business-wide ML projects from an engineering perspective.
• Develop and manage orchestration tools, governance tools, data discovery tools, and more.
• Work with other team members of the engineering group, including data architects, data analysts, and data scientists, to provide solutions using a use case-based approach that drives the construction of technical data flows.

Requirements

• BSc/BA in Computer Science, Engineering or a related field
• At least 4 years of experience with designing and building data pipelines, analytical tools and data lakes
• At least 4 years of development experience, using a general purpose programming language (Java, Scala, Kotlin, Go, etc.)
• Experience with the data engineering tech stack: ETL & orchestration tools (e.g. Airflow, Argo, Prefect), and distributed data processing tools (e.g Spark, Kafka, Presto)

Experience with Python

• Experience working with open-source products - big advantage
• Experience working in a containerized environment (e.g. k8s) – an advantage

Job No. 417036

CV to hilas@nisha.co.il