senior data engineer, network clustering

we are looking for an expert data engineer to build and evolve the data backbone for our r&d telemetry and performance analytics ecosystem. responsibilities include processing raw, large quantities of data from live systems at the cluster level: hardware, communication units, software, and efficiency indicators. youll be part of a fast paced r&d organization, where system behavior, schemas, and requirements evolve constantly. your mission is to develop flexible, reliable, and scalable data handling pipelines that can adapt to rapid change and deliver clean, trusted data for engineers and researchers.
what youll be doing:
build flexible data ingestion and transformation frameworks that can easily handle evolving schemas and changing data contracts
develop and maintain ETL /elt workflows for refining, enriching, and classifying raw data into analytics-ready form
collaborate with r&d, hardware, DevOps, ml engineers, data scientists and performance analysts to ensure accura

דרישות:

what we need to see:
b.sc. or m.sc. in Computer Science, computer engineering, or a related field
5+ years of experience in data engineering, ideally in telemetry, streaming, or performance analytics domains
confirmed experience with databricks and apache spark (pyspark or scala)
understanding of streaming processes and their applications (e.g., apache kafka for ingestion, schema registry, event processing)
proficiency in Python and sql for data transformation and automation
shown knowledge in schema evolution, data versioning, and data validation frameworks (e.g., delta lake, great expectations, iceberg, or similar)
experience working with cloud platforms (aws, gcp, or azure) - aws preferred
familiarity with data orchestration tools (airflow, prefect, or dagster)
experience handling time-series, telemetry, or Real-Time data from distributed systems המשרה מיועדת לנשים ולגברים כאחד.