Backend Software Engineer

Upstream is looking for a talented Software Engineer with a strong technical background in data infrastructure to join our engineering team.

You will be part of a growing team, responsible for building robust, scalable and accessible data solutions.

As a Software Engineer you will design, develop and implement core components such as our data lake, data pipelines and orchestration tools using modern big data architectures and frameworks.

The infrastructure you will develop will enable data consumers from across the entire company to produce data insights, implement ML models, query and process our data in a scalable and efficient manner.

This role is full-time and is based in Herzliya, Israel

Responsibilities:

• Build and expand our data infrastructure, including our data lake, analytics engine, data streaming and batch processing frameworks.
• Create robust infrastructure to enable automated pipelines that will ingest and process data into our analytical platforms, leveraging open-source, cloud-agnostic frameworks and toolsets.
• Develop and maintain our data lake and data warehouse architectures for efficient data access and advanced analytics.
• Build our ML platform and automate machine learning lifecycle.
• Lead business-wide ML projects from an engineering perspective.
• Develop and manage orchestration tools, governance tools, data discovery tools, and more.
• Work with other team members of the engineering group, including data architects, data analysts, and data scientists, to provide solutions using a use case-based approach that drives the construction of technical data flows.
  
  

• At least 5+ years of development experience, using a general purpose programming language (Python, Java, Scala, Kotlin, Go, etc.)
• At least 3+ years of experience with designing & building data pipelines, analytical tools and data lakes
• Experience with the data engineering tech stack: ETL & orchestration tools (e.g. Airflow, Argo, Prefect), data processing tools (e.g Spark, Kafka, Presto/Trino) - Big advantage
• Experience working in a containerized environments (e.g. k8s ) - Advantage
• Experience working with open-source products - Advantage