Real Time Data Engineer

Description

ppltx is recruiting for a dynamic B2C gaming startup in Tel Aviv. As a real time data engineer, you'll design, build, and maintain streaming pipelines that process billions of events per day, and provide complex real-time analytics and triggers that impact the user experience and company decisions.

Responsibilities

- Design and implement complex real time components written in Beam (Java) and run on GCP Dataflow, creating advanced processing, aggregations, windowing, filtering, etc.

- Connect to sources in Kafka, PubSub, and other real time sources.

- Build and maintain CDC pipelines replicating data from multiple source databases using Kafka and Debezium.

- Create pipelines that utilize ML models created by the data science team, as well as general LLM solutions, for product real time features.

- Develop monitoring, alerting, and self-healing mechanisms for streaming infrastructure.

- Create CICD processes for real time pipelines and IaC for related infra.

- Troubleshoot production issues across distributed systems under time pressure.

- collaborate with analytics, bi, science, qa, operation, product and more in order to drive real time based features to testing and production.

Requirements

- 2+ years of experience building and operating real-time data pipelines in production.

- Strong proficiency in Python and Java.

- Hands-on experience with real time processing tools (such as Apache Beam).

- Deep understanding of message broker concepts (such as Kafka, PubSub, Kinesis).

- experience with real time cache (Redis for example).

- Working experience with cloud environments (GCP experience preferred).

Working from the office only, 5 days a week.

Company located in Tel Aviv near Sarona Market