Senior Data Engineer

In your role as a Data Engineer, you'll be actively engaged in the most fascinating and demanding projects for our clients. Handling high-scale operations, heavy traffic, stringent SLAs, and massive data volumes is just the starting point. Our leadership in the realms of Big Data, Streaming Data, and Complex Event Processing (CEP) stems from our adept use of cutting-edge, optimal technologies.
Responsibilities:
Apply software-engineering methods and best practices (Medallion Architecture) in Data Lake management, using tools like DBT.
Develop Data pipelines to ingest and transform data.
Manage and enhance orchestrated pipelines across company
Data Modeling: Developing data models to ensure efficient storage, retrieval, and processing of data, often involving schema design for NoSQL databases, data lakes, and data warehouses. This will also include catalogs of data.
Scalability: Ensuring that the big data architecture can scale horizontally to handle large volumes of data and high traf

דרישות:

5 years of experience in Python (+Java / Scala)
Experience in Building data lakes, including ingestion and transformation (DBT)
Data Warehousing & ETL Pipelines
SQL & Advanced Query Optimization
Experience in data formats (Parquet, Avro, ORC)
Experience in streaming (Spark Streaming, Flink, Kafka Streams, Beam, etc.)
Experience in NoSQL and data storage (e.g. Elastic, Redis, MongoDB, CouchBase, BigQuery, Snowflake, Databricks)
Experience in messaging (Kafka, RabbitMQ, etc.)
Experience in cloud platforms (AWS, GCP) המשרה מיועדת לנשים ולגברים כאחד.