Senior Data Engineer

קוד משרה: JB-70080059 מיקום גיאוגרפי: שרון עיר: נתניה
תיאור המשרה
We are looking for Senior Data Engineer to join our innovative COMBATAI program
The program aim is to produce world-class capabilities for structured & un-structured data, AI lifecycle management that will introduce a new way to empower value from data & AI
We're looking for a skilled Data Engineer to join our team. You'll work with bid data lakehouse & warehouse, focusing on Data pipeline automation methods. Your role involves technical leading of data structure for contextual pipelines, and SQL knowledge
is a plus

:Key Responsibilities
Build smart logic data pipelines for processing data, manipulating, and deriving insights from data
Collaborate with data stewards to define and build metadata schemes
Implement data FAAS and workflows for example: metadata generator, metadata auto-extraction, validation procedures according to schemes and taxonomy, format conversions and more
Develop tools for audit, observability, and compliance analytics
Date feature Analyzing raw data and organizing it for Interpreting trends and patterns
Conducting complex data analysis and reporting on results
Preparing data for prescriptive and predictive modeling
Exploring ways to enhance data quality and reliability, data fusion from different sources
Developing analytical tools, algorithms and programs
Collaborating with data scientists and architects on several projects
דרישות המשרה
BSc in computer science or Data science
Five years of hands-on experience as lead big data engineer
Experience in developed and deployed Big Data & AI solutions with tools and frameworks such as Apache Nifi, Kafka, Spark, Graph DB, NoSQL DB, Airflow, microservices, S3 object storage
Coding: Python, Bash
:Advantages
Experience with concepts of managing metadata & Taxonomy beside big data objects
Experience with public cloud Big Data & AI services
Familiar with machine learning concepts