Senior Data Engineer

We are looking for Senior Data Engineer to join our innovative COMBATAI program The program aim is to produce world-class capabilities for structured & un-structured data, AI lifecycle management that will introduce a new way to empower value from data & AI We're looking for a skilled Data Engineer to join our team. You'll work with bid data lakehouse & warehouse, focusing on Data pipeline automation methods. Your role involves technical leading of data structure for contextual pipelines, and SQL knowledge  is a plus   :Key Responsibilities Build smart logic data pipelines for processing data, manipulating, and deriving insights from data Collaborate with data stewards to define and build metadata schemes Implement data FAAS and workflows for example: metadata generator, metadata auto-extraction, validation procedures according to schemes and taxonomy, format conversions and more Develop tools for audit, observability, and compliance analytics Date feature Analyzing raw data and organizing it for Interpreting trends and patterns Conducting complex data analysis and reporting on results Preparing data for prescriptive and predictive modeling Exploring ways to enhance data quality and reliability, data fusion from different sources Developing analytical tools, algorithms and programs Collaborating with data scientists and architects on several projects     :Requirements BSc in computer science or Data science Five years of hands-on experience as lead big data engineer Experience in developed and deployed Big Data & AI solutions with tools and frameworks such as Apache Nifi, Kafka, Spark, Graph DB, NoSQL DB, Airflow, microservices, S3 object storage Coding: Python, Bash   :Advantages Experience with concepts of managing metadata & Taxonomy beside big data objects Experience with public cloud Big Data & AI services Familiar with machine learning concepts