Senior Data Engineer, Data Fabric

תאור משרה
We are looking for
A Senior Data Engineer to take technical leadership in building and evolving the data fabric within our Military AI programYou will design and implement scalable data architectures, structure contextual data pipelines, and automate processes across big data environments. This is a unique opportunity to lead the technical design and innovation of advanced data infrastructure driving intelligence and operational decision-making
 
Main Responsibilities
Build smart logic data pipelines for processing, manipulating, and deriving insights from dataCollaborate with data stewards to define and build metadata schemesImplement Data Functions as a Service (FAAS) and workflows including metadata generation, auto extraction, validation procedures, and format conversionsDevelop tools for audit, observability, and compliance analyticsAnalyze raw data, organize, and prepare data for prescriptive and predictive modelingConduct complex data analysis and report resultsEnhance data quality, reliability, and manage data fusion from multiple sourcesDevelop analytical tools, algorithms, and data programsProvide technical leadership of data structuring for contextual pipelines
 
Requirements
M.Sc. in Computer Science or Data Science5+ years of hands-on experience as a lead Big Data Engineer – mustProven ability to develop and deploy Big Data & AI solutions using Apache Nifi, Kafka, Spark, Graph DB, NoSQL DB, and AirflowExperience with Microservices architecture and S3 object storage – mustStrong coding skills in Python and Bash – mustExperience with metadata management and taxonomy for big data structures – advantageExperience with public cloud Big Data & AI services – advantageFamiliarity with machine learning concepts – advantageExcellent technical skillsIndependent hands-on developer with a growth mindsetTeam player, open-minded, and eager to learn