Senior Big Data Engineer, Data Fabric & AI

We are seeking an experienced and highly skilled Senior Big Data Engineer to join our multidisciplinary team, leading the way in innovative products centered around Big Data and AI. In this critical role, you will be part of a dynamic group that includes architects, engineers, designers, UI experts, QA professionals, support staff, and product managers. Together, we aim to deliver world-class services for managing semi-structured and unstructured data across various scales. The product you will work on focuses on AI lifecycle management and aims to revolutionize the way value is extracted from data and AI capabilities

As a Senior Big Data Engineer, you will have the unique opportunity to contribute to a product that is not just a code base but a comprehensive solution affecting multiple aspects of the data and AI world. You will be responsible for designing, building, and implementing our product on a large scale, meeting both the challenges and the promises that come with big data and advanced AI technologies. Your work will have a direct impact on the core technology that drives our product, and by extension, our company's success

If the idea of contributing to a game-changing product in a fast-paced, innovative environment excites you, then this role is the perfect fit

Responsibilities

Make ideas happen. Convert ideas and needs from design into reality. Understand the needs and requirements, learn new domains, analyze alternatives, and provide solutions

Hands-on

Build services (software, Data engineering, AI engineering) which will be part of the whole product framework

Develop robust microservices for hybrid environments

Collaborate with data stewards to define and build metadata schemes using our unique unified language & taxonomy

Develop tools for audit, observability, and compliance analytics

Serve as domain expert

Be technical leader and mentor to a group of big data engineers

Be the main focal point on the above domains across the company

Grow. Explore recent technologies, learn new skills, and apply concepts in new ways

Collaborate. Work side by side with other Big Data & AI engineers (on core group and on customer groups), build & share documentation, and present outcomes at managerial level



Education

B.sc in computer science or equivalent

About You

A nice person – positive, good-natured, and pleasant to be around

Great learner, open-minded and a team player

Strong verbal and written communication skills

Experience

3+ years of hand-on implementation around hybrid Big Data & AI environment (on-prem and cloud) – Must

You have developed and deployed Big Data & AI solutions with tools and frameworks such as Kafka, Spark, Graph DB, NoSQL DB (ELK - Preferred), Airflow, object storage – Must

Coding: Python, pySpark/Scala – Must

Working with: Linux, Docker – Must, K8s - advantage

You have experience with concepts of managing metadata & taxonomy beside big data objects – advantage

Experience with public cloud Big Data & AI services – advantage

Familiar with machine learning concepts – advantage