Senior Big Data Engineer

Bigabid is an innovative technology company led by data scientists and engineers devoted to mobile app growth. Our proprietary ad platforms powered by machine learning are the outcome of that devotion.

We deliver valuable results and insights for a fast-growing clientele of major app developers using elite programmatic user acquisition and retargeting technologies.

Our ever evolving, state-of-the-art machine learning technology analyzes 50TB of raw data per day to produce millions of ad recommendations in real-time. This data is used to power our machine learning predictions, business critical metrics, and analytics to power our decision making.

Quality data is fundamental to Bigabid’s success. As a rapidly growing company, we are preparing for a future of tremendous growth and transformation. We are rebuilding our Data Engineering practice with a solid data foundation to enable further success.

As a Senior Big Data Engineer, you will be joining a team responsible for the management of the data layer at petabyte scale by creating and managing enterprise grade systems, and flows that process ~50TB of data every day and growing. The team will constantly evaluate new technologies in the field of stream processing and high scale high concurrency state store.

• Lead the implementation of scalable streaming data infrastructure that is architected for massive future growth in terms of amounts of data as well as types of fields and information managed in the system.
• Lead the implementation of a scalable serving layer that is architected for massive parallel throughput at extremely low latency
• Be involved in technology selection processes and implementation of new parts of the architecture using these technologies
• Have a deep understanding of our data, how it is acquired, managed, versioned, made discoverable, its relations, and how it can be consumed
  
  

• +6 years of proven experience as a Data/Backend Engineer
• Experience with at least one streaming technology: Spark structured streaming, Kafka streaming, Flink etc.
• Experience with designing and building ETL processes and data pipelines
• Experience with mission critical real-time systems.
• Good Analytical skills in SQL.
  
  

• Cloud native architecture with advantage to AWS
• Knowledge of Linux operating systems
• Good knowledge in containerization