Data Engineer

abra R&D solutions is seeking a Big Data & Data Engineering expert to join a cloud native observability team.

Be a part of something spectacular - join our lab on a journey to innovation of the coolest disruptive technologies.

An innovation lab is a unique work experience. Have the privilege of watching your ideas and vision developed and offered to tenants of a major cloud vendor. Here you will dabble in all the coolest new concepts, research new technologies, design cloud-scale, multi-tenant services, and of course, implement them.

As a big data expert, you will be expected to ideate, understand the value proposition and product vision, conceptualize the big picture, design complex distributed systems, select suitable tech stack and lead the implementation while mentoring others.

As a successful candidate you will have excellent grasp of massive scale SaaS offerings, outstanding verbal and written communication skills you will put to use both internally and externally to the lab and most importantly you will possess exceptional research capabilities, have an innovative mind & the ability to think out-of-the-box.

Responsibilities

• Design, build, and maintain Big Data workflows/pipelines using technologies like Spark, Flink, Pulsar, Hadoop, Kafka Streams, Druid, Clickhouse, and Apache Beam.
• Develop observability solutions for our cloud-native applications, focusing on areas such as application performance monitoring (APM), log analytics, application and infrastructure telemetry data.
• Design and model data at rest for massive scale, low latency read and reduced cost.
• Implement data validation procedures to ensure high data quality and process integrity.
• Leverage streaming and batch processing tools to provide real-time analytics insights.
• Work with NoSQL databases like HBase, Cassandra, Hive, Pig, and Impala to handle our data needs.
• Collaborate with cross-functional teams to define and implement data models that provide intuitive analytics.
• ng/LLM Ops)
  
  

• Proven experience as a Big Data/Data Engineer or similar role.
• Proven experience in data modeling for Multi-tenant SaaS Big-Data based offerings
• 10+ years of experience in architecting & implementing large scale, multi-tenant SaaS offerings
• Experience with cloud services (AWS, Google Cloud, Azure) and understanding of distributed systems.
• Knowledge of various Big Data frameworks and libraries (like Hadoop, Spark, Hive)
• Significant experience with Java, big data technologies and application state modelling
• Outstanding research capabilities
  
  

• Experience with Operations & Management and/or Performance Engineering
• Knowledge in other programming language e.g. TypeScipt, Python, Go, C++
• Experience with K8S/other virtualization orchestration technologies
• Experience with DevOPs in general & CI/CI pipelines in particular
• Experience with LLM (prompt/vector search/RAG/fine tuni