Data Engineer - RT Big Data Systems

abra R&D is looking for a Data engineer!

We are looking for a Data Engineer for RT Big Data Systems to join the team and design and deploy scalable, standardized, and maintainable data pipelines that enable efficient logging, error handling, and real-time data enrichment. The role requires strong ownership of both implementation and performance

The role includes:

• Optimize Splunk queries and search performance using best practices
• Build and manage data ingestion pipelines from sources like Kafka, APIs, and log streams
• Standardize error structures (error codes, severity levels, categories)
• Create mappings between identifiers such as session ID, user ID, and service/module components
• Implement real-time data enrichment processes using APIs, databases, or lookups
• Set up alerting configurations with thresholds, modules, and logic-based routing
• Collaborate with developers, DevOps, and monitoring teams to unify logging conventions
• Document flows and ensure traceability across environments
  
  

• Minimum 3 years of hands-on experience in Splunk – Mandatory
• Proficient in SPL, data parsing, dashboards, macros, and performance tuning – Mandatory
• Experience working with event-driven systems (e.g., Kafka, REST APIs) – Mandatory
• Deep understanding of structured/semi-structured data (JSON, XML, logs) – Mandatory
• Strong scripting ability with Python or Bash
• Familiar with CI/CD processes using tools like Git and Jenkins
• Experience with data modeling, enrichment logic, and system integration
• Advantage: familiarity with log schema standards (e.g., ECS, CIM)
• Ability to work independently and deliver production-ready, scalable solutions – Mandatory