Big Data Engineer

Avalor is applying its data expertise to security, enabling companies to better understand and manage risk. We have both a rich data platform and security modules that tap into the platform to provide immediate benefits to our customers. We have built and patented a Data Fabric for Security™. It curates data from hundreds of security and business tools and contextualizes that data to help companies understand and address their riskiest problems. Our Unified Vulnerability Management module is our first module to tap into the power of our data fabric, enabling companies to create custom contextualized risk scores and tailored workflows to reduce risk. If you’re passionate about data models and helping companies improve their security posture, we’d love to have your talents here at Avalor as we work to make the world a more secure place.

We are seeking a Big Data Engineer to join our team. In this role, you will play a crucial role in designing, building, and maintaining our query engine. You will play a crucial role in managing large volumes of data, optimizing data pipelines, optimizing data query, developing complex analytics capabilities on top of the data and ensuring the reliability and scalability of our data infrastructure.

Responsibilities:

• Develop our ETL and query engines using Java / Spring Boot / Scala / Zio
• Deliver high quality features for our high-scale web environment
• Participate in design from scratch to production
• Monitor the performance and cost of the services in production
• Stay up-to-date with industry trends and best practices to continuously improve our technology stack and development processes
• Integrate with other team members (backend/frontend/product/UX)

Requirements:

• 5+ years of proven backend development experience using Scala, Java, and Spring Boot
• 5+ years of experience working in a microservices environment, including AWS, Kubernetes, RabbitMQ, SQS, Terraform, Helm, and Tilt
• High proficiency in various data stores such as MongoDB, Elasticsearch, PostgreSQL, Snowflake, and Redshift
• An enthusiastic self-learner who can confidently embrace new technologies, conduct research to discover innovative solutions, and successfully deliver results
• Demonstrated excellent problem-solving skills and meticulous attention to detail.
• Strong communication and collaboration skills
• Ability to adapt quickly and effectively manage context switches