Data Engineer

We are looking for an experienced Data Engineer with expertise in modern data architectures, pipeline engineering, and cloud-based data ecosystems. In this role, you will design and build efficient, scalable data pipelines that enable robust data integration for AI, ML, SLM, LLM, and advanced analytics initiatives. You will also work closely with our data scientists and engineers to ensure high-quality, high-performance data flow and contribute to our data-driven culture.
Responsibilities:
Design, build, and maintain scalable ETL/ELT pipelines to integrate data from diverse sources, optimizing for performance and cost efficiency.
Leverage Databricks and other modern data platforms to manage, transform, and process data for ML and AI models, supporting both real-time and batch processing workflows.
Work with cross-functional teams to implement data solutions that support model training, monitoring, and production deployment.
Collaborate with a cybersecurity research team to understand emerging threats and develop solutions that leverage advanced data analytics.
Design and develop innovative prompts and instruction sets to enhance our autonomous LLM-based labeling platform.
Optimize prompts to generate high-quality, coherent, and contextually relevant responses.
Collaborate with software and data engineers to integrate ML/LLM techniques into production systems.