LLM Engineer

SoluGenAI is a rapidly growing company that develops a Generative AI platform for enterprise clients. Our platform streamlines and enhances employee access to their knowledge base, greatly improving productivity and efficiency. We are supported by experienced serial entrepreneurs and collaborate with top Israeli and Fortune 500 companies.

We are looking for a Senior Developers with significant Python experience to design and implement high-performance AI inference pipelines and accelerate large language model (LLM) inference in production environments.

Key Responsibilities

• Lead the design and implementation of AI inference pipelines, integrating Retrieval Augmented Generation (RAG) frameworks into a new AI chip's runtime environment.
• Implement and optimize inference acceleration techniques for LLMs, including methods such as flash attention, continuous batching, kv-caches, quantization, and attention with paging.
• Develop and lead the implementation of a dashboard that integrates with up-to-date organizational data, automates analysis, and makes the information accessible through conversational AI interfaces.
• Collaborate with hardware teams to optimize AI model inference for maximum throughput (tokens per second) across a large number of concurrent users

Requirements

• Hands-on experience with AI model inference. Experience with running LLMs and optimizing AI models for high throughput.
• At least 3 years of experience developing in Python for complex systems.
• Bachelor’s degree in a relevant engineering field (Computer Science, Electrical Engineering, etc.) is required.
• A Master's degree in engineering is an advantage.
• Experience with implementing complex software systems running on data centers is an advantage.