Runtime Software Architect

Job Summary: We are seeking an experienced Runtime Software Architect to lead the design and development of an advanced runtime framework tailored for Neureality servers. This role involves analyzing existing runtime systems and machine learning runtime frameworks such as ONNX Runtime onnxruntime.ai, TensorFlow Serving, PyTorch's TorchServe, Apache TVM tvm.apache.org, and Glow github.com, to inform the creation of a new framework that efficiently leverages low-level hardware components, including DSPs, video decoders, ARM cores, and DLAs. The ideal candidate will possess a deep understanding of these technologies and a proven track record in architecting software that harmonizes with complex hardware infrastructures.

Key Responsibilities:

• Framework Analysis: Conduct comprehensive evaluations of existing runtime systems and ML runtime frameworks, such as ONNX Runtime, TensorFlow Serving, TorchServe, Apache TVM, VLLM, TRT-LLM, and Glow, to identify their strengths, weaknesses, and applicability to our objectives.
• Architecture Design: Develop a robust runtime framework optimized for Neureality servers, ensuring seamless integration with hardware components such as DSPs, video & audio decoders, ARM cores, and DLAs.
• Hardware Integration: Collaborate with hardware engineers to adapt and optimize software components, maximizing performance and efficiency across various hardware modules.
• Performance Optimization: Implement strategies to enhance system performance, including reducing latency, increasing throughput, and improving resource utilization.
• Cross-Functional Collaboration: Work closely with cross-functional teams to align software architecture with business goals and technological advancements.
  
  

• Educational Background: Bachelor's or Master's degree in Computer Science, Computer Engineering, or a related field.
• Experience: Minimum of 5 years in software architecture, with a focus on runtime systems and hardware-software integration.
• Technical Proficiency: Extensive experience with runtime systems and ML runtime frameworks, such as ONNX Runtime, TensorFlow Serving, TorchServe, Apache TVM, VLLM, TRT-LLM, and Glow, and a deep understanding of different types of hardware components such as DSPs, hardware accelerators, ARM cores, and DLAs.
• Analytical Skills: Strong ability to analyze and compare complex frameworks to inform architectural decisions.
• Communication: Excellent verbal and written communication skills, with the ability to convey complex technical concepts to diverse stakeholders.
  
  

• Industry Knowledge: Familiarity with the competitive landscape in runtime frameworks and hardware acceleration technologies.
• Project Management: Experience in leading projects from conception through implementation, with a track record of successful delivery.
• Continuous Learning: Commitment to staying updated with the latest advancements in runtime systems and hardware integration.