PyTorch Distributed Developer

We are looking for a Software Developer for the High-Performance Computing Software development team. The team is responsible for developing high performance communication frameworks and applications running in production on the world’s largest supercomputers and datacenters. This position will embed you in an ambitious and diverse team that influences all areas of NVIDIA's AI platform as well as directly contributes to PyTorch, a premiere Deep Learning framework. In this role you will work with multiple teams at NVIDIA across fields, as well as collaborate internationally with the PyTorch community to develop the best AI platform in the world!The work environment is dynamic and challenging; we are innovating and inventing software products at the forefront of technology in terms of performance, scalability, and features. In addition, our team works closely with networking chip design teams in co-designing new hardware features and software APIs.

What You'll Be Doing

• Design and build core PyTorch components to maximize network performance and utilization.
• Collaborate with NVIDIA’s hardware and software teams to improve GPU and networking performance in PyTorch.
• Design, build and support production AI solutions used by enterprise customers and partners.
• Work with internal applied researchers to improve their AI tools.
• Advise design of new hardware generations.
• Be responsible for designing and implementing new protocols and algorithms that enable the best performance possible on NVIDIA networked supercomputers and datacenters.
• Deliver production grade solutions and get hands on with ground-breaking technology.
• Work closely with technical leaders solving some of the biggest challenges in machine learning, cloud computing, and system co-design.
  
  

• BS in computer science / software engineering (or equivalent experience)
• 5+ years experience of Programming in C/C++ and Python
• Deep knowledge of Networking Protocols InfiniBand and Ethernet
• Experience in network performance optimizations
• Experience with PyTorch
  
  

• You have positive attitude and work well with others.
• Demonstrated use of creative ideas, providing solutions to challenging problems.
• PhD in CS/EE/Math/Physics
• Experience with PyTorch distributed
• Background in RDMA technology
• Open Source Software Contributor
• Experience with parallel programming for CPU or GPU architectures.