LLM Inference Deployment Engineer

United States, Canada, Europe

EnCharge AI

AI Compute from the Edge-to-Cloud for Every Business. Transformative technology for AI computation, breaking records in efficiency and sustainability to enable state-of-the-art models uninhibited by power, space, and cost constraints.

View all jobs at EnCharge AI

Apply now Apply later

EnCharge AI is a leader in advanced AI hardware and software systems for edge-to-cloud computing. EnCharge’s robust and scalable next-generation in-memory computing technology provides orders-of-magnitude higher compute efficiency and density compared to today’s best-in-class solutions. The high-performance architecture is coupled with seamless software integration and will enable the immense potential of AI to be accessible in power, energy, and space constrained applications. EnCharge AI launched in 2022 and is led by veteran technologists with backgrounds in semiconductor design and AI systems.

About the Role

EnCharge AI is seeking an LLM Inference Deployment Engineer to optimize, deploy, and scale large language models (LLMs) for high-performance inference on its energy effiecient AI accelerators. You will work at the intersection of AI frameworks, model optimization, and runtime execution to ensure efficient model execution and low-latency AI inference.  

Responsibilities

  • Deploy and optimize LLMs (GPT, LLaMA, Mistral, Falcon, etc.) post-training from libraries like HuggingFace

  • Utilize inference runtimes such as ONNX Runtime, vLLM for efficient execution.

  • Optimize batching, caching, and tensor parallelism to improve LLM scalability in real-time applications.

  • Develop and maintain high-performance inference pipelines using Docker, Kubernetes, and other inference servers. 

Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or related field.

  • Experience in LLM inference deployment, model optimization, and runtime engineering.

  • Strong expertise in LLM inference frameworks (PyTorch, ONNX Runtime, vLLM, TensorRT-LLM, DeepSpeed).

  • In-depth knowledge of the Python programming language for model integration and performance tuning.

  • Strong understanding of high-level model representations and experience implementing framework-level optimizations for Generative AI use cases

  • Experience with containerized AI deployments (Docker, Kubernetes, Triton Inference Server, TensorFlow Serving, TorchServe).

  • Strong knowledge of LLM memory optimization strategies for long-context applications.

  • Experience with real-time LLM applications (chatbots, code generation, retrieval-augmented generation). 

EnchargeAI is an equal employment opportunity employer in the United States.

Apply now Apply later

* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰

Job stats:  0  0  0
Category: Engineering Jobs

Tags: Architecture Chatbots Computer Science Docker Engineering Generative AI GPT HuggingFace Kubernetes LLaMA LLMs ONNX Pipelines Python PyTorch RAG TensorFlow TensorRT vLLM

Regions: Europe North America
Countries: Canada United States

More jobs like this