Python Inference Engineer – Edge Cloud & AI

About the role

Gcore is looking for a Python Inference Engineer to join its Edge Cloud & AI team. You will help design, build, and improve the inference layer of the Gcore Inference platform, working with cutting‑edge AI infrastructure that powers real‑time communication, streaming, and enterprise AI worldwide.

Key responsibilities

• Build and enhance the inference layer of the Gcore Inference platform.
• Integrate and operate frameworks such as vLLM, SGLang, NVIDIA Dynamo, and TensorRT‑LLM.
• Bring new language and multimodal models into production.
• Improve latency, throughput, memory usage, GPU utilization and cost efficiency.
• Debug performance and reliability issues across model code, inference frameworks, GPU execution, networking, and Kubernetes.
• Collaborate with platform, infrastructure, product and customer‑facing teams to turn improvements into reliable features.
• Contribute to open‑source inference projects when appropriate.

Required profile

• 5+ years of experience writing reliable, well‑tested production code.
• Strong Python skills and experience designing production systems.
• Hands‑on experience with PyTorch and deploying machine‑learning models.
• Proficiency with Linux, Docker and Kubernetes.
• Experience in at least one area such as distributed systems, GPU computing, ML runtimes, model optimization or cluster scheduling.
• Ability to debug complex problems across software, infrastructure and hardware.
• Genuine interest in inference engineering and good communication skills.

Required skills

• Python
• PyTorch
• Linux
• Docker
• Kubernetes
• vLLM
• SGLang
• NVIDIA Dynamo
• TensorRT‑LLM
• CUDA
• Triton
• GPU computing
• Model optimization (quantization, continuous batching, speculative decoding, prefix caching, LoRA serving)

What we offer

• Competitive compensation and flexible working hours.
• Hybrid or remote options with up to 45 days per year of work‑from‑anywhere.
• Private medical insurance for you and your family.
• Extra paid vacation, sick leave and support for life events.
• Language courses, modern offices with snacks and drinks, and team sports/social activities.