Posted on March 24, 2025 by Jingkai He
This article provides a guide on how to serve large language models on multiple nodes running on Kubernetes. Challenges Large Language Model (LLM) serving is a challenging task. Namely:
Posted on September 13, 2024 by Jingkai He
Over the last year CECG has been working on an engagement within a client’s Advertising Technology division to deliver an Ad decision server solution. It comes with the following requirements:
Posted on June 13, 2023 by Jingkai He
For this engagement we built an MVP developer platform, based on Kubernetes, in a short timeframe (3 months) with 2 engineers. The goal was to get a small number of initial engineering teams’ application live.