Sponsored: Build an open source platform for ai/ml
Watch talk on YouTubeJorge Palma from Microsoft with a quick introduction.
Baseline
- Kubernetes is cool and all
- Challenges:
- Containerized models
- GPUs in the cluster (install, management)
Kubernetes AI Tool chain (KAITO)
- Kubernetes operator that interacts with
- Node provisioner
- Deployment
- Simple CRD that describes a model, infra and have fun
- Creates inference endpoint
- Models are currently 10 (Hugginface, LLMA, etc.)