The Cloudcast
The Cloudcast (@cloudcastpod) is the industry's #1 Cloud Computing podcast, and the place where Cloud meets AI. Co-hosts Aaron Delp (@aarondelp) & Brian Gracely (@bgracely) speak with technology and business leaders that are shaping the future of business. Topics will include Cloud Computing | AI | AGI | ChatGPT | Open Source | AWS | Azure | GCP | Platform Engineering | DevOps | Big Data | ML | Security | Kubernetes | AppDev | SaaS | PaaS .
The Cloudcast
Building Private GenAI stacks
Luke Marsden (@lmarsden, CEO @HelixML) talks about Private GenAI. What is it? Why do you need it? We also discuss integration into CI/CD pipelines, the layers of a Private GenAI Stack, and why most organizations are opting for RAG over fine-tuning LLMs.
SHOW: 943
SHOW TRANSCRIPT: The Cloudcast #943 Transcript
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS"
SPONSORS:
- [DoIT] Visit doit.com (that’s d-o-i-t.com) to unlock intent-aware FinOps at scale with DoiT Cloud Intelligence.
- [FCTR] Try FCTR.io (that's F-C-T-R dot io) free for 60 days. Modern security demands modern solutions. Check out Fctr's Tako AI, the first AI agent for Okta, on their website
- [VASION] Vasion Print eliminates the need for print servers by enabling secure, cloud-based printing from any device, anywhere. Get a custom demo to see the difference for yourself.
SHOW NOTES:
Topic 1 - Welcome to the show Luke. Give everyone a brief intro.
Topic 2 - Let’s start with Priavte GenAI. What is it? Why should organizations out there consider it? Why not just use OpenAI GPT’s and fine tune them?
Topic 2a Follow up - Regulatory Compliance - take the opposing forces in the EU for instance to using SaaS based services based in the United States.
Topic 3 - Let’s break down the layers in a typical Private AI stack. I’m seen various ways to represent this such as infrastructure layer, MLOps layer, models, data layer (typically RAG), etc. How do you break up the stack into individual components
Topic 4 - My mind immediately jumps to similarities in the DevOps space. Abstraction layers and components like Docker and containers comes to mind, integration into CI/CD pipelines, etc. I feel like MLOps is it’s own thing with specific tools and workflows. Does this all come together and if so how?
Topic 5 - Also, what does this mean for versioning and lifecycle management of the models and the data?
Topic 6 - We are seeing more and more data pipelines with backed by multiple models, sometimes in multiple locations. How do handle this from both a scheduling and interface standpoint? Is everything hidden behind APIs for instance?
FEEDBACK?
- Email: show at the cloudcast dot net
- Bluesky: @cloudcastpod.bsky.social
- Twitter/X: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
