AI infrastructure & technical architecture

Overview

Saket Kumar brings a rare practitioner's lens to AI infrastructure: the perspective of a financial-services executive who must make enterprise AI work under genuine regulatory and security constraints, not hypothetical ones. Across these conversations, the dominant tension is between capability and control — between deploying AI at meaningful scale and ensuring it doesn't leak sensitive data, hallucinate at the wrong moment, or embed PII in its weights. Kumar's framework is architecturally specific: zero-trust access controls, differential privacy, on-premise infrastructure where the cloud exposes too much surface area, and a calibrated tuning strategy that avoids both the rigidity of hyper-bespoke models and the generality of models that miss organizational KPIs. The editorial center of gravity is practical AI governance — not as a compliance checkbox, but as a first-class design constraint that shapes every architectural decision.

From the corpus

• Four techniques together control LLM hallucination risk in enterprise products: rule-based hybrid checkpoints, traceability/explainability of each step, human-in-the-loop review, and grounding through fine-tuning or retrieval-augmented generation. — Saket Kumar, Ep. 4 · cl-saket-kumar-007
• AI governance is not optional; zero-trust architecture with role-based access control is essential because a single model holding broad context creates real information-leakage risk. — Saket Kumar, Ep. 4 · cl-saket-kumar-010
• There must be a balance between a generalized AI that handles broad tasks and a fine-tuned AI grounded in the organization's specific context; tilting too far toward bespoke tuning yields a hard-coded system, while pure generality misses business-specific KPIs. — Saket Kumar, Ep. 4 · cl-saket-kumar-019
• Per-token computational cost makes large-scale LLM deployment expensive: every employee query carries cost, and scaling to thousands of users requires significant investment in digital infrastructure, servers, and memory. — Saket Kumar, Ep. 4 · cl-saket-kumar-017
• Differential privacy is a critical architectural pattern for the AI era: organizations can still understand consumers without ingesting sensitive personal data, and governments should push for its adoption. — Saket Kumar, Ep. 4 · cl-saket-kumar-014
• When data is highly sensitive — as in finance — organizations should not move infrastructure entirely to the cloud; on-premise solutions remain necessary to limit external leak and threat exposure. — Saket Kumar, Ep. 4 · cl-saket-kumar-012
• Personal sensitive information should always be removed before fine-tuning or training large language models; PII should not be part of what an LLM learns from. — Saket Kumar, Ep. 4 · cl-saket-kumar-011

Contributing guests

• Saket Kumar

Episodes in AI infrastructure & technical architecture

• Saket Kumar — From Data Pullers to Insight Architects: Gen AI in Enterprise Analytics — Ep. 4 · Mar 25, 2025

See AI & technology mapped across all quarters →