Choosing the right deployment architecture is one of the first decisions an organization makes before rolling out Herbie.AI. A good deployment architecture balances scalability, security, high availability and operational cost, and Herbie.AI is built to support cloud, on-premises, hybrid or fully air-gapped environments while keeping functionality consistent across all of them.
This guide walks through the four deployment architecture models Herbie.AI supports, the infrastructure sizing behind each one, and the network design that keeps a deployment both fast and secure, building on the same agentic AI and AI-RAG foundations covered elsewhere on this blog.
Four Deployment Architecture Models
Herbie.AI’s deployment architecture comes in four flavors, each suited to a different risk and compliance profile.
- Cloud deployment, for fast setup and elastic scaling on AWS, Azure or GCP
- On-premises deployment, for organizations that need full control over data and infrastructure
- Hybrid deployment, for mixing cloud scalability with on-premises data residency
- Air-gapped deployment, for maximum-security environments with no external network access
Inside a Herbie.AI Deployment
Every deployment architecture shares the same core building blocks: a load balancer or reverse proxy in front, the conversational engine and AI agents behind it, and a data layer underneath made up of a relational database for metadata and configuration, a cache for performance, a message queue for asynchronous processing, and a monitoring stack for observability. Connector services sit alongside these to reach enterprise systems.
Sizing a Deployment Architecture by Organization
Herbie.AI’s deployment architecture scales in three broad tiers, each sized to a different concurrent user load.
| Tier | Users | CPU | RAM | Storage | GPU |
|---|---|---|---|---|---|
| Small | Up to 100 | 8 Cores | 32 GB | 500 GB SSD | Optional |
| Medium | Up to 1,000 | 16-32 Cores | 64-128 GB | 2 TB SSD | NVIDIA RTX/A-Series |
| Enterprise | 5,000+ | 64+ Cores | 256-512 GB | NVMe Storage Cluster | Multiple Enterprise GPUs |
Enterprise-tier deployment architecture also requires a load balancer and Kubernetes for orchestration and resilience at that scale.
Kubernetes for Enterprise-Scale Deployment Architecture
For larger environments, Kubernetes orchestrates containers across nodes, handling auto-scaling, rolling updates and self-healing, so a spike in conversation volume does not become an outage. This is the same pattern that lets the workflow orchestration layer keep running smoothly under load.
Network Architecture and Security Zones
A secure deployment architecture segments the network into zones: the public internet reaches only the load balancer, which forwards traffic into an application zone containing the conversational engine and AI agents, which in turn reaches a protected data zone holding databases, caches and enterprise connectors. No zone talks directly to the internet except the load balancer itself.
Choosing the Right Deployment Architecture
Most organizations start cloud or hybrid for speed, then shift workloads on-premises or air-gapped as compliance requirements demand it. Whichever deployment architecture fits your organization, the surrounding security model and connector framework stay the same. Talk to the team about sizing a deployment architecture for your own user base.