Private AI Infrastructure
Enterprise-grade Local AI Deployment & System Integration.
Checking...
Private AI & Automation Builder
Specializing in building Self-Hosted AI Modules for enterprises. I focus on the infrastructure, API connectivity, and security layers that make LLMs deployable in business environments.
INFRASTRUCTURE
Host A (Main Ops) & Host B (Remote Chat) orchestration.
PARTNERSHIP MODEL
I build the Architecture & IO; My partner handles RAG Logic & Fine-tuning.
INTEGRATION
Seamless connection with Lark Base & Enterprise API Gateways.
System Architecture
Infrastructure Diagram
Multi-host architecture separating Operations (Host A) from External Chat Services (Host B).
▶
DEMO
%%{init: {'theme': 'dark', 'themeVariables': { 'fontSize': '16px', 'fontFamily': 'Inter', 'lineColor': '#aaa' }}}%%
graph TD
%% --- 樣式定義 (High Contrast Styles for Dark Mode) ---
classDef userLayer fill:#2c3e50,stroke:#fff,stroke-width:2px,color:#fff;
classDef proxyLayer fill:#d35400,stroke:#fff,stroke-width:2px,color:#fff;
classDef appLayer fill:#27ae60,stroke:#fff,stroke-width:2px,color:#fff;
classDef extLayer fill:#8e44ad,stroke:#fff,stroke-width:2px,color:#fff;
classDef apiLayer fill:#2980b9,stroke:#fff,stroke-width:2px,color:#fff;
classDef infraLayer fill:#7f8c8d,stroke:#fff,stroke-width:2px,color:#fff;
%% --- 外部使用者 ---
User((End User)):::userLayer
Admin((System Admin)):::userLayer
%% --- 主機 A: 核心運維中樞 ---
subgraph Host_A ["☁️ AWS EC2 (Main Host / Operations)"]
style Host_A fill:#1e1e24,stroke:#a855f7,stroke-width:2px,stroke-dasharray: 5 5,color:#fff
%% 核心路由
Caddy{{🛡️ Caddy Gateway
Port 443 / SSL}}:::proxyLayer %% 內部應用服務 subgraph Internal_Apps ["🖥️ Internal Applications"] style Internal_Apps fill:#2a2a30,stroke:#666,color:#eee Portal["🌐 Portal Landing Page
/var/www/portal"]:::appLayer DMail["💬 D-Mail (RAG Chat)
Streamlit :8501"]:::appLayer Mentor["🧠 Mentor (Training)
Streamlit :8502"]:::appLayer Watchyou["👁️ Watchyou (Monitor)
Streamlit :8503"]:::appLayer end %% 後端大腦 subgraph Core_Backend ["⚙️ Intelligence Engine"] style Core_Backend fill:#2a2a30,stroke:#666,color:#eee RAG_API["🚀 FastAPI Backend
Port :8000"]:::apiLayer SystemLog["📄 system.log"]:::infraLayer subgraph Knowledge_Store ["💾 Memory & Storage"] style Knowledge_Store fill:#1a1a1d,stroke:#444,color:#ccc Qdrant[("🗄️ Qdrant Vector DB")]:::infraLayer FileSys["📂 Files (knowledge_base)"]:::infraLayer Ollama_A[(" Ollama (Qwen 2.5)")]:::infraLayer Embed[("🔠 Embedding (BGE-M3)")]:::infraLayer end end end %% --- 主機 B: 外部聊天服務 --- subgraph Host_B ["☁️ AWS EC2 (Remote Host)"] style Host_B fill:#1e1e24,stroke:#e74c3c,stroke-width:2px,stroke-dasharray: 5 5,color:#fff Chat["💬 Chat Service
FastAPI"]:::extLayer Ollama_B[("External LLM")]:::extLayer end %% --- 連線關係 (Routing Logic) --- User ==>|HTTPS| Caddy Admin ==>|HTTPS| Caddy %% Caddy 路由分發 (關鍵路徑) Caddy -->|/| Portal Caddy -->|/dmail| DMail Caddy -->|/mentor| Mentor Caddy -->|/watchyou| Watchyou Caddy -->|/api| RAG_API %% 跨主機連線 (Cross-Instance Proxy) Caddy -.->|"/chat (Proxy Pass)"| Chat %% 內部服務溝通 DMail -->|POST| RAG_API Mentor -->|POST| RAG_API Watchyou -->|Read| SystemLog %% 後端運作 RAG_API --> Qdrant RAG_API --> Ollama_A RAG_API --> Embed RAG_API --> FileSys RAG_API --> SystemLog %% 外部主機運作 Chat --- Ollama_B
Port 443 / SSL}}:::proxyLayer %% 內部應用服務 subgraph Internal_Apps ["🖥️ Internal Applications"] style Internal_Apps fill:#2a2a30,stroke:#666,color:#eee Portal["🌐 Portal Landing Page
/var/www/portal"]:::appLayer DMail["💬 D-Mail (RAG Chat)
Streamlit :8501"]:::appLayer Mentor["🧠 Mentor (Training)
Streamlit :8502"]:::appLayer Watchyou["👁️ Watchyou (Monitor)
Streamlit :8503"]:::appLayer end %% 後端大腦 subgraph Core_Backend ["⚙️ Intelligence Engine"] style Core_Backend fill:#2a2a30,stroke:#666,color:#eee RAG_API["🚀 FastAPI Backend
Port :8000"]:::apiLayer SystemLog["📄 system.log"]:::infraLayer subgraph Knowledge_Store ["💾 Memory & Storage"] style Knowledge_Store fill:#1a1a1d,stroke:#444,color:#ccc Qdrant[("🗄️ Qdrant Vector DB")]:::infraLayer FileSys["📂 Files (knowledge_base)"]:::infraLayer Ollama_A[(" Ollama (Qwen 2.5)")]:::infraLayer Embed[("🔠 Embedding (BGE-M3)")]:::infraLayer end end end %% --- 主機 B: 外部聊天服務 --- subgraph Host_B ["☁️ AWS EC2 (Remote Host)"] style Host_B fill:#1e1e24,stroke:#e74c3c,stroke-width:2px,stroke-dasharray: 5 5,color:#fff Chat["💬 Chat Service
FastAPI"]:::extLayer Ollama_B[("External LLM")]:::extLayer end %% --- 連線關係 (Routing Logic) --- User ==>|HTTPS| Caddy Admin ==>|HTTPS| Caddy %% Caddy 路由分發 (關鍵路徑) Caddy -->|/| Portal Caddy -->|/dmail| DMail Caddy -->|/mentor| Mentor Caddy -->|/watchyou| Watchyou Caddy -->|/api| RAG_API %% 跨主機連線 (Cross-Instance Proxy) Caddy -.->|"/chat (Proxy Pass)"| Chat %% 內部服務溝通 DMail -->|POST| RAG_API Mentor -->|POST| RAG_API Watchyou -->|Read| SystemLog %% 後端運作 RAG_API --> Qdrant RAG_API --> Ollama_A RAG_API --> Embed RAG_API --> FileSys RAG_API --> SystemLog %% 外部主機運作 Chat --- Ollama_B
Deployment Specs
- Host A (Operations): Runs the core RAG Engine (FastAPI), Qdrant Vector DB, and internal Streamlit tools (D-Mail, Mentor, Watchyou).
- Host B (Remote): Dedicated to external chat services (Chat) to isolate public-facing traffic.
- Routing: Caddy handles all SSL termination and reverse proxies traffic to specific ports (8501-8503) or remote hosts.
- Data Flow: Internal apps communicate with the Intelligence Engine via local FastAPI calls for high-speed retrieval.
Infrastructure & Connectivity
I ensure the system is "accessible, secure, and monitored".
- Streamlit Suite: Deployed 3 distinct internal tools for Chat, Training, and Monitoring.
- Cross-Host Proxy: Configured Caddy to securely tunnel traffic to the secondary Host B.
- Observability: Built "Watchyou" app to visualize system logs and API health in real-time.
Live Demonstration
Below is a preview of the "D-Mail" RAG Chat system and the "Watchyou" monitoring dashboard running on Host A.
▶
DEMO
🌐 Visit Live Portal
Checking Service...
* Access requires enterprise VPN or whitelisted IP.