Ollama with OpenWebUI on Ubuntu 24.04 LTS

Self-Hosted AI Chat (Open WebUI + Ollama)

Private, secure AI on Ubuntu 24.04. No external APIs. Docker-based.

Azure ReadyPrivateFlexible

🔍 What you get

• Open WebUI – chat interface (like ChatGPT, self-hosted).
• Ollama – runtime for Llama, Mistral, Phi, etc.
• Auto-start – systemd service starts at boot.

🚀 First-Run Setup

• Open TCP port 3000 in Azure Network Security Group (inbound).
• Connect via SSH to the VM.
• Start the service (if not already running):

sudo systemctl enable --now hoiltd-ollama-openwebui.service

Step 4: Access the web UI – http://<your-vm-ip>:3000 – and start chatting!

✅ UI is all you need – model selection, chat history, RAG (document upload).

🧑‍💻 Advanced – Ollama CLI

SSH in and use docker exec:

# Pull model (e.g., llama3.2:3b)

sudo docker exec -it hoiltd-ollama ollama pull llama3.2:3b

# List models

sudo docker exec hoiltd-ollama ollama list

# Interactive chat

sudo docker exec -it hoiltd-ollama ollama run llama3.2:3b

💡 Models persist across reboots inside the container volume.

🔧 Remove `sudo` for Docker (optional)

To run Docker commands without sudo, add your user to the docker group:

sudo usermod -aG docker $USER

Then log out and back in (or run newgrp docker). After that, use docker exec ... without sudo.

🛠️ Service management

sudo systemctl status hoiltd-ollama-openwebui.service

sudo journalctl -u hoiltd-ollama-openwebui.service -f

sudo systemctl restart hoiltd-ollama-openwebui.service

📋 Which interface?

• Non-tech users: Web UI only.
• Developers: Web UI for daily use, CLI for pulling models / scripting.

❓ FAQs

Q: Start after boot? No – auto-starts.

Q: Add models? Via UI (if enabled) or SSH + docker exec ... pull.

Q: Data private? Yes – all runs in your VM.

Home & Office Improvements Ltd.

Cloud & AI Solutions • info@hoiltd.com