LocalAI - Hardened Self-Hosted OpenAI-Compatible API Server
Lynxroute · Software Development
Certification per Microsoft Marketplace.
Evidence tier Source Confirmed · 8 captures on record
What the publisher says
As described on Microsoft Marketplace.
What is LocalAI
LocalAI is an open-source, OpenAI-compatible inference server delivered as a single statically linked Go binary. It exposes the same REST surface as OpenAI - /v1/chat/completions, /v1/embeddings, /v1/audio/transcriptions, /v1/audio/speech, /v1/images/generations, /v1/models, plus a built-in /models gallery API - so any client written for OpenAI (openai-python, openai-node, LangChain OpenAI provider, LlamaIndex, IDE plugins) talks to it unchanged with one base URL swap. Inference runs entirely on CPU using llama.cpp and Whisper.cpp backends. GGUF model files load from local disk under /var/lib/localai/models and persist across restarts - no external database, no Redis, no cloud egress. Bring your own GGUF model (LLaMA family, Mistral, Qwen, Phi, Gemma) or install one in two clicks from the bundled gallery. MIT license.
Show the rest of the publisher’s description (34 more lines)
Why self-host LocalAI
Running LLM workloads on your own VM keeps prompts, embeddings, and generated content inside your Azure subscription - no per-call billing, no third-party model provider seeing your data. Ideal for teams with data residency requirements (GDPR, HIPAA), organisations evaluating private LLM deployments, and developers prototyping OpenAI-SDK clients against a locally controlled endpoint.
What this VM image adds
Security hardening:
- Per-instance LOCALAI_API_KEY - 24-character random key generated at first boot, enforced by LocalAI for every /v1/, /models, /embeddings, /audio, /images call. Never baked into the image.
- Single credential for SDK and Web UI - the same key authenticates OpenAI-SDK REST clients (Authorization: Bearer) and the Web UI "Login with API Token". One rotation surface.
- LocalAI bound to 127.0.0.1:8080 - the API process never listens on a public interface; only the hardened nginx reverse proxy is reachable externally.
- nginx TLS perimeter - port 443 with self-signed cert, HTTP redirects to HTTPS, security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy) applied.
- Certbot pre-installed - one command issues a Let's Encrypt certificate, no extra apt steps after deployment.
- UFW firewall - default deny; only 22, 80, 443 open. Internal 8080 explicitly blocked.
- fail2ban - SSH brute-force protection.
- AppArmor - mandatory access control.
- CVE scan - every image scanned with Trivy before release.
OS hardening (CIS Level 1):
- CIS Level 1 hardened - CIS Ubuntu 24.04 LTS Level 1 Benchmark via ansible-lockdown.
- auditd - system call auditing for critical paths.
- SSH hardening - PasswordAuthentication disabled, key-only access.
- Kernel hardening - SYN cookies, ASLR, rp_filter, TCP BBR.
- /tmp as tmpfs - nosuid, nodev, noexec.
- Azure IMDS endpoints - egress rules pre-configured (169.254.169.254, 168.63.129.16).
Compliance artifacts (inside the VM):
- SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json.
- CIS Conformance Report - OpenSCAP HTML at /etc/lynxroute/cis-report.html.
- Tailored CIS profile - /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md.
- Server credentials file - /root/localai-credentials.txt with the per-instance API key and public IP.
Quick Start
- Deploy VM from Azure Marketplace (Standard_D2s_v3 or larger recommended; CPU-only inference).
- Open NSG: TCP 443 from your client networks; SSH 22 from your management IPs only.
- SSH: ssh -i key.pem <username>@<PUBLIC_IP> (username set during VM creation, default: azureuser).
- Read credentials: sudo cat /root/localai-credentials.txt - contains the per-instance API key.
- Open https://<PUBLIC_IP>/ in your browser, accept the self-signed certificate, click "Login with API Token", paste the key.
- Install a model: Web UI "Models" tab - pick from the bundled gallery, or drop a GGUF file into /var/lib/localai/models and run sudo systemctl restart local-ai.
- REST API (OpenAI-compatible): point your OpenAI SDK base URL to https://<PUBLIC_IP>/v1 with header Authorization: Bearer <api-key>.
Models are NOT pre-loaded - install on demand. CPU-only - no GPU required. Replace the self-signed TLS certificate with a CA-signed one for production use (certbot pre-installed: sudo certbot --nginx -d your.domain.com).
Agent build and provenance
See the full provenance
The layer-by-layer build, the evidence behind each claim, the risk basis and the cross-marketplace links are open to any account. Some rows are disclosed, some the source leaves Unknown; a free account shows you which.
Compliance
- FedRAMPConfirmedNot listed90%, registry-checkedNo FedRAMP Marketplace entry matched this vendor's domain, checked 2026-08-27registry recordas observed 2026-08-27
Confirmed means matched to a public authoritative registry. Claimed means the vendor or its listing states it, not yet cross-checked. A framework not shown was not found in any source we hold, which is not evidence against it. Not listed means a scoped registry check found no match for this vendor's domain: a No is a scoped registry check, not a compliance judgment. Confidence bands: 95% domain-verified, 90% registry-checked, 80% self-attested, 70% weak signal. Self-attested items marked “vendor's site” are gathered from the vendor's own website and are not verified by us.
Vendor
External enrichment · as of 2026-08-29
Sources
Publisher resources
2 linksLinked repositories
Unknown means this listing does not publish a repository. It is not a statement that the code is closed, and a linked repository is not a claim that the publisher wrote it: the registry computes that relationship privately and does not publish it.
Evidence risk is the share of the build you cannot see before you deploy, not a security rating. Sign in to see the layer-by-layer basis for this band.