Vellum

Unified prompt management – Centralize all your prompts for easy collaboration and iteration.

Side-by-side model comparisons – Compare prompts across multiple models with version control and sharing.

Tool and output support – Define tools, structured outputs, and prompt caching with native support for OpenAPI specs, optimizing performance across all models.

Workflows: The IDE for Steerable Agents

Vellum provides a platform for both technical and non-technical stakeholders to experiment with prompts, models, retrieval-augmented generation (RAG) strategies, and AI architectures. Teams can iterate, debug, and deploy AI systems quickly.

All the building blocks needed – Construct graphs with provider-agnostic nodes that call any model, execute arbitrary Python/Typescript code, perform map/reduce functions, and more.

Graph-based system building – Use Vellum’s production-grade graph builder to model AI workflows, debug in real-time, and visualize inputs/outputs for faster iteration.

Advanced orchestration features – Build agentic systems with native looping, parallelism, error handling, and reusable components, ensuring your team creates scalable AI systems.

Evaluations: AI Meets Test-Driven Development

Vellum’s Evaluations framework helps teams measure and improve AI systems at scale. With customizable metrics and powerful testing tools, you can ensure your AI meets performance standards.

Scalable testing – Build a comprehensive test bank through the UI, CSV, API, or capture edge cases in real time, moving beyond subjective evaluations.

Comprehensive evaluation metrics – Use pre-made metrics or create custom ones using Python, TypeScript, or LLMs to handle non-deterministic AI cases.

Track improvements and regressions – Use quantitative evaluations to spot trends, catch regressions, and optimize systems for performance, cost, and latency.

Retrieval: RAG Without the Drag

Vellum simplifies retrieval-augmented generation (RAG), turning unstructured data into context-aware AI solutions. Whether starting with basic RAG or optimizing for complex use cases, Vellum streamlines your workflows.

Easy Setup – Quickly get started with two APIs: one for uploading unstructured data and another for searching it, removing the need for complex ingestion and filtering processes.

Advanced RAG – Modify chunking strategies, embedding models, and search weights to handle complex data workflows. Vellum can also process tables, images, and more.

Data format support – Upload PDFs, text files, CSVs, images, and more into our Vector DB, making it flexible for diverse data types.

Deployments: 10x Faster AI Updates

Vellum enables rapid AI updates, allowing you to make micro-adjustments in production without redeploying your entire application.

One-Click Deploy – Manage prompts and deployments for closed source, open source, or self-hosted models through a single interface, simplifying API integrations.

Staging Environment for AI – Run stable versions in production while testing experimental versions, promoting successful experiments with ease.

Observability: Never Fly Blind in Production

Vellum provides observability features to track how your AI system performs in production. Maintain detailed logs and metrics to quickly identify edge cases and make updates with confidence.

Audit Logging and Data Collection – Maintain logs for compliance and gather structured data for fine-tuning models. Customize retention policies as needed.

Measure AI Quality – Capture real-time user feedback via UI or API and monitor key metrics like request volume, cost, quality, latency, and errors to ensure optimal performance over time.