Build an AI-Powered App with Nhost: RAG on Postgres (pgvector), Real-Time GraphQL, and Serverless Functions

Architecture at a glance

You want Retrieval-Augmented Generation without duct-taping five services and a dozen dashboards. With Nhost, you get a cohesive, production-grade backend where pgvector lives inside Postgres next to your data, the GraphQL API streams real-time changes, Authentication protects every request, Storage handles your files, and serverless functions orchestrate AI workflows. If you need custom models or languages, Bring Your Own Containers (BYOC) lets you deploy services alongside your stack, all wired to the same auth and database. The result: RAG that behaves like one system, not a pile of adapters.

• Postgres with pgvector: documents, chunks, embeddings, and metadata in one transactional store
• GraphQL API: instant queries, mutations, and subscriptions with row-level security
• Auth: role- and claims-based access for users, teams, and AI agents
• Storage: file uploads with a CDN, triggering ingestion workflows on new content
• Serverless functions: low-latency APIs for retrieval, augmentation, and generation
• BYOC: run your own embedding or reranking model without leaving your project

Data model in Postgres (pgvector)

Model your knowledge base so search and permissions stay simple as you scale. A clean pattern for RAG is documents → chunks → embeddings:

• documents: one row per source (file, URL, note), with tenant_id, owner_id, version, and descriptive metadata like title and tags
• chunks: small, referenceable units (e.g., 200–400 tokens) linked to a document_id; include content, position, and section headings
• embeddings: one row per chunk per embedding model; store the vector column, embedding_model, dimensions, and hash for idempotency

Keep strong foreign keys and timestamps, and ensure every table has tenant_id for multi-tenant isolation through RLS. This layout keeps ingestion, re-embedding, and hybrid search efficient while the GraphQL API remains intuitive for your app.

Enable pgvector and create indexes

Turn on the extension once and align your vector dimensions with your chosen embedding model. For approximate nearest neighbor (ANN) performance, add the right index for your query style:

• ivfflat: great default for cosine similarity on high-dimensional vectors; tune lists and probes to balance recall and latency
• HNSW: strong for low-latency search with steady recall; tune M, efConstruction, and efSearch for your data scale

Practical tips:

• Pick a single default embedding model per environment to avoid mismatched dimensions
• Use separate indexes per embedding_model if you support multiple models
• Benchmark with realistic batch sizes and warm caches; then lock parameters in migrations

Row-Level Security for AI data

Enforce per-user and per-team isolation by pushing all access rules into Postgres. Define roles like user and admin with JWT claims. Write RLS policies that:

• Scope every read and write by tenant_id and owner_id
• Allow admins to act across the tenant while keeping users restricted
• Apply consistently to documents, chunks, and embeddings so vector queries inherit the same boundaries

Because vector search runs in SQL, RLS gates results before they reach your GraphQL API or serverless functions, preventing accidental cross-tenant leakage.

Ingestion via Storage and Functions

Make ingestion a one-click operation: upload to Storage, and let a serverless function do the rest. On file insert, the function can read the file, parse it (PDF, HTML, markdown), normalize headings, chunk it into the right token window, and enqueue embedding jobs. Store the original file path for preview and reprocessing, and write chunks immediately so clients can see progress in real time.

Best practices:

• Keep chunk size consistent across your dataset; avoid massive chunks that dilute retrieval quality
• Write idempotent ingestion logic using content hashes to skip duplicates
• Attach source metadata like author, version, and URL to support filtering later

Use the Nhost AI toolkit for embeddings

The fastest path to production is to call managed providers using the Nhost AI toolkit. From a function, batch texts for throughput, add exponential backoff, and set timeouts and maximum token guards. Log token counts per tenant so you can enforce quotas and forecast costs. The AI toolkit gives you stable interfaces as providers evolve, so your RAG pipeline stays resilient without chasing SDK updates.

BYOC embedding service

Prefer owning the model path, language, or runtime? Deploy a containerized embedding service with BYOC and talk to it over HTTP from your serverless functions. Co-locating this service with your database reduces latency, and your private weights never leave the project. This vendor-neutral setup lets you iterate on models, quantization, or rerankers without changing the rest of your stack. For a local-to-prod parity walkthrough, see Local Development with the Nhost CLI: Run Postgres, GraphQL, and Auth in Containers (BYOC Step-by-Step).

Real-time sync your knowledge base

Knowledge shifts. Keep embeddings fresh with change-data-driven reprocessing. Use database event triggers to detect content edits, mark affected chunks as stale, and enqueue re-embedding. Clients subscribe via the GraphQL API to watch document status, chunk counts, and last-embedded timestamps. When a user fixes a typo or adds a new section, updated results appear almost immediately, preserving trust in your RAG responses.

Hybrid search as a GraphQL action

Vector-only retrieval can miss exact keywords; keyword-only search can miss semantic paraphrases. Blend the two. Expose a SQL function that scores cosine similarity from pgvector alongside BM25 or trigram matches, then returns a ranked list with highlighted snippets and metadata. Publish that function as a GraphQL action so clients get a typed, cacheable endpoint through the same GraphQL API they already use.

Make it flexible:

• Accept filters for document type, tag, author, or update window
• Return facets (e.g., counts by tag) for instant UI filters
• Include provenance fields like document_id, chunk_id, and score breakdowns

Filters, facets, and per-tenant scoping

Push tenant and access filters into the SQL function and rely on RLS to enforce boundaries. Your function should never receive or return data for a tenant that the caller cannot access. This keeps filtering fast, consistent, and safe, while the GraphQL API remains simple for your frontend and agents.

Retrieval-Augmented Generation Function (streaming)

Package the full retrieve → augment → generate path into a single serverless function that streams tokens to the client. The flow:

• Normalize the query, run hybrid search, and select the top-k chunks
• Construct a compact context window with citations and route the prompt to your model
• Stream tokens to the client using Server-Sent Events (SSE) for smooth UX
• Log usage, store the trace, and record which chunks were used for auditability

You can apply per-user and per-team quotas mid-stream, fall back to a smaller context on provider errors, and redact sensitive fields before they ever reach a model. If your app needs agentic tools, the same function can call internal APIs for lookups, calculators, or structured actions while still streaming results.

Auth and security defaults

Nhost Auth issues JWTs with roles and claims that your GraphQL API, serverless functions, and Storage interpret consistently. Use it to:

• Enforce tenant scoping by default with RLS across all AI tables
• Protect embedding and generation endpoints with role checks and IP allowlists when needed
• Store model keys and provider secrets in environment-scoped secrets, never in code
• Apply per-user rate limits and request-size caps to keep abuse and costs in check

Building agents that call your backend? Follow the guidance in How to Add OAuth2 and JWT Authentication to Your AI Agents (Step-by-Step Guide) to securely issue and rotate tokens.

Performance and cost controls

RAG should be fast and predictable. Tune your pipeline with these proven levers:

• Chunking: 200–400 tokens per chunk balances recall and context size; over-large chunks reduce precision
• Vector dimensions: choose a compact model if latency and storage matter; align dimensions across environments
• Index parameters: for ivfflat, adjust lists and probes; for HNSW, tune M and efSearch; benchmark with real queries
• Batching: embed and upsert in batches sized to avoid timeouts; the AI toolkit can handle retries and backoff
• Cold starts: pre-warm serverless functions on deploys and schedule lightweight keep-alives during peak hours
• Background jobs: re-embed in the background after edits; expose “freshness” so users know when recalculation is pending
• Reranking: optionally apply a lightweight reranker on the top 50 results for precision without heavy compute
• Quotas: track token and request budgets per tenant; cap burst rates to prevent runaway costs

CI/CD and Git deployments

Ship your entire RAG stack through Git. Commit database migrations, metadata, and serverless functions. On push, Nhost builds and deploys functions and BYOC images and applies migrations safely. Secrets are environment-scoped so staging can use sandbox providers while production uses hardened keys. Observability is built in: logs, metrics, and traces help you catch outliers quickly.

Need to compare platforms before you commit? See the tradeoffs in Nhost vs Supabase vs Appwrite (2026): Open-Source Backend Comparison for Postgres, GraphQL, Auth, and Pricing.

What you’ll ship

With this architecture on Nhost, you get a multi-tenant, real-time RAG backend powered by pgvector. Your app exposes a typed GraphQL API for hybrid search, streams tokens from a serverless function, secures every call with Auth, and keeps files and metadata in sync. You can attach your own embedding or reranking models via BYOC without re-architecting anything. Most importantly, you will not babysit infrastructure while you scale.

Next steps

Once the core loop is live, extend it confidently using the same primitives:

• Add tools and agents that call internal APIs and enrich answers with actions
• Schedule re-ranking or reconciliation jobs to improve search quality over time
• Record audit logs of prompts, contexts, and outputs for compliance
• Ship analytics dashboards for retrieval quality, token spend, and latency
• Invite your team and scale tenants from a handful to thousands without rewrites

Conclusion

RAG goes from demo to dependable when your vectors, auth, files, and compute operate as one. Nhost gives you that single, production-ready backend: pgvector right in Postgres, a real-time GraphQL API, serverless functions for orchestration, Storage for ingestion, and BYOC for custom models—all deployable via Git. Use the Nhost AI toolkit to move fast on embeddings and generation today, and swap providers or models tomorrow without friction. Build smarter, ship sooner, and scale without limits.

Get started for free.

Frequently Asked Questions

How do I enable pgvector and create ANN indexes on Nhost’s managed Postgres?

Enable the pgvector extension in your database, then create a vector column on your embeddings table that matches your model’s dimension. Choose an ANN index such as ivfflat or HNSW. Tune parameters (for example, lists and probes for ivfflat; M and efSearch for HNSW) against real queries and data sizes. Commit these changes as migrations so your environments stay in sync.

What’s the best way to stream LLM tokens to the client with Nhost (GraphQL vs SSE in Functions)?

Use Server-Sent Events (SSE) from a serverless function for token streaming. GraphQL subscriptions are ideal for real-time data updates from Postgres, while SSE gives you low-latency, backpressure-friendly streams for generation. You can still publish final messages or trace IDs via the GraphQL API for chat history and analytics.

How do I isolate tenants so vector search can’t cross data boundaries?

Include tenant_id on documents, chunks, and embeddings. Turn on Row-Level Security and write policies that filter by tenant_id using JWT claims. Because vector queries execute in SQL, RLS applies before results leave the database, ensuring cross-tenant isolation for both hybrid search and pure vector lookups.

Can I run my own embedding or reranker model with BYOC, and how does it talk to Functions and Postgres?

Yes. Package your model server in a container with BYOC. Your serverless functions call it over HTTP for embeddings or reranking, then write vectors and metadata back to Postgres. Auth, secrets, and networking are unified within your Nhost project, keeping latency low and operations simple.

When should I keep vectors in Postgres vs use an external vector database with Nhost?

For most apps, keeping vectors in Postgres with pgvector is faster to ship, simpler to operate, and more secure under RLS. Choose an external vector database only if you have extreme scale or specialized features that outweigh the complexity of an extra system. With Nhost, you can start in Postgres and evolve later without rebuilding your GraphQL API or functions.