Will the assistant make things up?

Any LLM can produce wrong statements. A well-built RAG system reduces this sharply by answering only from retrieved passages, citing them, and saying “I don’t know” when nothing relevant is found. We measure faithfulness on your test set so you know the real rate for your data.

Can it respect our existing permissions?

Yes. We carry access-control information from the source systems into the index and filter at query time, so a user only gets answers built from documents they can already open.

How much data do we need?

RAG doesn’t need training data — it needs the documents people already use. A few hundred pages are enough for a meaningful proof of concept. The harder question is usually quality and ownership of those documents, not quantity.

Which model do you use?

Whichever meets your requirements for quality, cost, latency and data residency. The architecture is model-agnostic; we often use a smaller model for query rewriting and a stronger one for the final answer.

How long until it’s live?

It depends on the number of sources and integrations. Use our AI project effort estimator for an orientation; a proof of concept on one source usually comes first.

Service · Retrieval-augmented generation

An LLM that answers from your data — and shows where it found it.

Retrieval-augmented generation (RAG) connects a language model to your documents, wiki, tickets and databases, so answers come from your knowledge, not from the model’s memory. We build RAG systems that are accurate, permission-aware and measurable — not a chatbot that sounds confident and is wrong one time in five.

Two pipelines: ingestion keeps the index fresh; the question pipeline finds the right passages and answers from them.

When RAG is the right tool

RAG shines when people spend time finding and re-reading information that already exists somewhere in your company. Typical signals: support agents searching the knowledge base during calls, engineers asking the same questions about internal systems, sales teams digging through product docs, or compliance teams checking policies. If the problem is behaviour or format rather than missing knowledge, RAG is the wrong tool — our RAG vs fine-tuning guide explains the difference.

Support copilot

Draft replies to tickets from help articles, past resolutions and product docs, with links agents can verify.

Internal knowledge assistant

One place to ask about policies, processes and systems across Confluence, SharePoint, Notion or Google Drive.

Customer-facing help

Answers on your website or in your app, restricted to public documentation and with a clean hand-off to a human.

Sales & pre-sales

Answer security questionnaires and RFPs from approved content, with a reviewer in the loop.

Engineering assistant

Questions about your codebase, runbooks, incidents and architecture decisions.

Contract & policy search

Find clauses and obligations across hundreds of documents, with exact quotes.

Where RAG quality actually comes from

When a RAG system gives a bad answer, the model is rarely the culprit. Most failures happen earlier: a PDF table was parsed into gibberish, the relevant paragraph was split across two chunks, the search returned a similar-sounding but outdated document, or the answer lived in a spreadsheet nobody indexed. That’s why we spend most of the effort on the pipeline, not on the prompt.

Common failure points and how we handle them
Stage	What goes wrong	What we do
Parsing	Tables, scans and multi-column PDFs turn into noise	Layout-aware parsing, OCR where needed, table extraction checked on real files
Chunking	The answer is split across chunks or drowned in irrelevant text	Structure-aware chunks (headings, sections) with overlap tuned on your content
Search	Pure vector search misses exact terms: product codes, names, error IDs	Hybrid search (keyword + vector) and a reranker for the final ordering
Freshness	Old versions of a policy outrank the current one	Incremental re-indexing, version metadata and date-aware ranking
Permissions	The assistant quotes documents the user may not see	Access-control filters applied in the search query, not after the fact
Answering	The model fills gaps with plausible fiction	Answer-only-from-context instructions, “I don’t know” paths and mandatory citations

How we measure a RAG system

Before building, we collect a set of real questions with known good answers — typically 50 to 200, written with the people who will use the system. Every change to parsing, chunking, search, prompts or models is re-run against that set. You see the numbers, not just a few hand-picked demo questions. More on this in LLM evals for non-ML teams.

Retrieval recall: did the right passage make it into the context at all?
Answer correctness: is the answer right, judged against the reference?
Faithfulness: is every claim supported by the cited sources?
Refusal quality: does it say “I don’t know” when the answer isn’t in the data?
Latency and cost per question, tracked alongside quality so trade-offs are visible.

Reference architecture

Interfaces

Web chat
Slack / Teams bot
Helpdesk sidebar
API

Orchestration

Query rewriting
Hybrid retrieval
Reranking
Prompt assembly
Citations

Safety & quality

Permission filters
Prompt-injection checks
Evals in CI
Feedback capture

Ingestion

Connectors
Parsing & OCR
Chunking
Embeddings
Incremental sync

Storage (your cloud)

Vector / search index
Metadata DB
Object storage
Audit logs

Each layer is replaceable: models, vector stores and connectors can change without a rewrite.

For storage we pick what fits your stack: PostgreSQL with pgvector when you already run Postgres, Azure AI Search on Azure, OpenSearch or Elasticsearch where you have them, or a dedicated vector database for very large collections. The trade-offs are in our vector database comparison.

How a RAG project runs with Nythrex

1
Source audit (days, not weeks)
We look at your real documents and systems: formats, volume, permissions, how often they change and who owns them.
2
Test set
Together with future users, we write the questions and reference answers the system must get right.
3
Proof of concept
A thin end-to-end pipeline on a representative slice of your data. Output: accuracy, failure analysis, latency and cost per question — and a go / no-go.
4
Production build
Connectors for all sources, permissions, UI where people already work, monitoring, feedback loop, evals in CI and deployment in your cloud.
5
Operate and improve
Weekly review of unanswered and down-voted questions; they become new test cases and content fixes.

What you own at the end

0/5

Frequently asked questions

GuideRAG vs fine-tuning vs promptingRAG adds knowledge, fine-tuning changes behaviour, prompting is where you start. How to choose — with a decision tree, costs and pitfalls.Free toolRAG vs fine-tuning decision toolUp to four yes/no questions → a recommendation: RAG, fine-tuning, long-context prompting or better prompts, plus the next step to validate it.Example projectSupport copilot for a B2B SaaSIllustrative RAG project: a Zendesk support copilot drafting cited replies from help articles and tickets — discovery, PoC, architecture, lessons.ComparisonVector databases comparedVector stores for RAG: pgvector, Pinecone, Qdrant, Azure AI Search and OpenSearch compared on ops, hybrid search, filtering, scale and cost.GuideLLM evals for non-ML teamsHow to evaluate an LLM feature without a data-science team: build a test set, choose metrics, use LLM-as-judge carefully, run evals in CI and read the results.GuideAn AI support chatbot that doesn’t embarrass youBuild an AI support chatbot customers trust: grounded answers, clear scope, human hand-off, safe actions, abuse protection and honest metrics.

Want a second opinion on your project?

Tell us what you’re building and where you’re stuck. We’ll reply within one business day with the most practical next step — even if that step isn’t us.

Start a project