Halkali merkez neighborhood fatih st ozgur apt no 46 , Kucukcekmece , Istanbul , 34303 , TR
A RAG system contains multiple services from document ingestion to answer generation. Extraction, chunking, embeddings, vector DB, reranking, LLM and citations each carry different latency, quality and security risks. Production RAG is not one container.
It depends on workload. Generation can dominate GPU cost, bulk embedding can require significant compute, vector search consumes RAM/NVMe and reranking adds online latency. Each layer should be sized independently.
Documents are extracted/chunked, embedded and stored in a vector DB. Query embeddings retrieve candidates, a reranker selects top context, the LLM generates an answer and source metadata supports citations.
A single server can work initially; as you grow, separate the most constrained layer.
Measure retrieval recall, source precision, reranker gain, answer faithfulness and citation correctness with evaluation sets.
Tenant/user permissions should propagate into retrieval filters. Unauthorized chunk retrieval is a data leak even if the final LLM refuses to answer.
Adapt ports and model names to your stack.
Turkey (Turkish)
Germany (German)
Worldwide (English)
curl -s http://127.0.0.1:6333/healthzcurl -s http://127.0.0.1:8080/info | headcurl -s http://127.0.0.1:8000/v1/models | headnvidia-smidf -hIt is common for large semantic corpora, but small fixed datasets may use full-context or traditional search.
No. It can improve candidate ordering, but measure the gain against latency and compute cost.
Retrieved context is sent to the generation model. If that model is an external API, context may leave your infrastructure.
Share corpus/token volume, daily queries, vector DB, reranker and LLM; we can size ingestion and online serving separately.