A Company Brain does not automatically require a standalone vector database. For many mid-sized businesses, PostgreSQL with pgvector is the more economical architecture because operational data, permissions, metadata, and embeddings remain in one governed system. Pinecone, Weaviate, Qdrant, or Milvus become more compelling when retrieval scale, tenant isolation, workload volatility, or platform operations grow materially more demanding.
Why should the database not be the starting point?
Vector database selection often begins too early in a Company Brain initiative. The first questions should be which information employees or customers need to retrieve, which authorization rules must apply, and how the organization will judge answer quality. An internal assistant for operating procedures has a different workload from a customer-facing portal serving many independent accounts. A proposal assistant for a field service company also needs different filters from an enterprise search layer spanning contracts, inspection reports, service tickets, engineering drawings, and product documentation.
Make company knowledge easier to access
The KrambergAI Company Brain makes scattered knowledge from documents, projects, processes and internal sources easier to find and prepares answers with traceable context.
Implemented pragmatically · Source-based answers · Made in Germany
The vector store is only one component in a longer retrieval chain. Source documents must be ingested, normalized, versioned, segmented, enriched with metadata, and linked to access policies. Embeddings are then generated, the retrieval layer selects candidates, a reranker adjusts their order, and only the selected context reaches the language model. A more specialized database cannot compensate for poor chunking, missing metadata, outdated source material, or authorization checks applied too late.
The useful architecture question is therefore not “Which database wins the fastest benchmark?” A production Company Brain must also handle source traceability, deletion, re-indexing, disaster recovery, monitoring, cost allocation, model changes, and policy enforcement on every request. Those requirements usually influence the decision more than raw nearest-neighbor latency.
When is PostgreSQL with pgvector enough?
PostgreSQL (https://www.postgresql.org/) with pgvector (https://github.com/pgvector/pgvector) is often a strong fit when PostgreSQL already serves as the application database and the first Company Brain use cases are internal. User identities, business entities, source-document metadata, approval status, retention attributes, and embeddings can participate in the same transactional model. That reduces synchronization jobs, duplicate identifiers, and additional operational dependencies.
pgvector supports exact and approximate similarity search. HNSW and IVFFlat indexes are available for larger collections. According to the project documentation, HNSW generally offers a stronger search-speed and recall tradeoff but requires more memory and longer index builds. IVFFlat builds faster and uses less memory, while requiring more careful training and query tuning. PostgreSQL full-text search can also be combined with vector results through reciprocal rank fusion or a reranking stage.
This combination covers many practical Company Brain workloads: policies, procedures, service manuals, statements of work, inspection records, project folders, proposal language, product catalogs, and support knowledge. SQL filters can restrict retrieval by legal entity, business unit, facility, customer, product line, language, approval status, document type, or effective date. Existing database practices for backups, roles, encryption, observability, and change management can remain in use.
The technical range is also sufficient for common text-embedding models. The pgvector vector type can be indexed at up to 2,000 dimensions. This figure does not define the total amount of data that PostgreSQL can hold, but it demonstrates that ordinary embedding width alone does not force a separate vector platform.
Where do pgvector’s practical limits appear?
The limit rarely appears as a single threshold. It usually emerges from a combination of collection growth, concurrent requests, selective filters, frequent updates, index maintenance, and increasingly complex tenant boundaries. With approximate indexes, filtering can reduce the candidate pool after an index scan. pgvector provides iterative scans, partitioning, partial indexes, and separate tables to address this behavior, but those mechanisms require deliberate design, realistic testing, and ongoing tuning.
A shared PostgreSQL environment can also become problematic when transactional workloads and retrieval workloads compete for the same resources. Bulk ingestion, embedding refreshes, HNSW index builds, or sudden chatbot demand may interfere with customer portals, workflow transactions, or operational reporting. Read replicas, dedicated databases, and separate PostgreSQL clusters can reduce that contention. At some point, however, the organization may be building a specialized search platform inside a general-purpose database.
Tenant isolation is another boundary. The pgvector documentation notes that tenants sharing an approximate index can influence one another’s recall and performance. PostgreSQL list partitioning or separate tables provide stronger isolation, but administration becomes more involved as the number of customer accounts, project workspaces, subsidiaries, and protected departments grows.
The practical signal is not “PostgreSQL has become bad.” The signal is that retrieval has become an independently scaled service with its own availability, security, lifecycle, and capacity requirements. At that point, moving the vector workload can reduce coupling and give the platform team more specialized controls.
When does a specialized vector database become worthwhile?
A specialized vector database becomes worthwhile when retrieval itself is a shared platform capability. Typical indicators include multiple applications using the same search layer, highly variable demand, many isolated tenants, advanced hybrid retrieval, multi-stage ranking, separate scaling for storage and queries, or a requirement to operate search independently from business transactions.
Pinecone (https://www.pinecone.io/) is a fully managed option for teams that want to minimize database operations. Namespaces support tenant separation, and hybrid search combines semantic and lexical signals. The tradeoff is a deliberate dependency on an external platform, so procurement should evaluate data location, contract terms, portability, consumption behavior, and what metadata may be stored. Pinecone limits a query response to 4 MB. For a Company Brain, this mainly affects designs that return oversized records instead of compact identifiers, metadata, and relevant text passages.
Weaviate (https://weaviate.io/) combines vector, keyword, and hybrid search with a broader data model and built-in multi-tenancy. Tenant data can be assigned to dedicated shards, which fits platforms with many customer workspaces that share a common schema. Weaviate is available as a managed cloud service and through self-managed deployment options. That flexibility is valuable, but the organization must own the additional schema, indexing, and operational decisions.
Qdrant (https://qdrant.tech/) is often attractive when metadata filtering, deployment control, and composable query logic are central requirements. Payload indexes participate in the HNSW search path rather than operating only as a final filtering step. Qdrant also provides several tenancy patterns, including logical payload separation, tenant-aware indexing, and custom shards. Managed cloud, hybrid, and self-hosted deployment models are available.
Milvus (https://milvus.io/) is oriented toward distributed, very large, or infrastructure-intensive retrieval platforms. It supports hybrid search and several isolation levels, while its distributed deployment model is designed for Kubernetes. That can be excessive for an ordinary internal knowledge assistant, but it can fit high-volume, multimodal, or cross-application retrieval services. In an official benchmark using 10 million vectors, a compressed index variant retained more than 94 percent recall. This is not a universal product guarantee; it illustrates the kinds of compression and throughput tradeoffs that matter at larger scale.
How do pgvector, Pinecone, Weaviate, Qdrant, and Milvus compare?
The following assessment is not a universal ranking. It reflects common Company Brain architecture patterns for mid-sized organizations. Each candidate still needs to be tested with the organization’s documents, filters, security model, update frequency, and concurrency profile.
| Option | Typical Company Brain fit | Advantages | Drawbacks and operational impact |
|---|---|---|---|
| PostgreSQL with pgvector (https://github.com/pgvector/pgvector) | Internal Company Brain, limited tenant complexity, existing PostgreSQL expertise | Shared transactions for business data and embeddings, SQL filtering, established backup and access processes, fewer platform components | Retrieval may compete with application traffic; filtering and partitioning require tuning; horizontal search scaling is less specialized |
| Pinecone (https://www.pinecone.io/) | Managed retrieval service, lean platform team, variable demand | Low infrastructure burden, managed serverless model, namespaces, hybrid search | External service dependency, consumption costs, less control over database operations, migration work if the provider changes |
| Weaviate (https://weaviate.io/) | Knowledge platform requiring integrated schema, hybrid search, and many similarly structured tenants | Vector, keyword, and hybrid retrieval; tenant shards; managed and self-managed deployment | Additional schema and operations expertise; a broad feature set can create unnecessary platform scope |
| Qdrant (https://qdrant.tech/) | Filter-heavy RAG, self-hosting, hybrid and multi-stage retrieval | Strong payload filters, flexible query pipeline, managed, hybrid, and self-hosted options | Separate operating component; self-hosted replication, backups, upgrades, and capacity planning remain the customer’s responsibility |
| Milvus (https://milvus.io/) | Distributed retrieval platform, very large or multimodal collections | Distributed architecture, multiple index and tenancy models, hybrid retrieval | Highest operational entry cost among these options; commonly oversized for a limited internal knowledge system |
The comparison points to a practical rule: the best database is often the one whose operating model matches the organization. A mid-sized business without a platform engineering team gains little from a Kubernetes-based search cluster when PostgreSQL is already operated reliably. A software company serving many isolated customer workspaces may reach the opposite conclusion because a native tenancy model removes substantial custom application logic.
Which option fits each operating model?
For an internal Company Brain with a moderate source set, predictable usage, and an established PostgreSQL environment, pgvector is often the sensible starting point. This is particularly true when identities, permissions, document state, and business objects are already modeled relationally. A unified data path simplifies updates, deletions, audit trails, and incident investigation.
For a managed service with minimal DevOps ownership, Pinecone is a natural candidate. The application team can focus on ingestion, retrieval evaluation, and user experience while the vendor operates the search infrastructure. That convenience should be weighed against vendor dependency, cost governance, data-location requirements, and export strategy.
Weaviate or Qdrant often fit when retrieval behavior becomes a product differentiator. Weaviate provides a broad integrated search and data layer. Qdrant provides a focused engine with sophisticated filtering and query composition plus flexible deployment choices. Milvus belongs on the shortlist when distributed scale is a real requirement and the organization has the engineering capability to operate the surrounding platform.
A separate managed PostgreSQL instance with pgvector is also an important middle path. It isolates retrieval from transactional workloads without immediately introducing a new database technology. For many companies, that step captures most of the operational benefit while preserving familiar SQL, security tooling, backup procedures, and staff skills.
Why do hybrid search and metadata filters matter so much?
A Company Brain should not rely on semantic similarity alone. Part numbers, standards, contract clauses, error codes, equipment tags, customer names, and policy identifiers require lexical matching. At the same time, employees frequently phrase questions differently from the source documents. Hybrid search combines both signals and is generally more relevant to operational knowledge than an isolated vector benchmark.
Metadata filters are equally important. A result must not only be topically related; it must come from the correct subsidiary, facility, product generation, contract version, customer account, approval state, and effective period. Field service use cases may additionally require asset class, maintenance status, trade, site, and project phase. These filters must be tested in realistic combinations because their selectivity changes approximate-search behavior.
PostgreSQL with pgvector can implement hybrid retrieval by combining full-text search with vector ranking. Pinecone, Weaviate, Qdrant, and Milvus provide specialized hybrid or multi-stage query mechanisms. The difficult part is rarely the checkbox labeled “hybrid search.” It is choosing lexical and semantic weighting, defining reranking, handling language-specific tokenization, and validating how ranking changes when documents, models, or filters change.
For US businesses, this is also an auditability issue. When a system answers questions about procedures, customer commitments, safety instructions, or regulated operations, the team should be able to identify which source passage entered the prompt and why it outranked competing passages. Retrieval logging and evaluation therefore belong in the platform design, not only in model testing.
How should tenant isolation be designed for customers and departments?
Multi-tenancy is more than a metadata field named tenant_id. A Company Brain must prevent one customer’s, subsidiary’s, or project’s information from appearing in candidate lists, caches, logs, traces, or generated answers for another user. Authorization should constrain retrieval itself rather than being applied only after generation.
For internal deployments, role-based filtering inside a shared database may be sufficient. Contractually separated customer data may require partitions, schemas, tables, namespaces, collections, or dedicated shards. The right boundary depends on deletion obligations, restore procedures, key management, noisy-neighbor risk, and whether cross-tenant search is ever permitted.
Pinecone recommends namespaces for tenant separation. Weaviate associates tenants with dedicated shards. Qdrant distinguishes logical filtering, tenant-aware indexing, and custom shards. Milvus offers isolation at database, collection, partition, and partition-key levels. pgvector relies on PostgreSQL partitioning or separate tables for stronger isolation. These choices should be made before ingestion because retrofitting tenant boundaries usually requires re-indexing and data migration.
The application layer still matters. Authentication claims must map to retrieval filters, cache keys must include the relevant authorization context, and observability data must avoid leaking source text. Backup and restore procedures also need to preserve the selected isolation model. A database feature cannot repair an application that sends the wrong tenant identifier.
What usually goes wrong in Company Brain projects?
The first recurring failure is over-architecture. A team introduces a separate vector cluster, event streaming, multiple caches, and elaborate tenancy logic while the initial use case covers a limited internal document set. Infrastructure consumes the budget that should have gone into source preparation, permissions, evaluation questions, and user workflow integration.
The second failure is treating document count as the decisive sizing metric. A long manual can produce many chunks, while a large folder can become much smaller after deduplication and removal of obsolete versions. Actual vector count, index footprint, update frequency, filter distribution, and concurrent demand are more useful planning inputs.
The third failure is weak metadata. Without effective dates, source type, customer, product, language, ownership, and approval status, no database can reliably select the right context. Global caches are another risk when tenant, role, document version, or policy state is missing from the cache key.
The fourth failure is benchmarking without business questions. Random prompts do not represent production. A useful evaluation set should include sales, service, engineering, quality, operations, legal, and administrative questions. It should also contain difficult cases involving similar document versions, conflicting passages, exact identifiers, missing evidence, and users with different permissions.
The fifth failure is ignoring content lifecycle. A retrieval system can produce excellent initial results and still become unreliable when superseded documents remain searchable, deletions do not propagate, or changed source files generate duplicate chunks. Index maintenance must follow the same governance discipline as the source systems.
How can the choice be made without overengineering?
A defensible selection begins with a small but representative corpus. It should include different source types, authorization levels, languages, update scenarios, and search patterns. Evaluation should examine more than the generated answer: retrieved chunks, filter behavior, ranking, source attribution, latency, cost, and the system’s response when evidence is absent.
PostgreSQL with pgvector is often a useful reference architecture. The application stores document and chunk metadata relationally, generates embeddings through a separate service, and combines vector retrieval with full-text search. A specialized database should enter the same evaluation only after the team can describe a reproducible limitation.
The migration trigger should be measurable: unacceptable latency under real concurrency, declining recall under filters, excessive index-maintenance time, insufficient tenant isolation, missing horizontal scaling, or operational burden that exceeds the value of remaining on PostgreSQL. “The specialized platform sounds more future-proof” is not enough to justify another stateful system.
The team should also test reversibility. Export the source records, embeddings, metadata, and identifiers. Rebuild an index in a clean environment. Compare results before and after migration. A Company Brain should not become dependent on an undocumented index that cannot be reproduced from governed source data.
Which architecture is usually appropriate for a mid-sized business?
For many mid-sized organizations, a staged model is economically sound. The first production release uses an existing or separately operated PostgreSQL instance with pgvector, a documented ingestion pipeline, authorization filters, hybrid retrieval, reranking, and production monitoring. This keeps the number of new components limited while the business gathers real usage data.
As adoption grows, retrieval can be separated from the operational database. The relational source should usually remain authoritative for document state, permissions, and business entities. The specialized vector database receives only the fields required for retrieval, filtering, and source references. It becomes a replaceable search service rather than an uncontrolled second system of record.
A customer-facing platform with many tenants may justify a specialized database earlier. Tenant provisioning, isolation, data deletion, backups, restore, workload separation, and cost allocation then become product capabilities from the start. The deciding factor is less the company’s headcount than the role retrieval plays in its product and operating model.
For organizations with strict internal hosting requirements, Qdrant, Weaviate, Milvus, or a dedicated PostgreSQL deployment can be evaluated as self-managed options. The apparent control comes with responsibility for patching, vulnerability management, high availability, backups, observability, and capacity. Self-hosting is beneficial only when the organization can operate the service consistently.
What should a proof of concept demonstrate?
A proof of concept should demonstrate that the system retrieves the right sources under real authorization rules. It needs business-reviewed reference questions, expected documents, unacceptable sources, and cases where the correct behavior is to provide no answer. It should also measure how quickly new or changed content becomes searchable and whether deletion removes data from indexes, caches, and downstream artifacts.
Operational behavior matters as much as retrieval quality. How are failed imports detected? What happens when the embedding model changes? Can embeddings and indexes be rebuilt without a prolonged application outage? How are index versions tracked? Which metrics reveal degraded retrieval? Can a single tenant be restored without altering unrelated data?
Security testing should verify the full path from identity to retrieval. Role and tenant claims should be manipulated in negative tests, logs should be inspected for source leakage, and cache isolation should be validated. A model refusing to reveal data is not an adequate security boundary if unauthorized passages were already retrieved.
Only after these questions are answered does a comparison among pgvector, Pinecone, Weaviate, Qdrant, and Milvus become actionable. Without that foundation, the team is comparing product catalogs rather than suitability for its Company Brain.
Bring AI into daily operations in a structured way
The KrambergAI AI Introduction helps companies select suitable use cases, prepare workflows and integrate AI solutions into everyday operations in a controlled and practical way.
Structured implementation · Practical guidance · Made in Germany
Which sources support the figures used in this article?
Sources for the figures
- pgvector: supported dimensions for indexable vector types
https://github.com/pgvector/pgvector - Pinecone: query-result size limit
https://docs.pinecone.io/guides/index-data/indexing-overview - Milvus: RaBitQ benchmark, compression, and recall
https://milvus.io/blog/turboquant-rabitq-vector-database-cost.md
Which resources are useful for further study?
Further reading
- PostgreSQL: full-text search as a building block for hybrid retrieval
https://www.postgresql.org/docs/current/textsearch.html - Qdrant: production checklist for operating vector search
https://qdrant.tech/documentation/production-checklist/ - Weaviate: best practices for data modeling, tenants, and search
https://docs.weaviate.io/weaviate/best-practices
Frequently Asked Questions About a Vector Database for a Company Brain
Does every Company Brain need a separate vector database?
No. PostgreSQL with pgvector is sufficient for many internal knowledge systems, especially when user permissions, document metadata, and business entities already live in relational tables. A separate vector database becomes persuasive when retrieval must scale independently, many tenants require stronger isolation, or specialized search and operations features justify another stateful platform component.
Is pgvector suitable only for small collections?
No. pgvector can support substantial collections when indexes, memory, filters, and query patterns are designed appropriately. Its practical boundary depends on concurrency, update frequency, filter selectivity, index maintenance, and competition with transactional workloads. The organization should therefore test its own retrieval quality and operating profile instead of relying on a generic vector-count threshold.
When is Pinecone a good fit for a Company Brain?
Pinecone fits teams that want a fully managed retrieval service and prefer to minimize infrastructure ownership. Namespaces, managed scaling, and hybrid retrieval are relevant advantages. Before adoption, procurement and engineering should assess data location, contract terms, cost behavior, export options, service limits, and dependence on provider-specific features that may complicate a future migration.
When is Qdrant a good choice?
Qdrant is well suited to RAG systems that depend on sophisticated metadata filters, composable hybrid queries, and flexible deployment. It is especially attractive when filtering should participate directly in vector retrieval. With self-hosting, however, the customer still owns high availability, patching, backup verification, capacity planning, observability, and safe upgrades.
When does Weaviate fit better than pgvector?
Weaviate fits better when the Company Brain requires a dedicated search platform combining keyword, vector, and hybrid retrieval with a built-in tenant model. It can also suit many similarly structured customer workspaces. The added value appears only when those capabilities are used and the team can operate its schema, indexing choices, upgrades, and deployment model.
Which use cases justify Milvus?
Milvus is primarily relevant to very large, distributed, or multimodal retrieval platforms. It offers several index and isolation models plus a distributed operating mode. For a limited internal assistant, that architecture may be excessive. It becomes reasonable when scale, workload separation, and platform engineering are genuine core requirements rather than anticipated future possibilities.
Why is vector-only search often insufficient?
Operational questions frequently contain exact terms such as part numbers, standards, error codes, contract clauses, or equipment identifiers. Semantic similarity may not rank those terms reliably on its own. Hybrid retrieval combines lexical and semantic signals, while reranking and metadata filters promote sources that are relevant, current, authorized, and appropriate for the business context.
How important is multi-tenancy for a Company Brain?
Multi-tenancy is critical when the system processes knowledge from different customers, subsidiaries, departments, or protected projects. A metadata field alone may not be enough. Retrieval, caching, logging, deletion, backup, and restore must respect the same boundary. Filters, partitions, tables, namespaces, collections, or shards may be used depending on the required isolation.
Can the vector database be replaced later?
Yes, when the application and data model are intentionally decoupled. Governed documents, permissions, and business metadata should remain in an authoritative system. A retrieval service can hide database-specific queries behind a stable interface. Embeddings, chunk identifiers, metadata, and index versions must also be reproducible so a replacement index can be built and evaluated.
Which criteria matter more than a public benchmark?
Retrieval quality on the organization’s own documents, authorization behavior, ingestion delay, deletion, tenant isolation, operating effort, and total cost matter more. A public benchmark rarely reflects the same language, chunking strategy, filter distribution, update rate, and concurrency. Selection should rely on a representative corpus and business-reviewed reference questions.
Should a mid-sized company start directly with a specialized platform?
It can make sense when the roadmap already includes a customer-facing multi-tenant service, highly variable demand, or retrieval as a standalone product capability. For an internal Company Brain, PostgreSQL with pgvector is often the lower-risk starting point. The application should still isolate retrieval behind an interface so it can move later without rebuilding everything.
All articles about company brain
