Enterprise GPT

Who this document is written for

This whitepaper is aimed at managing directors, IT leaders, department heads and those responsible for digitalisation in small and medium-sized enterprises. It assumes no prior AI knowledge, yet deliberately avoids simplifications that turn out expensive in practice.

Structured in five parts

1. Starting point. Why access to knowledge is becoming the bottleneck today, and what market figures actually tell us.
2. Foundations. What an Enterprise GPT is, how it differs from the Company Brain and from AI employees, and where it is worth deploying.
3. Technology and protection. Architecture, retrieval methods, knowledge sources, permissions, attack surfaces, data protection and governance.
4. Implementation. Quality measurement, pilot selection, the 90-day approach and three detailed case examples.
5. Decision. Business case, acceptance, operating models, vendor selection, a maturity check and common wrong turns.

Corporate knowledge is becoming the bottleneck

Companies have more information at their disposal than ever before. And yet employees frequently have to ask colleagues, search through folders or open several systems before they have a reliable answer.

The scale of this effort can now be quantified. In an international survey of 12,000 knowledge workers and 200 executives at large companies, Atlassian found that teams and leaders spend around a quarter of their working week searching for information. 56 percent of respondents said they often only obtain the information they need by asking someone or scheduling a meeting. One in two knowledge workers also reported that teams within their own company unknowingly work on the same things.

Source: Atlassian, State of Teams 2025 (survey of 12,000 knowledge workers in six countries and 200 executives at Fortune 1000 companies). atlassian.com

The pressure comes from two sides

At the same time, expectations of performance keep rising. In its Work Trend Index 2025, based on a survey of 31,000 employees in 31 countries, Microsoft reports that 53 percent of leaders consider higher productivity necessary, while 80 percent of employees and leaders say they already lack the time or energy for their work. This gap cannot be closed by trying harder. It is a sign that the way work is organised is itself the bottleneck – and access to knowledge is a central part of it.

Source: Microsoft, Work Trend Index Annual Report 2025, published April 2025. microsoft.com/worklab

An Enterprise GPT can narrow this gap. It connects a language model with controlled internal knowledge sources and provides answers together with source references. The main value does not lie in the chat window, but in shortening recurring knowledge processes:

Five foundations decide whether it succeeds

The technology is now the easier part of the undertaking. A production-ready Enterprise GPT requires five foundations that cannot be replaced by choosing a better model.

1. A clearly defined business use case. Not "AI in the company", but a concrete work process with recurring questions, identifiable users and measurable effort.
2. Suitable and owned knowledge sources. Documents with a recognisable validity status, a designated business owner and no contradictory duplicates. Where this foundation is missing, the result is not a better answer but a faster wrong one.
3. A robust permission model. No one may obtain information through the Enterprise GPT that they could not access in the source system.
4. Measurable quality criteria. A reproducible test catalogue with real questions, edge cases and deliberately unanswerable questions. Without measurement, every assessment is a matter of taste.
5. An operating and governance model. Responsibilities, approvals, monitoring, costs and a procedure for errors and changes.

What experience from live programmes shows

Success depends less on the tool than on the organisation around the tool. In its Work Trend Index 2026, based on a survey of 20,000 AI-using employees in ten countries, Microsoft finds that organisational factors – culture, manager support and talent practices – account for around 67 percent of the reported AI impact, while individual factors such as mindset and usage behaviour account for 32 percent. Microsoft explicitly notes that these are statistical associations, not proven causes.

For planning an Enterprise GPT, this has an uncomfortable consequence: a project set up purely as an IT initiative forgoes the larger part of the possible value.

Source: Microsoft, Work Trend Index Annual Report 2026, published May 2026. microsoft.com/worklab

From a general AI chat to corporate knowledge

Generative AI has arrived in everyday business. The next step is to connect general-purpose language models with the context of the individual company.

According to Eurostat, around 20.0 percent of companies in the European Union with ten or more employees used at least one AI technology in 2025. In 2024 the figure was 13.5 percent, in 2023 only 8.1 percent. The gap by company size is considerable: 17 percent among small, 30.4 percent among medium and 55.0 percent among large enterprises. Anyone in the SME sector still waiting today is no longer waiting with the majority.

Source: Eurostat, Use of artificial intelligence in enterprises, reference year 2025, published December 2025. ec.europa.eu/eurostat

For Germany, the digital association Bitkom reported in March 2026 that 41 percent of companies with 20 or more employees use AI. A further 48 percent are planning or discussing its use, and only 11 percent reject it. Of the companies that use AI, 77 percent report an improved competitive position and 52 percent a measurable contribution to business success. At the same time, 33 percent state that AI has led to significantly higher costs than expected.

Source: Bitkom, press release of 11 March 2026, representative survey of 604 companies with 20+ employees, conducted in early 2026. bitkom.org

Adoption is not the same as maturity

An OECD survey of more than 5,000 small and medium-sized enterprises in Austria, Canada, Germany, Ireland, Japan, Korea and the United Kingdom shows that generative AI is used on average in 30.7 percent of the SMEs surveyed – most often in Germany, at 38.7 percent. The survey dates from 2024 and the analysis was published in 2025. One side finding is striking: of the SMEs that use generative AI, only 29 percent use it in their core activities. It is predominantly general-purpose tools at the edge of value creation, not yet controlled enterprise solutions.

Source: OECD (2025), Generative AI and the SME Workforce: New Survey Evidence, OECD Publishing, Paris. oecd.org

This is exactly where an Enterprise GPT comes in. It moves generative AI from the edge into the work process – and in doing so raises requirements that a publicly available chat tool does not have to meet.

What a public AI system cannot know

A general language model knows the world, but not your company. Typically it does not know:

Only access to approved corporate knowledge turns a general AI chat into a workable enterprise application. The difference is not one of degree but one of kind.

The strategic difference at a glance

This gap is the real opportunity. But it is also the reason such projects are more demanding than a website chatbot: internal knowledge is messier, more sensitive and more binding than external marketing material.

Documents are not yet a corporate memory

Many companies do not have an information problem. They have an access and structure problem.

The knowledge that is needed exists. It simply sits in many places at once, in formats created for different purposes and at different times:

Ownership, versions or consistent metadata are often missing. A document exists in four versions, three of them in different folders, and no one can say off the top of their head which one applies. This is not an individual failing. It is the normal consequence of file stores growing over the years while their ordering logic was fixed once at the start.

Why conventional search does not solve the problem

Full-text search finds documents in which a word appears. It does not find the answer to a question that would have to be assembled from three documents. It does not distinguish between valid and superseded. And it does not help when the person searching does not know the technical term the document was written with. These are exactly the three points an Enterprise GPT addresses: consolidation, validity context and language.

Conversely: where good search and a well-maintained file store already exist, the added value is smaller than hoped. The value arises where questions are keeping people busy today.

Typical symptoms in everyday work

Three kinds of knowledge loss

1. Operational knowledge loss. Employees spend time searching, asking and doing work twice. This loss is the most visible and the easiest to quantify.
2. Qualitative knowledge loss. Decisions are made on the basis of outdated or incomplete information. This loss is more expensive but is rarely attributed to access to knowledge, because it becomes visible only later.
3. Structural knowledge loss. When experienced employees leave, context and experience are lost. It cannot be compensated for in the short term – and in times of demographic change it hits the SME sector particularly hard.

What is an Enterprise GPT?

The term "GPT" has become a widely understood label for conversational generative AI. Technically it is imprecise: an Enterprise GPT need not be based on any particular model or vendor. In practice this is good news, because it means the architecture matters more than the brand.

Seven components that have to work together

1. User interface. Chat, search, Microsoft Teams, intranet, mobile app or embedding in a line-of-business system. The best interface is usually the one employees already have open.
2. Identity and permission management. Sign-in, roles, groups, tenants, departments and project-specific access rights – ideally taken from the existing directory services.
3. Knowledge layer. Documents, databases, wikis, line-of-business systems and other approved sources, including their metadata.
4. Search and retrieval component. Determines the content relevant to a question. In practice it drives answer quality more than the language model itself.
5. Language model. Formulates an answer based on the information found – not on its general world knowledge.
6. Orchestration logic. Rules for answer structure, use of sources, refusal and escalation. This is where technical possibilities become operational rules.
7. Quality and security functions. Logging, testing, feedback, monitoring and protection against malicious input.

What an Enterprise GPT is not

A clean scope prevents disappointment later in the project. The following points are regularly expected differently in first conversations.

And explicitly: not a replacement for professional responsibility

The Microsoft Work Trend Index 2026 reports that 86 percent of the AI users surveyed treat an AI system’s output as a starting point rather than a final answer. As the most important human skills they named quality control of AI output (50 percent) and critical thinking (46 percent). This matches practical experience: value rises when people check. It falls when they stop checking.

Source: Microsoft, Work Trend Index Annual Report 2026, survey of 20,000 AI-using employees in ten countries. microsoft.com/worklab

Enterprise GPT, Company Brain and AI employees

In practice these three terms are regularly mixed up. For planning, the distinction is helpful because it dictates the right sequence.

The recommended sequence

1. Organise corporate knowledge. Sources, owners, validity, rights.
2. Build the Enterprise GPT for controlled knowledge queries. Read, do not act.
3. Add recurring tasks. Drafts, summaries, preparation – with human approval.
4. Only then introduce agentic and semi-autonomous workflows.

Where an Enterprise GPT quickly creates value

The best entry point is rarely a company-wide universal solution. A more suitable choice is an area with many recurring knowledge questions and a manageable number of owned sources.

1. Technical service and maintenance

2. Construction and project business

3. Sales and quotations

Further fields and the suitability profile

Particularly suitable are processes in which …

• the same questions arise regularly,
• information has to be brought together from several documents,
• experienced employees are frequently interrupted,
• the answer is checkable because a source exists,
• a wrong result is relevant but controllable,
• there is a department willing to take ownership.

Less suitable are processes in which …

• individual cases dominate and questions barely recur,
• the answer comes solely from structured data and belongs in the ERP,
• no reliable sources exist, only experience in people’s heads,
• a wrong answer immediately endangers people,
• questions are so rare that no one will use the system.

From the user to a reliable answer

A single query passes through several technical and organisational layers. Knowing this flow reveals where quality is created – and where it is lost.

1. The user asks a question.
   Example: "Which maintenance work is scheduled for unit T-400 after 2,000 operating hours?"
2. Identity and permissions are checked.
   Who is asking? Which sites, customers or units may the person access? Which document classes are released for them?
3. The question is prepared.
   Product names, synonyms, abbreviations, document types and, where relevant, earlier conversation content are taken into account. "T-400" becomes the product series, "maintenance" becomes the relevant document classes.
4. Suitable sources are searched.
   A combination of semantic search, keyword search, metadata filters and re-ranking identifies the relevant passages – not the relevant documents, but the relevant passages within them.
5. The language model produces the answer.
   It receives only the selected passages and the operational answer rules. What is not supplied cannot be cited.
6. Sources and confidence cues are returned.
   Source document, chapter or section, document version, validity date and – where available – a note on uncertainty or contradictions.
7. Feedback and logging.
   Usage can be evaluated for quality assurance and improvement, subject to data protection and works agreements.

Target architecture and interchangeability

Why interchangeability should be a project goal

The model market moves faster than any implementation project. Models that lead today may be superseded or significantly cheaper in twelve months. Building your architecture so that the model remains an interchangeable component preserves negotiating room and lets you capture cost advantages without rebuilding the application.

In practice this means three things. The knowledge layer belongs to you and does not sit solely in a vendor’s index. The prompts and business rules are documented and versioned. And the model is connected through an interface behind which another model can sit.

Integration instead of a new interface

A frequent design mistake is to plan the Enterprise GPT as its own portal. Every additional portal competes with the tools employees already have open. In practice, embedding it in Microsoft Teams, in the intranet or directly in the line-of-business application is far more effective than a new address you first have to remember.

RAG, long context and fine-tuning

Several technical methods are available for an Enterprise GPT. They do not compete with one another but solve different tasks.

Retrieval-augmented generation (RAG)

For changeable corporate knowledge, RAG is usually the starting point. The language model is not retrained on all company data. Instead, for each query the system searches for relevant information and supplies it to the model for the answer. If a document changes, the answer changes – without touching a model.

OWASP lists RAG as one building block for connecting answers to verified information sources and reducing misinformation. At the same time, OWASP points out that RAG systems bring their own risks: prompt injection through embedded documents, manipulated or poisoned content, faulty embeddings and the disclosure of information via the retrieval path.

Source: OWASP, Top 10 for LLM Applications 2025 and related guidance on RAG security. genai.owasp.org

Long context

Here, complete documents or large amounts of information are passed directly to the model. This suits the analysis of a single contract, the comparison of a few documents, the summary of a project folder or one-off tasks with a limited amount of data. The limits are high processing costs, longer response times, irrelevant information in the context and limited scalability for large stores. Long context does not replace retrieval – it complements it for special cases.

Fine-tuning

In fine-tuning, a model is adapted using additional examples. This makes sense for fixed answer formats, classifications, company-specific writing styles and recurring structured tasks. It is not the preferred method for frequently changed documents, current prices, project states or individual internal facts. Whoever trains knowledge into a model has to train it in again at the next update – and cannot remove it selectively.

Retrieval in practice: the underrated discipline

In most projects, answer quality is decided not by the language model but by whether the right passage was found. This work is unglamorous and yet the core of the system.

Splitting documents (chunking)

Documents are split into passages that can be searched individually. Split too small, and context is lost – for instance when a table heading is separated from its values. Split too large, and too much irrelevant information ends up in the answer. In practice, splitting along the document’s logical structure works well: chapter, section, inspection step, item. This is exactly why well-structured source documents are a technical advantage, not a formality.

Hybrid search

Semantic search finds things that are similar in meaning, even when different words are used. Keyword search finds exact matches – and you need those for article numbers, fault codes, item numbers or standard designations. "E-47" means nothing semantically. In industrial settings, a combination of both methods is almost always the right answer.

Metadata and filters

A filter that searches only valid documents improves answer quality more than any model upgrade. Useful metadata includes: document type, product series, customer, project, site, validity date, release status, confidentiality level and business owner. These fields need not be perfect, but they must exist.

Re-ranking

After the first search, a manageable set of candidates is assessed again more closely and re-ordered. This costs little and often lifts hit quality noticeably – especially for questions that contain several aspects at once.

AI does not make poor sources better

A powerful language model cannot make up for missing document ownership. Before content is connected, companies should assess their knowledge sources.

Minimum requirements for a source used in production

• professionally reviewed
• assigned to an owner
• marked with a validity state
• released for a defined group of users
• machine-readable – a scanned fax only with extra effort
• sufficiently structured
• not overlaid by contradictory duplicates

Four source classes

Checklist and lifecycle of a knowledge source

Checklist before connecting

The lifecycle that begins afterwards

Connecting a source is not a one-off task. Without a defined lifecycle, the knowledge base ages just as quickly as the file store it is meant to replace – except that the answers are then perceived not as "found somewhere" but as "confirmed by the system". That makes the problem worse, not better.

An Enterprise GPT must not redistribute knowledge

A global knowledge index without differentiated access rights can undermine existing safeguards. What was separated for years by folder structures and group rights is brought back together by a single search interface – unnoticed, because no one intended it.

Critical types of information

Recommended permission model

1. Inherit existing rights. Where possible from Microsoft Entra ID, Active Directory, SharePoint, document management, ERP or CRM systems and project and tenant structures. A second, separate rights model is a permanent source of error.
2. Check rights on every query. Filtering must not happen only at import. The current user context must also take effect during the search, otherwise rights changes apply only at the next import.
3. Protect documents and passages. Not only complete files, but individual passages, metadata and search results can contain confidential information. Even the file name of a contract can be a piece of information.
4. Limit administration rights. Technical administrators should not automatically gain professional access to all content. Separate operations from insight.
5. Control exports and reuse. Check the copying of answers, the download of sources, the creation of summaries, the transfer into e-mails and the use by connected AI agents.

Attack surfaces that did not exist before

An Enterprise GPT connects a language-processing system with internal data. This creates risks that classic applications do not have. OWASP has described them systematically in its Top 10 for LLM and generative AI applications.

Sources: OWASP, Top 10 for LLM Applications 2025 (genai.owasp.org); OECD (2025), Generative AI and the SME Workforce (oecd.org).

Governance: the minimum components

ISO/IEC 42001 provides an international framework for setting up, operating and continually improving an AI management system. For companies already working to ISO 9001 or ISO/IEC 27001, the effort is manageable because structures and procedures can largely be aligned with what already exists.

Involve co-determination early

In companies with a works council, introducing an Enterprise GPT is regularly subject to co-determination. As soon as a system is technically capable of monitoring employees’ behaviour or performance, German law triggers co-determination – and a system that logs queries is objectively capable of this, regardless of whether any evaluation is intended.

Practice points to a clear recommendation: involve the works council not at the end of the pilot but when selecting the use case. A works agreement that limits the purpose of evaluation builds trust among users – and trust is the precondition for employees asking honest questions. Anyone who fears their searches will be assessed will ask cautiously. And cautious questions produce poor test data.

Not the demo but the test decides

Many AI demonstrations work with a handful of prepared questions. A production Enterprise GPT, by contrast, must cope with real phrasings, incomplete questions, abbreviations and contradictory sources.

Building a test catalogue

For a pilot, at least 50 to 150 typical questions should be compiled – by subject-matter experts, not by IT. The catalogue should contain:

Quality metrics

Scoring scheme for pilot questions

• 2 points: professionally correct, complete and traceably sourced
• 1 point: usable in principle, but incomplete or requiring rework
• 0 points: wrong, unsourced, misleading or unusable

Small enough to control, relevant enough to prove

A good pilot is not a non-committal trial access. It examines a real work process with real users and measurable goals.

Scoring matrix

Rate each candidate use case from 0 to 2 points per criterion.

The best pilot is not necessarily the technically simplest. It should create value that employees and management notice in everyday work. A pilot no one talks about has proven nothing – even when all the metrics look right.

Rollout in 90 days: a realistic approach

Technical service: fewer queries, faster preparation

Situation

A technical service provider maintains equipment from various manufacturers. The relevant information sits scattered across manufacturer manuals, work instructions, service reports, spare-part lists, customer folders and the e-mails of experienced technicians. For unusual faults, junior technicians call an experienced colleague, who is pulled out of ongoing tasks several times a day.

Necessary controls

• Safety instructions are prioritised over experience reports.
• Outdated manufacturer manuals are marked as such.
• Customer-specific information is visible only to authorised technicians.
• The system must not add or interpolate safety-relevant limits.
• Contradictory data triggers escalation to the technical owner.

Construction and projects: project knowledge available throughout

Situation

A mid-market company runs several construction and assembly projects in parallel. Information is spread across project drives, e-mails, minutes, plans, specifications and commercial systems. Project managers regularly have to reconstruct what was ordered, which changes were agreed, who made which decision, which documents are missing and which services have already been accepted.

Pilot application

The Enterprise GPT is first set up for three ongoing projects. Connected are the contract and specification, approved variations, site meeting minutes, schedules, acceptance and defect records, relevant correspondence and the project manual.

Typical questions

• Which site-setup services are included in the contract?
• When was the changed execution of item 4.3 agreed?
• Which documents are missing for the partial acceptance?
• Which open items were assigned to the client?
• Summarise the decisions from the last four site meetings.
• Create a draft for the project handover.

The business case: honest instead of impressive

An Enterprise GPT is an investment, not a licence cost. A serious business case names not only the benefit but also the ongoing effort – and does not calculate away the second half.

Cost blocks that are regularly forgotten

The benefit side – and its honest measurement

The benefit falls into two categories. Quantifiable benefit includes saved search and query time, faster onboarding and shorter throughput times. Qualitative benefit includes fewer wrong decisions, less knowledge loss, higher answer quality and greater employee satisfaction. Only the first category belongs in a return-on-investment figure. The second is real, but should not be dressed up with invented numbers – that damages credibility precisely with those who have to approve the budget.

Acceptance decides more than technology

The best system creates no value if it is not used. And whether it is used is decided less by its capabilities than by trust, everyday fit and the way it is introduced.

The Microsoft data cited earlier points in a clear direction: organisational factors account for the larger part of the reported AI impact. For an Enterprise GPT this means acceptance is not a soft accompanying topic but a hard success factor – on a par with retrieval and permissions.

What promotes acceptance

What destroys acceptance

• wrong answers without recognisable sources,
• confident tone despite a thin basis,
• a system that answers questions no one asks,
• an additional portal outside the daily workflow,
• the impression that queries are being assessed,
• a launch without support and without a point of contact.

Operating models: cloud, private or local

There is no universally correct operating model. The right choice depends on protection needs, existing infrastructure, available skills and cost expectations.

The question that structures the decision

Instead of "cloud or local?", ask more precisely: Which data categories occur? Which of them must not leave the company under any circumstances? Which model quality does the use case require? Which operating skills are available in-house? And which costs are plannable over three years? The answers usually point to a hybrid path – sensitive holdings under your own control, general tasks where model quality is best.

Selecting a vendor: the right questions

The market is confusing and moves quickly. The following questions separate a serious offer from an impressive presentation – regardless of the specific vendor.

On knowledge and retrieval

• How exactly are our documents split and prepared?
• Is exact search for article and fault numbers supported alongside semantic search?
• How are metadata and validity states taken into account?
• How does the system handle contradictory documents?

On permissions and security

• Are existing access rights inherited – and checked on every query?
• Does protection also apply at passage and metadata level?
• How does the system protect against prompt injection through documents?
• Where and by whom is our data processed?

On interchangeability and exit

• Can the model be swapped without rebuilding the application?
• In what format can we export the prepared data on a switch?
• Who owns the embeddings and the metadata?
• What happens to our data at the end of the contract?

On quality and operations

• How is answer quality measured – and can we see the method?
• Which logs are available for our own quality assurance?
• How are updates handled without degrading quality?
• Which support and which response times are contractually agreed?

Mistakes that cost time and trust

Most failed AI projects do not fail because of the model. They fail because of avoidable decisions in framing, data and operations.

Eight steps to a well-founded decision

An Enterprise GPT is worthwhile when it is built on the right foundations. The following path leads to a decision that holds – whichever way it turns out.

1. Identify the work process in which recurring knowledge questions cost the most time today.
2. Assess the knowledge sources for ownership, validity, permissions and structure.
3. Clarify responsibilities for the use case, sources, operations and governance.
4. Build a test catalogue with real questions, edge cases and unanswerable questions.
5. Run a limited pilot with real users and measurable goals.
6. Measure quality against defined criteria, not by impression.
7. Decide on the basis of data on expansion, adjustment or termination.
8. Set up operations before any rollout – maintenance, quality, costs and support.

An Enterprise GPT is not an end in itself. It is a means to make existing knowledge usable, to relieve employees and to make better decisions. It works where a company is prepared to create order, take responsibility and measure quality. And it is worth waiting where these foundations are not yet in place.

Sources

All figures are shown with their reference period. Where studies are cited, the respective publisher and the year of publication are given. The regulatory status refers to July 2026.