Why AI pilot projects stall
Most companies have already tried AI: a few chatbot licences, a proof of concept for reading invoices or a workshop with a vendor. Far less often does a solution move into everyday work, where it delivers measurable effects. The reasons repeat in practice and are rarely linked to the choice of model.
- Data is not ready. Documents exist in several versions, key data sits in personal spreadsheets, systems are not connected to each other. A model that gets poor input data gives unreliable answers, and user trust quickly disappears.
- Access rights are too broad. In a Microsoft 365 environment, over the years many folders, sites and groups build up that are shared more widely than necessary. As long as only a person searches them, the risk stays hidden. An AI assistant that can search everything exposes it quickly.
- The project has no owner. The pilot is run by IT or an enthusiastic individual, while the business department has neither the mandate nor the time. When the pilot ends, nobody decides on continuation, budget and expansion.
- No baseline for measurement. Without a measured state before the pilot, you cannot prove the solution improved anything, so it is hard to justify the investment in production.
The good news is that all four obstacles are organisational and can be removed with planning. Check where you stand today with the self-assessment on this page.
The readiness framework: six areas
The self-assessment follows six areas that together decide whether an AI rollout will succeed. Established frameworks such as the NIST AI Risk Management Framework and the OECD AI Principles use similar logic: technology is only one part, and people, processes and risk management are just as important.
| Area | Key question | Sign of readiness |
|---|---|---|
| Strategy and use cases | Which business problem are we solving? | A ranked list of use cases with measurable goals |
| Data | Is data of good quality, accessible and owned? | Data owners and classification by sensitivity |
| Technology and infrastructure | Is the environment secure and connectable? | Permissions in order, MFA, DLP, sensitivity labels, API |
| People and skills | Do employees know how to use AI safely and sensibly? | An AI literacy programme and a programme owner |
| Governance and compliance | Who may use what and under which conditions? | An AI use policy, an inventory of systems, GDPR assessment |
| Delivery and results | Do we know how to take a pilot to production? | Pilot experience, measurement of effects, budget |
The tool weights the questions: those that often stop a project count more, for example the state of access rights, data quality and a clear business goal. The overall score places you in one of four maturity stages: exploring, piloting, adopting or scaling.
Microsoft 365 Copilot: what to prepare before rollout
For many companies Copilot is the first contact with generative AI in everyday work, because it works in the tools employees already use: Outlook, Teams, Word, Excel and PowerPoint. Microsoft has since renamed the product Microsoft Copilot, although the earlier name still appears in licences and documentation. Rollout is technically simple, but preparation is not.
Licensing through a CSP partner
Basic Copilot Chat is included in suitable Microsoft 365 licences and answers mainly from the web and from content the user attaches themselves. For answers from your organisation's email, documents, meetings and chats you need the additional Microsoft 365 Copilot licence. You can arrange licences through a Microsoft CSP partner, who helps you choose the right combination and adjust it to the number of users. At Humanfrog we are a CSP partner, so we tie licensing to preparing the environment.
Access rights and oversharing
According to Microsoft's documentation, Copilot only shows a user content they have at least read access to. That is both a safeguard and a risk: if a salaries folder is shared with the whole organisation by mistake, Copilot will find it. Before rollout, therefore, review SharePoint sites and Teams shared with everyone, links that anyone can open, and inactive sites. Microsoft offers tools for this, such as SharePoint Advanced Management.
Sensitivity labels and DLP
Sensitivity labels in Microsoft Purview let you label confidential documents and encrypt them where needed. For encrypted content, Copilot respects the usage rights the user has. Data loss prevention (DLP) policies and mandatory multi-factor authentication complete this protection.
User adoption
A licence alone does not change the way people work. Start with a pilot group from several departments, prepare usage scenarios by role (meeting summary, draft offer, analysis in Excel), appoint champions and monitor usage in the admin centre reports. Only then extend the licences.
AI agents, Copilot or a custom solution
Not every problem suits Copilot, and not every problem has to be solved with development. The choice depends on whether it is about personal productivity, automating part of a process, or a company's specific knowledge and data.
| Copilot for Microsoft 365 | AI agents | Custom solution | |
|---|---|---|---|
| Purpose | Helping an individual write, summarise and search | More independent execution of process steps | A specific process, data or privacy requirements |
| Examples | Meeting summaries, email drafts, analysis in Excel | Handling requests, customer support, preparing orders | Document processing, an internal knowledge assistant (RAG), a local model |
| Data | Microsoft 365 content the user can access | Microsoft 365 and business systems through connections | Selected sources, including outside Microsoft 365 |
| Prerequisite | Permissions in order, labels, adoption | Integrations, clear rules, human oversight | A development team, maintenance, model management |
| Time to impact | Weeks | Weeks to months | Months |
Agents can be built with tools such as Microsoft Copilot Studio, or as standalone solutions connected to the ERP, CRM and document system. When data must not leave your environment, a custom solution with a local or private model makes sense.
From pilot to production in 90 days
For a clearly bounded use case with accessible data, 90 days is a realistic frame to get from selection to a first production version. The condition is that the decision and the owner are known from the start.
| Period | Activities | Result |
|---|---|---|
| Weeks 1–2 | Choosing the use case, owner, baseline measurements, risk and GDPR assessment | Confirmed scope and success criteria |
| Weeks 3–4 | Preparing data and permissions, security settings, choosing the technology | Environment ready |
| Weeks 5–8 | Pilot with real users and data, weekly improvements | Measured effects and feedback |
| Weeks 9–10 | Assessing results, decision on production, cost plan | Management decision |
| Weeks 11–13 | Production deployment, training, monitoring and support | The solution in everyday use |
How to measure return
The return on AI is easiest to prove for processes you already measure today. Before the pilot, record processing time, number of errors, response time or cost per transaction, and after rollout compare the same indicators. On the cost side, count everything: licences or consumption, integrations, data preparation, security, training and maintenance.
For personal productivity tools such as Copilot, time saved is harder to measure precisely. Track actual usage, user surveys and a few selected scenarios where the effect can be measured directly. An unused licence is a cost with no return, so reallocate licences regularly.
Governance and the basics of the AI Act
The AI Act (Regulation (EU) 2024/1689) also applies to companies that only use AI. Since 2 February 2025, all providers and deployers must ensure sufficient AI literacy among their staff (Article 4). Further obligations depend on the risk class, which in turn depends on the purpose of use, not on the tool. The same language model can be minimal risk in one case and high risk when used to rank job candidates.
To start, you need three things: a short AI use policy, an inventory of AI systems classified by risk, and a procedure for assessing the protection of personal data, including a data protection impact assessment (DPIA) where needed. To check the risk class of an individual system, read the guide EU AI Act: risk classes.
At Humanfrog we help companies from the readiness assessment and preparing the Microsoft 365 environment to developing agents, document processing and internal knowledge assistants. Write to us and we will define together the first use case that will deliver a measurable result.
