Table of content
Introduction
At a time when companies are beginning to integrate generative artificial intelligence (AI) tools, Microsoft is deploying this technology in its Microsoft 365 suite with an IA assistant called Copilot.
But beyond the hype and before we are all replaced by robots, what is actually the REAL impact of Copilot? What are the actionable use cases that can be applied not just for a demo, but for frequent tasks?
To demystify the fantasies surrounding AI in the workplace, we are sharing here feedback from multiple people who have reviewed Copilot extensively.
What is Copilot?
Copilot serves as a conversational chat interface enabling various tasks such as information retrieval, text generation for emails and summaries, and the creation of images based on written prompts.
Copilot can be accessed in different ways.
Go to the Copilot website (free)
Open your browser, then go to copilot.microsoft.com. Click on "Sign in", then enter your Microsoft account or Entra username and password.
Use Copilot in your Bing or Edge browser (free)
With Copilot in Bing, you can switch between asking Copilot or doing a regular web search.
You can also access Copilot from Edge Web Browser, in the sidebar.
Get Copilot on Microsoft 365 (PowerPoint, Excel, Word, Outlook) - $20 (individuals) to $30 (organizations) per user per month
Copilot for Microsoft 365 is designed to aid in tasks such as email drafting, data analysis in Microsoft Excel, and creating proposals in Word and presentations in PowerPoint. It integrates with your data to assist you.
Copilot is available on Microsoft 365 as Copilot Pro for individuals on a Microsoft 365 personal or family subscription and is also available to every organization with Copilot for Microsoft 365.
How does Microsoft 365 Copilot work?
Copilot combines the power of large language model (or LLM) technology with your data in the Microsoft Graph and the Microsoft 365 apps with a conversational interface.
It's a "generative AI" using the Open AI LLM of ChatGPT and integrating it into the Microsoft 365 apps (Word, Excel, PowerPoint, Outlook, and Teams) to serve as a virtual assistant.
For more information about the Copilot architecture, please refer to this article from the Microsoft Learning Center.
How does Copilot ensure data protection and privacy?
According to Marcel Broschk at Microsoft, Copilot encrypts your data at rest and in transit using industry-standard encryption protocols. Copilot also isolates your data from other tenants using Entra ID (former Azure Active Directory) authentication and authorization. Copilot only accesses your data when you explicitly request it and does not store or share your data with third parties.
Copilot is compliant with Microsoft's existing privacy, security, and compliance commitments to Microsoft 365 commercial customers, including the General Data Protection Regulation (GDPR) and European Union (EU) Data Boundary. Copilot respects your data residency preferences and does not transfer your data across regions without your consent.
How much does Copilot cost?
Below is a recap of Microsoft Copilot's pricing page. As you can see, the web version of Copilot might be free, but having Copilot your favorite in Microsoft 365 applications has a cost of $20 per month for individuals, and $30 per month for organizations.
There is actually no monthly plan, only an annual plan. And since there is also no trial version, it means you have to be ready to actually pay $240 or $360 to even just try it.
Copilot faces some big limitations
It appears right away that Copilot comes with some big limitations.
Copilot only works for files saved in OneDrive or SharePoint. If your files are not saved on OneDrive or SharePoint, the Copilot button will remain greyed out and will not be usable in any application.
Copilot can be very slow. Some actions can let you hanging for 30s before you see a result.
Prompting AI takes time! Don't be fooled by all the accelerated videos out there, writing sentences in natural language to ask Copilot a question takes time. In many, many cases it will take the users more time to write the question than to do it themselves by clicking the right button.
Copilot poses compliance and security issues. When you use Copilot, data from your emails, files and prompts is stored and can be viewed by your IT admin.
See this source for a more detailed list of these limitations.
Additional limitations of Copilot in PowerPoint
While Copilot can serve as a valuable source of inspiration for generating slides on generic topics, it seems inefficient when dealing with specialized and complex subjects requiring specific and original answers. The process of adjusting content with additional prompts and then refining the format in PowerPoint can become time-consuming, particularly for complex topics.
Copilot currently lacks the capability to format slides and charts efficiently. It doesn't know your organization's theme, so it will create presentations that don't comply with your corporate identity. And even if you ask it, it can’t change the slides formats like fonts, size.
While Copilot could help new users with private usage, like students, its utility may be limited for professionals and people with experience of PowerPoint. For them, it may be used more marginally for certain tasks and scenarios, particularly in providing summaries or generating initial drafts on broad topics.
Additional limitations of Copilot in Excel
One could think that because Excel contains mostly structured and quantitative data, Excel would be the ideal playground for Copilot. But surprisingly, Excel is not where Copilot works best, at least not for now.
In fact, Copilot in Excel is still advertised as being just a "Preview", to manage user expectations.
Copilot only works if your data is stored inside a table object. But most Excel users don't use tables (although they should, as explained why in this post on 12 reasons to use Excel tables)! If you don't work with tables, again Copilot will be simply unusable.
Copilot only works if AutoSave is ON. If its OFF, sorry but no Copilot.
Copilot doesn't work when you have more than a few rows of data. Even for a dataset of just 20 rows, Copilot seems to be struggling. But for larger sets in the hundreds, thousands or tens of thousands, it's just out of the question as Copilot will just crash.
Copilot says "sorry" way to often. In many situations, Copilot will be unable to perform the requested action, even for things that don't seem that complicated. Copilot doesn't know how to format a chart by changing a color or font for instance. It may not understand the question in some cases and do in some others, although from a user perspective it's not clear why.
See Chandoo's honest review of Excel Copilot below:
Copilot in Excel only understands English at the moment.
With its current capabilities, Copilot seems to have only the potential to help inexperienced Excel users perform a few tasks that they just wouldn't know how to do, like applying conditional formatting. But people with Excel experience already will find it much faster to click the right button to get the right result, rather than write prompts and hope AI will do it the way they hope.
Additional limitations of Copilot in Word, Teams and Outlook
Copilot performs better in Word, Teams and Outlook. It seems to be quite efficient at working with pure text, and that makes these applications the right place to get value using Copilot, assisting you with paraphrasing, summarizing, and rewriting.
However, Copilot won't work in the following cases:
Word documents with more than 18,000 words
Teams meetings that are hosted outside the participant's organization
The Teams meeting has not been transcribed
Very long Outlook threads, as the Summarize functionality may not use older messages
Review of Copilot use cases in Microsoft PowerPoint
Since the release of Copilot, multiple experts and consultants on Microsoft 365 have reviewed how it performs for multiple use cases in PowerPoint. We have compiled here a summary of the limitations it faces and the use cases it can address.
✅Crafting and updating a presentation on a broad subject topic
This is usually the main use case that is showcased on Copilot in PowerPoint: how good is it at creating presentations?
Check the below video From Lui Iacobellis, at 1:48.
When asked to create a presentation on the future of renewable energy with 5 slides integrating pictures and text in a logical order, Copilot was able to do so with a good enough result. The first draft could then be improved through another prompt, introducing additional text.
✅Organizing the presentation logically according to the content
In the same video, after disorganizing all the slides, the presenter asked Copilot to reorganize the presentation logically based on the content.
Copilot managed to rearrange the slides into cohesive sections and generated an agenda, ensuring a logical structure.
❌Generating speaking notes for your slides
Unfortunately, when asked to generate speaking notes for the slides, Copilot was unable to do so.
✅Summarizing a PowerPoint presentation into concise bullet points
At 4:40, Copilot successfully summarized a PowerPoint presentation into succinct bullet points, offering a quick overview of the key points covered.
✅Swapping an image with another one
In the below video, at 10:35, Copilot swapped an image with another one taken from the web, and readjusted the slide to account for the new image, changing the text position and format.
❌Translating a slide from one language to another one
When asked to translate a presentation, Copilot was not able to do the translatation. Additional textboxes were added on top of the slide, messing up the presentation.
It was however able to create an additional slide in a specific language.
❌Replacing a font with another
When asking Copilot to replace a specific font with another one, it did not manage to do it properly. Some shapes were replaced, some weren't, so the result didn't seem reliable. Yet Copilot stated that it successfully did, giving the user a false sense of security while the action had not been performed.
❌Searching for a slide
Asked to insert a slide from another file, Copilot failed to do so, even when provided with the path to the file where it was supposed to look.
Review of Copilot use cases in Microsoft Excel
Despite some serious limitations (see above), there are still some use cases that Copilot manages to address.
Check the below video from Kevin Stratvert.
✅Answering basic inquiries about the data
Copilot is able to answer some basic questions about the data, such as:
Giving the number of distinct values in a column.
Answering a question on which customer made the largest orders.
The answer can also be inserted sometimes in a new sheet, sometimes as a formula (althought not necessarily the best formula, as it appears it tends to over-complicate it), and sometimes as a PivotTable.
✅❌Adding some calculation in a new columns
Copilot was able to add a "Profit" column calculating the difference between 2 existing columns containing the revenues and costs.
But this will not always work. When asked to extract the domain from a column with email addresses, it was not able to do that.
✅Applying conditional formatting
When asked for basic conditional formatting, such as highlighting values above a certain threshold or the top 10 values, Copilot was able to successfully add that conditional formatting.
❌Filtering data
Asking Copilot to filter data might end up with surprising results, as sometimes it seems to filter the wrong information.
✅Creating charts
If you ask Copilot to create a chart, it will indeed be able to create a basic chart, with no specific formatting applied. It can even create a series of charts to give basic visualization of the data in your table.
However it remains to be seen how useful this is, except maybe for really non-analytical people, considering that it takes much long to ask Copilot to create a chart than to click on the ribbon to create it.
The charts may also be created including irrelevant data, such as blank cells that can make the chart inacurrate if you don't spot the issue.
❌Formatting charts
Most importantly, when asked to change the color of a line chart, Copilot was unable to do this very basic formatting action. More complex formatting are just even more out of the question.
❌Calculating more advanced statistics
If you ask Copilot to calculate financial aggregated or similar statistics from raw data, the AI fails to give a result.
❌Getting benchmark data
When asking Copilot to provide some benchmark data, like growth rate for the same market as your company, it was unable to do so.
Review of Copilot use cases in Microsoft Word
In Word, Copilot demonstrated the ability to perform several tasks. You can see some in the below video from Scott Brant.
✅Creating a specific document
With some basic information and Copilot, it’s possible to draft emails with different tones (direct, neutral, casual, formal) and lengths (short, medium, long). In the example, Copilot wrote a business plan for a Coffee Shop based in London targeting 20–40-year-old people.
✅Refining content
You can fine tune the draft and add some more details or specifications to regenerate another draft.
✅Creaing a table from text
Based on a text with data like market value, number of outlets and some growth rates, Copilot was able to generate tables.
✅Quickly summarizing documents
In the Copilot side bar, you can ask Copilot to summarize a document. With that summary, you get a quick overview of a long document content.
Review of Copilot use cases in Microsoft Teams
Copilot can listen to your meetings in Teams, and then restitute the information.
The below video from Gold Standard reviews Copilot's use cases in Teams.
✅Writing a summary of the meeting
After a Teams meeting, you can ask Copilot to summarize a meeting, and it will perform rather well, giving you a first draft should you write meeting minutes for instance. It can also be useful so that you don't have to take notes while participating in the meeting.
✅Listing action items
To make sure that the meeting is efficient, you can list the expected actions assigned to each participant.
✅Creating an email with a recap and the actions
Finally, you can ask Copilot to write an email including the meeting recap and actions to each participant.
Review of Copilot use cases in Microsoft Outlook
The below video from Jonathan Edwards features a review of Copilot in Outlook.
✅Drafting an email to a specific audience
With Copilot, it’s possible to draft emails with different tones (direct, neutral, casual, formal) and lengths (short, medium, long).
✅Giving advice on an existing email
Copilot can read your drafts and give you advice and suggestions in terms of tone, reader sentiment and clarity.
✅Summarizing long emails
With Summary by Copilot, it’s possible to create short recaps of emails. This can save time when you receive long emails and just want to extract the essentials.
Conclusion
Copilot comes with tremendous expectations. It works well on several use cases, but falls short on many others due to serious limitations.
As for now, this tool seems to be more appropriate to inexperienced users of PowerPoint and Excel, and to novice or experienced users of Word, Outlook and Teams due to its stronger suite being to handle text.
But one can expect the technology to improve over time, and to mitigate these limitations while adding more use cases to the tool.