Recent research from Harvard Business School (HBS) clarifies the complex dialogue surrounding the use of artificial intelligence in the business world and shows a clear path to how you can be 40% more productive in work.
The study, "Navigating the Jagged Technological Frontier" examined the productivity and output quality of 758 consultants at the Boston Consulting Group (BCG). It reveals how AI can increase both productivity and quality within certain domains, while leading to suboptimal results in others.
The study provides robust evidence for the value of training your employees in AI. As a company, it's important to find the right balance. Some tasks are simply better and more efficiently performed by or with AI, while others require the human touch. The challenge is to smartly combine these two worlds.
The key to maximising the benefits of AI lies in understanding and accurately identifying the limits of AI capabilities and strategically deploying AI where it is most effective.
The concept of a "Jagged Technological Frontier" is crucial to understanding the uneven progress of AI. While AI delivers remarkable performance in some areas, other areas remain unreachable. This inconsistency creates a landscape full of peaks and valleys, where the effectiveness of AI can drastically vary depending on the specific task. For example, AI models today excel in translating texts but sometimes fall short in answering simple questions.
This unevenness in AI capabilities showed significant implications for the productivity and quality of work among employees within the HBS study:
Within the boundaries of AI's capabilities, where tasks can be well performed by AI, there were significant improvements in both productivity and the quality of work. The consultants were given 18 realistic tasks including exercises in creativity, analytical thinking, writing skills, and persuasiveness to track changes in employee productivity and accuracy.
The study found that, compared to employees without AI access, those who used GPT-4:
- completed an average of 12% more tasks
- achieved results an average of 25% faster
- 40% of the test group produced higher quality results
These tasks benefited from the speed, efficiency, and capabilities of AI, leading to improved performance among the professionals who used AI.
In addition, the study included tasks that fell outside the boundaries of AI’s capabilities, and thus were more complex or required a deeper understanding that AI cannot yet provide. For example, the consultants were given a business case with spreadsheet data and customised interviews, where both the data and insights from the interviews required thorough analysis to uncover details that were not immediately visible. This led to conclusions that AI could not reach with only the task instructions and the available information.
This type of task showed a decrease in quality of as much as 19% among the consultant group with AI access, although they still completed more tasks and did so faster. Professionals had an excessive trust in AI for tasks for which AI was not adequately equipped, resulting in less accurate or less effective results.
This concept of a "jagged technological frontier" emphasises the importance of accurately identifying the boundaries of AI capabilities and strategically deploying AI where it is most effective.
Moreover, the study underscores the importance of preparatory training for users of GPT-4. The group of employees who were allowed to use GPT-4 were divided into two groups: one with prior training and one without. Although the offered training was limited to a short session of 5 to 10 minutes, there was already a clearly noticeable difference in the performance of those who had received this initial training and those who had not. They consistently scored better percentages than those who had no training.
Instead of a productivity increase of 38% among GPT-4 users, those with training had a 42.5% increase. This indicates the potential for significant improvements in efficiency and effectiveness with longer and more in-depth training programs.
The key to success thus lies in recognizing and identifying the "Jagged Technological Frontier" and strategically navigating within these boundaries. This way, companies can optimally benefit from the advantages of AI, while at the same time minimising the risks of excessive dependence and incorrect application.
Moreover, the benefits of using AI were not limited to individuals with high performance. Consultants who scored below the average performance level in the control tasks saw their scores increase by 43%, while those above the average saw an increase of 17% compared to their baseline scores. This suggests that AI can enhance the work performance of all employees, regardless of their starting level.
Training and awareness play a crucial role. Equipping employees with the knowledge to use AI technologies correctly not only helps them cope with its complexity but also how to optimally utilise it. This presents both a challenge and an opportunity for organisations to lead in the digital transformation.
Read more about our AI training programs for companies here ... (link).
There will always be tasks outside the ‘frontier’, the challenge being that we don't always intuitively know which tasks those are. Also, the ecosystem evolves quickly and the "Jagged Frontier" is always shifting.
Therefore, as a leader in an organisation, you need to develop good guidelines on how AI should be used within the organisation and regularly update these. These guidelines should not only indicate which types of tasks are suitable for AI support but must also be flexible enough to evolve with the changing capabilities of AI technology.
AI can be effectively used for tasks that are primarily language-based, such as generating text, summarising information, or translating between languages.
Given sufficient contextual information, AI can be a valuable tool for brainstorming. It can help generate ideas or provide inspiration for creative processes.
AI can provide support in decision-making processes, suggesting various options or simulating potential outcomes. This again depends on the availability of relevant and sufficiently detailed information.
AI can provide convincing but erroneous information, mixed with half-truths. Do you like using ChatGPT to google? Then Perplexity AI could be a good alternative.
There remains a difficulty around effectively navigating or modifying complex tasks, especially when multiple tasks are asked simultaneously or in a complex context. Here, prompt engineering can also be a useful skill to make better use of language models.
AI does not provide reliable expertise in specialised areas, where it can provide incorrect information. Think, for example, of fields such as law, finance, or medicine. Tools like ChatGPT can be used for inspiration in legal cases and even asked to consider specific legal texts, but even here they can hallucinate, so it is best to still have a human review come into play.
.png)
.png){kind=avatar}
.png){kind=avatar}