How to manage your AI projects: From Proof of Value to Production
“Every company is a technology company.”
-Gartner
This quote from Gartner represents businesses increasingly adopting sophisticated technologies to achieve their goals. Artificial Intelligence is one of the most important business opportunities in 2019. Knowing what it takes to apply these new technologies to work at scale and give a greater competitive edge, is the real differentiator.
In the following chapters, we provide a guide on how to manage your AI projects from proof of value to production.
As with any other technology, a business considering AI should undergo a Proof of Value (POV) to get started. With a POV you will have proof that an AI solution will, in fact, meet your business expectations. Most of these projects fail, mainly due to the fact that most teams get stuck building one or more POVs with little to no progression to an applied AI.
We previously talked about the 2 things to consider before starting an AI project, suggesting to clearly plan the impact your AI will have across the organisation, get buy-in from every department, and understand the limitations. In this blog, we identify the reasons why often AI projects don't continue beyond a POV and the challenges of productionising your solution.
Why Proof of Values fail?
According to a recent report from McKinsey 50% of enterprise claim to have “embedded some form of AI”, however, “only 21% of respondents report embedding AI into multiple business units”. From our experience in building multiple Applied AI solutions for enterprise, we usually start with a Proof of Value. However in some cases, the POV doesn't make it through to production, so we wanted to address the reasons why:
1. Misunderstood requirements. Make sure you know what the requirements are and what you are trying to solve/achieve from this implementation. The POV isn't the place to refine the problem, but where you prove the value, so there has to be a clear problem you're trying to solve or value you're looking to create in the first place.
2. Lack of ownership. If you're working with external parties, they will need clear communication as per point 1. However so will in-house teams where you'll need buy-in from multiple department stakeholders to ensure your AI solution will be adopted more widely. Clear ownership by each department is crucial, but so is clear direction from one project lead. Get buy-in but be clear on who has the final say.
3. Poor change management. Understand which teams will be affected by the implementation of the new AI system and try to communicate the long term vision and objective. Engage people early on who will be using the product. They're ultimately the ones who are affected, and also the ones who can make or break a POV.
4. Data modelling. The difference between generating the input data needed by a Machine Learning model for a POV and doing it continuously and at scale is important. The time and energy required to get the data needed are often underestimated.
In a POV, all forms of data modelling have to simplify the reality, therefore, some fidelity is always lost in the process. Ensure you cater for the drop in accuracy when moving to production, and have the right people on hand to retrain, and maintain the system while it deals with its new environment.
5. No definitive endpoint. A POV should end as soon as the desired objectives are archived, avoiding gold-plating the results to make it work if they aren't successful. Be clear when starting the project on what the Acceptance Criteria will be so that your data scientists and engineers have a clear goal in mind.
From POV to Production
In the Production step is not rare to completely change the way your system works. It's also safe to assume that new problems will arise as you're are getting closer to the final release of the implementation. Let's keep in mind that production systems won't work on sample data but they need real-life data used to solve a real-life problem.
Not only, moving towards the final stage of AI industrialisation, AI will be implemented in multiple areas of the business and perhaps made available to a few users/customers to try. This step's challenges include enterprise-scale infrastructure, security and support challenges.
Organisations should monitor and invest in many Proof of Values as they can relatively inexpensively learn about their potential, without getting lumbered with a useless piece of software. It's good practice to quickly kill the ones that aren't going anywhere, and identify the narrower group of promising ones to continue monitoring and investing resources into them.
Once you've identified the POV works and the project moves to a Production stage, new challenges arise in further data modelling and training.
As all the forms of data modelling are simplified for the POV, it might cause modelling issues during production. A solution to this is to minimise the gap between the real-world requirements and the POV data set. Add more detail to the model by having more fields, tables, relationships, etc., meaning that you'll potentially need access to more data. It's important to point out that with more data, the modelling becomes harder to carry out and understand. You'll need to make sure that the data scientists (in-house or external) have the right expertise to create an accurate model for your AI product.
There's an important step of any IT implementation which we like to think of as the AI's dress rehearsal. Getting the POV deployed into testing and then staging environment will submit it to near enough real production environments, that will allow you to iron out any bugs.
The most important part of the industrialisation of the applied AI is the people who will take it over. Your teams will need to train and upskill in the system. IT teams will have to work on a new set of skills and infrastructure to implement, to keep the AI running and be able to fix issues. Plus BAU teams will need to understand the systems early limitations, and what's a bug and what's not. Otherwise, people will give up on the system before its hand a chance to learn in the real world.
By choosing the right partner for your AI projects, you're ensuring solid forethought, planning, expertise and best practices. Experts will know how to treat your data without putting it at risk as well as strengthen competitive advantage by providing your teams with appropriate training and support.
Let us solve your impossible problem
Speak to one of our industry specialists about how Artificial Intelligence can help solve your impossible problem