2 things to consider before starting an AI project

Clients often contact us with questions and are curious to know about what AI can do in their organisations. However, they are often caught by headlines about AI changing this or that industry, hung up on the idea that AI will solve all their problems. Therefore, they jump on starting an AI project right away.

Most of these rushed projects fail, mainly due to the fact that most Machine Learning and AI consultancies get stuck building one or more Proof of Concepts (PoC) with little to no progression to an applied AI solution. Enterprises tend to isolate processes and departments from others, with a minimal budget and overlook the importance of having a complete AI roadmap that the entire organisation can benefit from.

Having a  clear plan across the whole organisation, and buy-in from every department, plus a clear understanding of the limitations, will set your AI up to succeed.

In this blog we focus on two main limitations, that we’ve come across from our experience in building AI for enterprise clients: people and data.

People

Two fundamental questions you should ask to start with:

  1. Do you have the right people to run and manage an AI project?
  2. Do the people in the organisation have the right mindset, processes and authority to make an AI project a success?

Whether or not an AI expert is running your AI project, you essentially need someone that can understand the limitations and constraints both on a technical side and the business side. Having stakeholders from all relevant departments jump on this project is extremely valuable for your AI solution. For example, a Content Recommendation Engine will require marketing, customer, and IT teams to be involved.

IT people won’t necessarily need to know the algorithms that will be used but will need to understand the principles behind them. They will need to understand the main participants to involve especially when using a Supervised Machine Learning model that will require the right kind of training from domain experts. For example, for a Content Recommendation Engine, this might be an administrator or content curator.
(Read our latest project with Breast Cancer Care: BECCA’s Recommendation Engine).

Be clear with all stakeholders what it’s going to take to make an AI project a success from the outset, and get everyone bought in with clear costs and timelines set against a Return On Investment.

Process limitations tie with the data limitations that will be mentioned in the second part of this blog. Manual processes to perform a task are valuable to the design and build of an AI solution, for example, let’s take Document Classification for use in an Incident Management system. Customer Support Level 1, writes a ticket which typically includes a brief description of the problem. If that L1 support ticket doesn’t include enough detail or is miss representing the real problem in some way, then when it comes to using those tickets for training a classifier, the Data Scientist involved will have to do an awful lot of data cleaning. It also makes the data less reliable, affecting the accuracy of the classifier.

If you ensure your teams follow the correct procedure, and the data capture is accurate, you will save yourself an awful lot of time and money when it comes to building an AI solution.

Data

Clive Humby surely is right by saying that “Data is the new oil”, but only if you know what to do with it. Data is useless if you have access only to a portion of it,  if it’s not stored and processed properly, there are compliance issues with using it, or you don’t even know you’ve got it.

Data can be broken down into two types, structured and unstructured. In short, structured data is clean, tagged and stored easily in a database (i.e., your date of birth or address at your doctor’s office). Unstructured data can be things like webpages, images or voice recordings. These are all made up of bytes, but a machine can’t understand the meaning of the data until its given structure in a database.

An example of a Structured data solution is Churn Prediction. Let’s pretend you have a mobile app, there are hundreds of thousands of users that have interactions on your app daily. They interact through ‘open’, ‘close’, ‘like’, ‘comment’ and ‘send’ interactions. These are structured in a database, i.e. Firebase. With those sequences of events, you can build a Machine Learning based Churn Prediction model that maps the behaviours to those likely to leave (churn) from your app.

According to IDC projections, in the next 5 years, 80% of enterprise data will be unstructured. Think of your emails, phone calls, invoices, customer service tickets, powerpoint presentations, that your teams work with on a daily basis, there’s so much untapped potential there. Although it is not just internal data that is unstructured, it can be found externally to your organisation as well, such as Tweets, Market Reports, or Stock information.

When considering AI and data solutions for your business you should consider all these types of data. However, the harder it is to reach the data, the higher the degree of complexity of the project. So, ask yourselves what type of data are you building your AI solutions with?

Dealing with the 80% of unstructured data in an organisation is a different ball game altogether. We often talk to our clients about the AI Creation Hierarchy Of Needs:

...

These steps show you what is needed to get your unstructured data into a position where AI or Deep Learning can take place. It’s a slow process that includes how the data is stored which is why we now have services such as Data Warehousing solutions or Data Lakes that hold vast amounts of data (i.e. Amazon’s Redshift). There’s a lot to be said on this, and not enough space for this post, so it might be something we write about in more detail soon. But without the right building blocks for your data, then you can never hope to have a successful AI project.

This might lead to a resource requirement from IT to get a Data Warehouse in place, so a People consideration again. However, getting this right at the beginning will have a huge cost saving in the long-term as you won’t have higher cost Data Scientists or Machine Learning Engineers working on database clean-up and instead on modelling. In addition, it will provide you with a capability for an Enterprise-wide AI that could lead to the development of further AI solutions inside the organization.

What else should you consider?

AI projects are complex, there are so many more facets to consider when starting an AI project. However, by having People and Data aligned correctly, you will have the highest chance of success.

There is much more on ethics and management to discuss AI projects, but you can imagine the impact of these projects on People who haven’t yet understood the benefits of these changes. Most of the solutions we design require human intervention to ensure not only a higher level of control and assurance but also acceptance. We always advocating bringing the people affected by the AI solution into the project early on. As they need to understand its benefits to their daily lives.

Disappointment due to AI projects failure from over-expectation can happen. Our most beneficial suggestion is to take baby steps and plan properly with the help of an expert to ensure everyone understands how it will work, what it needs to train and improve, and how long it might take.

We hope this article has been useful to you and your near or future AI projects.
To discuss further or if you have any more questions on the topic, feel free to contact us.

We help companies like yours win with AI

From finance to healthcare, from market research to media monitoring, we can help your people make better decisions. We work alongside companies like yours to help deliver successful AI and ML projects - to make a real business value impact

...
...

The challenge: Deloitte’s partners and account managers found they were drowning in news from sales support teams, and unable to react quickly to market changes

The solution: Deeper Insights built a prototype Automated Insights app allowing them to have better conversations with clients and close more business

The outcome: Account managers at Deloitte close more business thanks to actionable insights delivered straight to their phones

Client said: "There are a number of gems we’ve found that are far better than the standard services we use" - Dimitar Milanov, Partner, Deloitte

...
...

The challenge: Help the sales and marketing teams know more about their customers to enable them to drive deeper customer engagement in sales meetings

The solution: Deeper Insights developed a CMS that scraped the web and automatically identified and summarised customer events relating to key accounts at JLL

The outcome: The automation of the whole previous manual process, and being able to identify 60% more news stories than the manual process, enabling JLL have better and more informed conversations with their clients

Client said: "We have lots of researchers and people who generate insights for our clients, Deeper Insights™ (formerly Skim Technologies) helped us improve the speed at which we get insights and have better conversations with our clients." - Chris Zissis, CIO, Jones Lang LaSalle

...
...

The challenge: How to turn billions of valuable data into actionable insights that a human can read and understand? Quant Insight had developed a complex algorithm used by financial analysts, but now wanted to reach a consumer client.

The solution: Using Microsoft’s LUIS intent engine and some custom NLP models Deeper Insights developed a chatbot that could query the QI API and translate the financial data into natural language.

The outcome: QI’s retail investors were able to easily access stock information via a friendly chatbot. The chatbot was able to achieve over 94% accuracy for selecting the correct stock and factors related to a question. An impressive outcome.

Client said: "Deeper Insights™ were a pleasure to work with, and extremely knowledgeable in the field of NLP. Their ability to take our idea and turn it into a working product made them the perfect partner for our fast-growing business."

Talk to us to see if we can help you

Discuss your AI project with us and lets see if we can help. We can dive into the data you have, the data we can gather from the web and other data sources, how we can manipulate that data for you and how we can output it in a dashboard that your business can actually use.

Our Data Scientists will help you deliver your AI project

Our Data Science experts are recognised globally with over 500+ citations and patents

We have a combined experience of over 40 years in developing cutting edge, and innovative Artificial Intelligence from both academia and industry. We are specialists in Computational Linguistics, Natural Language Processing, Machine Learning, Deep Learning and Data Analytics

...

"I leverage my business, scientific and technical backgrounds to design and build high-impact AI solutions that turn data into something useful for our clients. My expertise lies in the fields of Network Science, NLP, Computer Vision, Imbalanced Learning and Unsupervised Learning."

Dr Márcia Oliveira, Lead Data Scientist

...

"I use Design Thinking as my approach to deliver invaluable technology."

Ygor Durães, Full Stack Engineer

...

"The development pipeline is my favorite technical and cultural tool. My mission is to help to achieve the balance between business and business materialization."

Eduardo Piairo, Operations Manager