Is this the right problem to solve – right now?

This is by far the most important question, and it lies squarely in the product management and business area. To answer it effectively, strong communication must be established between the data science team and the product and business teams. On one hand, it is important to have a process which enables ideas generated in the engineering world to be validated quickly with customers. This avoids situations when a product feature is developed on the premise of “Wouldn’t it be cool if…” and there’s no demand for the feature. On the other hand, there needs to be an efficient way to prototype and validate solutions for inbound ideas and requests from customers.

If we solved the problem, what would be the value?

The value is the net difference between the benefit of having the feature and the cost of building it. While it may be hard to estimate either of these quantities accurately, some idea of the financial or other impact should provide guidance on whether the benefit is worth the risk of investing in the project. The company is exposed to a potentially large opportunity and financial risk, for example, when a product feature is built without an estimate of either what it would cost to build and maintain, or what revenue impact it is expected to have.

Is machine learning the right tool to solve the problem?

Building a production machine learning system is hard and can be expensive. In many cases, an outcome that is good enough to solve the business problem can be achieved with simpler methods that are much easier to implement. During a panel discussion at H2O World 2015, Monica Rogati made this point beautifully: “My favorite data science algorithm is division because you can actually get very far with just division…” Understanding that using machine learning is not the goal and is not necessarily an appropriate tool can sometimes be disappointing to data scientists. However, the satisfaction of solving a real problem and having a business impact easily outweighs this disappointment.

What are the constraints?

Any project has its constraints, and machine learning systems are not an exception. The starting point is the available people and their skills. If the existing skills do not match the task at hand, the project will depend heavily on the ability to train, hire or outsource in order to fill the gaps. More on this in my post Four machine learning skills of a successful AI team.

Further, if machine learning models have to run in a production environment, the data scientists who build the models need to understand upfront the existing production environment architecture, the technology stack, and the requirements for scale, which typically limit the choice of programming language and algorithms that are acceptable in production. Scalability of machine learning systems is a large topic in itself, and I leave it for another blog post.

Summary

By solving the right problem, understanding the value of the solution, being confident that machine learning is the right tool to solve the problem, and defining the constraints upfront we drastically reduce the overall risk of machine learning projects. Answering the questions above helps avoid wasted funds, time, and effort, as well as frustration across the organization. Not all questions can be readily answered, and some discovery with customers and proof-of-concept projects may be needed. While it may appear as unnecessary extra work, the resulting clarity about the project is so powerful that it is worth the investment.

Photo by rawpixel on Unsplash

The post How Can We Reduce the Risk of Machine Learning Projects? by Sergei Izrailev appeared first on Life Around Data.

Kitchen builders, chefs, cooks, and microwave builders

In her blog Why businesses fail at machine learning, Cassie Kozyrkov, Chief Decision Scientist at Google, points out two main types of machine learning: research and applied, and draws a compelling analogy with cooking. In this analogy, machine learning researchers who develop new algorithms are likened to engineers who build microwaves and other appliances. The applied machine learning specialists, on the other hand, are cooks who use appliances to produce tasty dishes.

Extending this line of thought, just like individual dishes don’t necessarily make a meal, predictions coming from machine learning systems are usually not the end goal. Someone needs to define how to use them in a product that solves a specific customer or business problem. For example, if the AI system predicts a user’s music preferences, there’s still work on how this information is optimally used in a music streaming app, and how to measure its impact on key performance indicators. Such a person is similar to a chef in a restaurant, who is capable of selecting and combining dishes together to serve a full dinner.

Another important aspect of AI systems is their day-to-day operation, and so people who develop and integrate machine learning platforms are those who build automated and scalable kitchens. Kitchen builders are making the cooks efficient so that the chefs are able to deliver more meals.

To summarize, there are four broad AI skillsets:

• Development of new and improvement of existing algorithms (microwave and other appliance builders).
• Application of existing algorithms to build machine learning models and produce predictions that are useful in solving a business problem (cooks, who prepare individual dishes).
• Defining how predictions are used to solve specific business problems (chefs who create a full meal and dining experience).
• Building platforms that facilitate and automate machine learning (kitchen builders).

It is important to keep these skillsets in mind when building a team that is tasked with developing AI-driven products. So which of them are necessary, which are optional, and which can be outsourced?

How to assemble a successful AI team

Considering a large number of readily available machine learning algorithms, one would normally not start with developing a new algorithm or modifying an existing one. Therefore, we usually would not need microwave builders to start (unless it’s a microwave design business). Unfortunately, these are the most common skills taught in machine learning and data science courses.

Cooking skills are definitely required, and sometimes cooks specializing in certain dishes (pastries, sauces) may be needed. For example, if solving the business problem involves text analysis, knowledge of NLP could come in handy.

Having a chef, whose skills cross into the product management area, is absolutely critical to making AI valuable for the business. It doesn’t have to be a separate role, and it can be synthesized from more than one person.

Finally, if there is a lot of cooking to be done, kitchen builders are also required in order for the whole process to scale. While one can certainly rent a ready-to-go kitchen, some of the kitchen builder skills are usually needed in-house to make it operational. The main reason is that while platforms make certain aspects of machine learning easier, integration with existing production systems and processes remains a challenge.

To assemble a successful AI team, one first needs to evaluate what business problems need solving, which skill sets are needed, and which of them are required in-house. In some cases, it is sufficient to rent a kitchen, hire a chef who can also cook various dishes, and provide adequate engineering support to make the chef effective. In others, the competitive advantage comes from the ability to cook simple meals at scale and serve many customers. Then kitchen builders become a key to success. In yet other cases, none of the existing algorithms can solve the business problem well, and one has to develop better microwaves and hire microwave builders.

Image by olafBroeker on Pixabay

The post Four Machine Learning Skills of a Successful AI Team by Sergei Izrailev appeared first on Life Around Data.