Machine Learning Limitations

Apple recently added machine learning features in a framework called Core ML. The idea is that you can take machine learning models and plug them into any Xcode project to instantly add machine learning capabilities to your apps. While this is exciting since this makes adding artificial intelligence (AI) to any app quick and easy, it’s not without its drawbacks.

First, let’s talk about the drawback of machine learning in general. In the book “Weapons of Math Destruction,” author Cathy O’Neil explains the inherent limitations of machine learning. Someone must first create an algorithm and then that algorithm is only as good as the data that it trains on. Feed the algorithms he wrong or misleading data and it can’t help but reach incorrect conclusions.

For example, Microsoft once created an AI chat bot that could learn from comments people submitted. So people started submitting racist comments so the AI chat bot would repeat nonsense about the Holocaust never happening and blaming Jews, blacks, and Mexicans for the problems in the world. That example simply shows if you train a machine learning model with bad data, you get back bad results.

Second, machine learning only works when the machine learning model can learn and improve its decisions. Without feedback, the machine learning model has no idea whether it’s coming up with the right conclusions or not. Those twin problems are what Cathy O’Neil highlights in “Weapons of Math Destruction.”

One example she gives is how many school districts are using computer algorithms to determine the “best” teachers. To do this, the programmers who created the algorithm can’t take into account all possible factors so they choose the ones they think (rightly or wrongly) that matter most in evaluating a teacher.

This caused the algorithm to rate one teacher as a 6 (on a scale of 1 to 100) one year, and then rate that same teacher a 96 the following year. How can a teacher go from a dismal 6 rating to an excellent 96 rating in one year? It’s because the first year, the teacher taught advanced students so their test scores were high to begin with, and remained high by the end of the year so they showed no improvement. the algorithm incorrectly assumed that no improvement in test scores meant that the teacher wasn’t any good.

The following year, that same teacher taught ordinary students and their test scores went up, so the algorithm assumed that the teacher did a good job. By not taking all factors into account and even worse, not accepting any feedback for modifying the machine learning model, this algorithm for evaluating teachers generates false and misleading results. yet because a computer generates these results in a seemingly objective way, people wrongly assume the computer must be correct when it’s not.

Even worse, the results of these skewered answers affects the lives of people, often in negative ways. Trying to argue that a computer algorithm is wrong is often a pointless and fruitless effort, and that’s the problem with machine learning.

Now in the world of Xcode, Core ML seems to have several faults. First, it’s not easy to create a machine learning model. That’s why Apple offers several machine learning models on their website. Second, there are plenty of machine learning models available that aren’t stored in the Core ML format, so Apple offers a bunch of tools for converting these machine learning models into Core ML files.

Unfortunately, installing and using these tools is clumsy and error-prone, making the process difficult for all but hardened developers. Apple will likely fix this process, but right now, it’s a huge pain in the neck. Still if you can get it to work, you can theoretically convert most machine learning models into Core ML files so you can add them to any Xcode project. That means you can add AI capabilities to macOS, iOS, watchOS, or tvOS apps.

Perhaps the greatest limitation of machine learning models is that once you have them stored on Core ML format, can you modify these models to make them more intelligent so they actually learn over time with new data?

If you can modify a machine learning model over time, then how do you do it and how do you keep it from making mistakes such as what happened with Microsoft’s chat bot that learned to swear and offer racist comments?

If you can’t modify a machine learning model over time, then it’s inherently limited. If you try several of the machine learning models that Apple provides, you’ll find that they’re only average in recognizing images in pictures.

So machine learning has a great future in making apps smarter. The problem is that machine learning is still limited and difficult. That will only get better over time and machine learning models will get smarter over time as well.

Until then, machine learning is the future so you better know how to add machine learning models to your Xcode projects. Just don’t expect miracles just yet, and keep in mind that “Weapons of Math Destruction” highlights the danger of relying too much on algorithms that we don’t fully understand and can’t modify.

Machine learning has its place, and it’s firmly in assisting humans, not substituting actual thought for machine learning results.

July 30th, 2017 by
HTML Snippets Powered By :