The Twelve Truths of Machine Learning for the Real World

↑

⇐ ПредыдущаяСтр 11 из 13Следующая ⇒

The Twelve Truths of Machine Learning for the Real World

Last month I gave an informal talk to an intimate gathering of friends with this title that I am putting down in words. This post is mainly for people who are using machine learning to build something as opposed to people who are working on machine learning for its own sake (god bless them). Although the latter group will do well to listen to these truths and introspect their work. In the spirit of holiday baking, I will throw in a Truth Zero to make this a bakers’ dozen.

0. You are Not a Scientist.

Yes, that’s all of you building stuff with machine learning with a “scientist” in the title, including all of you with PhDs, has-been-academics, and academics with one foot in the industry. Machine learning (and other AI application areas, like NLP, Vision, Speech, …) is an engineering research discipline (as opposed to science research).

Tangent I won’t get into here: I think everything is art. The C. P. Snow “Two Cultures” is complete BS, but that’s for another day.

What’s the difference between science research and engineering research, you ask? I cannot improve upon what George A. Hazelrigg wrote in his “HONING YOUR PROPOSAL WRITING SKILLS” memo (emphasis mine):

Some scientists are taught how to frame research projects. Few engineers are, even PhD-level engineers. So let’s first try to understand the difference between science research and engineering research. To me, the difference is quite clear. The scientist seeks to understand nature at its core, to get to the fundamental essence. To do this, the scientist typically strips away extraneous effects and dives deeply into a very narrow element of nature. And from this look comes what we refer to as the laws of nature: energy and mass are the same thing, for every action there is an equal and opposite reaction, and so on. There are lots of laws of nature, and they apply everywhere all the time. Engineers live with the laws of nature. They have no choice. Their goal is to design things that work within what nature allows. To do this, they have to be able to predict the behavior of systems. So a big question for engineers is, how do we understand and predict the behavior of systems in which all the laws of nature apply everywhere all the time. This is an issue of integration, and it is every bit as difficult as finding the laws in the first place. To account for all the laws of nature everywhere all the time is an impossible task. So the engineer must find ways of determining which laws are important and which can be neglected, and how to approximate those laws that are important over space and time. Engineers do more than merely predict the future. They make decisions based in part on their predictions in the knowledge that their predictions cannot be both precise and certain. Understanding and applying the mathematics of this is also important. This includes the application of probability theory, decision theory, game theory, optimization, control theory, and other such mathematics in the engineering decision making context. This also is a legitimate area of research for engineering.

As an ML researcher and practitioner, you have to worry about the right models for the data you have as opposed to the right datasets for the models you have (like many research papers). If you’ve ever asked “what is the right dataset for this model”, then you are not in the Real World. What the heck is this Real World anyways? Real World is where you don’t have a choice about the data you have to deal with. Here the data defines the problem and not the other way around. Sometimes ML practitioners of the Real World pretend they are scientists by creating their own worlds as playgrounds for their modeling enterprise, such as “inventing” a language for doing NLP (hello BaBi!) or creating closed environments with simplifying assumptions for reinforcement learning. These produce interesting results, but their scope is limited to the worlds they emerge from, even if researchers like to sell it in their papers as something applicable for the Real World. In the Real World, the distribution of input more likely changes than not, “curve balls” from long-tails come out of nowhere, and you don’t always have an answer.

When working in the Real World, there are several truths one has to contend with, which is the main body of this post. But this prologue is essential. If you do ML research in the Real World, you are an engineer and not a scientist. Keep that in mind. There are a few recurring themes we find as we practice the craft. Interestingly, these themes are airlifted almost verbatim from another engineering research discipline — Networking — to make a point.

Познавательные статьи:



Организация работы процедурного кабинета

Статус республик в составе РФ

Понятие финансов, их функции и особенности

Сущность демографической политии