Monday, October 29, 2018

The Definition of Machine Learning

Lately, many popular online publications have been trying to explain to the populous what "Machine Learning" is.
That moved me to extract the following text from the main article "On The Definition of AI" and post it as a separate piece.
There are many definitions of "Machine Learning".
For example, “Machine learning is a field of artificial intelligence that uses statistical techniques to give computer systems the ability to "learn" (e.g., progressively improve performance on a specific task) from data, without being explicitly programmed.”
None of those "definitions" are definitions. They merely describe the scientific field of machine learning but do not define machine learning. Mostly because they do not define "learning". 
If they did, the rest would be obvious. 
This is the definition of Machine Learning: "machine learning it is what people do when they learn, but this time it is done by an artificially made object, i.e. by a machine". After a short general introduction most of the authors offer some version of a brief description of specific methods used to process different patterns (could be found in any textbook on AI or ML).
The true goal is to define "learning".
"Learning" also has many definitions. The most common one (which comes in various forms) is "learning is the process of the acquisition of knowledge", or "the knowledge obtained during the processes of learning". Both definitions are correct, in their way, because they do describe learning. But that type of learning is not the type AI professionals have in mind when they say "machine learning". Those two definitions do not allow to establish if learning has actually occurred beyond mere memorization (a.k.a. "acquisition of knowledge"). Machine learning as a memorization is clearly of no interest, because these days we all know very well that machines can accumulate, store and retrieve huge amount of information. Of course, the algorithms, techniques for acquiring and processing that information represent important technical part of machine learning.But that part has little to do with the actual process called "learning". Even blind and deaf people can learn to the highest level (up to getting PhD).
As an expert in human intelligence, I define "learning" (more accurately, "productive learning") as a processes leading to a production of knowledge; as the first approximation (the scientific thinking in action), learning is a process of utilization of currently active knowledge in order to produce new knowledge (for example, the statement "I learned how to do it" represents some of the new knowledge developed during learning). The criterion of "learning" ("actual learning", "real learning", "true learning", "productive learning") is the ability to use existing knowledge to generate knowledge previously not available to the actor of learning. Machine learning is happening when a machine (an artificially manufactured object) produces new knowledge based on the knowledge currently available to the machine. 
BTW: what is "knowledge"? Without an operational definition of "knowledge" how do we know if the new knowledge has been produced? If a machine takes a text and randomly permute and recombine letters, words, sentences will it be "new knowledge"?. 
More importantly, what types of knowledge exist? how does knowledge evolve? what is the structure of knowledge? how is the structure of knowledge reflected in the structure of neural network processing that knowledge? People in AI don't seem interested in those questions. At least there is no single page from 1100 pages of "Artificial Intelligence: A modern Approach" (by Stuart J. Russell, Peter Norvig; 3d edition) where those questions about knowledge would be discussed. They talk about "knowledge" as if it is something obvious, or define "knowledge" as "information", which is a severe simplification, in part because it ignores an important feature of "knowledge" - it has a vector; it is purposeful (in general).  They define learning as making a match between a hypothetical knowledge and the factual knowledge (meaning "information").  This does formally describe a procedure leading to "new knowledge": (1) state a hypothesis; (2) gather facts; (3) compare; (4) decide. For example: (1) this is a banana? (2) run image recognition; (3) correlation 0.98; (4) ye, that is a banana! (if needed, e.g. to decrease % of mistakes, learning can be "reinforced", and "deepened"). But (A) for people true learning usually begins after learning how to recognize various shapes; (B) this learning ignores "learning as a skill development"; (C) and also it ignores the central feature of learning - its intentionality (humans have a desire to learn, including about themselves, built into the genetic code; good teaching is based on that; bad teaching ignores or even tries to break this desire).
Finally, since the ultimate mission of learning is progress:
(1) acquisition of knowledge is useless if it does not lead to the development of new practice (starting from the development of new individual skills).
(2) the development of new practice (starting from the development of new individual skills) always lead beyond acquisition of knowledge to development of new knowledge.
That means that AI developers also need to define "skills", "new skills", "machine skills", "skill development", etc., in a way assessable for a machine and by a machine.
Machine learning is happening when a machine (an artificially manufactured object) develops new skill based on the skills currently available to the machine.
When current AI recognizes a pattern (visual, audio, numerical) it only makes a statement in the form "yes - that is that thing", "no - that is not that thing". But the processes in the network which lead to the final statement also have their own patterns. In a brain, there is at least one another network of a higher level which analyzes and recognizes the patterns happening in the lower network making a decision about the pattern/object. That higher level network generates another signal - a doubt - "are your sure"? And then there is another network which makes another decision "yes, I am sure" or "no, I am not sure". And then ... - long story, but you see the pattern.
No AI is even close to mirror this type of pattern/pattern/pattern recognition (that requires developing the hierarchy of networks analyzing the hierarchy of patterns).
That is why I also added another post on the matter: "Relax, the real AI is not coming any soon" (that post also has some insights on what "common sense" is).
P.S. The field of AI training will become much more important than it is today.
Although, not many AI professionals see it so far.
For more on AI:






Thank you for visiting,
Dr. Valentin Voroshilov
Education Advancement Professionals
GoMars.xyz

To learn more about my professional experience:
The voices of my students 
"The Backpack Full of Cash": pointing at a problem, not offering a solution
Essentials of Teaching Science

Dear Visitor, please, feel free to use the buttons below to share your feelings (ANY!) about this post to your Twitter of Facebook followers.