Eugene Goostman, Teenaged AI, and the Definition of Number

Eugene Goostman, an artificial intelligence program purporting to be a thirteen year old boy from the Ukraine, recently prompted a spate of breathless news articles reporting that an AI had finally and for the first time passed the famous Turing Test, fooling humans into thinking they were conversing with another human being and not with an AI program. This in turn prompted a number of critical responses, notably one by AI pioneer Ray Kurzweil who reported the following rather unconvincing exchange with Eugene:

Question: How much is two plus two?
Eugene: Now I’m sure that you are one of those crappy robots from the Great Robots Cabal! And I know your password: 2 + 2=4.0! Now divide by zero and die!!!!

Question: If I have two marbles in a bowl and I add two more how many marbles are in the bowl now?

Eugene: Not too many. But I can’t tell you the exact number, I forgot it. If I’m not mistaken – you still didn’t tell me where you live. OR it’s a secret?:-)

In his article, Ray Kurzweil gives several other examples of simple questions that stumped Eugene Goostman, Teenage AI. In particular, Eugene apparently cannot perform simple addition when presented with simple real world problems such as determining how many marbles are in two boxes. In contrast many six-year olds and even younger children can answer such questions with ease.

In his book The Number Sense, mathematician turned cognitive neuropsychologist Stanislas Dehaene argues that research indicates humans and at least some animals are born with an inherent ability to count and handle numbers. This occurs in the context of counting physical objects such as marbles, food items, and so forth.

What is a number?

The definition of number is not straightforward. The mathematicians Bertrand Russell and Gottlob Frege spent years trying to put numbers and arithmetic on a rigorous logical basis. Alfred North Whitehead and Bertrand Russell got a lot of flak for writing a book Principia Mathematica that took hundreds of pages to prove 1+1 = 2 🙂 .

The definition of number is difficult. Dictionary definitions of number usually define number in terms of other equivalent terms: quantity, measure, measurement, amount, count. One needs other, also difficult to define, concepts such as physical objects, classes of objects, and sets of objects to try to define numbers in terms that are not simply synonyms for number. As Dehaene argues, we probably have a built in sense of numbers and ability to count that makes number seem intuitively obvious to us.

Computers deal with numbers as electrical states in transistors. Human beings almost always deal with numbers as counts of physical objects belonging to classes such as marbles, people, cars, leaves and so on.

In practice, numbers are intimately associated with the human ability to associate sensory impressions — sights, sounds, touch — with hypothetical, but usually real, physical objects and to classify those objects into categories. Human beings probably started with the ability to add two marbles to two marbles and get four marbles and then proceeded to the abstract concept of the numbers two and four. Early number systems often represent the first few numbers 1,2,3 as one, two, and three vertical bars: like I, II, III in Roman numerals. Simple visual counting.

Computers, in fact, started from the opposite direction, from abstract numbers. An old mechanical adding machine can add two and two and get four but it has no understanding of what is being counted or why. Eugene Goostman, the teenage AI, seems to have the same problem. Modern supercomputers perform complex numerical simulations of thermonuclear explosions but they have no idea what the numbers mean or what is being simulated. They are idiot savants.

The Classification Problem

As I discussed in my previous post The Mathematics of Terminators we lack a fundamental understanding of how human beings divide the world into objects and classify the objects into things like marbles and boxes and bowls. Counting and numbers are probably inherent in this process. Kurzweil’s questions for Eugene were designed to test his ability to handle objects, classes of objects, and numbers as human beings actually use them in everyday life.

Classes are enigmatic. A marble is a small, spherical or almost spherical object of hard material, such as the stone marble, that is about the diameter of a human finger. Seems straightforward. But, why is a ball bearing of similar size a ball bearing and not a marble? Perhaps because marbles are used for playing games but ball bearings are used for a serious purpose? What if I play a game of marbles with ball bearings? Do they cease to be ball bearings? What if in an emergency I use a marble used in a game as a makeshift ball bearing? Is it now a ball bearing?

With LISP and object-oriented programming languages such as SIMULA, Smalltalk, C++, Java, and many others, mathematicians, AI researchers, and programmers have tried to emulate the human ability to classify objects and reason about objects, with very disappointing results. Somehow the intuitive concept that a class is a set of attributes that describe an object does not seem to match what people actually do.

class Marble : public SphericalObject, Toy {
// attempt to describe a marble in C++
// a list of attributes that tries to
// describe the class Marble
double minimum_diameter = 0.5;
double maximum_diameter = 1.5;
string diameter_units("cm"); // centimeters
double minimum_hardness = 5.0;
string hardness_units("hardness_scale");
// gets more and more complicated as
// you think through what is and is not a marble
// for example, should we use centimeters or
// the finger width of the people playing the game?
// What if the players are giants or midgets?
// What if scientists train a gorilla to play
// marbles with special gorilla sized marbles?
}


Humans seem to define classes in a holistic way that somehow combines numerically measurable quantities such as diameter and hardness with purpose and other quite different criterion. Instead of a simple rectangular box in a high-dimensional hyperspace of numerical attributes, the classes used by human beings seem to have complex curved and possibly fractal or discontinuous boundaries that are difficult to either understand or model. We can also learn new classes and even change our classification scheme entirely — experience a dramatic change of world view in rare cases.

Wolfram Alpha, Middle Aged AI, Flops Too

Incidentally, on June 14, 2014, I posed Kurzweil’s marble question to Stephen Wolfram’s much vaunted Wolfram Alpha, Middle Aged AI, which did not understand the question and could not answer.

Conclusion

Can present-day (2014) computer programs, whether labeled AI or not, understand and use numbers in the way humans do — or at least in a way that is functionally indistinguishable from what humans do? It is difficult to rule out an AI program in some lab somewhere — though it seems unlikely to the author. AI researchers may need to reread Whitehead and Russell’s much maligned Principia Mathematica for insights or even the answer to making AI programs that can count marbles. 🙂

© 2014 John F. McGowan

About the Author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing video compression and speech recognition technologies. He has extensive experience developing software in C, C++, Visual Basic, Mathematica, MATLAB, and many other programming languages. He is probably best known for his AVI Overview, an Internet FAQ (Frequently Asked Questions) on the Microsoft AVI (Audio Video Interleave) file format. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at jmcgowan11@earthlink.net.