An international award of Japanese origin, the Kyoto Prize is presented to individuals who have made significant contributions in the fields of science and technology, as well as arts and philosophy. Kyoto Prize laureates are those who have made an extra effort to plumb the depths of their chosen fields and profoundly inspired the scientific, cultural, and spiritual betterment of humankind through their achievements. In this series of “Unearthing the Words of Kyoto Prize Laureates,” we will interview the past laureates of the Kyoto Prize and take a closer look at the words that they delivered at their Commemorative Lectures to get to the heart of their unique ideas, thought process, and attitudes as inquirers. In the third installment of the series, we had the pleasure of interviewing Dr. Takeo Kanade, the 2016 Kyoto Prize laureate in the Advanced Technology category.

Previous article in “Unearthing the Words of Kyoto Prize Laureates”
#1 Dr. Toyoki Kunitake
You’re Sure to Find a Breakthrough in the Process of Going Back and Forth Between “Abstraction” and “Materialization”
#2 Dr. Takashi Mimura
“Being of Service” Means to Be Needed and Appreciated by the Overwhelming Majority

──────────────
Takeo Kanade
U. A. and Helen Whitaker University Professor, Carnegie Mellon University. After receiving a Ph.D. in Electrical Engineering from Kyoto University in 1974, Kanade moved to Carnegie Mellon University (CMU) in 1980. He served as Director of the Robotics Institute and Director of the Quality of Life Technology Engineering Research Center, CMU. His current and previous positions include Director of the Digital Human Research Center, National Institute of Advanced Industrial Science and Technology, Japan. Currently, he also serves as Invited Distinguished Professor, Kyoto University Institute for Advanced Study (KUIAS). Read more

Nishimura　 Could you start by giving us a brief self-introduction?

Kanade　I was born in Tamba-Sasayama in Hyogo Prefecture, spent my youth in Kobe and went on to Kyoto University. I got my Ph.D. from the graduate school and became assistant professor, and then associate professor before I moved to Carnegie Mellon University in the U.S. in 1980. So, I was at the midpoint of a typical Japanese researcher’s career path of moving up within a single university when I decided to go to America. People asked me, “Why are you quitting your job at a Japanese university to go to the United States?”

It’s true that job security is not guaranteed at American universities, but I knew, from my experience studying there when I was an assistant professor, that universities in the States had an outstanding computing environment. The gap now is no longer as big as it once was, but it was really like a paradise for someone from Japan. So, I thought this way, “They promised to keep me for at least five years at Carnegie Mellon University, and the environment is so marvelous that I will be able to do three times as much research there as I could in Japan. That means I will be able to advance my research as much as I would have done if I had stayed in Japan for 15 years.”

I was 35 years old then, so in 15 years, I would be 50. I determined, “At 50, maybe I will stop thinking about doing new things. All right, I can tell myself I will do what I would do thinking that my career will finish in five years.” However, as it has turned out, I have been in the States for 40 years.

Nishimura　I heard that after finishing the undergraduate program in electrical and electronic engineering at Kyoto University, you joined the laboratory of Dr. Toshiyuki Sakai^＊1 in 1968 and studied computer processing and recognition of image and sound. You were involved in global leading edge research at that time, which later turned out to be linked to your own research topics.

Kanade　My own work started out from research in image recognition by computer at Sakai Laboratory. We developed the first full-fledged computerized face recognition program in the world, so even now, I still receive interview requests mainly from the U.S. media.

Another research area of mine that is widely known is the project on autonomous land driving vehicles, something I worked on from the mid-80s. We fabricated a car equipped with a camera, computers, and a laser distance sensor that was able to drive itself automatically by detecting roads and obstacles. The highlight of this project was the “No Hands Across America ,” a self-driving trip across the North American continent in 1995. From Pittsburgh to San Diego, the car traveled at a speed of about 100 km/h, demonstrating to the world the potential of automated driving.

Another well-known project I was involved in was the development of “EyeVision,” a multi-camera based 3D replay system. The system was used in the broadcast of Super Bowl XXXV, the American football championship game held in 2001. The system synchronized 33 robotic cameras on the upper tier of the stadium positioned around the field and automatically controlled them to track a single target point of play. This allowed decisive moments in the game to be replayed from a variety of viewing angles, just like the film “The Matrix.”

Sugimoto　People called you “the only university professor to ever appear in the Super Bowl broadcast,” right?

Kanade　Yes. I appeared in the Super Bowl broadcast and explained about the EyeVision system. EyeVision is based on the research into multiple-camera systems I had been pursuing since around 1992. Back then, when cameras were very expensive and computers were not fast enough, the question was “how to get as much information as possible from a single camera.” For example, humans can infer “what they don’t see” based on information about the surroundings. What researchers were trying to do was to make computers do the same thing.

However, the first paragraph introducing my theories about computer vision states, “You cannot see what you cannot see.” Rather than inventing a program to imagine what you don’t see, I thought it was better to place cameras at the positions where you can see what you want to see.

Nishimura　Now, technologies using multiple cameras are very common. Your work has served as a theoretical foundation for them.

Kanade　The “Lucas-Kanade method” which I developed and published in 1981 with Bruce Lucas, a student at my laboratory then, is a widely used method of video processing in many applications including MPEG. I have pursued so many different research interests, just following whatever came to mind, that someone once called me “the robot researcher who has worked on the greatest variety of things.” When I saw someone doing something very well, I often had the outrageous thought, “Maybe I could do better,” and started working immediately. I was such type of person that I have done research on many different topics. This concludes my brief self-introduction.

Sugimoto　It must have taken a lot of courage to take on the challenge of developing the world’s first full-fledged face recognition program.

Kanade　Rather, I hated it the most when people asked me, “Haven’t you made it yet?” When I enrolled in the doctorate program, the idea of computers dealing with images was already coming around, and I had no doubt such technologies would be realized as a matter of course.

However, at that time, there were no such things as digital images. There was no handy way to digitize film photos, either. So, on a photo printed on paper, we drew precise grid lines using a ruler, visually checked each square against a reference pattern of brightness, and numbered all the pixels manually one by one. Then we typed them into a computer to produce a digital image. It was such a laborious process. We called it “a human digitizer.”

Sugimoto　You put such an unbelievable amount of time and effort into the process of digitizing images.

Kanade　Computers are machines that basically you can direct to do anything you want. To me, it was obvious that they would be able to recognize faces if you translate the information into a computer language and have them analyze the information. To me, the feasibility of self-driving in the “No Hands Across America” project was even more obvious. Around 1980, the idea of computerized image processing was already a very common concept, and there were research projects such as recognizing buildings and identifying roads by analyzing aerial photographs. You just needed to mount cameras on a car and write a program to drive along the route. Naturally, this was not easy, but the feasibility of automated driving in itself was obvious.

Kanade　Ever since learning about artificial intelligence as a third-year undergraduate student, I thought it was a foregone conclusion that artificial intelligence would beat humans at Go and Shogi. At that time, even Dr. Sakai said, “I don’t know. Humans are smart, and we have this thing called intuition.” When I said, “We already have artificial intelligence that can beat humans in chess,” some people even argued very seriously, “Chess is a Western game. Because it is logical, artificial intelligence can beat humans, but Shogi is based on an Eastern way of thinking, so it can never beat humans.”

Nishimura　I remember you said, “A human being is just a ‘machine’ that processes information in the brain.” If only we could conceptualize a human being, maybe the only issues to be solved are those of complexity—is that true?

Kanade　True, but it will never be easy because a human being is an extremely complex “machine.” Even so, I expect it will be only a matter of time before an artificial intelligence as complex as a human being appears. When I talk about these things, some people reject the possibility, saying, “Things such as love, intelligence, and intuition are qualities that only humans can have.” To me, however, this looks similar to the case of organic compounds in chemistry: people once believed that these compounds could only be produced by the “vital force” of an animal or plant and could never be artificially synthesized.

Before the synthesis of urea from inorganic substances by Friedrich Wöhler^＊2 in 1828, a fictitious boundary existed between “organic matter” that can be made only by natural living creatures and “inorganic matter” that is artificially produced by chemical synthesis. This boundary was based on “whether the matter has a vital force or not.” However, as you all know, nobody now believes in such a boundary. So, it is possible that someday, the way we conceive of a boundary between human beings and artificial intelligence will begin to change.

Nishimura　In your lecture, you mentioned that no other form of research is more useful than genuine basic research that can lead to a fundamental elucidation. I have the impression that our discussion about usefulness tends to focus on dimensions that are visible to everyone, but I suppose you are looking at the question from a different perspective.

Kanade　I place a great deal of importance on usefulness. I often wonder whether useless technology can really be called “technology.” It is very near-sighted to think that “useful” means only what is of use to us in our daily lives or what helps us to make money—these things are, of course, also important, though. Even research that wins the Nobel Prize does not become of use until it is used for curing a previously incurable disease, for preventing such a disease or for making a device which was previously impossible to realize. Maybe the word “useful” may sound less unconformable than the phrase “to become of use.”

I believe that the essence of research lies in the question “Why are you doing this research?” For example, there is sometimes a phenomenon which is well known but the mechanism through which it occurs is yet to be elucidated. To address this, some people may try to elucidate it in a more fundamental sense, while others may try to find out the mechanism superficially. In any case, I make it a rule to never say things such as “There is no meaning to any research other than fundamental elucidation” because that will have a highly demotivating effect on people.

However, if you get into the habit of questioning yourself, “Is this really the reason why the phenomenon occurs?” you will be more likely to be lucky enough to reach a better, more profound, and more essential solution. Even doing so, you may not necessarily be able to reach a fundamental solution. Even outstanding researchers do not succeed that often.

We all have the desire to do cool, basic research and achieve great results as quickly as possible. However, there is no magic method by which you can always reach a better conclusion. So, I believe it is more educational to lead students to think that by facing a question squarely this will eventually lead them to an essential understanding.

Nishimura　When you understand the fundamental mechanism of a phenomenon, it will help you understand other phenomena as well, and it can change the way you see things. Maybe it is not until you are here that you come up with the question “is the work I am doing now fundamental?”

Kanade　Exactly. Recently, artificial intelligence or deep learning tools have become available as packages, and you can readily access a variety of tools. Then, to my surprise, sometimes, not only students but even researchers put data and numerical values into such packages, write papers based on the answers thus obtained as “good results” and publish them. Because data are highly redundant, when you input a lot of data, you will certainly obtain a desired result in 60% to 70% of cases.

These tools are highly theoretical, and there are so many people who use them wrongly and say, “My method is theoretical,” but they don’t even consider what the package has been designed to do. I guess they don’t have the habit of thinking about the prerequisites for the package to work well when they have it in front of them.

Nishimura　When we get the desired result with a probability of 60% to 70％, we will become convinced that it’s all right to perform research in that way, and no longer question what we are doing. Then, what should you do to lead such people to seek a more essential solution?

Kanade　Well, I think it is important to train students to acquire the habit of trying to understand the essence of things at universities. In research, you need to turn your idea into a model in order to think about it conceptually. One of the best tools for constructing a model is mathematics. Building a computer program is also a very important way to develop a new model. The phenomena in this world are very complex, so you can represent them as a complex function called a program and see what will happen. That is the concept of simulation.

The ability to turn what you have reasoned or inferred into a model using such tools as mathematics and computer programs reflects your competence as a researcher. However, in a certain sense, what we do is no different from what we think in daily life. In that sense, essentially, I think teaching arithmetic at elementary schools in a good way should also lead to instilling the habit of looking into the essence of things.

Sugimoto　Dr. Kanade, you worked with Dr. Sakai and Dr. Makoto Nagao^＊3 at Kyoto University, and then with Dr. Allen Newell^＊4 and Dr. Raj Reddy ^＊5 at Carnegie Mellon University—you have encountered such exceptional people. For a researcher, I suppose it is very important to study under a good teacher.

Kanade　It also depends on luck whether you can meet a good teacher. I asked to join Dr. Sakai’s laboratory because I happened to hear him talk about artificial intelligence in his class. Dr. Sakai was an extraordinary individual, and Dr. Nagao was with his laboratory then as an associate professor. We really hit it off, or rather, he took very good care of me. This was also lucky, wasn’t it?

In reference to the verse from the Gospel of Matthew in the New Testament, “Whoever has will be given more, and they will have an abundance”, the “Matthew Effect” refers to the fact that a researcher who is working under good conditions can achieve excellent results and thereby gain even better conditions. Certainly, a good laboratory is likely to produce good researchers. Dr. Sakai’s laboratory had many excellent researchers in addition to Dr. Nagao, and I was very much inspired by them. In that sense, I would say I was really lucky.

Dr. Newell is virtually one of the pioneers in the field of artificial intelligence, and he happened to visit Kyoto University for a day during a long vacation. At that time, I was there as an assistant professor, so I talked about the research project I was working on, and he was interested in it. On the following day, Dr. Newell had planned a visit to the Shugaku-in Imperial Villa, and Dr. Sakai asked me to guide him. That also was a chance encounter.

I met Dr. Reddy at Carnegie Mellon University. Among the fields of artificial intelligence, he specialized in speech, but not really in images. However, at that time, he was the principal investigator leading a major research project on image understanding sponsored by the US Defense Advanced Research Projects Agency (DARPA). Perhaps about two months after my arrival in the U.S., at a project meeting, he suddenly said to me, “Takeo specializes in the study of images, so he will be the principal investigator from today.”

In the United States, to be the principal investigator of a project, usually you need to draw up a proposal and get the budget by yourself. If Dr. Reddy or some other researchers in his lab was specializing in the study of images, he would not have assigned the role to me. So, this again was a chance encounter.

Sugimoto　I suppose he chose you for the position because he had noticed your talent. After that, you took on a series of important roles including the director of the Robotics Institute, but I have the impression that you enjoyed these roles rather than feeling they put you under pressure.

Kanade　I really enjoyed them. Certainly, there were hard things, too: shortly after coming to the U.S., when I joined an overnight DARPA meeting for the first time, we discussed work from morning to night, even during meal times, without a break. I already spoke English then, but it was tough speaking it all day. When I took a shower in my hotel room, I saw a bunch of hair coming off onto my hands. I think I was under a lot of pressure at that time. Other than that, I managed to endure the pressure, as there were good people around me who supported me.

The four professors who guided me were already highly respected when I met them, so it was a simple thing for me in that all I had to do was to follow their lead. Although it’s hard to say an excellent researcher is always a person of character, these outstanding researchers were all people of great character as well as lots of fun. I learned from these four people how to speak in a way that caught the interest of an audience.

In particular, whenever I went to Professor Newell‘s office, he was really passionate and committed to listening to me talk about my research during the time we had scheduled. He asked many questions about every detail and enjoyed the conversation with me using as much of the knowledge he had as possible. He was a very busy person, but he wouldn’t answer phone calls while talking with a student, and if someone came to the office, he wouldn’t see the visitor until he had finished his discussion with the student. It’s difficult for me to follow his approach in this respect, even if I want to.

Sugimoto　I can imagine you have been teaching your students at your lab with the same passion and commitment.

Kanade　At discussions and meetings with students, I would never say “Let’s call it a day.” Even if it was two or three o’clock in the morning, I would not say, “Let’s call it a night” unless the other person said so. That’s a famous story.

Nishimura　As you train young researchers, what training and basic concepts do you think are necessary for educating them at universities?

Kanade　The most important thing is the ability to set a research question, that is, to think about what will happen by solving that problem. Generally speaking, your research advances rapidly when you think deeply about what will happen if you can do it, rather than when you focus on the approach, or how you can do it. Moreover, if you have a vision of what will happen after solving the problem, it will greatly motivate your research.

For example, it is said that Marie and Pierre Curie worked tirelessly day and night to separate and refine a mineral and ore called pitchblende to isolate radium chloride. The story is often told with the phrase “overcoming great hardships,” but I think they must have been enjoying it every day. From one ton of pitchblende, they could get only 0.1 grams of radium chloride. Even so, I imagine that they were able to continue because they believed that they could achieve a little more tomorrow.

It is because we have the story that “solving this problem should lead to the answer” that we enjoy research. In a sense, research is the process of enjoying the story. If the story you expected does not happen, then you can change the method or approach. That process is research, from which new discoveries emerge. You get a good result, and you can say that it is also good for human beings—I believe it is at such moments that scientists and engineers find the reason for being.

Nishimura　I can strongly empathize with what you were saying. When I joined RIKEN, I interviewed many researchers, and found they all had very interesting themes that each was fascinated in. If you look at the problem from the researcher’s viewpoint, it certainly looks like such a problem. And it was amazing to me they also had the ways to solve them.

Kanade　I agree. Every researcher has his or her own story.

Nishimura　To understand things and reach the essence, I think you need to have many frames of understanding. A frame of understanding is constructed with knowledge. I think it is definitely better to have as much knowledge as possible either in research or in daily life, but on the other hand, there are people in the world who say that you don’t have to study. Where do you think this sort of thinking comes from?

Kanade　Some people say you don’t need to have good college grades, citing such stories as “Dr. Albert Einstein didn’t have good grades when he was a student,” or “Dr. Masatoshi Koshiba, who won the Nobel Prize in Physics, graduated from the Department of Physics at the University of Tokyo at the bottom of his class.” However, even if these stories are true, it doesn’t mean they became great because they had bad grades.

It is true that those who thought up new things were not necessarily bound by old knowledge. However, it just means “they were not bound by the knowledge,” but doesn’t mean “it would have been better for them to have no knowledge,” does it? Their greatness resides in the fact that they did not constrain their perspectives with the knowledge they had. It’s not that they were great because they didn’t have knowledge.

Sugimoto　We could fall into the trap of trying to use the knowledge we have and end up narrowing our perspective.

Kanade　That’s what I mean. I often say, researchers and engineers like us are top specialists, but at the same time, I often say that we should “think like an amateur, do as an expert.” This is because trying to think as an expert can narrow the range of your ideas.

Sugimoto　How do you avoid “thinking as an expert” yourself?

Kanade　Above all, I try to start by asking myself “What do I want to do?” Anyone of us can think about what we want to do, and amateurs are better at coming up with ideas innocently, saying “Maybe I can do something like this.” However, I am not saying amateurs’ ideas are better—it’s important to have knowledge. Without any knowledge, in the first place, you would not even be able to come up with an idea of what you could possibly do.

However, in the case of researchers, as soon as we start thinking about how to realize an idea we have come up with, because we have knowledge, we tend to think ahead and say, “This will be difficult,” or “It’s not possible in the first place.” Another thing that is often said is that, because experts know very well how difficult it is to do what they do, they tend to expect doing something in their own field will be very difficult.

Nishimura　To me, what you have just said seems to suggest that we need different abilities to solve a question and set a question. To solve a question, we need to be based on the cumulative achievements of past research efforts and create something beyond them. On the other hand, when we are to set a question and choose a topic, an overly strong awareness of past research efforts can hinder an open-minded choice of topics.

Kanade　That is certainly right. In fact, those who do not know about past research cannot come up with anything important. I would say again, no one can do good research without knowledge. The problem is that your knowledge can help to advance your research in some cases, but in other cases, it can hinder your research, and there are possibilities for both even within the same researcher. I myself have experienced this many times. Too often, I felt frustrated to see the work of other researchers, wondering why I hadn’t noticed the things they noticed while doing something similar.

Nishimura　There can be many ways to solve a question, so even if you fail with one method, it does not necessarily mean that the question has been set the wrong way. Perhaps it needs the attempts of many people who try different approaches, and one of them will lead to the solution.

Kanade　True. When I give a speech, people often say, “You have been successful in everything you have done,” or “Have you ever failed?” To these comments, despite thinking “The audience will be happy if I say I have also failed,” I usually respond “My overriding principle is to continue until I succeed, so logically, my personal lexicon cannot contain the word failure.” I am rather mean. Then Ms. Shinobu Inamori-Kanazawa, who is Dr. Kazuo Inamori’s daughter, (current president of Inamori Foundation) told me that her father also said, “What you need is to do until you succeed.” I remember I felt very relieved at hearing that comment.

Nishimura　It is interesting that both a researcher and a businessperson with similar ideas have reached lofty goals. Thank you very much.

＊1. Toshiyuki Sakai　Professor Emeritus of Kyoto University and Ryukoku University. Information processing scientist. Born in Osaka in 1924. Graduated from the Department of Electrical Engineering, Faculty of Engineering, Kyoto University, Ph.D. in engineering. Sakai pioneered many fields, including character recognition, machine translation, natural language processing, and image processing, and mentored many talented researchers in these fields.

＊2. Friedrich Wöhler　A German chemist who, in 1828, discovered that urea crystallized while he was heating ammonium cyanate and succeeded in the first laboratory synthesis of the organic compound urea from an inorganic compound. Other achievements include discovering isomerism and beryllium.

＊3. Makoto Nagao　Professor Emeritus of Kyoto University. Computer scientist. Born in Mie Prefecture in 1936. Graduated from the Department of Electronic Engineering, Faculty of Engineering, Kyoto University. Completed the master’s degree program in electronic engineering and earned a Ph.D. in engineering from the Graduate School of Engineering, Kyoto University. Specialized in natural language processing, image processing, and pattern recognition. Nagao served as President of the Institute of Electronics, Information and Communication Engineers (IEICE) and the Information Processing Society of Japan (IPSJ), and also served as President of Kyoto University.

＊4. Allen Newell　An American researcher in artificial intelligence. Developed early artificial intelligence programs with Herbert A. Simon. In 1975, Newell and Simon received the Turing Award for their fundamental contributions to artificial intelligence and cognitive psychology.

＊5. Raj Reddy　An American computer scientist. In 1994, Reddy was awarded the Turing Award for the design and development of some of the first large-scale artificial intelligence systems, and for widely publicizing the practicality and commercial potential of artificial intelligence technology.

Click below to watch the YouTube video of Dr. Kanade’s Commemorative Lecture from 2016.

Previous article in “Unearthing the Words of Kyoto Prize Laureates”
#1 Dr. Toyoki Kunitake
You’re Sure to Find a Breakthrough in the Process of Going Back and Forth Between “Abstraction” and “Materialization”
#2 Dr. Takashi Mimura
“Being of Service” Means to Be Needed and Appreciated by the Overwhelming Majority
 

[About the interviewer]
Yuya Nishimura
Executive Director, MIRATUKU. Earned a Master’s Degree from the Osaka University Graduate School of Human Science. Built a cross-sector -business and -domain innovation platform, supports leading companies (about 30 annually) in creating new businesses, assists launch of R&D projects, designs future visions, and searches for future trends. Other positions include Innovation Designer, Innovation Design Office, RIKEN, Japan, and Specially Appointed Associate Professor, Social Solution Initiative (SSI), Osaka University　 MIRATUKU website

[About the writer]
Kyoko Sugimoto
Freelance writer. Earned a Master Degree in Media, Journalism & Communication, Graduate School of Letters, Doshisha University. Interviews researchers, business managers, Buddhist monks, urban designers on such topics as asylums, communities, and Buddhism. Authored Kyodaiteki Bunka Jiten: Jiyu to Kaosu no Seitaikei (Kyoto University Cultural Encyclopedia: Ecosystem of Freedom and Chaos), Film Art, Inc.　 writin’room