Melanie Mitchell, an AI researcher at the Santa Fe Institute, is also excited to see a whole new approach. “We really haven’t seen so much come out of the deep learning community,” she said. She also agrees with LeCun that big language models can’t be the whole story. “They lack memory and internal models of the world, which are really important,” she said.
Natasha Jacques, a researcher at Google Brain, believes that language models still have a role to play. It’s strange that language is completely missing from LeCun’s proposals, she says: “We know that big language models are super effective and based on a lot of human knowledge.”
Jacques, who is working on ways to get AI to share information and abilities with each other, points out that people don’t have to have direct experience of something to learn about it. We can change our behavior simply by telling us something, such as not touching a hot pan. “How do I update this global model that Ian offers if I don’t have a language?” She asks.
There is another problem. If they work, Lekun’s ideas will create a powerful technology that can be as transformative as the Internet. However, his proposal does not discuss how the behavior and motivation of his model will be controlled or who will control them. This is a strange omission, says Abhishek Gupta, founder of the Montreal AI Ethics Institute and a responsible AI expert at Boston Consulting Group.
“We need to think more about what it takes for AI to work well in society, and that requires thinking about ethical behavior, among other things,” Gupta said.
Still, Jacques notes that Lecun’s suggestions are still ideas rather than practical applications. Mitchell says the same thing: “There is certainly little risk that this will soon become intelligence on a human level.
Lekun would agree. His goal is to sow the seeds of a new approach in the hope that others will build on it. “It’s something that will take a lot of effort from a lot of people,” he says. “I’m putting it there because I think that’s the way to go in the end.” If nothing else, he wants to convince people that big language models and reinforced learning aren’t the only ways forward.
“I hate to see people waste their time,” he says.