On AI, language and sincerity - Cade Metz in NYT:

‘This summer, an artificial intelligence lab in San Francisco called OpenAI unveiled a technology several months in the making. This new system, GPT-3, had spent those months learning the ins and outs of natural language by analyzing thousands of digital books, the length and breadth of Wikipedia, and nearly a trillion words posted to blogs, social media and the rest of the internet.
Mckay Wrigley, a 23-year-old computer programmer from Salt Lake City, was one of the few invited to tinker with the system, which uses everything it has learned from that vast sea of digital text to generate new language on its own. Mr. Wrigley wondered if it could imitate public figures — write like them, perhaps even chat like them.
One of his experiments involved a pop psychologist, Scott Barry Kaufman. The system took in Mr. Kaufman’s name and a topic for discussion: creativity. Then, when asked “How do we become more creative?” GPT-3 responded instantly: I think creative expression is a natural byproduct of growing up in a diverse world. The more diverse the world is, the more you get exposed to different people, to different opportunities, to different places and to different challenges. And the more diverse that is, the more likely you’ll be to be able to put the dots together to form something new. And in many ways, I think if you want to be creative, you have to go for it. If you want to be a writer, you have to write, if you want to be a musician, you have to create music, if you want to be a comedian, you have to create comedy, and the more you create, the more likely it is that you’ll stumble onto some interesting stuff.
Later, when Mr. Wrigley posted the paragraph on Twitter, somebody looped in the real Scott Barry Kaufman. He was stunned. “It definitely sounds like something I would say,” the real Mr. Kaufman tweeted, later adding, “Crazy accurate A.I.”’

(…)

‘GPT-3 is far from flawless. It often spews biased and toxic language. And if you ask for 10 paragraphs in the style of Scott Barry Kaufman, it might give you five that are convincing — and five others that are not. When Mr. Kaufman actually chatted with his A.I. doppelgänger, he was not nearly as impressed as he was at first: Scott Barry Kaufman: What is imagination? “Scott Barry Kaufman”: What is imagination? You should tell me! Scott Barry Kaufman: You should know! “Scott Barry Kaufman”: Nope, you should know. You are the one who knows what imagination is. What do you think it is? But even this looked like real conversation. And notably, it showed what the system could do entirely on its own, without extra coding or fine-tuning.
GPT-3 is the culmination of several years of work inside the world’s leading artificial intelligence labs, including OpenAI, an independent organization backed by $1 billion dollars in funding from Microsoft, as well as labs at Google and Facebook. At Google, a similar system helps answer queries on the company’s search engine.’

(…)

‘As software designers, entrepreneurs, pundits and artists explore this system, each new experiment stokes an already heated debate over how powerful this breed of technology will ultimately be. While some say it may be a path toward truly intelligent machines, others argue that these experiments, while endlessly fascinating, are also misleading.
“It is very fluent,” said Mark Riedl, a professor and researcher at the Georgia Institute of Technology. “It is very articulate. It is very good at producing reasonable-sounding text. What it does not do, however, is think in advance. It does not plan out what it is going to say. It does not really have a goal.”’

(…)

‘Before asking GPT-3 to generate new text, you can focus it on particular patterns it may have learned during its training, priming the system for certain tasks. You can feed it descriptions of smartphone apps and the matching Figma code. Or you can show it reams of human dialogue. Then, when you start typing, it will complete the sequence in a more specific way. If you prime it with dialogue, for instance, it will start chatting with you.
“It has this emergent quality,” said Dario Amodei, vice president for research at OpenAI. “It has some ability to recognize the pattern that you gave it and complete the story, give another example.” Previous language models worked in similar ways. But GPT-3 can do things that previous models could not, like write its own computer code. And, perhaps more important, you can prime it for specific tasks using just a few examples, as opposed to the thousands of examples and several hours of additional training required by its predecessors. Researchers call this “few-shot learning,” and they believe GPT-3 is the first real example of what could be a powerful phenomenon.
“It exhibits a capability that no one thought possible,” said Ilya Sutskever, OpenAI’s chief scientist and a key figure in the rise of artificial intelligence technologies over the past decade. “Any layperson can take this model and provide these examples in about five minutes and get useful behavior out of it.” This is both a blessing and a curse.’

(…)

‘n the mid-1960s, Joseph Weizenbaum, a researcher at the Massachusetts Institute of Technology, built an automated psychotherapist he called ELIZA. Judged from our vantage point in 2020, this chatbot was exceedingly simple.
Unlike GPT-3, ELIZA did not learn from prose. It operated according to a few basic rules defined by its designer. It pretty much repeated whatever you said to it, only in the form of a question. But much to Dr. Weizenbaum’s surprise, many people treated the bot as if it were human, unloading their problems without reservation and taking comfort in the responses.’

(…)

‘For some researchers, the experiment indicates that such a system could ultimately handle tasks across multiple dimensions — language, sight, sound — much like humans do. Even when trained solely on language, they say, the system could already reach into other areas, whether computer programming, playing chess or generating guitar tabs.
But continuing to improve this technology is far from trivial. Processing all of that internet data requires a specialized supercomputer running for months on end, an undertaking that is enormously expensive. When asked if such a project ran into the millions of dollars, Sam Altman, OpenAI’s chief executive, said the costs were actually “higher,” running into the tens of millions.
Mr. Amodei, OpenAI’s vice president for research, said there was still room to improve the technique, using more processing power to analyze more data. But he also said the approach might be close to running out of “juice.” At the very least, GPT-3 is a new tool for a world of A.I. researchers and entrepreneurs, a way of building all sorts of new technologies and new products. Mr. Wrigley, the computer programmer, recently quit his day job to start a company called LearnFromAnyone, which aims to build a kind of automated tutor using GPT-3 that can assume the guise of everyone from scientist Douglas Hofstadter to venture capitalist Peter Thiel. Others are building companies that aim to automatically generate code for computer programmers and automatically write promotional emails and tweets for marketing professionals.
But it is unclear how effective these services will ultimately be. If GPT-3 generates the right text only half of the time, can it satisfy professionals? And it is unclear whether this technique is a path to truly conversational machines, let alone truly intelligent systems. Additional progress on the long road to machines that can mimic the human brain, Mr. Amodei said, will require entirely new ideas.
“It is kind of like a chemistry reaction,” he said. “We have this one ingredient. But other ingredients are required as well.”’

Read the article here.

So, GPT-3 is good, but how much better is it than Eliza?

Psychotherapists won’t like this but quite a few things they’re doing could be done by Eliza, GPT-3 or let’s say GPT-4. Of course, a lot of verbal human interaction is entirely predictable and could be produced by a system like GPT-3. The fact that the systems sometimes spew toxic language makes them only more human.
One could argue that the moment we humans will only use socially desirable language (whatever we mean by that, of course what exactly should be considered socially desirable language will differ from time to time, region to region) it will be easier for a system to copy that language.
Big problems for machines that produce language are conscious humor, irony and other sorts of subtleties. But human beings find these things difficult as well.

The better systems that produce language become the bigger the desire to know if the system is sincere.
If somebody tells us, ‘I love you’ fairly often we’d like to know if this person is serious. If it’s a meaningful statement or not, if the language has consequences.

The problem for the system is not the language, yes, for now, but sooner or later the system will be as fluent in English as an average American, and it will be as good in small talk as an average European.

But there comes a moment that language needs to have consequences.

The system can correctly say: ‘I will change your diaper, dear.’

But if my diaper needs to be changed, I hope that somebody will change my diaper. That it will be more than words, than an empty promise.

Or to use another example, the system might be able to say utterly correctly during a conversation: ‘You can do me with as you please. You can fuck me.’ But can we touch GPT-3, can we stroke the skin of GPT-3, does GPT-3 have genitals?

Not yet, but that’s only a matter of time.