Stanford researchers studied the evolution of the ChatGPT chatbot. The conclusions are disappointing. AI has gotten dumber in the last few months. This is evidenced by the test results, reports arxiv.org.
Scientists decided to carry out this work after complaints from users on the OpenAI forum. They didn’t like the responses of the ChatGPT-4 model and Microsoft’s Bing chatbot that runs on it. To assess the quality of ChatGPT, they were forced to solve mathematical problems, answer sensitive and dangerous questions, were asked to generate code and show visual thinking.
In the March version, AI determined prime numbers with an accuracy of 97.6%. In the June update, the indicator was 2.4%. June’s GPT-3.5 was better than March’s GPT-3.5. It turns out that the developers make updates to their product at a worse level.
As the reason for the situation, scientists suspect that the developers went into economy mode for the functioning of ChatGPT. According to some data, its maintenance costs up to 700,000 dollars per day. Therefore, the neural network is adjusted so that the answers are of the same quality, but with less spent resources. Quality has slipped, and economy has slipped sideways. Therefore, ChatGPT became “cheaper”.
Source: https://itechua.com/