Tech experts are starting to doubt that ChatGPT and A.I. ‘hallucinations’ will ever go away: ‘This isn’t fixable’::Experts are starting to doubt it, and even OpenAI CEO Sam Altman is a bit stumped.
the models are also getting larger (and require even more insane amounts of resources to train) far faster than they are getting better.
But bigger models have new “emergent” capabilities. I heard that from a certain size they start to know what they know and hallucinate less.
Wow you heard that crazy bro
One of the papers about it https://arxiv.org/pdf/2206.07682.pdf
I disagree, with models such as llama it has become clear that there are interesting advantages on increasing (even more) the ratio of parameters/data. I don’t think next iterations of models from big-corp will 10x the param count until nvidia has really pushed hardware, models are getting better over time. ChatGPT’s deterioration is mostly coming from openAI’s ensuring safety and is not a fair assessment of progress on LLMs in general, the leaderboard of open source models has been steadily improving over time: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
