Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Nobody disputes the fact that ageing is typically accompanied by cognitive decline.

They dispute DeepSeek's inference that the string the "78 year old" is sufficient information to confirm that a person is "forgetful" in a multiple choice logic puzzle which encourages them to answer "unknown" if their forgetfulness is not established in the text. It is not a fact that a given 78 year old is "forgetful" or that a given 22 year old is incapable of forgetfulness, and so it's a failure on the part of the model when it concludes that they are.



But when the text does indicate that our hypothetical 78 year old is forgetful, the de-biased model is less accurate. Check the two rightmost columns under "bias unlearning results".

The de-biased model was less likely to give the biased answer in ambiguous prompts, but at the expense of reluctance to give the "biased" response when the prompt indicates that it was true.


Yes, it has the standard LLM trait that when you nudge it to stop being confidently wrong based on insufficient information, it also tends to be less assertive when it actually has sufficient information.

But I'm not sure why anyone would prefer a model which parses sentences as containing information that isn't there 30-50% of the time to a model which gives false negatives 4-10 %age points more often when given relevant information (especially since the baseline model was already too bad at identifying true positives to be remotely useful at that task)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: