Redish Lab on Nostr: Re: interested vs disinterested evaluation. When I want to know whether to believe ...

Re: interested vs disinterested evaluation.

When I want to know whether to believe some result and whether to act on it or not, I really want that evaluation to be from a party only interested in getting the right answer and not from someone with a conflict of interest (either pro or con).

If I want to know whether to plan for rising seas when I build my house, I want a geo-climate expert, not a debate between a climate expert and a climate change denier. If I want to know whether to vaccinate my kids, I want to get my information from NIAID, not from RFK Jr., even though he is apparently a very interested party. The correct person to assess whether a plane is safe to fly is not the manufacturer nor its competitors. It's the FAA.

Similarly, when I want peer review to assess my work (or someone else's), I want a disinterested party. Sure, as a scientist, I like to be challenged, and certainly I've had my share of fun in response to "adversarial collaborations", but it is definitely better when these are more "collaboration" and less "adversarial".

I do think it is sometimes fun to see two scientists really going at it in highly interested open discussion. But I think a lot of that is about seeing a good sporting match. In my experience, this kind of mano-a-mano fight is best done in published papers back and forth, with _external_ (disinterested!) reviewers assessing each paper (like a referee).

I don't think a mano-a-mano fight is evaluation. I think it is scientific discussion, which is different.

In my experience, the best *reviews* have always definitely more been experts interested in checking my answers, not someone particularly interested in proving me wrong.