Maybe it’s worse. At least the formula is deterministic. The star thing isn’t, so it can be a fig leaf for all kinds all of the stuff you mention later in the thread like nepotism, bias, …
I don’t have a perfect solution either, but I’ll stick my neck out. When I review others, I think “Well, this institution hired a person in subsubfield X. Do other people in that subsubfield find their output, in whole, valuable?” So input from those experts, who have read (and hopefully cited) the work is useful. But I think at max 2 bits of dynamic range per reviewer. Finer grading than that is not reliable. Letters can be nice, but even just a three point scale rating (basically average or over a standard deviation above or below) can be helpful.
Spencer LaVere Smith
npub1yl…dq722
2025-07-31 01:24:18 UTC
in reply to nevent1q…ff7n
Author Public Key
npub1yl3knzsdqtxuzj8hn08zcrdpzrc2c50zu6tlnl08hnmnqugalt4smdq722Seen on
wss://relay.momostr.pinkPublished at
2025-07-31 01:24:18 UTCEvent JSON
{
"id": "de31cd56eb9e05ef48154530e559e7b2e10c0225a47d2f6e13d19af22f5d9a89",
"pubkey": "27e3698a0d02cdc148f79bce2c0da110f0ac51e2e697f9fde7bcf730711dfaeb",
"created_at": 1753925058,
"kind": 1,
"tags": [
[
"proxy",
"https://neuromatch.social/@sls/114945232633752183",
"web"
],
[
"p",
"ae2996b7856b79f43117388762ec45a556e8a844db6b739fb25792f41fed6923"
],
[
"p",
"8531cbd6c901be220064569020dac514b945ae75f9b9d612ebf3e889edfee63f"
],
[
"e",
"3339e8b7733e407d38ab08cf3031d018accdfcb3f11a87209b0a3c3c37552864",
"",
"reply",
"b126c437ff4890ef291aeebc26f59e12e9285c7a2fb0e5e26edb27d143e81e1e"
],
[
"p",
"b126c437ff4890ef291aeebc26f59e12e9285c7a2fb0e5e26edb27d143e81e1e"
],
[
"e",
"05a42f57b85be22fe3f6008e14721b61d639af173cde89076335a9e1f8c957b9",
"",
"root",
"8531cbd6c901be220064569020dac514b945ae75f9b9d612ebf3e889edfee63f"
],
[
"proxy",
"https://neuromatch.social/users/sls/statuses/114945232633752183",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://neuromatch.social/users/sls/statuses/114945232633752183",
"pink.momostr"
],
[
"-"
]
],
"content": "Maybe it’s worse. At least the formula is deterministic. The star thing isn’t, so it can be a fig leaf for all kinds all of the stuff you mention later in the thread like nepotism, bias, …\n\nI don’t have a perfect solution either, but I’ll stick my neck out. When I review others, I think “Well, this institution hired a person in subsubfield X. Do other people in that subsubfield find their output, in whole, valuable?” So input from those experts, who have read (and hopefully cited) the work is useful. But I think at max 2 bits of dynamic range per reviewer. Finer grading than that is not reliable. Letters can be nice, but even just a three point scale rating (basically average or over a standard deviation above or below) can be helpful.",
"sig": "32d4ef9c6c9c6d7e8a83512963767f270b00e1f383a5f76a81119d6f3f37c846d58c5552adeefd071c5d4cc82548241fe4094ea2d24c29e8fa1b58aea8c17920"
}