Bloom filters use sort of string comparisons. They use bitmask comparisons. I'm not sure how an index could work for such a bitmask but apparently redis does something there.
Pre-computing the hashes would make things super expensive again. In your example of a 67kB Bloom filter, pubkeys extended with hashes would be 67kB each, too.
I tried to do this myself but the best I got was to use probabilistic filters to save memory for active filters, not for querying the DB and thus not for the communications neither. The nice part is that you can mix event IDs and authors in the same filter.
https://github.com/Giszmo/NostrPostr/blob/master/nostrpostrlib/src/main/java/nostr/postr/Filter.kt#L195
Leo Wandersleb
npub1gm…78rf6
2024-08-19 18:11:46
in reply to nevent1q…6f6s
Author Public Key
npub1gm7tuvr9atc6u7q3gevjfeyfyvmrlul4y67k7u7hcxztz67ceexs078rf6Published at
2024-08-19 18:11:46Event JSON
{
"id": "71800a260c3155709925e6e9e50ebdc623d3bbe0e6301bf2c52716e9b937e9d2",
"pubkey": "46fcbe3065eaf1ae7811465924e48923363ff3f526bd6f73d7c184b16bd8ce4d",
"created_at": 1724091106,
"kind": 1,
"tags": [
[
"e",
"75f8b4e77cad6eb2d5dcdef11e195954baa6b0f97405e353a4a051af9291c80a",
"",
"root"
],
[
"e",
"9bbbb0e5202cb23702c8cecfb12334a0aca15c9be456ba9663e54f0a92e5a636",
"",
"reply"
],
[
"p",
"460c25e682fda7832b52d1f22d3d22b3176d972f60dcdc3212ed8c92ef85065c",
"",
"mention"
],
[
"p",
"5b459807bf935ddc2f83405821bced1c8a05dfee7c80b000c337ba6bd0ad78ed",
"",
"mention"
]
],
"content": "Bloom filters use sort of string comparisons. They use bitmask comparisons. I'm not sure how an index could work for such a bitmask but apparently redis does something there.\n\nPre-computing the hashes would make things super expensive again. In your example of a 67kB Bloom filter, pubkeys extended with hashes would be 67kB each, too.\n\nI tried to do this myself but the best I got was to use probabilistic filters to save memory for active filters, not for querying the DB and thus not for the communications neither. The nice part is that you can mix event IDs and authors in the same filter.\n\nhttps://github.com/Giszmo/NostrPostr/blob/master/nostrpostrlib/src/main/java/nostr/postr/Filter.kt#L195",
"sig": "d6d5162fc9ce54726d37a58ad87747b8dd9e293fe1bacadc49bce458593a1f630c62aea2738cc890e0766d9b730b71f7c3c6a77034833bde65784a542c0930a8"
}