I put .bitcoin/chainstate directory on ZFS, and quite satisfied with the result.
Before, processing a block could take 4 minutes, lightning complained about failed healtcheck 10 times a day. After 23 hours with ZFS, I observe just 1 healtcheck failure, and block processing is < 30s.
I also set /sys/module/zfs/parameters/zfs_arc_min and zfs_arc_max to 5.5 GB and 7 GB respectively. This is a laptop with 8 GB RAM and 2 HDDs, running just bitcoind, lnd is on another laptop. Just moving the chainstate to the other HDD did not help that much.
In theory, this should keep most of chainstate in cache, even when peers do a lot of historical queries. (The rest of .bitcoin is NOT on ZFS, but on Btrfs.)
Not the most elegant solution - ZFS help mostly because it has separate cache. But even if everything was on ZFS, I think ZFS's caching algo would notice that chainstate is read much more frequently than the rest, and keep most of it in cache - but I haven't tested this.
xmrk ₿ ⚡️ /
npub1zu…2qhzy
2023-10-08 11:09:48
Author Public Key
npub1zues2p7qdajk9hv7xn5ldntj3pufzl3jwzwa4dhlavxdkj27g92q62qhzyPublished at
2023-10-08 11:09:48Event JSON
{
"id": "419337d1ed440d5a39ed213f09ae14a54f059ed1e8b4729fe720b44f7ea1ae4f",
"pubkey": "17330507c06f6562dd9e34e9f6cd728878917e32709ddab6ffeb0cdb495e4154",
"created_at": 1696763388,
"kind": 1,
"tags": [],
"content": "I put .bitcoin/chainstate directory on ZFS, and quite satisfied with the result. \n\nBefore, processing a block could take 4 minutes, lightning complained about failed healtcheck 10 times a day. After 23 hours with ZFS, I observe just 1 healtcheck failure, and block processing is \u003c 30s. \n\nI also set /sys/module/zfs/parameters/zfs_arc_min and zfs_arc_max to 5.5 GB and 7 GB respectively. This is a laptop with 8 GB RAM and 2 HDDs, running just bitcoind, lnd is on another laptop. Just moving the chainstate to the other HDD did not help that much. \n\nIn theory, this should keep most of chainstate in cache, even when peers do a lot of historical queries. (The rest of .bitcoin is NOT on ZFS, but on Btrfs.) \n\nNot the most elegant solution - ZFS help mostly because it has separate cache. But even if everything was on ZFS, I think ZFS's caching algo would notice that chainstate is read much more frequently than the rest, and keep most of it in cache - but I haven't tested this.",
"sig": "59671386902f9aef2d77aa65540369aa8b412ed58a590203ff8a61afd6d0c58c3cca26c279e6201bbcb1d78737d8fc52eb5aa0dd708d0c24923d118a88a9ca63"
}