{"type":"rich","version":"1.0","title":"matejcik [ARCHIVE] wrote","author_name":"matejcik [ARCHIVE] (npub18g…sgac3)","author_url":"https://yabu.me/npub18g04tf4qudcsna6qfcrm55s39axx3ymrunr65gxen4rctm0zv24sasgac3","provider_name":"njump","provider_url":"https://yabu.me","html":"📅 Original date posted:2018-06-26\n📝 Original message:hello,\n\nin general, I agree with my colleague Tomas, the proposed changes are\ngood and achieve the most important things that we wanted. We'll review\nthe proposal in more detail later.\n\nFor now a couple minor things I have run into:\n\n- valid test vector 2 (\"one P2PKH input and one P2SH-P2WPKH input\")\nseems broken, at least its hex version; a delimiter seems to be missing,\nmisplaced or corrupted\n- at least the first signing vector is not updated, but you probably\nknow that\n- BIP32 derivation fields don't specify size of the \"master public key\",\nwhich would make it un-parsable :)\n- \"Transaction format is specified as follows\" and its table need to be\nupdated.\n\n\nI'm still going to argue against the key-value model though.\n\nIt's true that this is not significant in terms of space. But I'm more\nconcerned about human readability, i.e., confusing future implementers.\nAt this point, the key-value model is there \"for historical reasons\",\nexcept these aren't valid even before finalizing the format. The\noriginal rationale for using key-values seems to be gone (no key-based\nlookups are necessary). As for combining and deduplication, whether key\ndata is present or not is now purely a stand-in for a \"repeatable\" flag.\nWe could just as easily say, e.g., that the high bit of \"type\" specifies\nwhether this record can be repeated.\n\n(Moreover, as I wrote previously, the Combiner seems like a weirdly\nplaced role. I still don't see its significance and why is it important\nto correctly combine PSBTs by agents that don't understand them. If you\nhave a usecase in mind, please explain.\nISTM a Combiner could just as well combine based on whole-record\nuniqueness, and leave the duplicate detection to the Finalizer. In case\nthe incoming PSBTs have incompatible unique fields, the Combiner would\nhave to fail anyway, so the Finalizer might as well do it. Perhaps it\nwould be good to leave out the Combiner role entirely?)\n\nThere's two remaining types where key data is used: BIP32 derivations\nand partial signatures. In case of BIP32 derivation, the key data is\nredundant ( pubkey = derive(value) ), so I'd argue we should leave that\nout and save space. In case of partial signatures, it's simple enough to\nmake the pubkey part of the value.\n\nRe breaking change, we are proposing breaking changes anyway, existing\ncode *will* need to be touched, and given that this is a hand-parsed\nformat, changing `parse_keyvalue` to `parse_record` seems like a small\nmatter?\n\n---\n\nAt this point I'm obliged to again argue for using protobuf.\n\nThing is: BIP174 *is basically protobuf* (v2) as it stands. If I'm\nsuccesful in convincing you to switch to a record set model, it's going\nto be \"protobuf with different varint\".\n\nI mean this very seriously: (the relevant subset of) protobuf is a set\nof records in the following format:\n\u003crecord type\u003e\u003cvarint field length\u003e\u003cfield data\u003e\nRecord types can repeat, the schema specifies whether a field is\nrepeatable or not - if it's not, the last parsed value is used.\n\nBIP174 is a ad-hoc format, simple to parse by hand; but that results in\n_having to_ parse it by hand. In contrast, protobuf has a huge\ncollection of implementations that will do the job of sorting record\ntypes into relevant struct fields, proper delimiting of records, etc.\n...while at the same time, implementing \"protobuf-based-BIP174\" by hand\nis roughly equally difficult as implementing the current BIP174.\n\nN.B., it's possible to write a parser for protobuf-BIP174 without\nneeding a general protobuf library. Protobuf formats are designed with\nforwards- and backwards- compatibility in mind, so having a hand-written\nimplementation should not lead to incompatibilities.\n\nI did an experiment with this and other variants of the BIP174 format.\nYou can see them here:\n[1] https://github.com/matejcik/bip174-playground\nsee in particular:\n[2] https://github.com/matejcik/bip174-playground/blob/master/bip174.proto\n\nThe tool at [1] does size comparisons. On the test vectors, protobuf is\nslightly smaller than key-value, and roughly equal to record-set, losing\nout a little when BIP32 fields are used.\n(I'm also leaving out key-data for BIP32 fields.)\n\nThere's some technical points to consider about protobuf, too:\n\n- I decided to structure the message as a single \"PSBT\" type, where\n\"InputType\" and \"OutputType\" are repeatable embedded fields. This seemed\nbetter in terms of extensibility - we could add more sections in the\nfuture. But the drawback is that you need to know the size of\nInput/OutputType record in advance.\nThe other option is sending the messages separated by NUL bytes, same as\nnow, in which case you don't need to specify size.\n\n- in InputType, i'm using \"uint32\" for sighash. This type is not\nlength-delimited, so non-protobuf consumers would have to understand it\nspecially. We could also declare that all fields must be\nlength-delimited[1] and implement sighash as a separate message type\nwith one field.\n\n- non-protobuf consumers will also need to understand both protobuf\nvarint and bitcoin compact uint, which is a little ugly\n\nbest regards\nmatejcik\n\n\n[1] https://developers.google.com/protocol-buffers/docs/encoding#structure\n\n-------------- next part --------------\nA non-text attachment was scrubbed...\nName: signature.asc\nType: application/pgp-signature\nSize: 819 bytes\nDesc: OpenPGP digital signature\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180626/a1c4e6e5/attachment.sig\u003e"}