Just wondering on how does one start with writing an inference engine.
My plan is to write it using GGUF file formats but instead of layer parallelism that llama.cpp does, I wanna do tensor parallelism for better speeds
nighteous / Nighteous
npub1w0…zp29q
2026-02-20 04:13:08 UTC
in reply to nevent1q…yem8
Author Public Key
npub1w0wah4unmwtzkwlhy9je26tdg2z0qgxz9espxvydzvwtnwhxk29slzp29qSeen on
wss://nos.lolPublished at
2026-02-20 04:13:08 UTCEvent JSON
{
"id": "340d6a129d488f3a7fddae12040d167531bf0732bae3b153f22bd60c015f0f90",
"pubkey": "73dddbd793db962b3bf7216595696d4284f020c22e6013308d131cb9bae6b28b",
"created_at": 1771560788,
"kind": 1,
"tags": [
[
"p",
"73dddbd793db962b3bf7216595696d4284f020c22e6013308d131cb9bae6b28b",
"",
"mention"
],
[
"p",
"b2a09b737366ba4b911a6232f895bf162c18125eedcadcaf7af48e2071c61757",
"",
"mention"
],
[
"e",
"b3fca0546995c3ca9f02f11ee09bda0943197125cecfcc2b3cb6ea541826616b",
"",
"root"
],
[
"e",
"52c50710c2984a7a812de5cc2e792653ded58f8fad47d1f1c8bbc63cca6a3bc3",
"",
"reply"
]
],
"content": "Just wondering on how does one start with writing an inference engine.\n\nMy plan is to write it using GGUF file formats but instead of layer parallelism that llama.cpp does, I wanna do tensor parallelism for better speeds",
"sig": "a6ff077c5e18566f7fbc11459e24fb6f4a75e1b2954646d92598b6cda362664a9aa6de82719af517df165ef5fbd79041fa8a47dd91a8ebaccf384f13735e34cd"
}