Davide Eynard (+mala)

Dad of 3564020356.org, PicoGopher, BYOTA, and two amazing kids. Interested in open applications of ML on federated systems. Trustworthy AI at mozilla.ai. Genetically a teacher, forever a student. He/him.

Phlog: gopher://gopher.3564020356.org
Website: http://3564020356.org
Website: http://davide.eynard.it
GitHub: https://github.com/aittalam

Public Key

npub1xassj2fpkfy76nhxulxd2sn90nmg3dtx7lyv9hcda8ef84s4p7hsjaj5pq

NIP-05 Address

mala_at_fosstodon.org@momostr.pink

Profile Code

nprofile1qqsrwcgf9ysmyj0dfmnw0nx4gfjhea5gk4n00jxzmux7nu5n6c2sltcpz3mhxue69uhhyetvv9ujuerpd46hxtnfduqs6amnwvaz7tmwdaejumr0dsujsqce

Publishing to

yabu.me

relay.damus.io

nos.lol

Author Public Key

npub1xassj2fpkfy76nhxulxd2sn90nmg3dtx7lyv9hcda8ef84s4p7hsjaj5pq

Show more details

Last Notes

2025-03-10 13:05:02 UTC

Hey all, are there / will there be any recordings available from the #fediversehouse event at SXSW? My FOMO is killing me 🙈

2025-02-08 09:25:10 UTC

- reply

“What data do I have to share to run this?”
The answer is: none. The embedding servers run completely offline. And for the marimo notebook, the only remote connections you’ll do are those related to the marimo and WASM dependencies at bootstrap time, and those required to download posts from your marimo instance. You are always in control of which and how many messages you download before running the algorithm. From then on (embeddings, plots, search, re-ranking) everything runs on your device.
https://cdn.fosstodon.org/media_attachments/files/113/967/543/576/784/330/original/698211c82265c37d.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/543/638/839/400/original/9787568255113ad5.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/543/719/346/855/original/1b5c0918def31446.jpeg

2025-02-08 09:25:06 UTC

- reply

All-miniLM took 11 sec on M3, 52 sec on my 8yo laptop. And embeddings are already good! Larger models might provide extra perks (which I have not investigated yet) but at the price of higher compute. Interestingly, despite the fact that both ollama and llamafile are based on llama.cpp, ollama seems to be faster on newer hardware / smaller models, while llamafile becomes a better choice for larger models on older hardware. I used default params for both servers so there is space for improvement.
https://cdn.fosstodon.org/media_attachments/files/113/967/543/461/457/441/original/3a57eb46c4292049.jpeg

2025-02-08 09:25:04 UTC

- reply

How could we have a timeline algorithm without post re-ranking? As you see in the figures, by relying on sentence embeddings you can have a content-based re-ranking of a timeline. You basically apply the “style” of a set of statuses to another list, by putting on top those statuses which are overall closer to the ones you provided. For privacy purposes I did not show a re-ranked public timeline: I did it with my own messages, re-ranking them according to the “style” of #gopher and #bookstodon.
https://cdn.fosstodon.org/media_attachments/files/113/967/543/172/650/151/original/03ebed1ea8205b43.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/543/280/911/184/original/644ec1971c19f06e.jpeg

2025-02-08 09:25:04 UTC

- reply

“This looks nice, but how heavy is the calculation of these embeddings?”.
I embedded 800 statuses using four different models, from the 22M-parameters all-MiniLM to the 7B e5-mistral. I tested them with two different local servers, llamafile and ollama, and two different laptops, my 2016 MacBook Pro with Intel CPU and my 2024 one with M3 Max. The results are shown in the picture. The summary is: it can work on older hardware, and on recent one you will barely notice the calculation overhead.

2025-02-08 09:25:00 UTC

- reply

Semantic search is one of them! If you provide a status ID (which you can get by selecting a post in the map) you can look for the statuses (default is top 5) which are most similar, that is closest, to it. You can also just make up a sentence that describes what you are interested in (e.g. the figure shows the closest statuses to “I am a fan of free software and everything opensource”). Note that the results come from different timelines, in this case public, local, and tag/gopher.
https://cdn.fosstodon.org/media_attachments/files/113/967/543/033/707/381/original/b04a8f279d7129f6.jpeg

2025-02-08 09:24:58 UTC

- reply

You can also plot all embeddings together! In this picture I have selected, analyzed, and annotated a few areas of this map. What i find cool is that semantically similar statuses will always be close to each other regardless of which timeline they appeared in, so you can follow content across different timelines. This is simple to grasp and easy to interpret, as it is based on the actual text present in statuses. And this way of encoding semantic similarity allows us to do many other things.
https://cdn.fosstodon.org/media_attachments/files/113/967/542/861/553/835/original/8342b625accd6579.jpeg

2025-02-08 09:24:54 UTC

- reply

So, what can you do with #BYOTA? The first thing is embeddings visualization. In these pictures you can see a 2D plot of embeddings calculated on four different timelines: home (blue, only people I follow), local (orange, all posts from my instance, which is fosstodon.org), public (red, federated posts from people followed by users of my instance), and the timeline that I got searching for the #gopher hashtag (light blue).
https://cdn.fosstodon.org/media_attachments/files/113/967/542/279/684/510/original/7a7ad68554ca1818.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/542/383/694/906/original/b9fb462bd2fbbe2d.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/542/470/596/686/original/03435800e7afada3.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/542/705/545/589/original/1dbf8538b9b54467.jpeg

2025-02-08 09:24:46 UTC

- reply

Marimo is a reactive notebook for Python, that is also sharable as an application. It mixes code, markdown and UI widgets in the same interface so you can (1) develop as you’d do with other notebook environments, (2) share it as an application, by hiding the code and only displaying the UI components, (3) allow people to use it as they want, customizing both code and interface within a grid-based UI. Above all, marimo relies on WASM to run everything inside one’s browser.
https://cdn.fosstodon.org/media_attachments/files/113/967/541/800/929/064/original/0456300181d2fdaf.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/541/889/700/993/original/77eb875fb8903afe.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/541/990/378/031/original/477a6ed9766d236f.jpeg
https://cdn.fosstodon.org/media_attachments/files/113/967/542/129/000/155/original/05c2ce8e548f0eda.jpeg

2025-02-08 09:24:46 UTC

- reply

What does having a WASM-powered notebook mean? Consider #BYOTA: you can download it from its repo, pip-install its deps, and run it locally as any python notebook. But you can also deploy it as HTML+Javascript files, host it somewhere super cheap (coz the server won’t run any of your code), and people will be able to run it in their browser with no need to install anything else! Plus, this will work both with “my” algorithm and whatever alternative you might develop starting from BYOTA’s code.

2025-02-08 09:24:39 UTC

- reply

In #BYOTA, we use #llamafile to calculate *sentence embeddings*.
If you don’t know what embeddings are, just think about them as numerical descriptors of your Mastodon statuses, which are closer (as in two cities’ coordinates being close on a map) the more semantically similar their respective statuses are. We’ll get back later to this with a more visual description. If you are interested in embeddings and wanna delve deeper, see https://vickiboykis.com/what_are_embeddings/ by @npub1fzn…nnyk.

2025-02-08 09:24:38 UTC

- reply

#Llamafile is a Mozilla tool that packages a language model in a single executable file that will run on most platforms. It is 100% local and has been optimized to run on slower hardware, from Raspberry Pis to my 8yo laptop. It is based on llama.cpp which supports a plethora of models, not just LLMs: I chose all-minilm because it’s tiny (50MB) and has open code, research papers, and datasets. https://github.com/Mozilla-Ocho/llamafile https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
https://cdn.fosstodon.org/media_attachments/files/113/967/541/634/615/269/original/e7888f1ced808282.jpeg

2025-02-08 09:24:36 UTC

- reply

Mastodon.py is a python client library for Mastodon and I use it to download my own posts and my instance’s timeline updates. As what we are interested in are just status contents, you can swap this library with whatever works with your setup: a generic ActivityPub client, a specific one for a different social network that is smart enough to expose an API, an RSS feed reader… Even a mix of these if you want to play with cross-platform search and recommendations. https://mastodonpy.readthedocs.io

2025-02-08 09:24:36 UTC

- reply

BYOTA is built on this assumption: do something useful while preserving user privacy and control, using open models and local-first technologies. While the concept sounds straightforward, developing something which is both easy to use and customize, computationally light, and useful in my spare time was not trivial. Luckily, in the last year I found some tools which are the right ones for the task and now BYOTA exists thanks to these three main components: Mastodon.py, llamafile, and marimo.
https://cdn.fosstodon.org/media_attachments/files/113/967/541/436/686/260/original/ab511a699f3dc286.jpeg

2025-02-08 09:24:33 UTC

- reply

In practice, those are actual problems only when the objective is to retain users on your platform. However, we can break this assumption and decide we don’t need to compete with commercial algorithms: serving people’s purposes is more than enough. When you do this, you immediately realize there’s a solution to each of the above problems, one which relies on open, small, interpretable models, that can be run locally for ad-hoc tasks without requiring too much ML knowledge: local embeddings.
https://cdn.fosstodon.org/media_attachments/files/113/967/541/312/523/116/original/fb9b86ac57cc7a69.jpeg

2025-02-08 09:24:31 UTC

- reply

Every algorithm I saw was affected by one or more of the following problems: bias; lack of privacy, transparency, and user control; dependency on ML algorithms which are complex, computationally heavy and require to run in a centralized way. Note that reverse chronological itself -which many welcome as absence of an algorithm- is an algorithm and it is biased, as it generally favors statuses by people who write more and in the same timezone as me.
https://cdn.fosstodon.org/media_attachments/files/113/967/541/142/350/359/original/5d94f3945deab344.jpeg

2025-02-08 09:24:29 UTC

- reply

For algorithms specifically running on the fediverse, let me point you to the following in sparse order:
- https://searchtodon.social/
- https://github.com/hodgesmr/mastodon_digest
- https://www.tootfinder.ch/
- http://fediview.com/
- https://github.com/pkreissel/foryoufeed
I am grateful I could stand on the shoulders of these giants, learning from them about both the advantages and the limitations of running algorithms on the Fediverse.
https://cdn.fosstodon.org/media_attachments/files/113/967/540/980/075/257/original/67304a8f3e978322.jpeg

2025-02-08 09:24:26 UTC

- reply

But let us start with the TL algorithms we all know: they are usually not monolithic but made of smaller, more specific algorithms which contribute to decide whether and how posts appear on your timeline. If you are interested in this topic and want to delve deeper, let me point you to this excellent blog post (part of an excellent series!) by Luke Thorburn: https://medium.com/understanding-recommenders/how-platform-recommenders-work-15e260d9a15a
https://cdn.fosstodon.org/media_attachments/files/113/967/540/843/893/811/original/ee26455c3501be34.jpeg

2025-02-08 09:24:24 UTC

- reply

I like to say it all started with a sink, but for me it was about half a year before that. I was already fed up with what was happening around all social media and was looking for non-centralized alternatives. I joined the Fediverse and found some great communities of practice that resonated a lot with me. Each community had different characteristics, language, message frequency, number of participants. This is where I started to think about the idea of a “personal algorithm”.
https://cdn.fosstodon.org/media_attachments/files/113/967/540/726/290/297/original/af48f9a29c372054.jpeg

2025-02-08 09:24:22 UTC

- reply

What is BYOTA? Let’s start from what it is not: it’s not a new algorithm (you’ll see a few, but all coming from very classical ML), an end-user application, or a service.
To me BYOTA is a call for arms, starting with an imperative: BUILD. I like to see it as a playground where people can easily play with, create, and share their own algorithms. My main contribution here is just taking a few relatively recent and absolutely amazing tools and putting them together.
Where does this idea come from?
https://cdn.fosstodon.org/media_attachments/files/113/967/540/618/400/549/original/70f89ebbccd9e584.jpeg

2025-02-08 09:24:21 UTC

I got back home from the amazing #SocialWebFOSDEM and while I am drafting a “Build Your Own Timeline Algorithm” blog post to follow up on my talk I decided to keep the momentum going with a thread about it. You'll find my code at https://github.com/aittalam/byota and the talk slides here: https://fosdem.org/2025/schedule/event/fosdem-2025-5601-build-your-own-timeline-algorithm/ (a video with the recording of the talk should be available soon too). Now let’s dig into #BYOTA!
https://cdn.fosstodon.org/media_attachments/files/113/967/540/467/078/125/original/2702ece993a8f486.jpeg