Nostr notes by Peter Bloem

Mathematicians declare they have solved the crisis. The Strait of ...

2026-04-19T08:59:35Z

Mathematicians declare they have solved the crisis. The Strait of Hormuz is, in fact, clopen.

The fact that AI shows clear signs of being in a bubble does not ...

2026-04-18T07:45:17Z

In reply to nevent1q…3pp7
_________________________

The fact that AI shows clear signs of being in a bubble does not mean that the technology is necessarily on the same level as NFTs and Tulips.

US Railroads, the web, telecoms. All bubbles that burst and did damage, but not because the technology was fundamentally flawed.

This seems like a failure of memory to me. Cellphones were pushed ...

2026-04-18T07:42:47Z

This seems like a failure of memory to me.

Cellphones were pushed like nobody's business. The polarization around AI reminds me a lot of the attitudes to (pre-smartphone) cell phones. Especially the tedious repetitive arguments.

The telecoms industry was also extremely overvalued and overleveraged in the late 90s.

https://en.wikipedia.org/wiki/Telecoms_crash

RE: https://sigmoid.social/@pbloem/116250532724597189 4 days left ...

2026-04-13T09:44:23Z

RE: https://sigmoid.social/@pbloem/116250532724597189

4 days left to apply.

quoting
note1ly6…lulv

Do you feel like AI could learn to reason a bit better? Do you have ideas about how to make this happen? Do you feel like AI does reason, but we have no idea how and we should study it?

Well, we at the Vrije Universiteit Amsterdam are hiring an assistant professor to study just these kinds of questions. The profile is "learning and reasoning" in the broadest sense. If you can explain how your research touches these two topics, we want to hear about it.

https://werkenbij.vu.nl/vacatures/assistant-professor-learning-and-reasoning-career-track-amsterdam-1278904

Sorry if saying "do you?" came across as aggressive or ...

2026-04-09T11:29:20Z

In reply to nevent1q…7m7j
_________________________

Sorry if saying "do you?" came across as aggressive or combative. It wasn't meant that way. Perhaps I should have said "do we?"

I simply wanted to make the comparison between LLMs and humans. In most of these discussions people paint the LLMs as missing something fundamentally necessary for producing facts. This is usually by contrast with humans.

My point is that there are no _fundamental_ differences.

The innovations here are better finetuning and reasoning traces ...

2026-04-08T12:27:09Z

In reply to nevent1q…65v5
_________________________

The innovations here are better finetuning and reasoning traces (that is, having the model output drafts and "reflecting" on them). This is indeed what has reduced the hallucination so much.

That's a bit like how you summarize it (hidden prompts), but that doesn't quite capture how rich these reasoning traces can be. See, for example https://arxiv.org/abs/2601.10825 .

The "statistically probable" part refers to the ...

2026-04-08T12:22:06Z

In reply to nevent1q…9rw8
_________________________

The "statistically probable" part refers to the pre-trained model, not the fine-tuned one.

One example is asking "when's my birthday". The pre-trained model doesn't know, and so picks a random answer from all possible dates. That's hallucination.

The finetuned model takes that uncertainty, but expresses it as the answer "I don't know".

Sometimes the finetuning fails and we get hallucination in chatbots. This is because they are overtrained to be certain.

A rare misfire from Tufekci, I'd say. She's usually on ...

2026-04-08T09:28:51Z

A rare misfire from Tufekci, I'd say. She's usually on the money.

While the algorithm is always the same, AI is operating in substantially different modes when it's hallucinating to when it's retrieving factual information.

Hallucination is not a solved problem, but it's reducing substantially and because concrete innovations, not just "more data, more compute".

Do you feel like AI could learn to reason a bit better? Do you ...

2026-03-18T13:59:17Z

Do you feel like AI could learn to reason a bit better? Do you have ideas about how to make this happen? Do you feel like AI does reason, but we have no idea how and we should study it?

Well, we at the Vrije Universiteit Amsterdam are hiring an assistant professor to study just these kinds of questions. The profile is "learning and reasoning" in the broadest sense. If you can explain how your research touches these two topics, we want to hear about it.

https://werkenbij.vu.nl/vacatures/assistant-professor-learning-and-reasoning-career-track-amsterdam-1278904

David MacKay discusses this in Sustainable Energy without the Hot ...

2026-02-26T08:36:55Z

In reply to nevent1q…5tac
_________________________

David MacKay discusses this in Sustainable Energy without the Hot Air, and it's pretty disappointing (for fundamental rather than technical reasons). The UK can get at most 1.6% of its energy budget from geothermal.

If you divide the amount of heat the earth dissipates by the number of people, the resulting number is simply pretty low.

It works for places like Iceland, because a large area's worth of heat comes out at one point where the population is low.

https://www.withouthotair.com/c16/page_96.shtml

Also Mastodon/Fedi? I mean AI is pretty unwelcome, but the other ...

2026-02-22T11:59:16Z

In reply to nevent1q…evj5
_________________________

Also Mastodon/Fedi? I mean AI is pretty unwelcome, but the other disciplines seem to be having a good time here.

I won't do it. I won't become a LinkedIn guy.

2026-02-22T11:40:08Z

In reply to nevent1q…wkpd
_________________________

I won't do it. I won't become a LinkedIn guy.

It looks like the AI research community is finally moving away ...

2026-02-22T11:38:45Z

It looks like the AI research community is finally moving away from X and of all the possible platforms, they've picked LinkedIn. These fucking people...

Explaining the fediverse is simple. It's social media for ...

2026-02-14T11:30:24Z

Explaining the fediverse is simple. It's social media for people who hate social media.

There aren't that many people on it? Feature.
It's not that addictive or even engaging? Feature.
You can't make money on it and it won't help your career? Feature.

The thing about vibe coding is, I was never really on board with ...

2026-01-27T08:56:53Z

The thing about vibe coding is, I was never really on board with my IDE closing my brackets for me.

RE: https://mastodon.social/@nixCraft/115835690251372850 This is, ...

2026-01-04T13:19:41Z

RE: https://mastodon.social/@nixCraft/115835690251372850

This is, of course, a problem for AI as well. StackExchange is a hugely important part of the pre-training data. Maybe even more important than Wikipedia.

For anything that happened after ~2024, that pre-training data is now no longer available.

Models have got pretty good at working novel problems in-context, but not nearly as good as they are for in-distribution problems (stuff that SE had questions on). We'll have to see if AI keeps working as well on really new topics.
nostr:note1jy070xg48gxe78eqh2gvwheretvuvmgl7xcplkzhmvxqd485fetqe7xyua

In a way all that pre-training power is just initialization. Our ...

2025-11-28T15:10:03Z

In reply to nevent1q…xl3q
_________________________

In a way all that pre-training power is just initialization. Our initial brain state is also the result of a huge power investment.

What matters is how well they learn in context.

Obviously, an M4 running Llama 13B is definitely nowhere near human intelligence. Still, it doesn't seem at all obvious to me that we would need completely new hardware to get to human intelligence @ 20 Watts.

I made the familiar point yesterday during a panel discussion ...

2025-11-28T15:06:56Z

I made the familiar point yesterday during a panel discussion that even if AI were approaching human intelligence, we're nowhere near human power efficiency (in "intelligence per watt")

But is that actually true anymore? We need huge power to train models, but once we have them trained, distilled and quantized, what do we have?

The human brain draws abt 20 Watts. An M4 doing local inference on a heavily quantized LLM won't draw more than 40 Watts.

Solving ARC (well... up to 20%) with inference-time compression. ...

2025-06-29T13:23:33Z

Solving ARC (well... up to 20%) with inference-time compression. Another indication that #MDL and #AlgorithmicComplexity are becoming relevant in the age of Deep Learning.

https://iliao2345.github.io/blog_posts/arc_agi_without_pretraining/arc_agi_without_pretraining.html

I asked ChatGPT to transport me to the airport and it ...

2025-06-11T12:54:11Z

I asked ChatGPT to transport me to the airport and it couldn't do it. My 30 year old car gets me there just fine and uses much less energy. This technology is useless.

If you want to help people in #academia who are maybe less ...

2025-06-09T21:41:44Z

If you want to help people in #academia who are maybe less fortunate than you, who have less famous supervisors, or work at less prestigious universities, here's one simple thing you can do:

Do proper literature research.

That means complete forward and backward snowballing from a decent seed set. Find everything that is relevant to your paper and cite it. Budget a couple of full days for it.

An 8TB corpus of copyright-free text for training AI models. ...

2025-06-06T09:51:57Z

An 8TB corpus of copyright-free text for training AI models. Comes with two 7B models trained as proof-of-concept.

https://github.com/r-three/common-pile/blob/main/paper.pdf

I'm glad somebody has finally done this. The "we need to break copyright or AI won't work argument" feels super-dodgy to me and we can just evaluate whether it's true. 1/n

New lecture on transformers for my #MachineLearning BSc course. ...

2025-03-09T18:38:59Z

New lecture on transformers for my #MachineLearning BSc course.

We go all the way from simple self-attention to a full-fledged ChatGPT. Includes position embedding, tokenization, instruction tuning and much more.

https://mlvu.github.io/transformers/

Is the cute cartoon a dark pattern? I like cute things (and this ...

2025-02-28T14:39:36Z

In reply to nevent1q…hh3r
_________________________

Is the cute cartoon a dark pattern? I like cute things (and this fox is well-drawn to be fair).

But I think it's a deliberate attempt to infantilize the user and to put a harmless face on something insidious.

Aral Balkan (npub1p09…m5vl) noted this more than ten years ago. Make big brother a cartoon dinosaur instead of a mustachioed man and suddenly we will swallow anything.

https://ar.al/notes/spyware-2.0/

Moving away from Firefox shows how much the rot has already set ...

2025-02-28T14:35:44Z

Moving away from Firefox shows how much the rot has already set in. After dismissing the "set as default browser" dialog, and explicitly clicking "don't ask again", next time it shows me a different pop-up, that is much easier to accidentally click.

All the dark patterns are there: high contrast easily-clickable default option. "No now" instead of "No" as low contrast alternative answer. Cute little cartoon.

I cannot express my frustration with Intel and AMD at this point. ...

2024-12-23T10:22:11Z

I cannot express my frustration with Intel and AMD at this point. It's 10 years and counting since deep learning should have been on everybody's radar and they still cannot seem to grasp that you need _software support_ to make people use your chips.

This NVidia monopoly on #ai hardware is not good for anybody. We need a little competition in this space.

https://semianalysis.com/2024/12/22/mi300x-vs-h100-vs-h200-benchmark-part-1-training/

If you get this sort of shit, I made a handy website a while ago. ...

2024-12-16T18:29:22Z

In reply to nevent1q…f58v
_________________________

If you get this sort of shit, I made a handy website a while ago.

https://dontmail.it

my insurance company: Don't fall for phishing mails, always ...

2024-12-16T18:28:41Z

my insurance company: Don't fall for phishing mails, always check that the sender is us. That's how you can tell who emails are from!

also my insurance company: Time to pay the bill, here's a handy link to some third party company you've never heard of.

Why is this not quoted more among computer scientists and ...

2024-12-16T16:18:20Z

In reply to nevent1q…y7yq
_________________________

Why is this not quoted more among computer scientists and programmers? Why is it not played on a loop throughout the year, from speakers around campus?

Even that didn't really split the fediverse, I think. I'm ...

2024-12-15T15:42:41Z

In reply to nevent1q…7rl2
_________________________

Even that didn't really split the fediverse, I think. I'm pretty sure I'm still talking to people on either side of the divide.

I think that's the key. Because there are so many instances, there will always be little bridges between any two sides of a split.

We never really talk to all of the fediverse, but any message has a chance to reach a good chunk of it.

I think it probably wouldn't. Unless people actively ...

2024-12-15T14:51:33Z

In reply to nevent1q…g0xy
_________________________

I think it probably wouldn't. Unless people actively defederated from instances that would defederate over hosting a wedge user. But if the aim is to defend freedom of speech, that wouldn't make much sense.

The result is probably that the wedge user gets their platform on an instance somewhere, but only reaches a limited slice of the fediverse.

It's all a bit messy, but I do think the fediverse is more robust to this sort of thing than a centralized platform with a single policy.

Bluesky has been shaken up by the joining of somebody who half ...

2024-12-15T14:47:43Z

Bluesky has been shaken up by the joining of somebody who half the people strongly feel should be banned and half the people strongly rally behind. A kind of wedge-user.

Without discussing the specifics, I wonder if the #fediverse is more resilient to this. If a wedge-user joined a large instance, and they were banned (which would prob happen in this specific case), they would then move to an instance that would take them.

Would this cause a split into effectively two fediverses?

This quote is showing itself to be more and more prescient.

2024-12-09T18:46:50Z

In reply to nevent1q…ehmr
_________________________

This quote is showing itself to be more and more prescient.

If you are an #artist worried about #ai being used to copy your ...

2024-12-08T14:14:12Z

If you are an #artist worried about #ai being used to copy your style, it is a miscalculation to worry only about training data.

Many artists are explicitly forbidding use of their work in training generative AI models.

However, a large amount of research is looking into in-context use of examples. For example, you give ChatGPT three examples of poems and then ask it to generate a new poem in the same style.

(1/6)

Rest of thread: https://sigmoid.social/@pbloem/113617628089668394

#artistsofmastodon

I thought it was a kind of reluctance to collective action and ...

2024-12-01T15:34:46Z

In reply to nevent1q…5hxl
_________________________

I thought it was a kind of reluctance to collective action and general inertia, and taking the road of least resistance.

I'm now beginning to think most of them are fully on board with the whole enterprise.

It's true, that it's surprising. However, as an AI ...

2024-12-01T15:33:30Z

In reply to nevent1q…2fhw
_________________________

It's true, that it's surprising. However, as an AI researcher, I'm slowly beginning to learn how naive I've been about my community

I thought they were all basically scientists raised on Star Trek. So how bad could it be? Then they all went to work for Google, OpenAI and FB. Then OpenAI went private and they all kept working there. Then Musk took over X and they all stayed there. Then X put Trump on the throne and they're all still there, some of them applauding.

I don't know about FB, but during the Twitter takeover, it ...

2024-12-01T12:28:12Z

In reply to nevent1q…2xu4
_________________________

I don't know about FB, but during the Twitter takeover, it was pretty clear that Musk gutted the company of anybody who wasn't absolutely dependent on job security for their VISA.

Everybody who didn't leave in disgust was going to do whatever he said with no questions asked.

https://onesquareminesweeper.com/

2024-10-23T07:24:00Z

https://onesquareminesweeper.com/

I wrote a new article. ⬬ Understanding Gaussians ⬬ ...

2024-10-21T19:00:44Z

I wrote a new article.

⬬ Understanding Gaussians ⬬
https://gestalt.ink/gaussians

The idea is that Gaussians can seem pretty daunting, especially if you look at the density function. However, they're actually simple to understand if you forget about all that and focus on _linear transformations of the standard Gaussian_

Then, the density function gets really simple, and the rest follows from basic rules of linear algebra.

#tutorial #machinelearning #statistics #explainer #math

Assuming you're an average meat eater, you take two ...

2024-10-14T08:04:46Z

Assuming you're an average meat eater, you take two intercontinental flights per year, you drive a car daily, and use ChatGPT/Copilot regularly throughout the day.

Which of these activities should you cut to have the biggest positive climate impact?

But would either of these lower the ecological footprint? I think ...

2024-09-22T16:48:20Z

In reply to nevent1q…na4t
_________________________

But would either of these lower the ecological footprint? I think the models would just grow. How much energy and money is spent is currently determined by the companies' future expected reward, not by environmental concerns or hardware limitations.

And the expected reward is probably inflated by the arms race and them drinking their own Kool aid.

I agree that #AI has an environmental problem, but Mastodon's ...

2024-09-20T13:10:49Z

I agree that #AI has an environmental problem, but Mastodon's default position of "anything to do with AI is inherently evil" does not feel particularly productive.

I find a lot of AI plenty useful, even LLMs. If I'm going to take the environmental costs seriously as a user, I need to talk about it much more precisely, and carefully than what's happening now.

Saying "AI is an environmental crime of gigantic proportions." is so over-the-top, there's no meaningful conversation left to have.

I tried Obsidian, but it made me realize that all I need is a ...

2024-08-17T10:19:27Z

In reply to nevent1q…w5sl
_________________________

I tried Obsidian, but it made me realize that all I need is a dropbox directory with markdown files.

If I'm on my laptop I can edit them in an IDE or text editor, and on my phone I use markor & dropsync.

I get no lock-in, all the formatting offered by markdown and search through spotlight. If I ever need more organization, I can fire up Obsidian again and it'll load it all.