Bartosz Milewski

Physicist, programmer (Haskell, C++), mathematician, category theorist. Author of Category Theory for Programmers and The Dao of Functional Programming

Public Key

npub15c9gsd6d3cw0pyk8a2fkv248snan8vl8t0nhy5qhqvhxj20tch2swwczp5

NIP-05 Address

BartoszMilewski@mathstodon-xyz.mostr.pub

Profile Code

nprofile1qqs2vz5gxaxcu88sjtr74ymx92ncf7enk0n4hemj2qtsxtnf984ut4gpz3mhxue69uhhyetvv9ujuerpd46hxtnfduqs6amnwvaz7tmwdaejumr0dss2u4xn

Publishing to

relay.damus.io

nos.lol

Author Public Key

npub15c9gsd6d3cw0pyk8a2fkv248snan8vl8t0nhy5qhqvhxj20tch2swwczp5

Show more details

Last Notes

2026-05-11 20:38:41 UTC

I just realized that Chekhov's Gun is an example of linear logic. It's a premise that has to be consumed at some point in the plot.

2026-02-19 13:55:08 UTC

The paradox of the stone: "Could God create a stone so heavy that even he could not lift it?" was solved by Grothendieck by introducing an infinite tower of gods. The God of the n'th universe can only create stones that he can lift. But the God from the n+1 universe can create an unliftable stone for the God from the n'th universe, and so on.

2026-02-17 19:29:05 UTC

Democracy is like health: You don't appreciate it until you lose it.

2026-01-28 13:48:40 UTC

The piñata effect: At the final moment of a black hole evaporation through Hawking radiation, all objects that were swallowed by it suddenly come back, thus restoring the conservation of information required by quantum mechanics.

2026-01-27 22:44:52 UTC

- reply

@nprofile…5pn7
I took my inspiration from this paper, where they define isolated points in HoTT.
https://arxiv.org/abs/2512.17484

2026-01-27 18:57:37 UTC

- reply

@nprofile…5pn7
This is why I said "isolated" objects. It could be just objects with no connection with the rest of the category, or objects whose removal leaves a subcategory of the original.

2026-01-24 13:30:09 UTC

- reply

We can take a derivative of a container with respect to an isolated object \(p_0 \colon P s\). The shapes in the derivative are pairs of objects \( (s \colon S, p_0 \colon P s ) \) and the positions are functors to \(P s \textbackslash p_0 \) (the category \(P s\) with the (isolated) object \(p_0\) removed).
One can even define container optics:
\[ O\langle a, b \rangle \langle s, t \rangle = \int^{S \triangleleft P} (s \to T_{S \triangleleft P} a) \times (T_{S \triangleleft P} b \to t) \]
3/3

2026-01-24 13:29:33 UTC

- reply

Now replace sets with categories. A categorical container is a category of shapes \(S\) and a (pseudo-) functor \(P \colon S \to \mathbf{Cat}\). It's the same functor that appears in the Grothendieck construction.
An arrow from \(S \triangleleft P \) to \(T \triangleleft Q \) is a functor \(S \to T\) and a natural transformation \(Q \to P\).
We can fill the container with data from a category \(X\) using a coend:
\[ T_{S \triangleleft P} X = \int^{s \colon S} (P s \to X) \]
2/3

2026-01-24 13:28:41 UTC

I have found an interesting link between containers and the Grothendieck construction. Here's a sketch.
A container \(S \triangleleft P \) in a category \(\mathbb C\) consists of an object of shapes \(S\) and an arrow \(P \colon S \to \mathbb C\) of positions. For instance, in \(\mathbf{Set}\), the shape of lists is \(\mathbb N\) (lengths), and for each shape \(n \in \mathbb N \), positions are ordinals less than \(n\). To fill a shape with data, we construct the extension functor. E.g., in \(\mathbf{Set}\):
\[ T_{S \triangleleft P} X = \Sigma_{s \in S} (P s \to X) \]
(think of a list of \(X\)).
1/3

2026-01-10 08:42:01 UTC

Geoffrey Hinton came up with the same conclusion.
https://www.reddit.com/r/mathematics/comments/1q4jrph/geoffrey_hinton_says_mathematics_is_a_closed/

2026-01-07 14:43:46 UTC

I see parallels between the current situation in Iran and the Solidarity movement in Poland in 1980. Back then I went on a business trip to the Soviet Union. A Russian friend told me that we were lucky their troops were busy in Afghanistan. They are now busy in Ukraine, so I have high hopes for Iran.

2025-12-31 12:08:08 UTC

- reply

@nprofile…p3nn
I'm not saying that there won't be mathematical olympiads for humans. But just like chess is a hobby and an entertainment, so will math.

2025-12-30 22:47:53 UTC

- reply

@nprofile…hzey
I agree. In the future math will be done for fun by hobbyists.

2025-12-30 19:27:06 UTC

- reply

@nprofile…p3nn
I don't know. Memorizing thousands of chess openings is not my idea of fun.

2025-12-30 19:24:50 UTC

- reply

@nprofile…hzey
Lee Sedol, a go master said: "losing to AI, in a sense, meant my entire world was collapsing. ... I could no longer enjoy the game. So I retired."

2025-12-30 16:43:51 UTC

Math is next on the chopping block. The AI was able to master chess and go because it was given the rules of the game and an oracle to evaluate the result of each game. We now have several formalizations of math and theorem checkers to evaluate the results. So it's just a matter of time.

2025-12-26 22:45:39 UTC

How sure are we that there were never any intelligent dinosaurs? How much of fossil evidence would survive 66 million years? Would pyramids survive this long?

2025-12-25 15:55:33 UTC

- reply

Turns out, last week there was a massive rename of the two fields in Iso involving 281 files. Hence my problem.

2025-12-25 14:49:05 UTC

- reply

Apparently I'm supposed to use v.09 of the library, but I'm unable to clone it

2025-12-25 14:10:25 UTC

The latest cubical Agda repository has this definition:
record Iso {ℓ ℓ'} (A : Type ℓ) (B : Type ℓ')
...
sec : section fun inv
ret : retract fun inv
but the paper I'm reading uses
leftInv
rightInv
instead. Chat GPT insists that the "modern" library uses the latter, but I can't seem to find it. I'm using
https://github.com/agda/cubical

2025-12-24 13:43:36 UTC

I hate this notation:
\[ List (X) = \mu Y.1 + X \times Y \]
Lots of space around + and none between the dot and the 1. This is much more readable:
\[ List (X) = \mu Y \to 1 + X \times Y \]

2025-12-21 22:20:48 UTC

You can catch more dark matter with honey than with vinegar.
https://arxiv.org/pdf/2510.00068

2025-12-20 17:19:58 UTC

- reply

@nprofile…p3nn
Is Agda any better? I started using it because Lean doesn't do HoTT. Plus there is Cubical Agda.

2025-12-19 21:28:43 UTC

I like the attention to detail in Pluribus. I think it's the first time that an intravenous injection was shown correctly on camera (usually they just stab the patient with the needle perpendicular to the surface). We also learned that Zosia was from Gdańsk --and she pronounced it correctly (Karolina Wydra is a Polish-American actress).

2025-12-18 11:50:25 UTC

I'm wondering if there is a maximal category, such that every small category is a subcategory of.

2025-12-14 09:30:35 UTC

A physicist's brain is trained on the Standard Model data. No wonder we are constantly looking for more gauge groups and Higgs fields.

2025-12-11 20:48:24 UTC

The US is experiencing a Tourette's syndrome with Donald Trump being its mouthpiece.

2025-12-09 08:33:59 UTC

- reply

@nprofile…stct
For me Blindsight has a creepy atmosphere. But in general, in the era of jump-scare movies, I don't think books can be as scary. We are too inoculated by movies and, unlike previous generations, our lives are too comfortable to develop real existential fears.

2025-12-06 12:06:39 UTC

- reply

@nprofile…8eua What's fascinating about burds is that they have to be very thrifty with their brains, or they wouldn't be able to lift them into the air. And their brains have to deal with 3-d navigation and superior vision. Compared to birds, human brains are extremely wasteful.

2025-12-01 20:22:31 UTC

Instead of terraforming Mars, humanity opted for terradeforming Earth

2025-11-29 08:39:01 UTC

- reply

@nprofile…8eua @nprofile…drul @nprofile…stct I'm imagining a taffy machine running for billions of years, with an emerging Taffian civilization discovering Taffian physics, when suddenly the movement is reversed.

2025-11-27 22:19:24 UTC

- reply

@nprofile…8eua @nprofile…stct I wonder if it could be undone.
https://www.youtube.com/watch?v=UpJ-kGII074

2025-11-25 22:10:22 UTC

- reply

@nprofile…stct Intimidation works!

2025-11-25 14:53:26 UTC

Suppose you were outfitting a school in the middle of Africa. What would be the best way to provide it with internet access? Is there an alternative to StarLink?

2025-11-24 20:16:54 UTC

Type theory and category theory are the product of highly refined abstractions. Since every level of abstraction strips a layer of intuition, the final product is sometimes called abstract nonsense. The only way to navigate such vast halls of abstraction is to keep in mind where those abstractions come from, or how they can be modeled in more familiar surroundings, be it computer programs, logic, or set theory. When studying an elephant, it makes sense to occasionally focus on its trunk, legs, or ears,
Learning category theory is a little like reading a novel that has many hidden layers of meaning.

2025-11-24 20:06:27 UTC

I think we should immediately start ostacizing people on social media who publish AI slop without proper attribution. It's not funny!

2025-11-10 13:57:47 UTC

- reply

@nprofile…0775 @nprofile…p3nn
Your prop B.4 works for presheaves, if you replace \(M(m\otimes n, p)\) with \(M(p, m \otimes n \) in Day convolution

2025-11-10 12:45:10 UTC

- reply

@nprofile…0775 : Wouldn't a co-para construction correspond to a locally graded category over co-presheaves? I haven't done the calculation, but it seems intuitive.
@nprofile…p3nn

2025-11-10 11:29:46 UTC

- reply

@nprofile…p3nn Except in your blog?

2025-11-10 10:23:34 UTC

@nprofile…p3nn
Who came up with this idea that the para construction can be described as a locally graded category?

2025-11-02 16:01:25 UTC

- reply

Without looking at the sky, the stars, the Moon, or the Sun, you just count the steps

2025-11-02 15:49:46 UTC

Clocks are the instruments of dead reckoning in time

2025-10-30 13:09:52 UTC

- reply

@nprofile…8eua @nprofile…y0ah
I think its the Kolmogorov-Arnold theorem
https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold_representation_theorem

2025-10-23 15:47:21 UTC

- reply

In its purest unadulterated form, type theory is a set of rules with no axioms. One can then add such axioms as axiom K (uniqueness of identity proofs) or its opposite, the axiom of univalence.

2025-10-23 14:08:31 UTC

Is it correct to say that the Martin Lof type theory, unlike set theory, is not axiomatic?

2025-10-06 08:35:18 UTC

- reply

@nprofile…fa4k @nprofile…3k8q
On the bright side, they don't burn scientists at the stake any more.

2025-10-05 19:16:53 UTC

- reply

@nprofile…fa4k @nprofile…upuh
Great video! I think this is not just a physics problem, but a more general societal problem. We are living in times of unprecedented inequality. This is obvious when you look at the ownership of resources, with all these billionaires monopolizing financial wealth. But there is a similar divergence in the distribution of knowledge. Only the top one percent have the means, the dedication, and the ability to get basic understanding of the current knowledge in any given area of science. The physics-bashing or anti-vax propaganda is just a reflection of the rise of populism in politics. Uneducated people are easier to manipulate.

2025-09-08 08:51:38 UTC

Normalizing probabilities between zero and one breaks the rule of making impossible states unrepresentable. Negative or superunerary probabilites are impossible.
This normalization is the result of the frequentist interpretation. Bayesians should use numbers between negative and positive infinity. Zero means no prior knowledge. Negative infinity means total disinformation.

2025-08-22 16:17:59 UTC

- reply

@nprofile…4aqs Do you mean this paper?
https://arxiv.org/abs/1709.02562

2025-08-22 14:11:31 UTC

Evolutionary science invalidated most of philosophy. All those Platonic ideals and a priori knowledge can be simply explained by natural selection. No need for metaphysics. It's a hill I'm willing to die on.

2025-08-22 09:26:09 UTC

I was wondering why the Goedel's incompleteness theorem didn't apply to Euclidean geometry. After all we can encode natural numbers on a blackboard by drawing digits, which are geometric figures. Turns out the problem is that we can't pick the encoding using first order logic.
https://math.stackexchange.com/questions/90393/why-euclidean-geometry-cannot-be-proved-incomplete-by-g%C3%B6dels-incompleteness-the

2025-07-02 15:48:42 UTC

- reply

@nprofile…4aqs @nprofile…fa4k
More precisely, take an infinite number of drawers, each containing a pair of socks. The AoC says that it's possible to pick one sock from each drawer.
So in AoC you have to assume that the drawers are not empty because you can't pick a sock from an empty drawer.

2025-07-02 12:10:13 UTC

Axiom of Choice in HoTT. For any family of sets Y parameterized by elements of a set X, there exists a function below.
Naively, this says that if all sets Y(x) are inhabited then all sets Y(x) are inhabited. Duh!
The tricky part is the vertical bars. They signify 'propositional truncation', meaning: we know it's true, but we forgot why. So we know that every Y(x) is not empty, but we don't have the witness of this fact.
The axiom of choice says that the witnesses exist: if you know that the sets are non-empty, it's possible to choose one element from each set.
The propositional truncation on the right means that, trust me!, the witnesses exist, but you can't get at them.
https://media.mathstodon.xyz/media_attachments/files/114/783/493/006/659/908/original/cc4314a7f9b195e2.png

2025-06-29 10:13:28 UTC

The main idea of HoTT/UF is to narrow the gap between equality and isomorphism. The problem is that (set-thoretic) equality is a yes/no proposition, whereas there could be many different isomorphisms betwee two objects. So if we want to merge the two notions, we have to extend equality to allow for multiple witnesses. Thus in MLTT, (propositional) equality is a type whose elements are witnesses.
Set-theoretic isomorphism is itself defined using equality (two compositions must be equal to identities). So if we modify equality, we should replace isomorphism with a more general type of equivalence.
The final step is to identify the new equality with the new equivalence in the axiom of univalence, which states that equality is equivalent to equivalence.

2025-06-19 12:52:34 UTC

Another blog post on homotopy almost ready.
https://media.mathstodon.xyz/media_attachments/files/114/710/121/616/384/789/original/82dfc2978a757955.png

2025-06-17 20:52:23 UTC

It seems like most explosives use nitrogen to separate oxygen from carbon. I wonder what's so special about nitrogen. Could it be substituted by phosphorus (same group)?

2025-06-11 16:08:25 UTC

Amazon AI in action: "Algebraic Topology" is related to the bestseller "Cleaning Lady". Is it the graphics on the cover?
https://media.mathstodon.xyz/media_attachments/files/114/665/589/221/518/068/original/38a20545cfe16628.png

2025-06-11 11:26:33 UTC

- reply

@nprofile…den0
HoTT is definitely using the language of homotopy, and a lot of intuitions about equality types come from topology. Contractible types, equivalence (defined in terms of contractibility), infinity groupoids, they all have roots in topology.

2025-06-11 07:48:04 UTC

I was doing it all wrong. I should have started by studying topology and homotopy. Only then can you fully appreciate category theory and HoTT.

2025-05-21 14:36:25 UTC

How common is it to use the notation \(2^S\) for the power set of \(S\)?

2025-05-12 20:53:14 UTC

- reply

@nprofile…fa4k @nprofile…40e0
"More electrons will land on one side than the other, by chance."
I think they land on the side that faces the negative pole of the current source. It's not random. They get stuck there for a while. They are repelled by the negative pole, so the drop will start turning.
"When they reach the bottom of the droplet, some will fall off and continue moving through the oil."
Actually, the droplet is also a conductor, albeit slower than the oil. So the current inside the droplet will slowly neutralize these surface charges--as the droplet turns.
The key is that there is a difference in conductivity and permeability between the two mediums.

2025-05-11 11:20:05 UTC

- reply

@nprofile…fa4k
It seems like the charges move slower in the droplet than they do in the liquid. This leads to charges accumulating on the surface, creating a dipole moment. But this dipole is oriented in the "wrong" direction, leading to instability. The dipole tips over in a random direction. The viscosity means that the droplet starts slowly rotating in that direction.That triggers the dipolization process again, trying to recreate the "wrong" dipole by moving charges. The bulk of the droplet rotates in one direction, while the charges are moving in the opposite direction.

2025-05-04 09:34:17 UTC

- reply

@nprofile…fa4k
Here are his thought on Intelligent Design:
"I was an atheist during most of my younger years, but a long study of mathematics has slowly led me towards a mild form of ID ; one that does not insist on a biblical rewriting of science, but which wants to acknowledge the remarkable coincidences, startling beauty and overpowering rigidity, structure and rightness in the mathematics underlying our world. This is an aspect of existence that is sadly unknown to a good part of the population, biological scientists perhaps included, so it deserves some press."

2025-05-04 09:03:33 UTC

- reply

@nprofile…fa4k I used to watch his YouTube videos, but I got gradually disillusioned by his ideology. He's actually motivating his philosophy of mathematics by his religious beliefs. So when he says a number "exists" he means it was created by God.

2025-04-23 10:12:27 UTC

Stupid math question: In one of my blogs I claimed that if there are two parallel morphism f and g, whose composites on both left and right are always equal, that there is no way to distinguish between them, in THAT category. But you can always use a functor to a larger category, in which their images can compose differently, giving rise to different buidups of composites. (Below is an example of a 3-object category and its functorial mapping.)
Does it make sense? Is there a formal way of saying that two different morphisms are indistinguishable in a particular category? Are morphisms distinguishable if their sieves are not isomorphic?
https://media.mathstodon.xyz/media_attachments/files/114/386/720/076/270/361/original/3c7f4b6d62a91fee.png

2025-04-18 10:21:20 UTC

- reply

@nprofile…zj80
To say that they will benefit would require that you have some measure of performance. A larger orchestra will produce more decibels, but will it produce better music?

2025-04-17 14:03:42 UTC

Scaling arguments are tricky. Naively, the bigger the orchestra, the better the music should be. Yet the size of the symphonic orchestra stalled at about a 100, and it didn't eliminate string quartets or solo performances. So what's all this hype about artificial intelligence?

2025-04-09 13:39:25 UTC

- reply

@nprofile…yu7x
She hands me a cigarette lighter. "Here, take this. Its flintstone contains cerium and lanthanum, rare earth elements."

2025-04-08 09:43:35 UTC

- reply

My conjecture is that, beyond a certain level of complexity, neural networks start hallucinating. So there is a natural barrier to increasing intelligence.
This worked okay for our hunter-gatherer ancestors--it's better to occasionally run away from an imaginary demon than to miss subtle clues that a saber-toothed tiger is stalking you. The result was widespread magical thinking and animism which, with the brain shrinkage following the agricultural revolution, turned into religion.

2025-04-07 17:39:29 UTC

I used some "back of the envelope" calculations (really, asking Chat GPT) to come up with the analog of the Moore's law for hominins. Our brains double in complexity roughly 2-4 million years. It follows that we are on average 0.5% smarter than our ice-age cave-dwelling ancestors.

2025-04-06 18:06:27 UTC

I'm so fed up with Bluetooth headphones!
They take a few seconds to connect. When I'm streaming YouTube and I disconnect, it sometimes stops the playback, other times it doesn't. I'm using B&O headphones. Is there a better alternative?

2025-04-05 19:05:41 UTC

- reply

@nprofile…59k6
It could be worse than that.
https://en.wikipedia.org/wiki/Jonestown

2025-03-31 20:21:51 UTC

- reply

@nprofile…zj80 Essentially, compile-time C++ is a purely functional language that uses the worst possible syntax of template metaprogramming. Constexpr/consteval tries to use the more familiar C++ constructs to implement it. It's putting lipstick on a pig.

2025-03-28 13:33:07 UTC

- reply

However, if you also scale peculiar velocities of galaxies, the temperature of this hydrogen gas would be almost absolute zero. Chat GPT calculates it to be \(4.6 \times 10^{-56} K \).

2025-03-28 10:43:28 UTC

I asked Chat GPT to do this calculation, so I can't guarantee the correctness:
If a galaxy were the size of the hydrogen atom, the size of the Universe would be 0.1 mm and the pressure of this gas under normal conditions would be close to the atmospheric pressure.

2025-03-26 11:44:21 UTC

- reply

@nprofile…7u88
Yes, Twin Peaks immediately crossed my mind when I was posting this. But Twin Peaks resists pigeonholing into a pre-existing genre.

2025-03-25 22:19:59 UTC

Imagine the creators of Severance or Lost producing a crime fiction. There'd be lots of murders and conflicting clues. At the end, instead of getting the reveal, you'd be left with no idea who did what and why.

2025-03-23 12:22:59 UTC

Bayes' theorem can save you a lot of aggravation. Avoid getting into arguments with people whose prior beliefs are 100% (absolute certainty), because no amount of evidence can change their beliefs. https://www.youtube.com/watch?v=R13BD8qKeTg

2025-03-21 08:36:51 UTC

- reply

@nprofile…fa4k
I have 34k followers and I'm almost exclusively posting about politics. I have a feeling that Musk (if he paid any attention) would rather have me quit. It makes no difference that I'm doing it from France. Living in the US was just too depressing.

2025-03-20 18:57:07 UTC

- reply

@nprofile…fa4k
The reason I think engagement is so important is that people will trust personal exchanges more than they trust mass media. A tweet from John Baez is more trustworthy than an article in NYT because you don't have an economic incentive (maybe even the opposite). You do it because you are genuinely worried about the state of the country. And your voice also encourages other people to speak their mind.
You can always block the idiots who are disrespectful.

2025-03-20 18:28:42 UTC

- reply

@nprofile…fa4k
The same goes for emigration--it makes sense only in extreme conditions, when you're actually facing persecution (that was my reason for leaving Poland). Of course, moving to Europe for other reasons is a personal choice (which I'm glad I made). Not to mention that Europe needs all the help it can get, now that it can't count on the US.

2025-03-20 18:17:57 UTC

- reply

@nprofile…fa4k
I'm worried about people reacting to Trump excesses by withdrawing to safe environments--ceding territory to the enemy. It makes sense to boycott Tesla, but it's counterproductive to withdraw from Twitter, which is a huge public forum. As long as Musk is not blocking critical voices, we can engage and be heard by lots of people. Our voices have to be louder, not quieter. Every negative action by Trump, Vance, or Musk must be met with strong reaction.

2025-03-20 14:40:20 UTC

New post.
https://bartoszmilewski.com/2025/03/20/subobject-classifier/

2025-03-20 10:44:31 UTC

- reply

@nprofile…59k6 @nprofile…fa4k @nprofile…hszg
Speaking as a survivor or a totalitarian system:
The things that work are protests and solidarity.
They will try to go after individuals, so solidarity is very important. An attack on one is an attack on all. Don't be a bystander.
The more public the protests are, the more effective they are. Protesting inside gated communities and ivory towers doesn't work.
This is why I see no sense in boycotting Twitter.
You might feel weak as an individual against a mob, but as a community we are very strong. A society would collapse without scientists.

2025-03-18 22:15:26 UTC

So, as expected, Putin totally pwned Trump. We can expect a major spin cycle to follow "the best phone conversation ever" (since Montjoy called King Henry V)

2025-03-17 21:42:16 UTC

I find it a little disturbing when chefs in cooking videos demostrate which part of the animal a certain cut of meat comes from by pointing at their own body parts

2025-03-17 10:10:58 UTC

- reply

@nprofile…59k6 I think physics has accumulated much more technical debt than mathematics. It made a lot of sense as long as we could fall back on experimental verification. But the ice is getting thinner.

2025-03-16 21:14:23 UTC

There is a common misconception that physicists create mathematical models of reality. In actuality they use a system of heuristics that imitates mathematics. It can be used to calculate things while skilfully ignoring internal inconsistencies. Physicists hallucinate mathematics.

2025-03-09 12:23:08 UTC

- reply

@nprofile…v2av I think this is purely an optimization issue. The matrix \(K^T\) is just too huge. This is where heads of attention came into play. The vectors \(\ve Q\) and \(\vec K_i\) are chopped into smaller chunks and they are processed in parallel in reduced dimensional spaces. The query, key, and value matrices rearrange the input, so that each chunk depends on each input vector. The question is, what would this algorithm look like if we didn't have resource constraints. And would the training still converge?

2025-03-08 23:58:03 UTC

I think I'm going to stop using Chrome. They just disabled uBlock Origin, so I'm being spammed by ads.

2025-03-08 16:46:14 UTC

- reply

@nprofile…v2av Good point. If the matrix W_V were invertible, than we could absorb it into the definition of E's and the two other matrices. My guess is that all these matrices project onto smaller-dimensional subspaces, thus cannot be inverted. But I don't know how much of it is just optimization heuristics.

2025-03-08 11:28:10 UTC

Trump apologists remind me of a story by Stanisław Lem from his collection A Perfect Vacuum. "A family of three consisted of a mother, a father, and an idiot son. The idiot son caused them many troubles, but the parents kept up an act and reinterpreted everything he did to deny the fact that their son was an idiot."

2025-03-06 09:55:33 UTC

New blog post about AI. https://bartoszmilewski.com/2025/03/06/understanding-attention-in-llms/

2025-03-05 17:29:32 UTC

I know we made a secret pact not to unleash the prime finder, but I think the current political situation might be reason enough to override our earlier reservations.

2025-03-05 07:24:29 UTC

- reply

@nprofile…59k6 There is one car missing, so maybe the second?

2025-03-04 09:03:01 UTC

- reply

@nprofile…9gmq @nprofile…q7zp @nprofile…5a92
I always rely on multiple sources

2025-03-04 08:10:54 UTC

- reply

@nprofile…q7zp @nprofile…9gmq @nprofile…5a92
I agree. I'm studying the LLM architecture and it's clear that it's just Eliza on steroids.
On the other hand, Google isn't reliable either. We don't say that Google is hallucinating answers--it's the humanity that's hallucinating.

2025-03-03 22:38:30 UTC

- reply

@nprofile…5a92
I don't think it's capable of making stuff up. It's just repeating whatever was available in its training set. Just like Google search can lead you astray, so can GPT.

2025-03-03 21:35:03 UTC

- reply

@nprofile…59k6
I don't think ChatGPT was doing any introspection. I didn't ask "How many heads do you have?"

2025-03-03 17:59:36 UTC

I had a long conversation with ChatGPT testing my understanding of attention patterns in LLM. (I hope it wasn't lying to me.) Getting answers to questions like "how many attention heads does GPT-3 use" was extremely useful.