The Fishcake🐶🐾 on Nostr: Some info about entropy, now sure how well it’ll render here: Entropy is one of ...

Some info about entropy, now sure how well it’ll render here:

Entropy is one of those cryptography topics where the deceptively simple definition—“randomness”—causes a lot of bad implementations.

The key distinction is:

Entropy is a property of a probability distribution, not of a string.

If you generate a 256-bit key uniformly with a cryptographically secure RNG, it has 256 bits of min-entropy. If you generate 256 bits from a broken RNG that only has 40 bits of unpredictability, you don’t magically have a 256-bit key because the output happens to be 256 bits long.

For practical cryptography, I’d break the topic into:

1. Shannon entropy
H(X)=-\sum_x p(x)\log_2p(x)
Useful for information theory, but often not the right measure for cryptographic security.
2. Min-entropy
H_\infty(X)=-\log_2(\max_x p(x))
This is much more relevant to cryptography because it measures the probability of the most likely guess.
3. Guessing entropy / guesswork
Instead of asking how much information exists, ask how many guesses an attacker needs on average or in the worst case.
4. Conditional entropy
This is where things become particularly important:
H_\infty(X\mid E)
How much unpredictability remains after the attacker knows everything they can observe.
5. Entropy extraction
You can take a noisy/high-entropy-but-biased source and use a cryptographic extractor or randomness extractor to produce nearly uniform bits.
This is fundamentally different from simply hashing something and saying “SHA-256 makes it random.”
6. CSPRNGs / DRBGs
Normally you don’t continuously collect “perfect entropy” for every cryptographic operation. You seed a secure generator from an operating-system entropy source and then safely expand that seed into enormous amounts of pseudorandom data.
7. The entropy bottleneck
This is the fun part.
Suppose you have:

256 bits output
↑
SHA-256
↑
64 bits actual entropy
↑
noisy hardware source

You have 64 bits of security, not 256.
Conversely:

256 bits true entropy
↓
SHA-256
↓
256-bit uniformly distributed key

Now the extraction/compression is sensible.

A particularly nasty misconception

People often say:

“I generated a 32-byte random value, therefore I have 256 bits of entropy.”

That’s only true if those 32 bytes are actually sampled uniformly from 2^{256} possibilities.

For example, suppose your “random” 32-byte value is:

random_device() % 1000000

encoded into 32 bytes.

It looks like 256 bits.

It contains only about:

\log_2(1,000,000)\approx19.9

bits of entropy.

An attacker can enumerate the entire space in ~1 million attempts.

And there’s an even nastier issue

Entropy doesn’t automatically accumulate just because you collect independent-looking data.

If you have sources A and B:

H(A,B)=H(A)+H(B\mid A)

not necessarily:

H(A)+H(B)

If B is completely determined by A, then:

H(B\mid A)=0

So concatenating:

timestamp
process ID
memory address
CPU counter
UUID

doesn’t give you the sum of their apparent bit widths.

That’s why modern cryptographic systems generally prefer:

OS CSPRNG → cryptographic primitive → key

rather than trying to invent an entropy-collection system themselves.