Nostr notes by Andrej Karpathy / @karpathy (RSS Feed)

The high-order bit that changed in AI: "I'll give you 10X ...

2023-08-03T23:45:59Z

The high-order bit that changed in AI:

"I'll give you 10X bigger computer"

\- 10 years ago: I'm not immediately sure what to do with it

\- Now: Not only do I know exactly what to do with it but I can predict the metrics I will achieve

Algorithmic progress was necessity, now bonus.

https://nitter.moomoo.me/karpathy/status/1687248476508487681#m

Who’s getting how many H100s and when is top gossip of the ...

2023-08-02T23:22:10Z

Who’s getting how many H100s and when is top gossip of the valley rn

nitter.moomoo.me/lpolovets/status/1686545776246390784#m (https://nitter.moomoo.me/lpolovets/status/1686545776246390784#m)

https://nitter.moomoo.me/karpathy/status/1686880098417508353#m

I like how all the Meissner effect photos/videos are right around ...

2023-08-02T05:39:23Z

I like how all the Meissner effect photos/videos are right around the threshold of convincing. A kind of mix of intriguing but also a bit confusing and lacking, strangely scarce, grainy… almost exactly like photos/videos of flying saucers. And just the same - I want to believe.

https://nitter.moomoo.me/karpathy/status/1686612638187552768#m

Neat, didn't realize llama2.c made it to the top of Github ...

2023-07-27T17:13:25Z

Neat, didn't realize llama2.c made it to the top of Github trending. Also more generally Github trending is a great place to keep an eye on for projects that are seeing traction, either as following this account and its xeets, or as bookmark.

nitter.moomoo.me/trending\_repos/status/1684488232862732289#m (https://nitter.moomoo.me/trending_repos/status/1684488232862732289#m)

https://nitter.moomoo.me/karpathy/status/1684612972034011136#m

Filmmaking 2.0 ...

2023-07-27T16:57:10Z

Filmmaking 2.0

nitter.moomoo.me/terhavlova/status/1684265329277403149#m (https://nitter.moomoo.me/terhavlova/status/1684265329277403149#m)

https://nitter.moomoo.me/karpathy/status/1684608881920802816#m

Excellent TED talk from Sal Khan: \- many inspiring examples of ...

2023-05-02T16:38:39Z

Excellent TED talk from Sal Khan:

\- many inspiring examples of GPTs finetuned into socratic tutors, assisting without giving away answers.

\- none of it "out of the box", requires prompt engineering, finetuning, data collection, iteration.

\- sense of barely scratching the surface.

nitter.moomoo.me/salkhanacademy/status/1653070197056585732#m (https://nitter.moomoo.me/salkhanacademy/status/1653070197056585732#m)

https://nitter.moomoo.me/karpathy/status/1653438865880023041#m

FaceTime with ChatGPT callannie.ai (https://callannie.ai) Fun, ...

2023-04-27T18:48:31Z

FaceTime with ChatGPT callannie.ai (https://callannie.ai)

Fun, qualitatively different experience over "texting"

Ty @CallAnnieAI (https://nitter.moomoo.me/CallAnnieAI) for shoutout to my tweet a ~year ago pitching this as an idea nitter.moomoo.me/CallAnnieAI/stat… (https://nitter.moomoo.me/CallAnnieAI/status/1651379993413558272)

With improvements to personality, latency, ASR/TTS could be magical ✨

nitter.moomoo.me/frantzfries/status/1651316031762071553#m (https://nitter.moomoo.me/frantzfries/status/1651316031762071553#m)

https://nitter.moomoo.me/karpathy/status/1651659606844928000#m

Response to @karpathy: All of that is just one agent/thread. ...

2023-04-02T19:30:16Z

**Response to @karpathy:**

All of that is just one agent/thread. People coalesce into organizations so they can specialize and parallelize work towards shared goals. Imo this is likely to happen to AutoGPTs and for the same reasons, strung into AutoOrgs, with AutoCEO, AutoCFO, AutoICs, etc.

https://nitter.moomoo.me/karpathy/status/1642610417779490816#m

Response to @karpathy: 1 GPT call is a bit like 1 thought. ...

2023-04-02T19:19:09Z

**Response to @karpathy:**

1 GPT call is a bit like 1 thought. Stringing them together in loops creates agents that can perceive, think, and act, their goals defined in English in prompts.

For feedback / learning, one path is to have a "reflect" phase that evaluates outcomes, saves rollouts to memory,… nitter.moomoo.me/i/web/status/164… (https://nitter.moomoo.me/i/web/status/1642607620673634304)

https://nitter.moomoo.me/karpathy/status/1642607620673634304#m

Less publicized but highly awesome aspect of GPT-4 launch was ...

2023-03-16T20:07:42Z

Less publicized but highly awesome aspect of GPT-4 launch was that OpenAI open sourced an evals framework, allowing us to crowdsource model evaluations at scale 📈. The repo is getting some very high quality PRs (rewarded with GPT-4 access).

I <3 evals; \`pip install evals\`

nitter.moomoo.me/kondrich2/status/1635687409668227072#m (https://nitter.moomoo.me/kondrich2/status/1635687409668227072#m)

https://nitter.moomoo.me/karpathy/status/1636459245184106497#m

Dropout layers in a Transformer leak the phase bit (train/eval) - ...

2023-03-12T22:46:03Z

Dropout layers in a Transformer leak the phase bit (train/eval) - small example. So an LLM may be able to determine if it is being trained and if backward pass follows. Clear intuitively but good to see, and interesting to think through repercussions of

colab.research.google.com/dr… (https://colab.research.google.com/drive/1286r553N8drh6-VeZjZA1vbUBY9Z1fps?usp=sharing)

https://nitter.moomoo.me/pic/card_img%2F1634876168636874752%2FCaT1snPL%3Fformat%3Dpng%26name%3D280x280_2

https://nitter.moomoo.me/karpathy/status/1635049541534879745#m

File reading under the "horror" genre. reality vs ...

2023-03-12T16:31:08Z

File reading under the "horror" genre.

reality vs expectation

nitter.moomoo.me/bilalmahmood/status/1634595490968006661#m (https://nitter.moomoo.me/bilalmahmood/status/1634595490968006661#m)

https://nitter.moomoo.me/karpathy/status/1634955190964219905#m

"The hot mess theory of AI misalignment" a favorite talk ...

2023-03-09T16:55:16Z

"The hot mess theory of AI misalignment"

a favorite talk from a recent alignment workshop turned article; offers a unique and imo fairly realistic framework for superintelligent system futures that departs from your stock paperclip maximizers.

nitter.moomoo.me/jaschasd/status/1633864184697081858#m (https://nitter.moomoo.me/jaschasd/status/1633864184697081858#m)

https://nitter.moomoo.me/karpathy/status/1633874103672406017#m

ControlNet is 🔥 github.com/lllyasviel/Contro… Allows for ...

2023-03-01T18:00:29Z

ControlNet is 🔥 github.com/lllyasviel/Contro…
Allows for very fine control over stable diffusion process, has taken over r/stablediffusion and friends
nitter.moomoo.me/bhutanisanyam1/status/1630980866775330819#m

https://nitter.moomoo.me/karpathy/status/1630991410387378176#m

(random) I appreciate the work @GrowSF is doing and recommend ...

2023-02-26T16:59:32Z

(random) I appreciate the work @GrowSF is doing and recommend their newsletter to people living in SF. Have been subscribed for a while and find it helpful
Main site: growsf.org
Substack: growsf.substack.com/p/the-gr…

https://nitter.moomoo.me/karpathy/status/1629888907914678272#m

Watching a lot more Korean TV/content recently (Netflix and such) ...

2023-02-25T19:06:40Z

Watching a lot more Korean TV/content recently (Netflix and such) and finding it very refreshing compared to US equivalents. People are so much nicer, more courteous, respectful with each other, it’s beautiful and calming.

https://nitter.moomoo.me/karpathy/status/1629558513914769408#m