Andrej Karpathy / @karpathy (RSS Feed)

Twitter feed for: @karpathy. Generated by nitter.moomoo.me

https://nitter.moomoo.me/karpathy

Last Notes

2023-08-03 23:45:59 UTC

The high-order bit that changed in AI:
"I'll give you 10X bigger computer"
\- 10 years ago: I'm not immediately sure what to do with it
\- Now: Not only do I know exactly what to do with it but I can predict the metrics I will achieve
Algorithmic progress was necessity, now bonus.
https://nitter.moomoo.me/karpathy/status/1687248476508487681#m

2023-08-02 23:22:10 UTC

Who’s getting how many H100s and when is top gossip of the valley rn
nitter.moomoo.me/lpolovets/status/1686545776246390784#m (https://nitter.moomoo.me/lpolovets/status/1686545776246390784#m)
https://nitter.moomoo.me/karpathy/status/1686880098417508353#m

2023-08-02 05:39:23 UTC

I like how all the Meissner effect photos/videos are right around the threshold of convincing. A kind of mix of intriguing but also a bit confusing and lacking, strangely scarce, grainy… almost exactly like photos/videos of flying saucers. And just the same - I want to believe.
https://nitter.moomoo.me/karpathy/status/1686612638187552768#m

2023-07-27 17:13:25 UTC

Neat, didn't realize llama2.c made it to the top of Github trending. Also more generally Github trending is a great place to keep an eye on for projects that are seeing traction, either as following this account and its xeets, or as bookmark.
nitter.moomoo.me/trending\_repos/status/1684488232862732289#m (https://nitter.moomoo.me/trending_repos/status/1684488232862732289#m)
https://nitter.moomoo.me/karpathy/status/1684612972034011136#m

2023-07-27 16:57:10 UTC

Filmmaking 2.0
nitter.moomoo.me/terhavlova/status/1684265329277403149#m (https://nitter.moomoo.me/terhavlova/status/1684265329277403149#m)
https://nitter.moomoo.me/karpathy/status/1684608881920802816#m

2023-05-02 16:38:39 UTC

Excellent TED talk from Sal Khan:
\- many inspiring examples of GPTs finetuned into socratic tutors, assisting without giving away answers.
\- none of it "out of the box", requires prompt engineering, finetuning, data collection, iteration.
\- sense of barely scratching the surface.
nitter.moomoo.me/salkhanacademy/status/1653070197056585732#m (https://nitter.moomoo.me/salkhanacademy/status/1653070197056585732#m)
https://nitter.moomoo.me/karpathy/status/1653438865880023041#m

2023-04-27 18:48:31 UTC

FaceTime with ChatGPT callannie.ai (https://callannie.ai)
Fun, qualitatively different experience over "texting"
Ty @CallAnnieAI (https://nitter.moomoo.me/CallAnnieAI) for shoutout to my tweet a ~year ago pitching this as an idea nitter.moomoo.me/CallAnnieAI/stat… (https://nitter.moomoo.me/CallAnnieAI/status/1651379993413558272)
With improvements to personality, latency, ASR/TTS could be magical ✨
nitter.moomoo.me/frantzfries/status/1651316031762071553#m (https://nitter.moomoo.me/frantzfries/status/1651316031762071553#m)
https://nitter.moomoo.me/karpathy/status/1651659606844928000#m

2023-04-02 19:30:16 UTC

**Response to @karpathy:**
All of that is just one agent/thread. People coalesce into organizations so they can specialize and parallelize work towards shared goals. Imo this is likely to happen to AutoGPTs and for the same reasons, strung into AutoOrgs, with AutoCEO, AutoCFO, AutoICs, etc.
https://nitter.moomoo.me/karpathy/status/1642610417779490816#m

2023-04-02 19:19:09 UTC

**Response to @karpathy:**
1 GPT call is a bit like 1 thought. Stringing them together in loops creates agents that can perceive, think, and act, their goals defined in English in prompts.
For feedback / learning, one path is to have a "reflect" phase that evaluates outcomes, saves rollouts to memory,… nitter.moomoo.me/i/web/status/164… (https://nitter.moomoo.me/i/web/status/1642607620673634304)
https://nitter.moomoo.me/karpathy/status/1642607620673634304#m

2023-03-16 20:07:42 UTC

Less publicized but highly awesome aspect of GPT-4 launch was that OpenAI open sourced an evals framework, allowing us to crowdsource model evaluations at scale 📈. The repo is getting some very high quality PRs (rewarded with GPT-4 access).
I <3 evals; \`pip install evals\`
nitter.moomoo.me/kondrich2/status/1635687409668227072#m (https://nitter.moomoo.me/kondrich2/status/1635687409668227072#m)
https://nitter.moomoo.me/karpathy/status/1636459245184106497#m

2023-03-12 22:46:03 UTC

Dropout layers in a Transformer leak the phase bit (train/eval) - small example. So an LLM may be able to determine if it is being trained and if backward pass follows. Clear intuitively but good to see, and interesting to think through repercussions of
colab.research.google.com/dr… (https://colab.research.google.com/drive/1286r553N8drh6-VeZjZA1vbUBY9Z1fps?usp=sharing)
https://nitter.moomoo.me/pic/card_img%2F1634876168636874752%2FCaT1snPL%3Fformat%3Dpng%26name%3D280x280_2
https://nitter.moomoo.me/karpathy/status/1635049541534879745#m

2023-03-12 16:31:08 UTC

File reading under the "horror" genre.
reality vs expectation
nitter.moomoo.me/bilalmahmood/status/1634595490968006661#m (https://nitter.moomoo.me/bilalmahmood/status/1634595490968006661#m)
https://nitter.moomoo.me/pic/media%2FFrCGsSMaYAIEH2l.jpg
https://nitter.moomoo.me/karpathy/status/1634955190964219905#m

2023-03-09 16:55:16 UTC

"The hot mess theory of AI misalignment"
a favorite talk from a recent alignment workshop turned article; offers a unique and imo fairly realistic framework for superintelligent system futures that departs from your stock paperclip maximizers.
nitter.moomoo.me/jaschasd/status/1633864184697081858#m (https://nitter.moomoo.me/jaschasd/status/1633864184697081858#m)
https://nitter.moomoo.me/karpathy/status/1633874103672406017#m

2023-03-01 18:00:29 UTC

ControlNet is 🔥 github.com/lllyasviel/Contro…
Allows for very fine control over stable diffusion process, has taken over r/stablediffusion and friends
nitter.moomoo.me/bhutanisanyam1/status/1630980866775330819#m
https://nitter.moomoo.me/karpathy/status/1630991410387378176#m

2023-02-26 16:59:32 UTC

(random) I appreciate the work @GrowSF is doing and recommend their newsletter to people living in SF. Have been subscribed for a while and find it helpful
Main site: growsf.org
Substack: growsf.substack.com/p/the-gr…
https://nitter.moomoo.me/karpathy/status/1629888907914678272#m

2023-02-25 19:06:40 UTC

Watching a lot more Korean TV/content recently (Netflix and such) and finding it very refreshing compared to US equivalents. People are so much nicer, more courteous, respectful with each other, it’s beautiful and calming.
https://nitter.moomoo.me/karpathy/status/1629558513914769408#m