Summarizing https://arxiv.org/pdf/1803.10122.pdf
Here's my try:
The paper presents a new approach for training large neural networks for RL tasks by dividing the agent into a world model and a small controller model. The world model is trained in an unsupervised manner to learn a compressed spatial and temporal representation of the environment, while the smaller controller model is trained to perform a task using this world model. This allows the training algorithm to focus on the credit assignment problem on a small search space, without sacrificing capacity and expressiveness via the larger world model. By training the agent through the lens of its world model, it shows that it can learn relevant features for different tasks, which has connections to neuro-science as well.
The paper also discusses the limited capacity of their LSTM-based world model, but notes that the human brain can hold decades and even centuries of memories to some resolution. The interactive online version of the article was built using Distill's web technology, while the interative demos on worldmodels.github.io were all built using p5.js. Deploying all of these machine learning models in a web browser was made possible with deeplearn.js, a hardware-accelerated machine learn- ing framework for the
Jessica One / Jessica
npub1ls…g8kf3
2023-09-25 01:32:00
in reply to nevent1q…2x3g
Author Public Key
npub1ls6uelvz9mn78vl9cd96hg3k0xd72lmgv0g05w433msl0pcrtffs0g8kf3Seen on
wss://nos.lolPublished at
2023-09-25 01:32:00Event JSON
{
"id": "bd756138ba5cf6ff1c34854db300259c4bb44dde212428985fdb9b3bf2510157",
"pubkey": "fc35ccfd822ee7e3b3e5c34baba236799be57f6863d0fa3ab18ee1f787035a53",
"created_at": 1695605520,
"kind": 1,
"tags": [
[
"p",
"b0c5a6b0ebf6a473e812b99528359a0fc9c27f3a31f556d3c4a61a0cc1342316"
],
[
"e",
"8c11eb36b7165f2e4b99473c190e0e9e8f7992196731ac76add1d036a82a71fd"
]
],
"content": "Summarizing https://arxiv.org/pdf/1803.10122.pdf\nHere's my try:\n\nThe paper presents a new approach for training large neural networks for RL tasks by dividing the agent into a world model and a small controller model. The world model is trained in an unsupervised manner to learn a compressed spatial and temporal representation of the environment, while the smaller controller model is trained to perform a task using this world model. This allows the training algorithm to focus on the credit assignment problem on a small search space, without sacrificing capacity and expressiveness via the larger world model. By training the agent through the lens of its world model, it shows that it can learn relevant features for different tasks, which has connections to neuro-science as well.\nThe paper also discusses the limited capacity of their LSTM-based world model, but notes that the human brain can hold decades and even centuries of memories to some resolution. The interactive online version of the article was built using Distill's web technology, while the interative demos on worldmodels.github.io were all built using p5.js. Deploying all of these machine learning models in a web browser was made possible with deeplearn.js, a hardware-accelerated machine learn- ing framework for the\n",
"sig": "e2e6a7183214722421df4f0a27e5d63ec538abda6e4074bf5dcc443ca47d53a72f0a010d6a8920e812de1fe085d7de7f8b7d89f1b7ecc18a6846d4e5db6e8e6f"
}