![](\"https://habrastorage.org/webt/3s/1c/4-/3s1c4-gqcwdn-dvy4w03g3wwmru.png\")
I found these videos on BERT by Chris McCormick very helpful (I know Chris for the most comprehensive intro into Word2vec).
\n\nChris tries to explain transformers without the need for a demanding prerequisite of knowing all nitty-gritty details of RNNs.
\n\n
The inner workings of transformers and BERT are described based on wonderful posts by Jay Alammar. However, there's some added value to just reading Jay's posts yourself, as Chris makes some clarifying sketches and also raises questions on the go.
\n\nFor example, showing in parallel the BERT architecture described in Jay's post (left in the picture below) and corresponding BERT weights printed in PyTorch (right in the picture below). \n![](\"https://habrastorage.org/webt/sb/gl/ed/sbgledj6xbgn-nwmyg21xm59qb0.png\")
There are also some examples given with PyTorch implementations.
\n\nI watched all of the videos, well, I was already familiar with the topic but still found it useful to consolidate my knowledge. And for those only starting the journey, maybe it's the best source.
\n\nHappy learning!
\n\nPS. I think the material is good enough to spread some tweets about it.
", "messages": [ { "id": "805946", "postDate": "04/13/2020 09:21:30", "content": "I found these videos on BERT by Chris McCormick very helpful (I know Chris for the most comprehensive intro into Word2vec).
\n\nChris tries to explain transformers without the need for a demanding prerequisite of knowing all nitty-gritty details of RNNs.
\n\n
The inner workings of transformers and BERT are described based on wonderful posts by Jay Alammar. However, there's some added value to just reading Jay's posts yourself, as Chris makes some clarifying sketches and also raises questions on the go.
\n\nFor example, showing in parallel the BERT architecture described in Jay's post (left in the picture below) and corresponding BERT weights printed in PyTorch (right in the picture below). \n
There are also some examples given with PyTorch implementations.
\n\nI watched all of the videos, well, I was already familiar with the topic but still found it useful to consolidate my knowledge. And for those only starting the journey, maybe it's the best source.
\n\nHappy learning!
\n\nPS. I think the material is good enough to spread some tweets about it.
", "rawMarkdown": "I found [these videos](https://youtube.com/channel/UCoRX98PLOsaN8PtekB9kWrw/videos) on BERT by Chris McCormick very helpful (I know Chris for the most comprehensive [intro](http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/) into Word2vec). \n\n\n\nChris tries to explain transformers without the need for a demanding prerequisite of knowing all nitty-gritty details of RNNs. \n\n\n\nThe inner workings of transformers and BERT are described based on [wonderful posts](http://jalammar.github.io) by Jay Alammar. However, there's some added value to just reading Jay's posts yourself, as Chris makes some clarifying sketches and also raises questions on the go. \n\nFor example, showing in parallel the BERT architecture described in Jay's post (left in the picture below) and corresponding BERT weights printed in PyTorch (right in the picture below). \n\n\nThere are also some examples given with PyTorch implementations. \n\nI watched all of the videos, well, I was already familiar with the topic but still found it useful to consolidate my knowledge. And for those only starting the journey, maybe it's the best source. \n\nHappy learning!\n\nPS. I think the material is good enough to spread [some tweets](https://twitter.com/ykashnitsky/status/1249265938903576576) about it.",
"votes": null
},
{
"id": "805968",
"postDate": "04/13/2020 09:59:13",
"content": "Thanks for sharing. He also has a blog, which seems to be a written form of his videos. I have read a couple of his blog posts and they are very informative and helpful.
", "rawMarkdown": "Thanks for sharing. He also has [a blog](https://mccormickml.com/), which seems to be a written form of his videos. I have read a couple of his blog posts and they are very informative and helpful.", "votes": null }, { "id": "806314", "postDate": "04/13/2020 15:51:10", "content": "Thanks for sharing
", "rawMarkdown": "Thanks for sharing", "votes": null }, { "id": "807713", "postDate": "04/14/2020 20:26:34", "content": "What an excellent use of BERT! Where did you find the videos?
", "rawMarkdown": "What an excellent use of BERT! Where did you find the videos?", "votes": null }, { "id": "808208", "postDate": "04/15/2020 08:02:31", "content": "on YouTube :)\nWell, actually a colleague of mine recommended.
", "rawMarkdown": "on YouTube :)\nWell, actually a colleague of mine recommended.", "votes": null }, { "id": "808355", "postDate": "04/15/2020 10:45:54", "content": "Superb videos. Thanks for sharing!
", "rawMarkdown": "Superb videos. Thanks for sharing!", "votes": null }, { "id": "815511", "postDate": "04/21/2020 15:42:04", "content": "This is also a great video explaining the paper, Attention is All You Need with cool slides!
\n\n", "rawMarkdown": "This is also a great video explaining the paper, Attention is All You Need with cool slides!\n\nhttps://youtu.be/yCdl2afW88k", "votes": null }, { "id": "816818", "postDate": "04/22/2020 16:04:56", "content": "It is a great video explaining the paper,
", "rawMarkdown": "It is a great video explaining the paper,", "votes": null }, { "id": "862181", "postDate": "05/26/2020 13:08:20", "content": "This is good. Thank you so much for sharing.
", "rawMarkdown": "This is good. Thank you so much for sharing.", "votes": null } ], "comments": [ { "id": 805968, "author_name": "tanlikesmath", "author_url": "", "post_date": "04/13/2020 09:59:13", "content": "Thanks for sharing. He also has a blog, which seems to be a written form of his videos. I have read a couple of his blog posts and they are very informative and helpful.
", "votes": null, "replies": [] }, { "id": 806314, "author_name": "jagannathrk", "author_url": "", "post_date": "04/13/2020 15:51:10", "content": "Thanks for sharing
", "votes": null, "replies": [] }, { "id": 807713, "author_name": "larsen0966", "author_url": "", "post_date": "04/14/2020 20:26:34", "content": "What an excellent use of BERT! Where did you find the videos?
", "votes": null, "replies": [ { "id": 808208, "author_name": "kashnitsky", "author_url": "", "post_date": "04/15/2020 08:02:31", "content": "on YouTube :)\nWell, actually a colleague of mine recommended.
", "votes": null, "replies": [] } ] }, { "id": 808355, "author_name": "", "author_url": "", "post_date": "04/15/2020 10:45:54", "content": "Superb videos. Thanks for sharing!
", "votes": null, "replies": [] }, { "id": 815511, "author_name": "adityaecdrid", "author_url": "", "post_date": "04/21/2020 15:42:04", "content": "This is also a great video explaining the paper, Attention is All You Need with cool slides!
\n\n", "votes": null, "replies": [] }, { "id": 816818, "author_name": "dssant85", "author_url": "", "post_date": "04/22/2020 16:04:56", "content": "It is a great video explaining the paper,
", "votes": null, "replies": [] }, { "id": 862181, "author_name": "abhinand05", "author_url": "", "post_date": "05/26/2020 13:08:20", "content": "This is good. Thank you so much for sharing.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "805946": "I found [these videos](https://youtube.com/channel/UCoRX98PLOsaN8PtekB9kWrw/videos) on BERT by Chris McCormick very helpful (I know Chris for the most comprehensive [intro](http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/) into Word2vec). \n\n\n\nChris tries to explain transformers without the need for a demanding prerequisite of knowing all nitty-gritty details of RNNs. \n\n\n\nThe inner workings of transformers and BERT are described based on [wonderful posts](http://jalammar.github.io) by Jay Alammar. However, there's some added value to just reading Jay's posts yourself, as Chris makes some clarifying sketches and also raises questions on the go. \n\nFor example, showing in parallel the BERT architecture described in Jay's post (left in the picture below) and corresponding BERT weights printed in PyTorch (right in the picture below). \n\n\nThere are also some examples given with PyTorch implementations. \n\nI watched all of the videos, well, I was already familiar with the topic but still found it useful to consolidate my knowledge. And for those only starting the journey, maybe it's the best source. \n\nHappy learning!\n\nPS. I think the material is good enough to spread [some tweets](https://twitter.com/ykashnitsky/status/1249265938903576576) about it.",
"805968": "Thanks for sharing. He also has [a blog](https://mccormickml.com/), which seems to be a written form of his videos. I have read a couple of his blog posts and they are very informative and helpful.",
"806314": "Thanks for sharing",
"807713": "What an excellent use of BERT! Where did you find the videos?",
"808208": "on YouTube :)\nWell, actually a colleague of mine recommended.",
"808355": "Superb videos. Thanks for sharing!",
"815511": "This is also a great video explaining the paper, Attention is All You Need with cool slides!\n\nhttps://youtu.be/yCdl2afW88k",
"816818": "It is a great video explaining the paper,",
"862181": "This is good. Thank you so much for sharing."
},
"source": "meta"
}