{ "id": 192138, "title": "New to transformers? Some resources", "url": "/competitions/riiid-test-answer-prediction/discussion/192138", "author_name": "", "post_date": "2020-10-20T10:43:58.954290500Z", "votes": 32, "comment_count": 4, "views": 0, "content": "

Transformers are models that some competitors might encounter for the first time. Myself included, not so long ago 😊 Some current state-of-the-art constitutes of transformers (in Natural Language Processing, Knowledge tracing, …) so it's worth to learn.

\n

If you are still puzzled about components of transformers (like the queries, keys, values below), I highly recommend The Illustrated Transformer blog
\n\"\"

\n

I like to scroll the blog down together with a great Youtube series on BERT. It focuses on the blog and explains the matrices multiplications within transformers in more detail.

\n

You can also recap your knowledge on transformers with these videos:

\n\n

And some started code to play with in Tensorflow:
\nhttps://www.tensorflow.org/tutorials/text/transformer

\n

Have fun learning!

", "messages": [ { "id": "1054956", "postDate": "10/20/2020 10:43:58", "content": "

Transformers are models that some competitors might encounter for the first time. Myself included, not so long ago 😊 Some current state-of-the-art constitutes of transformers (in Natural Language Processing, Knowledge tracing, …) so it's worth to learn.

\n

If you are still puzzled about components of transformers (like the queries, keys, values below), I highly recommend The Illustrated Transformer blog
\n\"\"

\n

I like to scroll the blog down together with a great Youtube series on BERT. It focuses on the blog and explains the matrices multiplications within transformers in more detail.

\n

You can also recap your knowledge on transformers with these videos:

\n\n

And some started code to play with in Tensorflow:
\nhttps://www.tensorflow.org/tutorials/text/transformer

\n

Have fun learning!

", "rawMarkdown": "Transformers are models that some competitors might encounter for the first time. Myself included, not so long ago 😊 Some current state-of-the-art constitutes of transformers (in Natural Language Processing, Knowledge tracing, ...) so it's worth to learn.\n\nIf you are still puzzled about components of transformers (like the queries, keys, values below), I highly recommend [The Illustrated Transformer blog](https://jalammar.github.io/illustrated-transformer/)\n![](https://jalammar.github.io/images/t/transformer_self_attention_vectors.png)\n\nI like to scroll the blog down together with a great [Youtube series on BERT](https://www.youtube.com/watch?v=FKlPCK1uFrc&list=PLam9sigHPGwOBuH4_4fr-XvDbe5uneaf6&ab_channel=ChrisMcCormickAI). It focuses on the blog and explains the matrices multiplications within transformers in more detail.\n\nYou can also recap your knowledge on transformers with these videos:\n- [Transformer Neural Networks - EXPLAINED! (Attention is all you need)](https://www.youtube.com/watch?v=TQQlZhbC5ps&ab_channel=CodeEmporium)\n- [Illustrated Guide to Transformers Neural Network: A step by step explanation\n](https://www.youtube.com/watch?v=4Bdc55j80l8&ab_channel=TheA.I.Hacker-MichaelPhi)\n- [Attention is All You Need](https://www.youtube.com/watch?v=iDulhoQ2pro&ab_channel=YannicKilcher)\n\nAnd some started code to play with in Tensorflow:\nhttps://www.tensorflow.org/tutorials/text/transformer\n\nHave fun learning!", "votes": null }, { "id": "1054964", "postDate": "10/20/2020 11:03:40", "content": "

I like ChrisMcCormickAI detail explanation videos. MichaelPhi is awesome Thanks for sharing

", "rawMarkdown": "I like ChrisMcCormickAI detail explanation videos. MichaelPhi is awesome Thanks for sharing", "votes": null }, { "id": "1054971", "postDate": "10/20/2020 11:26:19", "content": "

Same! He makes it very clear

", "rawMarkdown": "Same! He makes it very clear", "votes": null }, { "id": "1073777", "postDate": "11/09/2020 22:50:37", "content": "

Transformers with PyTorch - https://www.youtube.com/watch?v=U0s0f995w14

", "rawMarkdown": "Transformers with PyTorch - https://www.youtube.com/watch?v=U0s0f995w14", "votes": null }, { "id": "2366556", "postDate": "07/31/2023 04:55:31", "content": "

Thank you so much for this!
\nThis has helped me a lot

", "rawMarkdown": "Thank you so much for this!\nThis has helped me a lot", "votes": null } ], "comments": [ { "id": 1054964, "author_name": "seshurajup", "author_url": "", "post_date": "10/20/2020 11:03:40", "content": "

I like ChrisMcCormickAI detail explanation videos. MichaelPhi is awesome Thanks for sharing

", "votes": null, "replies": [ { "id": 1054971, "author_name": "rafiko1", "author_url": "", "post_date": "10/20/2020 11:26:19", "content": "

Same! He makes it very clear

", "votes": null, "replies": [] } ] }, { "id": 1073777, "author_name": "rashmibanthia", "author_url": "", "post_date": "11/09/2020 22:50:37", "content": "

Transformers with PyTorch - https://www.youtube.com/watch?v=U0s0f995w14

", "votes": null, "replies": [] }, { "id": 2366556, "author_name": "aatmanvaidya", "author_url": "", "post_date": "07/31/2023 04:55:31", "content": "

Thank you so much for this!
\nThis has helped me a lot

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1054956": "Transformers are models that some competitors might encounter for the first time. Myself included, not so long ago 😊 Some current state-of-the-art constitutes of transformers (in Natural Language Processing, Knowledge tracing, ...) so it's worth to learn.\n\nIf you are still puzzled about components of transformers (like the queries, keys, values below), I highly recommend [The Illustrated Transformer blog](https://jalammar.github.io/illustrated-transformer/)\n![](https://jalammar.github.io/images/t/transformer_self_attention_vectors.png)\n\nI like to scroll the blog down together with a great [Youtube series on BERT](https://www.youtube.com/watch?v=FKlPCK1uFrc&list=PLam9sigHPGwOBuH4_4fr-XvDbe5uneaf6&ab_channel=ChrisMcCormickAI). It focuses on the blog and explains the matrices multiplications within transformers in more detail.\n\nYou can also recap your knowledge on transformers with these videos:\n- [Transformer Neural Networks - EXPLAINED! (Attention is all you need)](https://www.youtube.com/watch?v=TQQlZhbC5ps&ab_channel=CodeEmporium)\n- [Illustrated Guide to Transformers Neural Network: A step by step explanation\n](https://www.youtube.com/watch?v=4Bdc55j80l8&ab_channel=TheA.I.Hacker-MichaelPhi)\n- [Attention is All You Need](https://www.youtube.com/watch?v=iDulhoQ2pro&ab_channel=YannicKilcher)\n\nAnd some started code to play with in Tensorflow:\nhttps://www.tensorflow.org/tutorials/text/transformer\n\nHave fun learning!", "1054964": "I like ChrisMcCormickAI detail explanation videos. MichaelPhi is awesome Thanks for sharing", "1054971": "Same! He makes it very clear", "1073777": "Transformers with PyTorch - https://www.youtube.com/watch?v=U0s0f995w14", "2366556": "Thank you so much for this!\nThis has helped me a lot" }, "source": "meta" }