![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fb579990d060e0066c1ff151bae8832d3%2Flandmarks_0.gif?generation=1678874361243265&alt=media\")
Hi. I made a notebook to animate landmark data. Hope it helps to understand data in more intuitive way.
\nHave fun!
\nISLR: Animated 250 Sampled Signs
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fa1843a7c264969e538186f327468872d%2Flandmarks_1.gif?generation=1678874497337976&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F82aa4c362b613e07bb20ab0b377d7e49%2Flandmarks_2.gif?generation=1678874517148729&alt=media\")
Hi. I made a notebook to animate landmark data. Hope it helps to understand data in more intuitive way.
\nHave fun!
\nISLR: Animated 250 Sampled Signs
\n
I also published a dataset of 250 unique signed sampled from the landmark data.
\n(Files are same as the output of the notebook.)
\nhttps://www.kaggle.com/datasets/tatamikenn/islr-animation-250-signs
i want animation in stable diffusion + control net!!!!
\n", "rawMarkdown": "i want animation in stable diffusion + control net!!!!\n\nhttps://www.reddit.com/r/StableDiffusion/comments/11jy7b7/controlnet_models_based_on_mediapipe_prototype/", "votes": 2, "replies": [ { "id": 2183026, "postDate": "2023-03-15T12:39:08.020Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2185012, "postDate": "2023-03-16T18:42:21.697Z", "content": "I managed to manipulated the following GIF. Although pose is good, the hand shape is sometimes good but sometimes far from original. I think natural pose as a photo model is easy to generate, but rare hand shape (that is not unusual in hand signs) are hard to generate due to lack of training image.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff67ea9ee99ea241f04c5173d9a733b78%2Fconv.gif?generation=1678991967371522&alt=media\")
\"hand signs are hard to generate due to lack of training image.
\napply mediapipline to back transalte image to pose.
\nmaybe these are good noise for training data
i think we can modify control net to have more than one contraints :
\ne.g. pose + different human signer (different body, arm length)
\ne.g. pose + depth (different viewpoint)
\ne.g. pose + different framerate /keframe (i think you are ask stable diffusion to interpolate )
\ne.g. mix 2 pose of same class
Are you saying the process below?
\nIt's difficult to generate a lot of data because plenty of human loop exists when generating image from stable diffusion & ControlNet part (because I used WebUI).
\nBesides, we should care the effect of domain-shift of the back-translated landmark data of hand: it must change the meaning of hand signs if we use SD & ControlNet with poor hand shape generation.
However I think It's interesting idea to use the back-translated mediapipe data for external train data or interpolate missing data.
", "rawMarkdown": "Are you saying the process below?\n\n- media pipe (of this competition data) -> stable diffusion + ControlNet -> media pipe (of generated image)\n\nIt's difficult to generate a lot of data because plenty of human loop exists when generating image from stable diffusion & ControlNet part (because I used WebUI).\nBesides, we should care the effect of domain-shift of the back-translated landmark data of hand: it must change the meaning of hand signs if we use SD & ControlNet with poor hand shape generation.\n\nHowever I think It's interesting idea to use the back-translated mediapipe data for external train data or interpolate missing data." }, { "id": 2185319, "postDate": "2023-03-17T00:21:37.787Z", "content": "For augmentation, I think the following approach is more promising.
\nBefore animation as I'm trying to plot, the graph is inverted how do we rotate it by 180 degree
", "rawMarkdown": "Before animation as I'm trying to plot, the graph is inverted how do we rotate it by 180 degree", "replies": [ { "id": 2185451, "postDate": "2023-03-17T04:35:33.733Z", "content": "You can simply invert y axis of plot library.
\n matplotlib.pyplot plt\n\nx = [, , , , ]\ny = [, , , , ]\n\nplt.plot(x, y)\n\nax = plt.gca()\nax.invert_yaxis()\n\nplt.show()\nI also published a dataset of 250 unique signed sampled from the landmark data.
\n(Files are same as the output of the notebook.)
\nhttps://www.kaggle.com/datasets/tatamikenn/islr-animation-250-signs
i want animation in stable diffusion + control net!!!!
\n", "votes": 2, "replies": [ { "id": 2183026, "author_name": "", "author_url": "", "post_date": "2023-03-15T12:39:08.020000", "content": "", "votes": 0, "replies": [] }, { "id": 2185012, "author_name": "Bilzard", "author_url": "", "post_date": "2023-03-16T18:42:21.697000", "content": "I managed to manipulated the following GIF. Although pose is good, the hand shape is sometimes good but sometimes far from original. I think natural pose as a photo model is easy to generate, but rare hand shape (that is not unusual in hand signs) are hard to generate due to lack of training image.
\n
\"hand signs are hard to generate due to lack of training image.
\napply mediapipline to back transalte image to pose.
\nmaybe these are good noise for training data
i think we can modify control net to have more than one contraints :
\ne.g. pose + different human signer (different body, arm length)
\ne.g. pose + depth (different viewpoint)
\ne.g. pose + different framerate /keframe (i think you are ask stable diffusion to interpolate )
\ne.g. mix 2 pose of same class
Are you saying the process below?
\nIt's difficult to generate a lot of data because plenty of human loop exists when generating image from stable diffusion & ControlNet part (because I used WebUI).
\nBesides, we should care the effect of domain-shift of the back-translated landmark data of hand: it must change the meaning of hand signs if we use SD & ControlNet with poor hand shape generation.
However I think It's interesting idea to use the back-translated mediapipe data for external train data or interpolate missing data.
", "votes": 0, "replies": [] }, { "id": 2185319, "author_name": "Bilzard", "author_url": "", "post_date": "2023-03-17T00:21:37.787000", "content": "For augmentation, I think the following approach is more promising.
\nBefore animation as I'm trying to plot, the graph is inverted how do we rotate it by 180 degree
", "votes": 0, "replies": [ { "id": 2185451, "author_name": "Bilzard", "author_url": "", "post_date": "2023-03-17T04:35:33.733000", "content": "You can simply invert y axis of plot library.
\n matplotlib.pyplot plt\n\nx = [, , , , ]\ny = [, , , , ]\n\nplt.plot(x, y)\n\nax = plt.gca()\nax.invert_yaxis()\n\nplt.show()\n