{
  "id": 312001,
  "title": "My Approach and Experiment Registry [ONGOING]",
  "url": "/competitions/ultra-mnist/discussion/312001",
  "author_name": "",
  "post_date": "2022-03-09T22:57:03.530726900Z",
  "votes": 14,
  "comment_count": 1,
  "views": 0,
  "content": "<p><strong>[UPDATED - MARCH 10, 2022]</strong></p>\n<hr>\n<p><br></p>\n<p>Hi all, I figured I would make a document detailing my approaches for this competition. </p>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Basic Approach</strong></p>\n<ul>\n<li>Use OpenCV to identify contours and crop and resize individual digits to a format expected by various pre-trained models</li>\n<li>Use pre-trained models (ensemble) to perform inference</li>\n<li>Post-process summation, clipping and other</li>\n</ul>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Details</strong></p>\n<ul>\n<li>To naively detect the digits, I did the following:<ul>\n<li>I resize to 1000x1000 to speed up the computation</li>\n<li>I set all 255 pixels to black and used the residual information (with some blurring and expand/contract filling) to create large contour blobs.</li>\n<li>I then found the bounding rectangle for each blob</li>\n<li>I cropped each 'digit' from the original 1000x1000 resized image using the blob bounding rectangle coordinates (injecting black padding if desired)</li>\n<li>I am currently creating a 64x64 pixel dataset for this</li></ul></li>\n<li>More sophisticated digit detection<ul>\n<li>Same as previously, except I apply Sector Cleaning as suggested by <a href=\"https://www.kaggle.com/lukaszborecki\" target=\"_blank\">Lucasz</a> <a href=\"https://www.kaggle.com/lukaszborecki/digit-cleaner-concept\" target=\"_blank\">in his notebook suggesting this approach.</a>.</li>\n<li>I also don't set all 255 pixels to black and I adjust some of my kernels to be less extreme.</li>\n<li>I am currently creating a 28x28 pixel dataset for this</li></ul></li>\n<li>What Pre-trained Models?<ul>\n<li><a href=\"https://tfhub.dev/tensorflow/tfgan/eval/mnist/logits/1\" target=\"_blank\">tfhubs tfgan model</a> used to validate tfgan mnist models</li>\n<li>This inference notebook is currently running and will be made public shortly after… after running a small validation I would expect an accuracy of 20-25% using this technique.</li>\n<li><a href=\"https://github.com/EscVM/Efficient-CapsNet\" target=\"_blank\">Efficient-CapsNet</a> MNIST model</li>\n<li>I have not validated this but will create and update an ablation study table in this post shortly.</li></ul></li>\n<li>What Post-Processing?<ul>\n<li>For my V1 dataset approach I was performing inference with both the original crop and the inverted crop and taking the label with the highest prediction confidence. (this helps as the models were probably only trained with the digit being one color and background being a different color).</li>\n<li>I simply take the individual digit crop predictions and sum them</li>\n<li>This seems to yield an MAE of around 4 and an accuracy of around 22-23%.</li>\n<li>For V2 dataset, I probably won't need to do the trick from V1 and will report on scores later</li>\n<li>I also clip all summed predictions to 27 (as that is the maximum possible value)… I tried submitting all 28s to validate and received a score of 0.</li>\n<li>I also think the test distribution is identical to the train distribution. This could open up the idea of some sort of post-processing to ensure a maximum of 1000 predictions per class… or something.</li>\n<li>I also think TTA and ensembling will be massively helpful</li>\n<li>I also think identifying edge cases where the OpenCV digit detection fails will be critical.</li></ul></li>\n</ul>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Ablation Study - Experiment Registry</strong></p>\n<p><br></p>\n<table>\n<thead>\n<tr>\n<th>Trial #</th>\n<th>Image Shape</th>\n<th>Experiment Name</th>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>28x28</td>\n<td>V1 Data Version &amp; TFHUB TFGAN Model</td>\n<td>22%</td>\n<td>25%</td>\n</tr>\n<tr>\n<td>2</td>\n<td>28x28</td>\n<td>V2 Data Version &amp; TFHUB TFGAN Model</td>\n<td>44%</td>\n<td>44%</td>\n</tr>\n<tr>\n<td>3</td>\n<td>28x28</td>\n<td>V3 Data Version w/ TTA &amp; TFHUB TFGAN Model</td>\n<td>71%</td>\n<td>70%</td>\n</tr>\n<tr>\n<td>4</td>\n<td>28x28</td>\n<td>V4 Data Version w/ TTA &amp; Capsule Network</td>\n<td>84%</td>\n<td>?</td>\n</tr>\n</tbody>\n</table>\n<ul>\n<li>V1 Data Version is my first attempt at using cv2.findContours and various thresholding techniques to identify blobs and crop the digits</li>\n<li>V2 Data Version is a more sophisticated approach that uses my original approach augmented w/  <a href=\"https://www.kaggle.com/lukaszborecki\" target=\"_blank\">Lucasz</a> <a href=\"https://www.kaggle.com/lukaszborecki/digit-cleaner-concept\" target=\"_blank\">notebook suggesting this approach</a></li>\n<li>V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result.</li>\n<li>V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result. I also switch to connected components but keep much of my original logic intact. I also perform TTA here with basic augmentations to elicit a more robust prediction.</li>\n<li>V4 Data Version includes updates from V1, V2 &amp; V3 but adds in additional heuristics to try and catch some of the mistakes from the previous iterations. (ie. rules around the minimum area, aspect ratio, etc.). I also perform TTA here with basic augmentations to elicit a more robust prediction.</li>\n</ul>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Relevant Notebooks</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-eda-baseline/settings\" target=\"_blank\">EDA Notebook</a></li>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation\" target=\"_blank\">Dataset Creation Notebook V1 - 64x64 - No Sector Cleaning</a></li>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation-v2\" target=\"_blank\">Dataset Creation Notebook V2 - 28x28 - W/ Sector Cleaning</a><ul>\n<li>Thanks to <a href=\"https://www.kaggle.com/lukaszborecki\" target=\"_blank\">Lucasz</a> <a href=\"https://www.kaggle.com/lukaszborecki/digit-cleaner-concept\" target=\"_blank\">notebook suggesting this approach</a>. Please give it an upvote.</li></ul></li>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-baseline-inference\" target=\"_blank\">Inference Notebook - Inline Using TFHUB - 28x28 - No Sector Cleaning</a></li>\n</ul>",
  "messages": [
    {
      "id": "1717410",
      "postDate": "03/09/2022 22:57:03",
      "content": "<p><strong>[UPDATED - MARCH 10, 2022]</strong></p>\n<hr>\n<p><br></p>\n<p>Hi all, I figured I would make a document detailing my approaches for this competition. </p>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Basic Approach</strong></p>\n<ul>\n<li>Use OpenCV to identify contours and crop and resize individual digits to a format expected by various pre-trained models</li>\n<li>Use pre-trained models (ensemble) to perform inference</li>\n<li>Post-process summation, clipping and other</li>\n</ul>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Details</strong></p>\n<ul>\n<li>To naively detect the digits, I did the following:<ul>\n<li>I resize to 1000x1000 to speed up the computation</li>\n<li>I set all 255 pixels to black and used the residual information (with some blurring and expand/contract filling) to create large contour blobs.</li>\n<li>I then found the bounding rectangle for each blob</li>\n<li>I cropped each 'digit' from the original 1000x1000 resized image using the blob bounding rectangle coordinates (injecting black padding if desired)</li>\n<li>I am currently creating a 64x64 pixel dataset for this</li></ul></li>\n<li>More sophisticated digit detection<ul>\n<li>Same as previously, except I apply Sector Cleaning as suggested by <a href=\"https://www.kaggle.com/lukaszborecki\" target=\"_blank\">Lucasz</a> <a href=\"https://www.kaggle.com/lukaszborecki/digit-cleaner-concept\" target=\"_blank\">in his notebook suggesting this approach.</a>.</li>\n<li>I also don't set all 255 pixels to black and I adjust some of my kernels to be less extreme.</li>\n<li>I am currently creating a 28x28 pixel dataset for this</li></ul></li>\n<li>What Pre-trained Models?<ul>\n<li><a href=\"https://tfhub.dev/tensorflow/tfgan/eval/mnist/logits/1\" target=\"_blank\">tfhubs tfgan model</a> used to validate tfgan mnist models</li>\n<li>This inference notebook is currently running and will be made public shortly after… after running a small validation I would expect an accuracy of 20-25% using this technique.</li>\n<li><a href=\"https://github.com/EscVM/Efficient-CapsNet\" target=\"_blank\">Efficient-CapsNet</a> MNIST model</li>\n<li>I have not validated this but will create and update an ablation study table in this post shortly.</li></ul></li>\n<li>What Post-Processing?<ul>\n<li>For my V1 dataset approach I was performing inference with both the original crop and the inverted crop and taking the label with the highest prediction confidence. (this helps as the models were probably only trained with the digit being one color and background being a different color).</li>\n<li>I simply take the individual digit crop predictions and sum them</li>\n<li>This seems to yield an MAE of around 4 and an accuracy of around 22-23%.</li>\n<li>For V2 dataset, I probably won't need to do the trick from V1 and will report on scores later</li>\n<li>I also clip all summed predictions to 27 (as that is the maximum possible value)… I tried submitting all 28s to validate and received a score of 0.</li>\n<li>I also think the test distribution is identical to the train distribution. This could open up the idea of some sort of post-processing to ensure a maximum of 1000 predictions per class… or something.</li>\n<li>I also think TTA and ensembling will be massively helpful</li>\n<li>I also think identifying edge cases where the OpenCV digit detection fails will be critical.</li></ul></li>\n</ul>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Ablation Study - Experiment Registry</strong></p>\n<p><br></p>\n<table>\n<thead>\n<tr>\n<th>Trial #</th>\n<th>Image Shape</th>\n<th>Experiment Name</th>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>28x28</td>\n<td>V1 Data Version &amp; TFHUB TFGAN Model</td>\n<td>22%</td>\n<td>25%</td>\n</tr>\n<tr>\n<td>2</td>\n<td>28x28</td>\n<td>V2 Data Version &amp; TFHUB TFGAN Model</td>\n<td>44%</td>\n<td>44%</td>\n</tr>\n<tr>\n<td>3</td>\n<td>28x28</td>\n<td>V3 Data Version w/ TTA &amp; TFHUB TFGAN Model</td>\n<td>71%</td>\n<td>70%</td>\n</tr>\n<tr>\n<td>4</td>\n<td>28x28</td>\n<td>V4 Data Version w/ TTA &amp; Capsule Network</td>\n<td>84%</td>\n<td>?</td>\n</tr>\n</tbody>\n</table>\n<ul>\n<li>V1 Data Version is my first attempt at using cv2.findContours and various thresholding techniques to identify blobs and crop the digits</li>\n<li>V2 Data Version is a more sophisticated approach that uses my original approach augmented w/  <a href=\"https://www.kaggle.com/lukaszborecki\" target=\"_blank\">Lucasz</a> <a href=\"https://www.kaggle.com/lukaszborecki/digit-cleaner-concept\" target=\"_blank\">notebook suggesting this approach</a></li>\n<li>V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result.</li>\n<li>V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result. I also switch to connected components but keep much of my original logic intact. I also perform TTA here with basic augmentations to elicit a more robust prediction.</li>\n<li>V4 Data Version includes updates from V1, V2 &amp; V3 but adds in additional heuristics to try and catch some of the mistakes from the previous iterations. (ie. rules around the minimum area, aspect ratio, etc.). I also perform TTA here with basic augmentations to elicit a more robust prediction.</li>\n</ul>\n<p><br></p>\n<hr>\n<p><br></p>\n<p><strong>Relevant Notebooks</strong></p>\n<ul>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-eda-baseline/settings\" target=\"_blank\">EDA Notebook</a></li>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation\" target=\"_blank\">Dataset Creation Notebook V1 - 64x64 - No Sector Cleaning</a></li>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation-v2\" target=\"_blank\">Dataset Creation Notebook V2 - 28x28 - W/ Sector Cleaning</a><ul>\n<li>Thanks to <a href=\"https://www.kaggle.com/lukaszborecki\" target=\"_blank\">Lucasz</a> <a href=\"https://www.kaggle.com/lukaszborecki/digit-cleaner-concept\" target=\"_blank\">notebook suggesting this approach</a>. Please give it an upvote.</li></ul></li>\n<li><a href=\"https://www.kaggle.com/dschettler8845/ultramnist-baseline-inference\" target=\"_blank\">Inference Notebook - Inline Using TFHUB - 28x28 - No Sector Cleaning</a></li>\n</ul>",
      "rawMarkdown": "**[UPDATED - MARCH 10, 2022]**\n\n---\n\n<br>\n\nHi all, I figured I would make a document detailing my approaches for this competition. \n\n<br>\n\n---\n\n<br>\n\n**Basic Approach**\n* Use OpenCV to identify contours and crop and resize individual digits to a format expected by various pre-trained models\n* Use pre-trained models (ensemble) to perform inference\n* Post-process summation, clipping and other\n\n<br>\n\n---\n\n<br>\n\n**Details**\n* To naively detect the digits, I did the following:\n  * I resize to 1000x1000 to speed up the computation\n  * I set all 255 pixels to black and used the residual information (with some blurring and expand/contract filling) to create large contour blobs.\n  * I then found the bounding rectangle for each blob\n  * I cropped each 'digit' from the original 1000x1000 resized image using the blob bounding rectangle coordinates (injecting black padding if desired)\n    * I am currently creating a 64x64 pixel dataset for this\n* More sophisticated digit detection\n  * Same as previously, except I apply Sector Cleaning as suggested by [Lucasz](https://www.kaggle.com/lukaszborecki) [in his notebook suggesting this approach.](https://www.kaggle.com/lukaszborecki/digit-cleaner-concept).\n  * I also don't set all 255 pixels to black and I adjust some of my kernels to be less extreme.\n  * I am currently creating a 28x28 pixel dataset for this\n* What Pre-trained Models?\n  * [tfhubs tfgan model](https://tfhub.dev/tensorflow/tfgan/eval/mnist/logits/1) used to validate tfgan mnist models\n    * This inference notebook is currently running and will be made public shortly after... after running a small validation I would expect an accuracy of 20-25% using this technique.\n  * [Efficient-CapsNet](https://github.com/EscVM/Efficient-CapsNet) MNIST model\n    * I have not validated this but will create and update an ablation study table in this post shortly.\n* What Post-Processing?\n  * For my V1 dataset approach I was performing inference with both the original crop and the inverted crop and taking the label with the highest prediction confidence. (this helps as the models were probably only trained with the digit being one color and background being a different color).\n    * I simply take the individual digit crop predictions and sum them\n    * This seems to yield an MAE of around 4 and an accuracy of around 22-23%.\n  * For V2 dataset, I probably won't need to do the trick from V1 and will report on scores later\n  * I also clip all summed predictions to 27 (as that is the maximum possible value)... I tried submitting all 28s to validate and received a score of 0.\n  * I also think the test distribution is identical to the train distribution. This could open up the idea of some sort of post-processing to ensure a maximum of 1000 predictions per class... or something.\n  * I also think TTA and ensembling will be massively helpful\n  * I also think identifying edge cases where the OpenCV digit detection fails will be critical.\n   \n<br>\n\n ---\n\n<br>\n\n**Ablation Study - Experiment Registry**\n\n<br>\n\n| Trial # | Image Shape | Experiment Name | CV | LB |\n| --- | --- | --- | --- |\n| 1 | 28x28 | V1 Data Version & TFHUB TFGAN Model | 22% | 25% |\n| 2 | 28x28 | V2 Data Version & TFHUB TFGAN Model | 44% | 44% |\n| 3 | 28x28 | V3 Data Version w/ TTA & TFHUB TFGAN Model | 71% | 70% |\n| 4 | 28x28 | V4 Data Version w/ TTA & Capsule Network | 84% | ? |\n\n* V1 Data Version is my first attempt at using cv2.findContours and various thresholding techniques to identify blobs and crop the digits\n* V2 Data Version is a more sophisticated approach that uses my original approach augmented w/  [Lucasz](https://www.kaggle.com/lukaszborecki) [notebook suggesting this approach](https://www.kaggle.com/lukaszborecki/digit-cleaner-concept)\n* V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result.\n* V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result. I also switch to connected components but keep much of my original logic intact. I also perform TTA here with basic augmentations to elicit a more robust prediction.\n* V4 Data Version includes updates from V1, V2 & V3 but adds in additional heuristics to try and catch some of the mistakes from the previous iterations. (ie. rules around the minimum area, aspect ratio, etc.). I also perform TTA here with basic augmentations to elicit a more robust prediction.\n\n<br>\n\n ---\n\n<br>\n\n**Relevant Notebooks**\n* [EDA Notebook](https://www.kaggle.com/dschettler8845/ultramnist-eda-baseline/settings)\n* [Dataset Creation Notebook V1 - 64x64 - No Sector Cleaning](https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation)\n* [Dataset Creation Notebook V2 - 28x28 - W/ Sector Cleaning](https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation-v2)\n  * Thanks to [Lucasz](https://www.kaggle.com/lukaszborecki) [notebook suggesting this approach](https://www.kaggle.com/lukaszborecki/digit-cleaner-concept). Please give it an upvote.\n* [Inference Notebook - Inline Using TFHUB - 28x28 - No Sector Cleaning](https://www.kaggle.com/dschettler8845/ultramnist-baseline-inference)",
      "votes": null
    },
    {
      "id": "1723167",
      "postDate": "03/15/2022 07:18:12",
      "content": "<p>Thanks for your kind help and efforts </p>",
      "rawMarkdown": "Thanks for your kind help and efforts",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1723167,
      "author_name": "gazu468",
      "author_url": "",
      "post_date": "03/15/2022 07:18:12",
      "content": "<p>Thanks for your kind help and efforts </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1717410": "**[UPDATED - MARCH 10, 2022]**\n\n---\n\n<br>\n\nHi all, I figured I would make a document detailing my approaches for this competition. \n\n<br>\n\n---\n\n<br>\n\n**Basic Approach**\n* Use OpenCV to identify contours and crop and resize individual digits to a format expected by various pre-trained models\n* Use pre-trained models (ensemble) to perform inference\n* Post-process summation, clipping and other\n\n<br>\n\n---\n\n<br>\n\n**Details**\n* To naively detect the digits, I did the following:\n  * I resize to 1000x1000 to speed up the computation\n  * I set all 255 pixels to black and used the residual information (with some blurring and expand/contract filling) to create large contour blobs.\n  * I then found the bounding rectangle for each blob\n  * I cropped each 'digit' from the original 1000x1000 resized image using the blob bounding rectangle coordinates (injecting black padding if desired)\n    * I am currently creating a 64x64 pixel dataset for this\n* More sophisticated digit detection\n  * Same as previously, except I apply Sector Cleaning as suggested by [Lucasz](https://www.kaggle.com/lukaszborecki) [in his notebook suggesting this approach.](https://www.kaggle.com/lukaszborecki/digit-cleaner-concept).\n  * I also don't set all 255 pixels to black and I adjust some of my kernels to be less extreme.\n  * I am currently creating a 28x28 pixel dataset for this\n* What Pre-trained Models?\n  * [tfhubs tfgan model](https://tfhub.dev/tensorflow/tfgan/eval/mnist/logits/1) used to validate tfgan mnist models\n    * This inference notebook is currently running and will be made public shortly after... after running a small validation I would expect an accuracy of 20-25% using this technique.\n  * [Efficient-CapsNet](https://github.com/EscVM/Efficient-CapsNet) MNIST model\n    * I have not validated this but will create and update an ablation study table in this post shortly.\n* What Post-Processing?\n  * For my V1 dataset approach I was performing inference with both the original crop and the inverted crop and taking the label with the highest prediction confidence. (this helps as the models were probably only trained with the digit being one color and background being a different color).\n    * I simply take the individual digit crop predictions and sum them\n    * This seems to yield an MAE of around 4 and an accuracy of around 22-23%.\n  * For V2 dataset, I probably won't need to do the trick from V1 and will report on scores later\n  * I also clip all summed predictions to 27 (as that is the maximum possible value)... I tried submitting all 28s to validate and received a score of 0.\n  * I also think the test distribution is identical to the train distribution. This could open up the idea of some sort of post-processing to ensure a maximum of 1000 predictions per class... or something.\n  * I also think TTA and ensembling will be massively helpful\n  * I also think identifying edge cases where the OpenCV digit detection fails will be critical.\n   \n<br>\n\n ---\n\n<br>\n\n**Ablation Study - Experiment Registry**\n\n<br>\n\n| Trial # | Image Shape | Experiment Name | CV | LB |\n| --- | --- | --- | --- |\n| 1 | 28x28 | V1 Data Version & TFHUB TFGAN Model | 22% | 25% |\n| 2 | 28x28 | V2 Data Version & TFHUB TFGAN Model | 44% | 44% |\n| 3 | 28x28 | V3 Data Version w/ TTA & TFHUB TFGAN Model | 71% | 70% |\n| 4 | 28x28 | V4 Data Version w/ TTA & Capsule Network | 84% | ? |\n\n* V1 Data Version is my first attempt at using cv2.findContours and various thresholding techniques to identify blobs and crop the digits\n* V2 Data Version is a more sophisticated approach that uses my original approach augmented w/  [Lucasz](https://www.kaggle.com/lukaszborecki) [notebook suggesting this approach](https://www.kaggle.com/lukaszborecki/digit-cleaner-concept)\n* V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result.\n* V3 Data Version includes updates from V1 and V2 but uses smarter logic around centring the digit, padding with black and cropping to ascertain a better result. I also switch to connected components but keep much of my original logic intact. I also perform TTA here with basic augmentations to elicit a more robust prediction.\n* V4 Data Version includes updates from V1, V2 & V3 but adds in additional heuristics to try and catch some of the mistakes from the previous iterations. (ie. rules around the minimum area, aspect ratio, etc.). I also perform TTA here with basic augmentations to elicit a more robust prediction.\n\n<br>\n\n ---\n\n<br>\n\n**Relevant Notebooks**\n* [EDA Notebook](https://www.kaggle.com/dschettler8845/ultramnist-eda-baseline/settings)\n* [Dataset Creation Notebook V1 - 64x64 - No Sector Cleaning](https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation)\n* [Dataset Creation Notebook V2 - 28x28 - W/ Sector Cleaning](https://www.kaggle.com/dschettler8845/ultramnist-dataset-creation-v2)\n  * Thanks to [Lucasz](https://www.kaggle.com/lukaszborecki) [notebook suggesting this approach](https://www.kaggle.com/lukaszborecki/digit-cleaner-concept). Please give it an upvote.\n* [Inference Notebook - Inline Using TFHUB - 28x28 - No Sector Cleaning](https://www.kaggle.com/dschettler8845/ultramnist-baseline-inference)",
    "1723167": "Thanks for your kind help and efforts"
  },
  "source": "meta"
}