{
  "id": 19631,
  "title": "34th place solution, end-to-end learning with a single neural net. ",
  "url": "/competitions/second-annual-data-science-bowl/writeups/florian-muellerklein-34th-place-solution-end-to-en",
  "author_name": "",
  "post_date": "2016-03-18T11:18:56.870Z",
  "votes": 4,
  "comment_count": 9,
  "views": 1890,
  "content": "<p>First of all, congrats to the winners and all of the competitors. Lots of really great performances! </p>\n\n<p>Also, big big thanks to  Marko Jocic, Bing Xu, and the Booz Allen Hamilton/NVIDIA Team. I joined this competition with only a month left and relied on those starter scripts and tips really heavily. </p>\n\n<p><a href=\"http://florianmuellerklein.github.io/DSB/\">http://florianmuellerklein.github.io/DSB/</a></p>",
  "messages": [
    {
      "id": "112141",
      "postDate": "03/18/2016 11:18:56",
      "content": "<p>First of all, congrats to the winners and all of the competitors. Lots of really great performances! </p>\n\n<p>Also, big big thanks to  Marko Jocic, Bing Xu, and the Booz Allen Hamilton/NVIDIA Team. I joined this competition with only a month left and relied on those starter scripts and tips really heavily. </p>\n\n<p><a href=\"http://florianmuellerklein.github.io/DSB/\">http://florianmuellerklein.github.io/DSB/</a></p>",
      "rawMarkdown": "First of all, congrats to the winners and all of the competitors. Lots of really great performances! \r\n\r\nAlso, big big thanks to  Marko Jocic, Bing Xu, and the Booz Allen Hamilton/NVIDIA Team. I joined this competition with only a month left and relied on those starter scripts and tips really heavily. \r\n\r\nhttp://florianmuellerklein.github.io/DSB/",
      "votes": null
    },
    {
      "id": "112171",
      "postDate": "03/18/2016 14:52:15",
      "content": "<p>Thank you Florian! Have you published your code?</p>",
      "rawMarkdown": "Thank you Florian! Have you published your code?",
      "votes": null
    },
    {
      "id": "112182",
      "postDate": "03/18/2016 16:21:09",
      "content": "<p>@rcarson</p>\n\n<p>I don't really feel right posting my code since its pretty much just Marko's Keras tutorial with the changes that I outlined in my post. It should be easy enough to just make the same tweaks. </p>",
      "rawMarkdown": "rcarson\r\n\r\nI don't really feel right posting my code since its pretty much just Marko's Keras tutorial with the changes that I outlined in my post. It should be easy enough to just make the same tweaks.",
      "votes": null
    },
    {
      "id": "112210",
      "postDate": "03/18/2016 20:40:25",
      "content": "<p>@Florian, <br>\nYes, Many Thanks to @Marko Jocic, @Bing Xu, and the Booz Allen Hamilton/NVIDIA Team, I've followed the same path as you. Here is my quick summary from my notes(in case people wants to use these samples):\nmxnet sample ~ 0.039, I have tried augmentation but couldn't able to manage it easily\nKeras sample ~ 0.033 (around 230 iteration)\nProcess Input Images to Get ROI: Used Fourier tutor code to find the center and cropped 20 cm squares:\nthat gave ~ 0.025\nAdded extra Layer in sample ~ 0.0227\nUpgrade the model to something similar to VGG (smaller version very similar to yours) ~ 0.022 after iteration 90. I tried various higher droputs /L2 regularization but cannot able to avoid overfitting so never really managed to go under .02. I thought about appling Patient orientation which @ZTurbo explained how really to do this in another post but I couldn't able to solve it in time.</p>",
      "rawMarkdown": "Florian,  \r\nYes, Many Thanks to @Marko Jocic, @Bing Xu, and the Booz Allen Hamilton/NVIDIA Team, I've followed the same path as you. Here is my quick summary from my notes(in case people wants to use these samples):\r\nmxnet sample ~ 0.039, I have tried augmentation but couldn't able to manage it easily\r\nKeras sample ~ 0.033 (around 230 iteration)\r\nProcess Input Images to Get ROI: Used Fourier tutor code to find the center and cropped 20 cm squares:\r\nthat gave ~ 0.025\r\nAdded extra Layer in sample ~ 0.0227\r\nUpgrade the model to something similar to VGG (smaller version very similar to yours) ~ 0.022 after iteration 90. I tried various higher droputs /L2 regularization but cannot able to avoid overfitting so never really managed to go under .02. I thought about appling Patient orientation which @ZTurbo explained how really to do this in another post but I couldn't able to solve it in time.",
      "votes": null
    },
    {
      "id": "112227",
      "postDate": "03/18/2016 21:46:49",
      "content": "<p>Awesome, thanks! How did you combine the outputs of the model to make predictions about a patient; did you just take the average of the output for all the patients SAX slices? Also, you said you used zero padding to make sure the output feature maps were the same size as the input feature maps; why was making sure the input and output feature maps were the same size important?</p>",
      "rawMarkdown": "Awesome, thanks! How did you combine the outputs of the model to make predictions about a patient; did you just take the average of the output for all the patients SAX slices? Also, you said you used zero padding to make sure the output feature maps were the same size as the input feature maps; why was making sure the input and output feature maps were the same size important?",
      "votes": null
    },
    {
      "id": "112239",
      "postDate": "03/19/2016 00:14:39",
      "content": "<p>@Alex Risman</p>\n\n<p>For the padding, lets say you have an input size 7x7 and without padding your output would be 5x5. So when you try to stack a whole bunch of convolution layers together you'll continuously decrease your image size before you really want to. So you could miss out on some convolution layers.</p>\n\n<p>Combining the the outputs it was just a simple average. </p>",
      "rawMarkdown": "Alex Risman\r\n\r\nFor the padding, lets say you have an input size 7x7 and without padding your output would be 5x5. So when you try to stack a whole bunch of convolution layers together you'll continuously decrease your image size before you really want to. So you could miss out on some convolution layers.\r\n\r\nCombining the the outputs it was just a simple average.",
      "votes": null
    },
    {
      "id": "112425",
      "postDate": "03/21/2016 05:58:02",
      "content": "<p>Thanks!</p>",
      "rawMarkdown": "Thanks!",
      "votes": null
    },
    {
      "id": "113444",
      "postDate": "03/31/2016 23:11:38",
      "content": "<p>@Ertuka,\nNice job! You mentioned you used the Keras tutorial, and you also used Fourier tutorial code to find the ROI. I've been using the Keras tutorial as a base as well, and it takes the 30 time steps for each image, but I noticed the ROI code only returns one ROI image for each slice, not 30. Are you just using the 1 ROI image per slice or did you find a way to get the ROI in each time step?\nThanks,\nAlex</p>",
      "rawMarkdown": "Ertuka,\r\nNice job! You mentioned you used the Keras tutorial, and you also used Fourier tutorial code to find the ROI. I've been using the Keras tutorial as a base as well, and it takes the 30 time steps for each image, but I noticed the ROI code only returns one ROI image for each slice, not 30. Are you just using the 1 ROI image per slice or did you find a way to get the ROI in each time step?\r\nThanks,\r\nAlex",
      "votes": null
    },
    {
      "id": "113453",
      "postDate": "04/01/2016 03:36:25",
      "content": "<p>@Alex,\nI've modified Fourier tutorial to handle 30 timeslices, I have attached my patch applied to segment.py from Fourier tutorial. There are couple slices fail with this, either due to different sizes or near the end, I ignored them since i didn't have time to write special handling code. Probably processing them could give better results.</p>",
      "rawMarkdown": "Alex,\r\nI've modified Fourier tutorial to handle 30 timeslices, I have attached my patch applied to segment.py from Fourier tutorial. There are couple slices fail with this, either due to different sizes or near the end, I ignored them since i didn't have time to write special handling code. Probably processing them could give better results.",
      "votes": null
    },
    {
      "id": "113765",
      "postDate": "04/04/2016 21:02:40",
      "content": "<p>@Ertuka,\nAwesome, thanks!</p>",
      "rawMarkdown": "Ertuka,\r\nAwesome, thanks!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 112171,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "03/18/2016 14:52:15",
      "content": "<p>Thank you Florian! Have you published your code?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 112182,
      "author_name": "florianm",
      "author_url": "",
      "post_date": "03/18/2016 16:21:09",
      "content": "<p>@rcarson</p>\n\n<p>I don't really feel right posting my code since its pretty much just Marko's Keras tutorial with the changes that I outlined in my post. It should be easy enough to just make the same tweaks. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 112210,
      "author_name": "esorar",
      "author_url": "",
      "post_date": "03/18/2016 20:40:25",
      "content": "<p>@Florian, <br>\nYes, Many Thanks to @Marko Jocic, @Bing Xu, and the Booz Allen Hamilton/NVIDIA Team, I've followed the same path as you. Here is my quick summary from my notes(in case people wants to use these samples):\nmxnet sample ~ 0.039, I have tried augmentation but couldn't able to manage it easily\nKeras sample ~ 0.033 (around 230 iteration)\nProcess Input Images to Get ROI: Used Fourier tutor code to find the center and cropped 20 cm squares:\nthat gave ~ 0.025\nAdded extra Layer in sample ~ 0.0227\nUpgrade the model to something similar to VGG (smaller version very similar to yours) ~ 0.022 after iteration 90. I tried various higher droputs /L2 regularization but cannot able to avoid overfitting so never really managed to go under .02. I thought about appling Patient orientation which @ZTurbo explained how really to do this in another post but I couldn't able to solve it in time.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 112227,
      "author_name": "kaggalex",
      "author_url": "",
      "post_date": "03/18/2016 21:46:49",
      "content": "<p>Awesome, thanks! How did you combine the outputs of the model to make predictions about a patient; did you just take the average of the output for all the patients SAX slices? Also, you said you used zero padding to make sure the output feature maps were the same size as the input feature maps; why was making sure the input and output feature maps were the same size important?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 112239,
      "author_name": "florianm",
      "author_url": "",
      "post_date": "03/19/2016 00:14:39",
      "content": "<p>@Alex Risman</p>\n\n<p>For the padding, lets say you have an input size 7x7 and without padding your output would be 5x5. So when you try to stack a whole bunch of convolution layers together you'll continuously decrease your image size before you really want to. So you could miss out on some convolution layers.</p>\n\n<p>Combining the the outputs it was just a simple average. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 112425,
      "author_name": "kaggalex",
      "author_url": "",
      "post_date": "03/21/2016 05:58:02",
      "content": "<p>Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 113444,
      "author_name": "kaggalex",
      "author_url": "",
      "post_date": "03/31/2016 23:11:38",
      "content": "<p>@Ertuka,\nNice job! You mentioned you used the Keras tutorial, and you also used Fourier tutorial code to find the ROI. I've been using the Keras tutorial as a base as well, and it takes the 30 time steps for each image, but I noticed the ROI code only returns one ROI image for each slice, not 30. Are you just using the 1 ROI image per slice or did you find a way to get the ROI in each time step?\nThanks,\nAlex</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 113453,
      "author_name": "esorar",
      "author_url": "",
      "post_date": "04/01/2016 03:36:25",
      "content": "<p>@Alex,\nI've modified Fourier tutorial to handle 30 timeslices, I have attached my patch applied to segment.py from Fourier tutorial. There are couple slices fail with this, either due to different sizes or near the end, I ignored them since i didn't have time to write special handling code. Probably processing them could give better results.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 113765,
      "author_name": "kaggalex",
      "author_url": "",
      "post_date": "04/04/2016 21:02:40",
      "content": "<p>@Ertuka,\nAwesome, thanks!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "112141": "First of all, congrats to the winners and all of the competitors. Lots of really great performances! \r\n\r\nAlso, big big thanks to  Marko Jocic, Bing Xu, and the Booz Allen Hamilton/NVIDIA Team. I joined this competition with only a month left and relied on those starter scripts and tips really heavily. \r\n\r\nhttp://florianmuellerklein.github.io/DSB/",
    "112171": "Thank you Florian! Have you published your code?",
    "112182": "rcarson\r\n\r\nI don't really feel right posting my code since its pretty much just Marko's Keras tutorial with the changes that I outlined in my post. It should be easy enough to just make the same tweaks.",
    "112210": "Florian,  \r\nYes, Many Thanks to @Marko Jocic, @Bing Xu, and the Booz Allen Hamilton/NVIDIA Team, I've followed the same path as you. Here is my quick summary from my notes(in case people wants to use these samples):\r\nmxnet sample ~ 0.039, I have tried augmentation but couldn't able to manage it easily\r\nKeras sample ~ 0.033 (around 230 iteration)\r\nProcess Input Images to Get ROI: Used Fourier tutor code to find the center and cropped 20 cm squares:\r\nthat gave ~ 0.025\r\nAdded extra Layer in sample ~ 0.0227\r\nUpgrade the model to something similar to VGG (smaller version very similar to yours) ~ 0.022 after iteration 90. I tried various higher droputs /L2 regularization but cannot able to avoid overfitting so never really managed to go under .02. I thought about appling Patient orientation which @ZTurbo explained how really to do this in another post but I couldn't able to solve it in time.",
    "112227": "Awesome, thanks! How did you combine the outputs of the model to make predictions about a patient; did you just take the average of the output for all the patients SAX slices? Also, you said you used zero padding to make sure the output feature maps were the same size as the input feature maps; why was making sure the input and output feature maps were the same size important?",
    "112239": "Alex Risman\r\n\r\nFor the padding, lets say you have an input size 7x7 and without padding your output would be 5x5. So when you try to stack a whole bunch of convolution layers together you'll continuously decrease your image size before you really want to. So you could miss out on some convolution layers.\r\n\r\nCombining the the outputs it was just a simple average.",
    "112425": "Thanks!",
    "113444": "Ertuka,\r\nNice job! You mentioned you used the Keras tutorial, and you also used Fourier tutorial code to find the ROI. I've been using the Keras tutorial as a base as well, and it takes the 30 time steps for each image, but I noticed the ROI code only returns one ROI image for each slice, not 30. Are you just using the 1 ROI image per slice or did you find a way to get the ROI in each time step?\r\nThanks,\r\nAlex",
    "113453": "Alex,\r\nI've modified Fourier tutorial to handle 30 timeslices, I have attached my patch applied to segment.py from Fourier tutorial. There are couple slices fail with this, either due to different sizes or near the end, I ignored them since i didn't have time to write special handling code. Probably processing them could give better results.",
    "113765": "Ertuka,\r\nAwesome, thanks!"
  },
  "source": "meta"
}