{
  "id": 56858,
  "title": "Go-based starter kit",
  "url": "/competitions/trackml-particle-identification/discussion/56858",
  "author_name": "",
  "post_date": "2018-05-15T22:22:50.996025300Z",
  "votes": 7,
  "comment_count": 2,
  "views": 0,
  "content": "<p>hi there,</p>\n\n<p>I have just translated the <a href=\"https://github.com/LAL/trackml-library\">LAL/trackml-library</a> python starter kit into a <a href=\"https://golang.org/\">Go</a>-based one:</p>\n\n<ul>\n<li><a href=\"https://github.com/sbinet/go-trackml\">https://github.com/sbinet/go-trackml</a></li>\n</ul>\n\n<p>As stated on the <code>README</code>, I get nice performance improvements:</p>\n\n<h2>Python version</h2>\n\n<pre><code>$&gt; time python trkml-hough.py ./example_standard/dataset/event000000200\n   hit_id          x         y       z  volume_id  layer_id  module_id\n0       1 -62.663200  -3.05090 -1502.5          7         2          1\n1       2 -66.124702  -1.36730 -1502.5          7         2          1\n2       3 -63.697701   1.73267 -1502.5          7         2          1\n3       4 -82.501801 -14.09150 -1502.5          7         2          1\n4       5 -74.343399   0.84469 -1502.5          7         2          1\nYour score:  0.13153644878592863\nScore for event 200: 0.132\nScore for event 201: 0.133\nMean score: 0.132\n\nreal  7m12.351s\nuser  7m10.400s\nsys   0m0.828s\n</code></pre>\n\n<h2>Go version (single-threaded)</h2>\n\n<pre><code>$&gt; time trkml-hough ./example_standard/dataset/event000000200\ntrkml-hough: loading [./example_standard/dataset/event000000200]...\ntrkml-hough: loading [./example_standard/dataset/event000000200]... [done]\ntrkml-hough: score for event 200: 0.1316012364071201\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"...\ntrkml-hough: score for event 200: 0.1316012364071201\ntrkml-hough: score for event 201: 0.1332602513710427\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\ntrkml-hough: mean score: 0.13243074388908138\n\nreal  1m21.033s\nuser  1m22.541s\nsys   0m0.569s\n</code></pre>\n\n<h2>Go version (multi-threaded)</h2>\n\n<pre><code>$&gt; time trkml-hough -ncpus=-1 ./example_standard/dataset/event000000200\ntrkml-hough: loading [./example_standard/dataset/event000000200]...\ntrkml-hough: loading [./example_standard/dataset/event000000200]... [done]\ntrkml-hough: score for event 200: 0.13160123640712013\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"...\ntrkml-hough: score for event 200: 0.1316012364071201\ntrkml-hough: score for event 201: 0.13326025137104267\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\ntrkml-hough: mean score: 0.13243074388908138\n\nreal  0m30.081s\nuser  1m26.658s\nsys   0m0.741s\n</code></pre>\n\n<p>hth,</p>\n\n<p>-s</p>",
  "messages": [
    {
      "id": "329164",
      "postDate": "05/15/2018 22:22:50",
      "content": "<p>hi there,</p>\n\n<p>I have just translated the <a href=\"https://github.com/LAL/trackml-library\">LAL/trackml-library</a> python starter kit into a <a href=\"https://golang.org/\">Go</a>-based one:</p>\n\n<ul>\n<li><a href=\"https://github.com/sbinet/go-trackml\">https://github.com/sbinet/go-trackml</a></li>\n</ul>\n\n<p>As stated on the <code>README</code>, I get nice performance improvements:</p>\n\n<h2>Python version</h2>\n\n<pre><code>$&gt; time python trkml-hough.py ./example_standard/dataset/event000000200\n   hit_id          x         y       z  volume_id  layer_id  module_id\n0       1 -62.663200  -3.05090 -1502.5          7         2          1\n1       2 -66.124702  -1.36730 -1502.5          7         2          1\n2       3 -63.697701   1.73267 -1502.5          7         2          1\n3       4 -82.501801 -14.09150 -1502.5          7         2          1\n4       5 -74.343399   0.84469 -1502.5          7         2          1\nYour score:  0.13153644878592863\nScore for event 200: 0.132\nScore for event 201: 0.133\nMean score: 0.132\n\nreal  7m12.351s\nuser  7m10.400s\nsys   0m0.828s\n</code></pre>\n\n<h2>Go version (single-threaded)</h2>\n\n<pre><code>$&gt; time trkml-hough ./example_standard/dataset/event000000200\ntrkml-hough: loading [./example_standard/dataset/event000000200]...\ntrkml-hough: loading [./example_standard/dataset/event000000200]... [done]\ntrkml-hough: score for event 200: 0.1316012364071201\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"...\ntrkml-hough: score for event 200: 0.1316012364071201\ntrkml-hough: score for event 201: 0.1332602513710427\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\ntrkml-hough: mean score: 0.13243074388908138\n\nreal  1m21.033s\nuser  1m22.541s\nsys   0m0.569s\n</code></pre>\n\n<h2>Go version (multi-threaded)</h2>\n\n<pre><code>$&gt; time trkml-hough -ncpus=-1 ./example_standard/dataset/event000000200\ntrkml-hough: loading [./example_standard/dataset/event000000200]...\ntrkml-hough: loading [./example_standard/dataset/event000000200]... [done]\ntrkml-hough: score for event 200: 0.13160123640712013\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"...\ntrkml-hough: score for event 200: 0.1316012364071201\ntrkml-hough: score for event 201: 0.13326025137104267\ntrkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\ntrkml-hough: mean score: 0.13243074388908138\n\nreal  0m30.081s\nuser  1m26.658s\nsys   0m0.741s\n</code></pre>\n\n<p>hth,</p>\n\n<p>-s</p>",
      "rawMarkdown": "hi there,\n\nI have just translated the [LAL/trackml-library](https://github.com/LAL/trackml-library) python starter kit into a [Go](https://golang.org)-based one:\n\n- https://github.com/sbinet/go-trackml\n\nAs stated on the `README`, I get nice performance improvements:\n\n## Python version\n\n    $&gt; time python trkml-hough.py ./example_standard/dataset/event000000200\n       hit_id          x         y       z  volume_id  layer_id  module_id\n    0       1 -62.663200  -3.05090 -1502.5          7         2          1\n    1       2 -66.124702  -1.36730 -1502.5          7         2          1\n    2       3 -63.697701   1.73267 -1502.5          7         2          1\n    3       4 -82.501801 -14.09150 -1502.5          7         2          1\n    4       5 -74.343399   0.84469 -1502.5          7         2          1\n    Your score:  0.13153644878592863\n    Score for event 200: 0.132\n    Score for event 201: 0.133\n    Mean score: 0.132\n    \n    real  7m12.351s\n    user  7m10.400s\n    sys   0m0.828s\n\n## Go version (single-threaded)\n\n    $&gt; time trkml-hough ./example_standard/dataset/event000000200\n    trkml-hough: loading [./example_standard/dataset/event000000200]...\n    trkml-hough: loading [./example_standard/dataset/event000000200]... [done]\n    trkml-hough: score for event 200: 0.1316012364071201\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"...\n    trkml-hough: score for event 200: 0.1316012364071201\n    trkml-hough: score for event 201: 0.1332602513710427\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\n    trkml-hough: mean score: 0.13243074388908138\n    \n    real  1m21.033s\n    user  1m22.541s\n    sys   0m0.569s\n\n## Go version (multi-threaded)\n\n    $&gt; time trkml-hough -ncpus=-1 ./example_standard/dataset/event000000200\n    trkml-hough: loading [./example_standard/dataset/event000000200]...\n    trkml-hough: loading [./example_standard/dataset/event000000200]... [done]\n    trkml-hough: score for event 200: 0.13160123640712013\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"...\n    trkml-hough: score for event 200: 0.1316012364071201\n    trkml-hough: score for event 201: 0.13326025137104267\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\n    trkml-hough: mean score: 0.13243074388908138\n    \n    real  0m30.081s\n    user  1m26.658s\n    sys   0m0.741s\n\nhth,\n\n-s",
      "votes": null
    },
    {
      "id": "329484",
      "postDate": "05/16/2018 15:16:31",
      "content": "<p>Latest version <a href=\"https://github.com/sbinet/go-trackml/commit/d4dc0b32d0e17da75006cae51b4f6c8182ac7805\">d4dc0b3</a> got:</p>\n\n<ul>\n<li>support for ZIP datasets</li>\n<li>creation of <code>submission.csv.gz</code> files</li>\n</ul>\n\n<p>Enjoy :)</p>",
      "rawMarkdown": "Latest version [d4dc0b3](https://github.com/sbinet/go-trackml/commit/d4dc0b32d0e17da75006cae51b4f6c8182ac7805) got:\n\n- support for ZIP datasets\n- creation of `submission.csv.gz` files\n\nEnjoy :)",
      "votes": null
    },
    {
      "id": "329533",
      "postDate": "05/16/2018 16:55:31",
      "content": "<p>Version <a href=\"https://github.com/sbinet/go-trackml/tree/v0.1.1\">v0.1.1</a> clicks at <code>0.1445</code>.</p>\n\n<p>Interestingly, running the full procedure on my desktop gives:</p>\n\n<pre><code>$&gt; time trkml-hough -ncpus=-1 -submit ./train_sample.zip event000001000 ./test.zip \n[...]\ntrkml-hough: processing event 124...\ntrkml-hough: loading test dataset \"./test.zip\"... [done]\n\nreal    19m40.461s\nuser    57m14.608s\nsys      0m10.350s\n</code></pre>\n\n<p>running on server with twice the number of cores:</p>\n\n<pre><code>real     11m35.166s\nuser    101m36.791s\nsys       0m16.070s\n</code></pre>\n\n<p>so the scaling is quite OK, even if there's still room for improvement (<em>e.g.</em> the <code>Dataset</code> iterator could start to unpack the next event's data once the <code>Next()</code> method has been called, or even keep a pre-fetch collection of events)</p>",
      "rawMarkdown": "Version [v0.1.1](https://github.com/sbinet/go-trackml/tree/v0.1.1) clicks at `0.1445`.\n\nInterestingly, running the full procedure on my desktop gives:\n\n\n    $&gt; time trkml-hough -ncpus=-1 -submit ./train_sample.zip event000001000 ./test.zip \n    [...]\n    trkml-hough: processing event 124...\n    trkml-hough: loading test dataset \"./test.zip\"... [done]\n    \n    real\t19m40.461s\n    user\t57m14.608s\n    sys      0m10.350s\n\nrunning on server with twice the number of cores:\n\n    real\t 11m35.166s\n    user\t101m36.791s\n    sys       0m16.070s\n\nso the scaling is quite OK, even if there's still room for improvement (_e.g._ the `Dataset` iterator could start to unpack the next event's data once the `Next()` method has been called, or even keep a pre-fetch collection of events)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 329484,
      "author_name": "sbinet",
      "author_url": "",
      "post_date": "05/16/2018 15:16:31",
      "content": "<p>Latest version <a href=\"https://github.com/sbinet/go-trackml/commit/d4dc0b32d0e17da75006cae51b4f6c8182ac7805\">d4dc0b3</a> got:</p>\n\n<ul>\n<li>support for ZIP datasets</li>\n<li>creation of <code>submission.csv.gz</code> files</li>\n</ul>\n\n<p>Enjoy :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 329533,
      "author_name": "sbinet",
      "author_url": "",
      "post_date": "05/16/2018 16:55:31",
      "content": "<p>Version <a href=\"https://github.com/sbinet/go-trackml/tree/v0.1.1\">v0.1.1</a> clicks at <code>0.1445</code>.</p>\n\n<p>Interestingly, running the full procedure on my desktop gives:</p>\n\n<pre><code>$&gt; time trkml-hough -ncpus=-1 -submit ./train_sample.zip event000001000 ./test.zip \n[...]\ntrkml-hough: processing event 124...\ntrkml-hough: loading test dataset \"./test.zip\"... [done]\n\nreal    19m40.461s\nuser    57m14.608s\nsys      0m10.350s\n</code></pre>\n\n<p>running on server with twice the number of cores:</p>\n\n<pre><code>real     11m35.166s\nuser    101m36.791s\nsys       0m16.070s\n</code></pre>\n\n<p>so the scaling is quite OK, even if there's still room for improvement (<em>e.g.</em> the <code>Dataset</code> iterator could start to unpack the next event's data once the <code>Next()</code> method has been called, or even keep a pre-fetch collection of events)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "329164": "hi there,\n\nI have just translated the [LAL/trackml-library](https://github.com/LAL/trackml-library) python starter kit into a [Go](https://golang.org)-based one:\n\n- https://github.com/sbinet/go-trackml\n\nAs stated on the `README`, I get nice performance improvements:\n\n## Python version\n\n    $&gt; time python trkml-hough.py ./example_standard/dataset/event000000200\n       hit_id          x         y       z  volume_id  layer_id  module_id\n    0       1 -62.663200  -3.05090 -1502.5          7         2          1\n    1       2 -66.124702  -1.36730 -1502.5          7         2          1\n    2       3 -63.697701   1.73267 -1502.5          7         2          1\n    3       4 -82.501801 -14.09150 -1502.5          7         2          1\n    4       5 -74.343399   0.84469 -1502.5          7         2          1\n    Your score:  0.13153644878592863\n    Score for event 200: 0.132\n    Score for event 201: 0.133\n    Mean score: 0.132\n    \n    real  7m12.351s\n    user  7m10.400s\n    sys   0m0.828s\n\n## Go version (single-threaded)\n\n    $&gt; time trkml-hough ./example_standard/dataset/event000000200\n    trkml-hough: loading [./example_standard/dataset/event000000200]...\n    trkml-hough: loading [./example_standard/dataset/event000000200]... [done]\n    trkml-hough: score for event 200: 0.1316012364071201\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"...\n    trkml-hough: score for event 200: 0.1316012364071201\n    trkml-hough: score for event 201: 0.1332602513710427\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\n    trkml-hough: mean score: 0.13243074388908138\n    \n    real  1m21.033s\n    user  1m22.541s\n    sys   0m0.569s\n\n## Go version (multi-threaded)\n\n    $&gt; time trkml-hough -ncpus=-1 ./example_standard/dataset/event000000200\n    trkml-hough: loading [./example_standard/dataset/event000000200]...\n    trkml-hough: loading [./example_standard/dataset/event000000200]... [done]\n    trkml-hough: score for event 200: 0.13160123640712013\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"...\n    trkml-hough: score for event 200: 0.1316012364071201\n    trkml-hough: score for event 201: 0.13326025137104267\n    trkml-hough: loading the whole dataset \"./example_standard/dataset\"... [done]\n    trkml-hough: mean score: 0.13243074388908138\n    \n    real  0m30.081s\n    user  1m26.658s\n    sys   0m0.741s\n\nhth,\n\n-s",
    "329484": "Latest version [d4dc0b3](https://github.com/sbinet/go-trackml/commit/d4dc0b32d0e17da75006cae51b4f6c8182ac7805) got:\n\n- support for ZIP datasets\n- creation of `submission.csv.gz` files\n\nEnjoy :)",
    "329533": "Version [v0.1.1](https://github.com/sbinet/go-trackml/tree/v0.1.1) clicks at `0.1445`.\n\nInterestingly, running the full procedure on my desktop gives:\n\n\n    $&gt; time trkml-hough -ncpus=-1 -submit ./train_sample.zip event000001000 ./test.zip \n    [...]\n    trkml-hough: processing event 124...\n    trkml-hough: loading test dataset \"./test.zip\"... [done]\n    \n    real\t19m40.461s\n    user\t57m14.608s\n    sys      0m10.350s\n\nrunning on server with twice the number of cores:\n\n    real\t 11m35.166s\n    user\t101m36.791s\n    sys       0m16.070s\n\nso the scaling is quite OK, even if there's still room for improvement (_e.g._ the `Dataset` iterator could start to unpack the next event's data once the `Next()` method has been called, or even keep a pre-fetch collection of events)"
  },
  "source": "meta"
}