The original research by Dr Brent Seales' team into ink detection, in particular the work by Stephen Parsons (@stephenrparsons) has led to a program called ink-id. This might be useful to play around with. We haven't included tutorials for this, since it's a bit involved to set up, but I'll explain the basics here, and perhaps people can contribute notebooks or Pull Requests to the repo to make it easier to set up with Kaggle.
\nI hope this is somewhat useful, and perhaps we can together make it possible to run ink-id in Kaggle. :)
", "messages": [ { "id": 2188470, "postDate": "2023-03-19T16:54:28.670Z", "content": "The original research by Dr Brent Seales' team into ink detection, in particular the work by Stephen Parsons (@stephenrparsons) has led to a program called ink-id. This might be useful to play around with. We haven't included tutorials for this, since it's a bit involved to set up, but I'll explain the basics here, and perhaps people can contribute notebooks or Pull Requests to the repo to make it easier to set up with Kaggle.
\nI hope this is somewhat useful, and perhaps we can together make it possible to run ink-id in Kaggle. :)
", "rawMarkdown": "The original research by Dr Brent Seales' team into ink detection, in particular the work by Stephen Parsons (@stephenrparsons) has led to a program called [ink-id](https://github.com/educelab/ink-id). This might be useful to play around with. We haven't included tutorials for this, since it's a bit involved to set up, but I'll explain the basics here, and perhaps people can contribute notebooks or Pull Requests to the repo to make it easier to set up with Kaggle.\n\n1. You first have to turn the surface volume data from Kaggle into a .volpkg (see [Tutorial 3](https://scrollprize.org/tutorial3) on our main site).\n2. You then have to create some .txt and .json files indicating what you're training and predicting on. E.g. see [this file](https://github.com/educelab/ink-id/blob/b5dee91d0bb81b15d475a5dd226d42ef908cbc1d/inkid/examples/DummyTest.volpkg/working/DummyTest_grid1x2.txt) and the JSON files it refers to.\n3. You need a dummy .ppm file that is an identity transformation. You can generate this using a [script in the repo](https://github.com/educelab/ink-id/blob/b5dee91d0bb81b15d475a5dd226d42ef908cbc1d/inkid/scripts/generate_surface_volume_ppm.py).\n\nI hope this is somewhat useful, and perhaps we can together make it possible to run ink-id in Kaggle. :)", "votes": 32 }, { "id": 2189943, "postDate": "2023-03-20T22:50:14.473Z", "content": "Does this repo relate to the LB team ink-id.csv? What is the baseline ink-id.csv?
Yeah, ink-id.csv is the baseline produced by the ink-id repo.
", "rawMarkdown": "Yeah, ink-id.csv is the baseline produced by the ink-id repo.", "votes": 3, "replies": [ { "id": 2196160, "postDate": "2023-03-25T07:23:34.783Z", "content": "Can you share which ink-id model was used for the baseline?
", "rawMarkdown": "Can you share which ink-id model was used for the baseline?" }, { "id": 2196471, "postDate": "2023-03-25T12:43:39.957Z", "content": "It's the InkClassifier3DCNN https://github.com/educelab/ink-id/blob/develop/inkid/model/model.py#L536
", "rawMarkdown": "It's the InkClassifier3DCNN https://github.com/educelab/ink-id/blob/develop/inkid/model/model.py#L536", "votes": 2 }, { "id": 2208409, "postDate": "2023-04-04T03:27:26.200Z", "content": "Hi Stephen Parsons, If I only use the model structure InkClassifier3DCNN in ink-id repo with data preprocessing method offered by kaggle notebook, but not create something like .volpkg or .ppm, can I get the baseline score about 0.48 in LB? Thanks !
", "rawMarkdown": "Hi Stephen Parsons, If I only use the model structure InkClassifier3DCNN in ink-id repo with data preprocessing method offered by kaggle notebook, but not create something like .volpkg or .ppm, can I get the baseline score about 0.48 in LB? Thanks !" }, { "id": 2208411, "postDate": "2023-04-04T03:29:30.717Z", "content": "Now I can only get about 0.2 score using only train/1 dataset.
", "rawMarkdown": "Now I can only get about 0.2 score using only train/1 dataset." } ] } ] }, { "id": 2190823, "postDate": "2023-03-21T14:11:13.433Z", "content": "Thank you for sharing this! Does ink-id give the baseline score of 0.51 when trained and tested on the same data as here on Kaggle? For me, the interpretation of this value depends a lot on whether or not the testing set is contained within the training set (as is the case with the public leaderboard), so I am wondering what is the case for the benchmark result.
", "rawMarkdown": "Thank you for sharing this! Does ink-id give the baseline score of 0.51 when trained and tested on the same data as here on Kaggle? For me, the interpretation of this value depends a lot on whether or not the testing set is contained within the training set (as is the case with the public leaderboard), so I am wondering what is the case for the benchmark result.", "votes": 2, "replies": [ { "id": 2191047, "postDate": "2023-03-21T16:55:43.667Z", "content": "Yes exactly, it gives that result when trained on the same data as here on Kaggle. The test set was not included in the training (that would be cheating 😁).
", "rawMarkdown": "Yes exactly, it gives that result when trained on the same data as here on Kaggle. The test set was not included in the training (that would be cheating 😁).", "replies": [ { "id": 2191427, "postDate": "2023-03-22T01:02:21.607Z", "content": "Can test data be used for unsupervised/self-supervised pre-training?
", "rawMarkdown": "Can test data be used for unsupervised/self-supervised pre-training?" }, { "id": 2191447, "postDate": "2023-03-22T01:30:53.660Z", "content": "Yes, but to win we have to be able to reproduce your methods.
", "rawMarkdown": "Yes, but to win we have to be able to reproduce your methods.", "votes": 2 } ] } ] }, { "id": 2190182, "postDate": "2023-03-21T05:04:35.680Z", "content": "A healthy competition to start with. Thanks @jpposma
", "rawMarkdown": "A healthy competition to start with. Thanks @jpposma ", "votes": 2 }, { "id": 2205127, "postDate": "2023-04-01T09:10:53.800Z", "content": "I am setting up an environment on Kaggle, but I cannot run the Ink-id program because I cannot change the Python version to 3.8 on Kaggle Notebook. Do you know how to change the Python version to 3.8 on Kaggle Notebook?
", "rawMarkdown": "I am setting up an environment on Kaggle, but I cannot run the Ink-id program because I cannot change the Python version to 3.8 on Kaggle Notebook. Do you know how to change the Python version to 3.8 on Kaggle Notebook?", "replies": [ { "id": 2205478, "postDate": "2023-04-01T15:49:03.417Z", "content": "I'm not sure, perhaps someone else knows.
\nBut since we haven't spent time trying to get ink-id to run in a Kaggle notebook, there's a good chance you'll have to make modifications to the program to make it compatible.
", "rawMarkdown": "I'm not sure, perhaps someone else knows.\n\nBut since we haven't spent time trying to get ink-id to run in a Kaggle notebook, there's a good chance you'll have to make modifications to the program to make it compatible.", "votes": 1, "replies": [ { "id": 2206255, "postDate": "2023-04-02T12:43:50.533Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2206257, "postDate": "2023-04-02T12:44:18.477Z", "content": "How about this notebook?
\nhttps://www.kaggle.com/code/taylorsamarel/change-python-version-on-kaggle-taylor-amarel/notebook
you can fork the ink-id github repo and change the file setup.cfg from python_requires = >=3.8 to python_requires = >=3.7
Does this repo relate to the LB team ink-id.csv? What is the baseline ink-id.csv?
Yeah, ink-id.csv is the baseline produced by the ink-id repo.
", "votes": 3, "replies": [ { "id": 2196160, "author_name": "Peleg Shilo", "author_url": "", "post_date": "2023-03-25T07:23:34.783000", "content": "Can you share which ink-id model was used for the baseline?
", "votes": 0, "replies": [] }, { "id": 2196471, "author_name": "Stephen Parsons", "author_url": "", "post_date": "2023-03-25T12:43:39.957000", "content": "It's the InkClassifier3DCNN https://github.com/educelab/ink-id/blob/develop/inkid/model/model.py#L536
", "votes": 2, "replies": [] }, { "id": 2208409, "author_name": "clwclw", "author_url": "", "post_date": "2023-04-04T03:27:26.200000", "content": "Hi Stephen Parsons, If I only use the model structure InkClassifier3DCNN in ink-id repo with data preprocessing method offered by kaggle notebook, but not create something like .volpkg or .ppm, can I get the baseline score about 0.48 in LB? Thanks !
", "votes": 0, "replies": [] }, { "id": 2208411, "author_name": "clwclw", "author_url": "", "post_date": "2023-04-04T03:29:30.717000", "content": "Now I can only get about 0.2 score using only train/1 dataset.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2190823, "author_name": "Sasha Fedchin", "author_url": "", "post_date": "2023-03-21T14:11:13.433000", "content": "Thank you for sharing this! Does ink-id give the baseline score of 0.51 when trained and tested on the same data as here on Kaggle? For me, the interpretation of this value depends a lot on whether or not the testing set is contained within the training set (as is the case with the public leaderboard), so I am wondering what is the case for the benchmark result.
", "votes": 2, "replies": [ { "id": 2191047, "author_name": "JP Posma", "author_url": "", "post_date": "2023-03-21T16:55:43.667000", "content": "Yes exactly, it gives that result when trained on the same data as here on Kaggle. The test set was not included in the training (that would be cheating 😁).
", "votes": 0, "replies": [ { "id": 2191427, "author_name": "Sid", "author_url": "", "post_date": "2023-03-22T01:02:21.607000", "content": "Can test data be used for unsupervised/self-supervised pre-training?
", "votes": 0, "replies": [] }, { "id": 2191447, "author_name": "JP Posma", "author_url": "", "post_date": "2023-03-22T01:30:53.660000", "content": "Yes, but to win we have to be able to reproduce your methods.
", "votes": 2, "replies": [] } ] } ] }, { "id": 2190182, "author_name": "Tariq Mahmood", "author_url": "", "post_date": "2023-03-21T05:04:35.680000", "content": "A healthy competition to start with. Thanks @jpposma
", "votes": 2, "replies": [] }, { "id": 2205127, "author_name": "Muj!rush!", "author_url": "", "post_date": "2023-04-01T09:10:53.800000", "content": "I am setting up an environment on Kaggle, but I cannot run the Ink-id program because I cannot change the Python version to 3.8 on Kaggle Notebook. Do you know how to change the Python version to 3.8 on Kaggle Notebook?
", "votes": 0, "replies": [ { "id": 2205478, "author_name": "JP Posma", "author_url": "", "post_date": "2023-04-01T15:49:03.417000", "content": "I'm not sure, perhaps someone else knows.
\nBut since we haven't spent time trying to get ink-id to run in a Kaggle notebook, there's a good chance you'll have to make modifications to the program to make it compatible.
", "votes": 1, "replies": [ { "id": 2206255, "author_name": "", "author_url": "", "post_date": "2023-04-02T12:43:50.533000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2206257, "author_name": "tbt", "author_url": "", "post_date": "2023-04-02T12:44:18.477000", "content": "How about this notebook?
\nhttps://www.kaggle.com/code/taylorsamarel/change-python-version-on-kaggle-taylor-amarel/notebook
you can fork the ink-id github repo and change the file setup.cfg from python_requires = >=3.8 to python_requires = >=3.7