I am creating a learning model, but TPU and GPU reach QUOTA immediately every week.
\nAre the top players using their own high-performance PCs or expensive clouds?
I'm sorry for the poor man's distortion
", "messages": [ { "id": 1522123, "postDate": "2021-09-23T21:43:47.820Z", "content": "I would encourage you to really rethink your way of solving this problem. How often do you do mathematics? Not programming, MATHEMATICS? How often do you write proofs? Do you have a journal of proofs? Do you write them just for fun even when you don't have a specific problem to solve? Do you ever contemplate the relationships between things, or funny numerical coincidences, or boolean logic, or trig identities, or solutions to the Traveling Salesman Problem, or the basic properties of sets and topologies?
\nIf you do, you will sometimes find funny little equivalences, funny little cheats, that mother nature left us embedded deep in the nature of reality itself, like holes in the sheer face of rock for grasping on to the side of Matterhorn as you climb to the summit, thousands of feet above the ground. You will sometimes find that the sum of i from 1 to n is just n*(n+1)/2. Mathematical Induction is especially good for such little cheats, but so is proof by contradiction or contrapositive.
\nWhen Guinness found that they couldn't afford to a \"statistically valid\" sample of each batch of beer, they discovered the Students T-Distribution, still a crucial tool of statistics today, and also a beautiful insight in to the nature of the convergence of binomial sums to normal curves. Maybe you should start regarding your poverty as a blessing? You don't have the option of the \"standard way\". Let it direct you to something greater, something that will truly change the world, something bigger than this competition!
", "rawMarkdown": "I would encourage you to really rethink your way of solving this problem. How often do you do mathematics? Not programming, MATHEMATICS? How often do you write proofs? Do you have a journal of proofs? Do you write them just for fun even when you don't have a specific problem to solve? Do you ever contemplate the relationships between things, or funny numerical coincidences, or boolean logic, or trig identities, or solutions to the Traveling Salesman Problem, or the basic properties of sets and topologies?\n\nIf you do, you will sometimes find funny little equivalences, funny little cheats, that mother nature left us embedded deep in the nature of reality itself, like holes in the sheer face of rock for grasping on to the side of Matterhorn as you climb to the summit, thousands of feet above the ground. You will sometimes find that the sum of i from 1 to n is just n*(n+1)/2. Mathematical Induction is especially good for such little cheats, but so is proof by contradiction or contrapositive.\n\nWhen Guinness found that they couldn't afford to a \"statistically valid\" sample of each batch of beer, they discovered the Students T-Distribution, still a crucial tool of statistics today, and also a beautiful insight in to the nature of the convergence of binomial sums to normal curves. Maybe you should start regarding your poverty as a blessing? You don't have the option of the \"standard way\". Let it direct you to something greater, something that will truly change the world, something bigger than this competition!", "votes": 5, "replies": [ { "id": 1522792, "postDate": "2021-09-24T15:30:51.787Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 1523006, "postDate": "2021-09-24T20:32:02.803Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1540330, "postDate": "2021-10-10T11:45:08.560Z", "content": "Awesome observation, really motivated us with renewed energy, thanks @ahfretheim !
", "rawMarkdown": "Awesome observation, really motivated us with renewed energy, thanks @ahfretheim !" } ] }, { "id": 1521227, "postDate": "2021-09-23T03:30:32.207Z", "content": "Sorry to say but the universe is filled with bell shaped curves. It's kind of pesky that mother nature did not create all things to be exact and equal.
\nML problems have one of those pesky bell curves with regards to the compute required. Vision problems using today's methods need a lot of compute - in the full Google Landmark data set I believe there are 4.5 million images with 200K+ classes. So they clearly have dumbed it down a bit to try and keep it within the capability of the free compute that they offer us with 1.5 million images and 83K classes in this competition.
\nI am currently training my latest model on 4 high-performance PC's (with dual GPU's), with training for 4 folds likely to take 3-4 days. I am doing nothing special with regards to my code and not real sure that my model will achieve metal status.
\nWorked a lot of Kaggle competitions over the last couple of years - in every one them two things are required.
\nFor any problem you try to solve as a data scientist you need to evaluate the problem (IMO) and determine it's resource requirements. When your short on compute than you need to be long on brain power.
\nSo my suggestion - put on a pair of reality glasses to correct the distortion. Solve and learn with what you have.
", "rawMarkdown": "Sorry to say but the universe is filled with bell shaped curves. It's kind of pesky that mother nature did not create all things to be exact and equal. \n\nML problems have one of those pesky bell curves with regards to the compute required. Vision problems using today's methods need a lot of compute - in the full Google Landmark data set I believe there are 4.5 million images with 200K+ classes. So they clearly have dumbed it down a bit to try and keep it within the capability of the free compute that they offer us with 1.5 million images and 83K classes in this competition.\n\nI am currently training my latest model on 4 high-performance PC's (with dual GPU's), with training for 4 folds likely to take 3-4 days. I am doing nothing special with regards to my code and not real sure that my model will achieve metal status.\n\nWorked a lot of Kaggle competitions over the last couple of years - in every one them two things are required.\n1. Brain power - supplied by you.\n2. Compute power - supplied by Kaggle or your own resources.\n\nFor any problem you try to solve as a data scientist you need to evaluate the problem (IMO) and determine it's resource requirements. When your short on compute than you need to be long on brain power. \n\nSo my suggestion - put on a pair of reality glasses to correct the distortion. Solve and learn with what you have.\n\n\n\n", "votes": 6 }, { "id": 1521113, "postDate": "2021-09-22T22:45:21.120Z", "content": "I am creating a learning model, but TPU and GPU reach QUOTA immediately every week.
\nAre the top players using their own high-performance PCs or expensive clouds?
I'm sorry for the poor man's distortion
", "rawMarkdown": "I am creating a learning model, but TPU and GPU reach QUOTA immediately every week. \nAre the top players using their own high-performance PCs or expensive clouds?\n\nI'm sorry for the poor man's distortion", "votes": 6 }, { "id": 1522367, "postDate": "2021-09-24T05:49:36.083Z", "content": "Colab Pro is a cost-effective option for TPU. However, you need to like TensorFlow, as TPU currently is misbehaving with PyTorch. As I mentioned before, there is a space for quite a successful library that would enable Pytorch on TPU. Maybe one of the \"poor\" (as you describe them) Pytorchers will become frustrated enough to build it.
", "rawMarkdown": "Colab Pro is a cost-effective option for TPU. However, you need to like TensorFlow, as TPU currently is misbehaving with PyTorch. As I mentioned before, there is a space for quite a successful library that would enable Pytorch on TPU. Maybe one of the \"poor\" (as you describe them) Pytorchers will become frustrated enough to build it.", "votes": 4, "replies": [ { "id": 1525973, "postDate": "2021-09-27T18:56:39.597Z", "content": "Problem is that TPU is Google only, and Google is pushing TF development, and noone is really supporting Pytorch XLA development as far as I know. But at the same time, Google could win a lot of Pytorch people over if Pytorch would better support TPUs. But then there is Nvidia, who are pushing GPUs… Politics :/
", "rawMarkdown": "Problem is that TPU is Google only, and Google is pushing TF development, and noone is really supporting Pytorch XLA development as far as I know. But at the same time, Google could win a lot of Pytorch people over if Pytorch would better support TPUs. But then there is Nvidia, who are pushing GPUs... Politics :/", "votes": 5 }, { "id": 1526118, "postDate": "2021-09-27T21:42:55.533Z", "content": "Thank you very much. Due to the political background, TPU cannot be used from PYTORCH. From the developer's point of view, it doesn't matter, so I wan to support PYTORCH.
", "rawMarkdown": "Thank you very much. Due to the political background, TPU cannot be used from PYTORCH. From the developer's point of view, it doesn't matter, so I wan to support PYTORCH." }, { "id": 1526545, "postDate": "2021-09-28T07:26:45.507Z", "content": "Im not necessarily saying you cant use it, I am myself not sure what the current state of XLA is, but there has been quite some development going on.
", "rawMarkdown": "Im not necessarily saying you cant use it, I am myself not sure what the current state of XLA is, but there has been quite some development going on.", "votes": 1 }, { "id": 1526630, "postDate": "2021-09-28T08:40:09.467Z", "content": "thank you so much. I didn't know that.
", "rawMarkdown": "thank you so much. I didn't know that." }, { "id": 1528361, "postDate": "2021-09-29T15:25:46.813Z", "content": "Agreed it is all about Politics. So let's consider… the TPU department at Google should be interested in broadening usage of TPU, no matter the framework (TF or PyTorch). So hopefully, the TPU department in Google will exercise their agenda over the TF department, and support XLA to the point where it works with Pytorch. 😃
\nEven for the sake of TF quality, TF should compete with Pytorch on the basis of fair competition, and not have an extra advantage of TPU integration.
\nAlternatively, NVIDIA might introduce its own TPU. I don't know if that would be accessible on Kaggle though :)
@tensorchoko - congratulations on your two bronze medals.
", "rawMarkdown": "@tensorchoko - congratulations on your two bronze medals.", "votes": 2, "replies": [ { "id": 1540415, "postDate": "2021-10-10T12:55:40.417Z", "content": "thank you for that.
", "rawMarkdown": "thank you for that.", "votes": 3 }, { "id": 1540434, "postDate": "2021-10-10T13:10:21.880Z", "content": "Congratulations @tensorchoko ! Very well done !
", "rawMarkdown": "Congratulations @tensorchoko ! Very well done !", "votes": 2 } ] }, { "id": 1521575, "postDate": "2021-09-23T11:00:02.937Z", "content": "Why don't you use Colab Pro?
\nThe champion of Google Landmark Retrieval 2020 used Colab Pro only.
Ow. thank you . I'll try that.
", "rawMarkdown": "Ow. thank you . I'll try that." } ] } ], "comments": [ { "id": 1522123, "author_name": "AlexFromIdaho", "author_url": "", "post_date": "2021-09-23T21:43:47.820000", "content": "I would encourage you to really rethink your way of solving this problem. How often do you do mathematics? Not programming, MATHEMATICS? How often do you write proofs? Do you have a journal of proofs? Do you write them just for fun even when you don't have a specific problem to solve? Do you ever contemplate the relationships between things, or funny numerical coincidences, or boolean logic, or trig identities, or solutions to the Traveling Salesman Problem, or the basic properties of sets and topologies?
\nIf you do, you will sometimes find funny little equivalences, funny little cheats, that mother nature left us embedded deep in the nature of reality itself, like holes in the sheer face of rock for grasping on to the side of Matterhorn as you climb to the summit, thousands of feet above the ground. You will sometimes find that the sum of i from 1 to n is just n*(n+1)/2. Mathematical Induction is especially good for such little cheats, but so is proof by contradiction or contrapositive.
\nWhen Guinness found that they couldn't afford to a \"statistically valid\" sample of each batch of beer, they discovered the Students T-Distribution, still a crucial tool of statistics today, and also a beautiful insight in to the nature of the convergence of binomial sums to normal curves. Maybe you should start regarding your poverty as a blessing? You don't have the option of the \"standard way\". Let it direct you to something greater, something that will truly change the world, something bigger than this competition!
", "votes": 5, "replies": [ { "id": 1522792, "author_name": "", "author_url": "", "post_date": "2021-09-24T15:30:51.787000", "content": "", "votes": 2, "replies": [] }, { "id": 1523006, "author_name": "", "author_url": "", "post_date": "2021-09-24T20:32:02.803000", "content": "", "votes": 0, "replies": [] }, { "id": 1540330, "author_name": "Old Monk", "author_url": "", "post_date": "2021-10-10T11:45:08.560000", "content": "Awesome observation, really motivated us with renewed energy, thanks @ahfretheim !
", "votes": 0, "replies": [] } ] }, { "id": 1521227, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2021-09-23T03:30:32.207000", "content": "Sorry to say but the universe is filled with bell shaped curves. It's kind of pesky that mother nature did not create all things to be exact and equal.
\nML problems have one of those pesky bell curves with regards to the compute required. Vision problems using today's methods need a lot of compute - in the full Google Landmark data set I believe there are 4.5 million images with 200K+ classes. So they clearly have dumbed it down a bit to try and keep it within the capability of the free compute that they offer us with 1.5 million images and 83K classes in this competition.
\nI am currently training my latest model on 4 high-performance PC's (with dual GPU's), with training for 4 folds likely to take 3-4 days. I am doing nothing special with regards to my code and not real sure that my model will achieve metal status.
\nWorked a lot of Kaggle competitions over the last couple of years - in every one them two things are required.
\nFor any problem you try to solve as a data scientist you need to evaluate the problem (IMO) and determine it's resource requirements. When your short on compute than you need to be long on brain power.
\nSo my suggestion - put on a pair of reality glasses to correct the distortion. Solve and learn with what you have.
", "votes": 6, "replies": [] }, { "id": 1522367, "author_name": "narsil (jobs-in-data.com)", "author_url": "", "post_date": "2021-09-24T05:49:36.083000", "content": "Colab Pro is a cost-effective option for TPU. However, you need to like TensorFlow, as TPU currently is misbehaving with PyTorch. As I mentioned before, there is a space for quite a successful library that would enable Pytorch on TPU. Maybe one of the \"poor\" (as you describe them) Pytorchers will become frustrated enough to build it.
", "votes": 4, "replies": [ { "id": 1525973, "author_name": "Psi", "author_url": "", "post_date": "2021-09-27T18:56:39.597000", "content": "Problem is that TPU is Google only, and Google is pushing TF development, and noone is really supporting Pytorch XLA development as far as I know. But at the same time, Google could win a lot of Pytorch people over if Pytorch would better support TPUs. But then there is Nvidia, who are pushing GPUs… Politics :/
", "votes": 5, "replies": [] }, { "id": 1526118, "author_name": "tensor choko", "author_url": "", "post_date": "2021-09-27T21:42:55.533000", "content": "Thank you very much. Due to the political background, TPU cannot be used from PYTORCH. From the developer's point of view, it doesn't matter, so I wan to support PYTORCH.
", "votes": 0, "replies": [] }, { "id": 1526545, "author_name": "Psi", "author_url": "", "post_date": "2021-09-28T07:26:45.507000", "content": "Im not necessarily saying you cant use it, I am myself not sure what the current state of XLA is, but there has been quite some development going on.
", "votes": 1, "replies": [] }, { "id": 1526630, "author_name": "tensor choko", "author_url": "", "post_date": "2021-09-28T08:40:09.467000", "content": "thank you so much. I didn't know that.
", "votes": 0, "replies": [] }, { "id": 1528361, "author_name": "narsil (jobs-in-data.com)", "author_url": "", "post_date": "2021-09-29T15:25:46.813000", "content": "Agreed it is all about Politics. So let's consider… the TPU department at Google should be interested in broadening usage of TPU, no matter the framework (TF or PyTorch). So hopefully, the TPU department in Google will exercise their agenda over the TF department, and support XLA to the point where it works with Pytorch. 😃
\nEven for the sake of TF quality, TF should compete with Pytorch on the basis of fair competition, and not have an extra advantage of TPU integration.
\nAlternatively, NVIDIA might introduce its own TPU. I don't know if that would be accessible on Kaggle though :)
@tensorchoko - congratulations on your two bronze medals.
", "votes": 2, "replies": [ { "id": 1540415, "author_name": "tensor choko", "author_url": "", "post_date": "2021-10-10T12:55:40.417000", "content": "thank you for that.
", "votes": 3, "replies": [] }, { "id": 1540434, "author_name": "Old Monk", "author_url": "", "post_date": "2021-10-10T13:10:21.880000", "content": "Congratulations @tensorchoko ! Very well done !
", "votes": 2, "replies": [] } ] }, { "id": 1521575, "author_name": "sukekiyo", "author_url": "", "post_date": "2021-09-23T11:00:02.937000", "content": "Why don't you use Colab Pro?
\nThe champion of Google Landmark Retrieval 2020 used Colab Pro only.
Ow. thank you . I'll try that.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1522123": "I would encourage you to really rethink your way of solving this problem. How often do you do mathematics? Not programming, MATHEMATICS? How often do you write proofs? Do you have a journal of proofs? Do you write them just for fun even when you don't have a specific problem to solve? Do you ever contemplate the relationships between things, or funny numerical coincidences, or boolean logic, or trig identities, or solutions to the Traveling Salesman Problem, or the basic properties of sets and topologies?\n\nIf you do, you will sometimes find funny little equivalences, funny little cheats, that mother nature left us embedded deep in the nature of reality itself, like holes in the sheer face of rock for grasping on to the side of Matterhorn as you climb to the summit, thousands of feet above the ground. You will sometimes find that the sum of i from 1 to n is just n*(n+1)/2. Mathematical Induction is especially good for such little cheats, but so is proof by contradiction or contrapositive.\n\nWhen Guinness found that they couldn't afford to a \"statistically valid\" sample of each batch of beer, they discovered the Students T-Distribution, still a crucial tool of statistics today, and also a beautiful insight in to the nature of the convergence of binomial sums to normal curves. Maybe you should start regarding your poverty as a blessing? You don't have the option of the \"standard way\". Let it direct you to something greater, something that will truly change the world, something bigger than this competition!", "1521227": "Sorry to say but the universe is filled with bell shaped curves. It's kind of pesky that mother nature did not create all things to be exact and equal. \n\nML problems have one of those pesky bell curves with regards to the compute required. Vision problems using today's methods need a lot of compute - in the full Google Landmark data set I believe there are 4.5 million images with 200K+ classes. So they clearly have dumbed it down a bit to try and keep it within the capability of the free compute that they offer us with 1.5 million images and 83K classes in this competition.\n\nI am currently training my latest model on 4 high-performance PC's (with dual GPU's), with training for 4 folds likely to take 3-4 days. I am doing nothing special with regards to my code and not real sure that my model will achieve metal status.\n\nWorked a lot of Kaggle competitions over the last couple of years - in every one them two things are required.\n1. Brain power - supplied by you.\n2. Compute power - supplied by Kaggle or your own resources.\n\nFor any problem you try to solve as a data scientist you need to evaluate the problem (IMO) and determine it's resource requirements. When your short on compute than you need to be long on brain power. \n\nSo my suggestion - put on a pair of reality glasses to correct the distortion. Solve and learn with what you have.\n\n\n\n", "1521113": "I am creating a learning model, but TPU and GPU reach QUOTA immediately every week. \nAre the top players using their own high-performance PCs or expensive clouds?\n\nI'm sorry for the poor man's distortion", "1522367": "Colab Pro is a cost-effective option for TPU. However, you need to like TensorFlow, as TPU currently is misbehaving with PyTorch. As I mentioned before, there is a space for quite a successful library that would enable Pytorch on TPU. Maybe one of the \"poor\" (as you describe them) Pytorchers will become frustrated enough to build it.", "1537460": "@tensorchoko - congratulations on your two bronze medals.", "1521575": "Why don't you use Colab Pro?\nThe champion of Google Landmark Retrieval 2020 used Colab Pro only." } }