Looking at the signals I suppose you can augment the dataset by flipping them horizontally or vertically.
\n\nDepending on how you use phases I believe you want to perform the same transformation to all phases to keep all events synchronized, though you may re-order them when flipping horizontally.
\n\nI've seen Rand Xie's great code but I'm under the impression that shifting things around as done in the code may break causal effects. Though I may be plain wrong ;-)
\n\nI have not done anything of that yet but I'll surely try one way or the other when time permits.
\n\nWhat are your thoughts on that ?
", "messages": [ { "id": 464759, "postDate": "2019-02-01T12:47:34.930Z", "content": "Looking at the signals I suppose you can augment the dataset by flipping them horizontally or vertically.
\n\nDepending on how you use phases I believe you want to perform the same transformation to all phases to keep all events synchronized, though you may re-order them when flipping horizontally.
\n\nI've seen Rand Xie's great code but I'm under the impression that shifting things around as done in the code may break causal effects. Though I may be plain wrong ;-)
\n\nI have not done anything of that yet but I'll surely try one way or the other when time permits.
\n\nWhat are your thoughts on that ?
", "rawMarkdown": "Looking at the signals I suppose you can augment the dataset by flipping them horizontally or vertically.\n\nDepending on how you use phases I believe you want to perform the same transformation to all phases to keep all events synchronized, though you may re-order them when flipping horizontally.\n\nI've seen [Rand Xie's great code](https://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/76302) but I'm under the impression that shifting things around as done in the code may break causal effects. Though I may be plain wrong ;-)\n\nI have not done anything of that yet but I'll surely try one way or the other when time permits.\n\nWhat are your thoughts on that ?", "votes": 13 }, { "id": 464816, "postDate": "2019-02-01T14:52:42.013Z", "content": "I think I've read somewhere (maybe @tvantuch 's paper) that there DP has more chance to occur while the signal is increasing between 0 -> max or decreasing between 0 -> min.\nI don't remember on which paper but it was something similar to those images, basically you can see that there are some specific time frame where the risk is higher. \nhttps://www.fastcomtec.com/applications/multiparameter-systems/high-voltage-partial-discharge/\nhttp://www.whitelegg.com/products/electric-motor/condition-monitoring/partial-discharge-testing
\n\nSo it this is true, then flipping the signal horizontally won't make sense. Vertically should be ok...
", "rawMarkdown": "I think I've read somewhere (maybe @tvantuch 's paper) that there DP has more chance to occur while the signal is increasing between 0 -> max or decreasing between 0 -> min.\nI don't remember on which paper but it was something similar to those images, basically you can see that there are some specific time frame where the risk is higher. \nhttps://www.fastcomtec.com/applications/multiparameter-systems/high-voltage-partial-discharge/\nhttp://www.whitelegg.com/products/electric-motor/condition-monitoring/partial-discharge-testing\n\nSo it this is true, then flipping the signal horizontally won't make sense. Vertically should be ok...\n", "votes": 5 }, { "id": 464800, "postDate": "2019-02-01T14:26:58.987Z", "content": "It doesn't work for me. I can understand why it doesn't work for CNN, but RNNs accuracy suffers so much too. Quite strange. I guess a kind of leak must be here, but it's not found yet.
", "rawMarkdown": "It doesn't work for me. I can understand why it doesn't work for CNN, but RNNs accuracy suffers so much too. Quite strange. I guess a kind of leak must be here, but it's not found yet.", "votes": 4, "replies": [ { "id": 464808, "postDate": "2019-02-01T14:35:11.433Z", "content": "If you understand it doesn't work for CNN, I think I don't :(
\n\nFor sure RNNs (bidirectional) should not suffer or that means sample order is meaningful, when I think you don't really care what direction voltage or current goes :/
\n\nhmm Leak ? ;()
", "rawMarkdown": "If you understand it doesn't work for CNN, I think I don't :(\n\nFor sure RNNs (bidirectional) should not suffer or that means sample order is meaningful, when I think you don't really care what direction voltage or current goes :/ \n\nhmm Leak ? ;()", "votes": 2 }, { "id": 464810, "postDate": "2019-02-01T14:42:13.970Z", "content": "Thanks for your feedback Sergei !!!!
", "rawMarkdown": "Thanks for your feedback Sergei !!!!", "votes": 1 }, { "id": 464811, "postDate": "2019-02-01T14:43:47.550Z", "content": "PD location is very important and doesn't vary so much in our dataset in terms of starting point. PD pattern looks like a series of waves with increasing amplitude as for me (I didn't know anything about PD before the contest, so I can be absolutely wrong). When we work with text classification we also use bidirectional rnn but \"reverse words order\" augmentation doesn't work too.
\n\nUPD. PD structure is not so simple, yes. But I think something like that is a problem.
", "rawMarkdown": "PD location is very important and doesn't vary so much in our dataset in terms of starting point. PD pattern looks like a series of waves with increasing amplitude as for me (I didn't know anything about PD before the contest, so I can be absolutely wrong). When we work with text classification we also use bidirectional rnn but \"reverse words order\" augmentation doesn't work too. \n\nUPD. PD structure is not so simple, yes. But I think something like that is a problem.", "votes": 2 }, { "id": 464814, "postDate": "2019-02-01T14:51:40.887Z", "content": "Oh yeah that make sense ! Thanks.
", "rawMarkdown": "Oh yeah that make sense ! Thanks.", "votes": 1 } ] }, { "id": 465184, "postDate": "2019-02-02T14:48:39.070Z", "content": "Does it make sense to make an augmented data by linearly combine two PD signals (for positive cases) or two non-PD signals (for negative cases) ?
", "rawMarkdown": "Does it make sense to make an augmented data by linearly combine two PD signals (for positive cases) or two non-PD signals (for negative cases) ?", "votes": 2, "replies": [ { "id": 465200, "postDate": "2019-02-02T15:46:20.183Z", "content": "Due to difference in phases you may have trouble doing this , or you may need to synchronize samples before you can combine.
", "rawMarkdown": "Due to difference in phases you may have trouble doing this , or you may need to synchronize samples before you can combine.", "votes": 1 }, { "id": 470040, "postDate": "2019-02-12T09:34:42.460Z", "content": "As we can see, voltage in all three phases obeys to harmonic function (Sin or Cos). Usually we can not apply linear combinations to thus signals, because harmonic functions are non-linear ones.
", "rawMarkdown": "As we can see, voltage in all three phases obeys to harmonic function (Sin or Cos). Usually we can not apply linear combinations to thus signals, because harmonic functions are non-linear ones. ", "votes": 2 }, { "id": 470388, "postDate": "2019-02-12T21:22:44.557Z", "content": "You could probably high pass the signals (remove only very low frequencies) to remove the sinusoidal and then add two signals.\nGenerally you would then expect the Signal to Noise Ratio of the transients (pulses, PD regions) to be half.
", "rawMarkdown": "You could probably high pass the signals (remove only very low frequencies) to remove the sinusoidal and then add two signals.\nGenerally you would then expect the Signal to Noise Ratio of the transients (pulses, PD regions) to be half.", "votes": 2, "isDeleted": true }, { "id": 470397, "postDate": "2019-02-12T21:48:34.773Z", "content": "Interesting, could you explain how that would augment the data?
", "rawMarkdown": "Interesting, could you explain how that would augment the data?" }, { "id": 475241, "postDate": "2019-02-20T13:46:46.217Z", "content": "Thanks everybody!! Now there are some good public kernels that just do phase syncing, so this idea should be more feasible now.
", "rawMarkdown": "Thanks everybody!! Now there are some good public kernels that just do phase syncing, so this idea should be more feasible now." } ] } ], "comments": [ { "id": 464816, "author_name": "Antoine", "author_url": "", "post_date": "2019-02-01T14:52:42.013000", "content": "I think I've read somewhere (maybe @tvantuch 's paper) that there DP has more chance to occur while the signal is increasing between 0 -> max or decreasing between 0 -> min.\nI don't remember on which paper but it was something similar to those images, basically you can see that there are some specific time frame where the risk is higher. \nhttps://www.fastcomtec.com/applications/multiparameter-systems/high-voltage-partial-discharge/\nhttp://www.whitelegg.com/products/electric-motor/condition-monitoring/partial-discharge-testing
\n\nSo it this is true, then flipping the signal horizontally won't make sense. Vertically should be ok...
", "votes": 5, "replies": [] }, { "id": 464800, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2019-02-01T14:26:58.987000", "content": "It doesn't work for me. I can understand why it doesn't work for CNN, but RNNs accuracy suffers so much too. Quite strange. I guess a kind of leak must be here, but it's not found yet.
", "votes": 4, "replies": [ { "id": 464808, "author_name": "olivier", "author_url": "", "post_date": "2019-02-01T14:35:11.433000", "content": "If you understand it doesn't work for CNN, I think I don't :(
\n\nFor sure RNNs (bidirectional) should not suffer or that means sample order is meaningful, when I think you don't really care what direction voltage or current goes :/
\n\nhmm Leak ? ;()
", "votes": 2, "replies": [] }, { "id": 464810, "author_name": "olivier", "author_url": "", "post_date": "2019-02-01T14:42:13.970000", "content": "Thanks for your feedback Sergei !!!!
", "votes": 1, "replies": [] }, { "id": 464811, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2019-02-01T14:43:47.550000", "content": "PD location is very important and doesn't vary so much in our dataset in terms of starting point. PD pattern looks like a series of waves with increasing amplitude as for me (I didn't know anything about PD before the contest, so I can be absolutely wrong). When we work with text classification we also use bidirectional rnn but \"reverse words order\" augmentation doesn't work too.
\n\nUPD. PD structure is not so simple, yes. But I think something like that is a problem.
", "votes": 2, "replies": [] }, { "id": 464814, "author_name": "olivier", "author_url": "", "post_date": "2019-02-01T14:51:40.887000", "content": "Oh yeah that make sense ! Thanks.
", "votes": 1, "replies": [] } ] }, { "id": 465184, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2019-02-02T14:48:39.070000", "content": "Does it make sense to make an augmented data by linearly combine two PD signals (for positive cases) or two non-PD signals (for negative cases) ?
", "votes": 2, "replies": [ { "id": 465200, "author_name": "olivier", "author_url": "", "post_date": "2019-02-02T15:46:20.183000", "content": "Due to difference in phases you may have trouble doing this , or you may need to synchronize samples before you can combine.
", "votes": 1, "replies": [] }, { "id": 470040, "author_name": "Fedor", "author_url": "", "post_date": "2019-02-12T09:34:42.460000", "content": "As we can see, voltage in all three phases obeys to harmonic function (Sin or Cos). Usually we can not apply linear combinations to thus signals, because harmonic functions are non-linear ones.
", "votes": 2, "replies": [] }, { "id": 470388, "author_name": "", "author_url": "", "post_date": "2019-02-12T21:22:44.557000", "content": "You could probably high pass the signals (remove only very low frequencies) to remove the sinusoidal and then add two signals.\nGenerally you would then expect the Signal to Noise Ratio of the transients (pulses, PD regions) to be half.
", "votes": 2, "replies": [] }, { "id": 470397, "author_name": "Joop", "author_url": "", "post_date": "2019-02-12T21:48:34.773000", "content": "Interesting, could you explain how that would augment the data?
", "votes": 0, "replies": [] }, { "id": 475241, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2019-02-20T13:46:46.217000", "content": "Thanks everybody!! Now there are some good public kernels that just do phase syncing, so this idea should be more feasible now.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "464759": "Looking at the signals I suppose you can augment the dataset by flipping them horizontally or vertically.\n\nDepending on how you use phases I believe you want to perform the same transformation to all phases to keep all events synchronized, though you may re-order them when flipping horizontally.\n\nI've seen [Rand Xie's great code](https://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/76302) but I'm under the impression that shifting things around as done in the code may break causal effects. Though I may be plain wrong ;-)\n\nI have not done anything of that yet but I'll surely try one way or the other when time permits.\n\nWhat are your thoughts on that ?", "464816": "I think I've read somewhere (maybe @tvantuch 's paper) that there DP has more chance to occur while the signal is increasing between 0 -> max or decreasing between 0 -> min.\nI don't remember on which paper but it was something similar to those images, basically you can see that there are some specific time frame where the risk is higher. \nhttps://www.fastcomtec.com/applications/multiparameter-systems/high-voltage-partial-discharge/\nhttp://www.whitelegg.com/products/electric-motor/condition-monitoring/partial-discharge-testing\n\nSo it this is true, then flipping the signal horizontally won't make sense. Vertically should be ok...\n", "464800": "It doesn't work for me. I can understand why it doesn't work for CNN, but RNNs accuracy suffers so much too. Quite strange. I guess a kind of leak must be here, but it's not found yet.", "465184": "Does it make sense to make an augmented data by linearly combine two PD signals (for positive cases) or two non-PD signals (for negative cases) ?" } }