{ "id": 275841, "title": "13th Place Solution ", "url": "/competitions/g2net-gravitational-wave-detection/writeups/solvers-club-4curry-1lasagna-13th-place-solution", "author_name": "", "post_date": "2021-10-01T19:06:54.161448100Z", "votes": 14, "comment_count": 1, "views": 0, "content": "

I would like to thank my team members @sakshamaggarwal @darkravager @callmeb and @mrigendraagrawal, organizers and the Kaggle community for the amazing experience we had during this competition.

\n

Preprocessing:

\n

thanks to @callmeb

\n\n
train0 = train[train.target==0].reset_index(drop=True)\nDET = None\nfor idx, (id, target, path) in tqdm(train0.iterrows(), total=len(train0)):\n    ts = np.load(path)\n    ts = ts_window(ts)\n    ts = ts_whiten(ts)\n    ts = torch.tensor(ts)\n    fs = torch.fft.fft(ts)\n    if DET == None:\n        DET = fs.abs()\n    else:\n        DET = (fs.abs() + idx * DET) / (idx+1)\n
\n\n
WINDOW=signal.tukey(4096, 1/4)[None,:]\ndef ts_window(ts):\n    return ts * WINDOW\ndef ts_whiten(ts, lf=24, hf=364, order=4):\n    sos = signal.butter(order, [lf, hf], btype=\"bandpass\", output=\"sos\", fs=2048)\n    normalization = np.sqrt((hf - lf) / (2048 / 2))\n    return signal.sosfiltfilt(sos, ts) / normalization\n
\n\n
ts = ts_window(ts)\nts = ts_whiten(ts)\nts = torch.fft.ifft(fs).real\nfs = fs / DET\nts = torch.fft.ifft(fs).real\n(you can also reuse the window here)\n
\n

Augmentations

\n\n

Approaches

\n

Used stratified 5-fold training strategy. We had some ideas which gave a little boost to its native counterparts. The approaches were:

\n\n

NOTE : Parallel keyword denotes a modified architecture where we trained two CNN backbones simultaneously and concatenated their GAP’s output followed by two FC layers.

\n

We trained backbones of different complexities which may use a specific approach or a combination of them. The models also varied with image sizes and frequency range.
\n\"model

\n

Ensembing

\n\n

Special thanks to Solvers Club for the generous GPU resources they provided. Alone Kaggle resources would not have have suffice for the competition (forever queued TPUs)

\n

Although the shakeup was little, it was enough to bring us down from gold to silver.
\nHope had trained 1D CNNs :)

", "messages": [ { "id": "1531258", "postDate": "10/01/2021 19:06:54", "content": "

I would like to thank my team members @sakshamaggarwal @darkravager @callmeb and @mrigendraagrawal, organizers and the Kaggle community for the amazing experience we had during this competition.

\n

Preprocessing:

\n

thanks to @callmeb

\n\n
train0 = train[train.target==0].reset_index(drop=True)\nDET = None\nfor idx, (id, target, path) in tqdm(train0.iterrows(), total=len(train0)):\n    ts = np.load(path)\n    ts = ts_window(ts)\n    ts = ts_whiten(ts)\n    ts = torch.tensor(ts)\n    fs = torch.fft.fft(ts)\n    if DET == None:\n        DET = fs.abs()\n    else:\n        DET = (fs.abs() + idx * DET) / (idx+1)\n
\n\n
WINDOW=signal.tukey(4096, 1/4)[None,:]\ndef ts_window(ts):\n    return ts * WINDOW\ndef ts_whiten(ts, lf=24, hf=364, order=4):\n    sos = signal.butter(order, [lf, hf], btype=\"bandpass\", output=\"sos\", fs=2048)\n    normalization = np.sqrt((hf - lf) / (2048 / 2))\n    return signal.sosfiltfilt(sos, ts) / normalization\n
\n\n
ts = ts_window(ts)\nts = ts_whiten(ts)\nts = torch.fft.ifft(fs).real\nfs = fs / DET\nts = torch.fft.ifft(fs).real\n(you can also reuse the window here)\n
\n

Augmentations

\n\n

Approaches

\n

Used stratified 5-fold training strategy. We had some ideas which gave a little boost to its native counterparts. The approaches were:

\n\n

NOTE : Parallel keyword denotes a modified architecture where we trained two CNN backbones simultaneously and concatenated their GAP’s output followed by two FC layers.

\n

We trained backbones of different complexities which may use a specific approach or a combination of them. The models also varied with image sizes and frequency range.
\n\"model

\n

Ensembing

\n\n

Special thanks to Solvers Club for the generous GPU resources they provided. Alone Kaggle resources would not have have suffice for the competition (forever queued TPUs)

\n

Although the shakeup was little, it was enough to bring us down from gold to silver.
\nHope had trained 1D CNNs :)

", "rawMarkdown": "I would like to thank my team members @sakshamaggarwal @darkravager @callmeb and @mrigendraagrawal, organizers and the Kaggle community for the amazing experience we had during this competition.\n\n# Preprocessing: \nthanks to @callmeb\n - **Mean PSD**: We calculated the mean PSD as simple running mean and store it for later normalization:\n```\ntrain0 = train[train.target==0].reset_index(drop=True)\nDET = None\nfor idx, (id, target, path) in tqdm(train0.iterrows(), total=len(train0)):\n ts = np.load(path)\n ts = ts_window(ts)\n ts = ts_whiten(ts)\n ts = torch.tensor(ts)\n fs = torch.fft.fft(ts)\n if DET == None:\n DET = fs.abs()\n else:\n DET = (fs.abs() + idx * DET) / (idx+1)\n```\n - **Data whitening** : you can use any whitening you want. This is the one of many solutions we used.\n```\nWINDOW=signal.tukey(4096, 1/4)[None,:]\ndef ts_window(ts):\n return ts * WINDOW\ndef ts_whiten(ts, lf=24, hf=364, order=4):\n sos = signal.butter(order, [lf, hf], btype=\"bandpass\", output=\"sos\", fs=2048)\n normalization = np.sqrt((hf - lf) / (2048 / 2))\n return signal.sosfiltfilt(sos, ts) / normalization\n```\n - After loading the data we **Normalize** the signal as follows:\n```\nts = ts_window(ts)\nts = ts_whiten(ts)\nts = torch.fft.ifft(fs).real\nfs = fs / DET\nts = torch.fft.ifft(fs).real\n(you can also reuse the window here)\n```\n# Augmentations\n - Random channel shuffle\n - Random rolling (time shift)\n# Approaches\nUsed stratified 5-fold training strategy. We had some ideas which gave a little boost to its native counterparts. The approaches were:\n - **3 Channel (native)** - stacked the detectors channel wise. \n - **6 Permutations** - We horizontally stacked the 3 detectors with different permutations. Then stacked these 6 permutations channel wise.\n - **CQT+CWT** - We took 3 channels(for 3 detectors) from CQT transformation and 3 from CWT transformation and stacked them.\n - **Double CWT** - created two parallel CWT transformations with different parameters, with the idea of better frequency resolution by one and better time resolution by another.\n\n**NOTE** : **Parallel** keyword denotes a modified architecture where we trained two CNN backbones simultaneously and concatenated their GAP’s output followed by two FC layers.\n\nWe trained backbones of different complexities which may use a specific approach or a combination of them. The models also varied with image sizes and frequency range. \n![model list](https://drive.google.com/file/d/1cWQpfQw1ghp6ZPqTR6OlmLnPCEe6Sb8r/view?usp=sharing)\n# Ensembing\n - We had around 50 models at the end with our best model scoring 0.8808.\n - With weighted averaging, we were able to achieve around 0.8823 public LB ( 0.88047 private).\n - Switching to Stacking Ensembling with sklearn’s MLP classifier significantly boosted our public LB to 0.8830 ( 0.88133 private).\n - Also tried stacking with different classifiers like LGB, logistic etc and different model selections like top 10, top 20 etc, but they performed relatively poorer in both public and private LB.\n\nSpecial thanks to [Solvers Club](https://www.solversclub.com/) for the generous GPU resources they provided. Alone Kaggle resources would not have have suffice for the competition (forever queued TPUs)\n\nAlthough the shakeup was little, it was enough to bring us down from gold to silver. \nHope had trained 1D CNNs :)", "votes": null }, { "id": "1559901", "postDate": "10/27/2021 08:07:18", "content": "

Hey All,

\n

Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.

\n

Thank you all,

\n

Regards,
\nChris

", "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris", "votes": null } ], "comments": [ { "id": 1559901, "author_name": "zerafachris", "author_url": "", "post_date": "10/27/2021 08:07:18", "content": "

Hey All,

\n

Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.

\n

Thank you all,

\n

Regards,
\nChris

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1531258": "I would like to thank my team members @sakshamaggarwal @darkravager @callmeb and @mrigendraagrawal, organizers and the Kaggle community for the amazing experience we had during this competition.\n\n# Preprocessing: \nthanks to @callmeb\n - **Mean PSD**: We calculated the mean PSD as simple running mean and store it for later normalization:\n```\ntrain0 = train[train.target==0].reset_index(drop=True)\nDET = None\nfor idx, (id, target, path) in tqdm(train0.iterrows(), total=len(train0)):\n ts = np.load(path)\n ts = ts_window(ts)\n ts = ts_whiten(ts)\n ts = torch.tensor(ts)\n fs = torch.fft.fft(ts)\n if DET == None:\n DET = fs.abs()\n else:\n DET = (fs.abs() + idx * DET) / (idx+1)\n```\n - **Data whitening** : you can use any whitening you want. This is the one of many solutions we used.\n```\nWINDOW=signal.tukey(4096, 1/4)[None,:]\ndef ts_window(ts):\n return ts * WINDOW\ndef ts_whiten(ts, lf=24, hf=364, order=4):\n sos = signal.butter(order, [lf, hf], btype=\"bandpass\", output=\"sos\", fs=2048)\n normalization = np.sqrt((hf - lf) / (2048 / 2))\n return signal.sosfiltfilt(sos, ts) / normalization\n```\n - After loading the data we **Normalize** the signal as follows:\n```\nts = ts_window(ts)\nts = ts_whiten(ts)\nts = torch.fft.ifft(fs).real\nfs = fs / DET\nts = torch.fft.ifft(fs).real\n(you can also reuse the window here)\n```\n# Augmentations\n - Random channel shuffle\n - Random rolling (time shift)\n# Approaches\nUsed stratified 5-fold training strategy. We had some ideas which gave a little boost to its native counterparts. The approaches were:\n - **3 Channel (native)** - stacked the detectors channel wise. \n - **6 Permutations** - We horizontally stacked the 3 detectors with different permutations. Then stacked these 6 permutations channel wise.\n - **CQT+CWT** - We took 3 channels(for 3 detectors) from CQT transformation and 3 from CWT transformation and stacked them.\n - **Double CWT** - created two parallel CWT transformations with different parameters, with the idea of better frequency resolution by one and better time resolution by another.\n\n**NOTE** : **Parallel** keyword denotes a modified architecture where we trained two CNN backbones simultaneously and concatenated their GAP’s output followed by two FC layers.\n\nWe trained backbones of different complexities which may use a specific approach or a combination of them. The models also varied with image sizes and frequency range. \n![model list](https://drive.google.com/file/d/1cWQpfQw1ghp6ZPqTR6OlmLnPCEe6Sb8r/view?usp=sharing)\n# Ensembing\n - We had around 50 models at the end with our best model scoring 0.8808.\n - With weighted averaging, we were able to achieve around 0.8823 public LB ( 0.88047 private).\n - Switching to Stacking Ensembling with sklearn’s MLP classifier significantly boosted our public LB to 0.8830 ( 0.88133 private).\n - Also tried stacking with different classifiers like LGB, logistic etc and different model selections like top 10, top 20 etc, but they performed relatively poorer in both public and private LB.\n\nSpecial thanks to [Solvers Club](https://www.solversclub.com/) for the generous GPU resources they provided. Alone Kaggle resources would not have have suffice for the competition (forever queued TPUs)\n\nAlthough the shakeup was little, it was enough to bring us down from gold to silver. \nHope had trained 1D CNNs :)", "1559901": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris" }, "source": "meta" }