{
  "id": 126648,
  "title": "Help for TTA Solution ",
  "url": "/competitions/bengaliai-cv19/discussion/126648",
  "author_name": "",
  "post_date": "2020-01-19T05:27:17.565217200Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Can anyone please help with TTA implementation. I am not able to do so with multiout </p>",
  "messages": [
    {
      "id": "722784",
      "postDate": "01/19/2020 05:27:17",
      "content": "<p>Can anyone please help with TTA implementation. I am not able to do so with multiout </p>",
      "rawMarkdown": "Can anyone please help with TTA implementation. I am not able to do so with multiout",
      "votes": null
    },
    {
      "id": "723159",
      "postDate": "01/19/2020 15:10:26",
      "content": "<p>You could do something like this in pytorch,  it does multiple model + TTA voting.\nAll you need to do then is select the mode for each row of the outputs.</p>\n\n<p>If you want to average the probabilities, it is fairly easy to adapt the code by getting rid of the <code>max</code> part.</p>\n\n<p>Hope it helps,</p>\n\n<p>Cheers</p>\n\n<p>```\ndef predict_TTA_multiple_models(models, dataloader, n_samples, n_tta=10):\n    \"\"\"Voting TTA with different models</p>\n\n<pre><code>Parameters\n----------\nmodels : list\n    List of models in the voting\ndataloader : torch.DataLoader\n    data loader with (random) augmentation\nn_samples : int\n    number of images in total for prediction (len(dataset))\nn_tta : int (default=10)\n    Number of test augmentation you are willing to perform\n\nReturns\n-------\nall_root_preds, all_vowel_preds, all_consonant_preds : np.arrays\n    arrays of size n_samples x n_tta*nb_models that you can use to perform the voting\n\"\"\" \nnb_models = len(models)\nall_root_preds = np.empty((n_samples, n_tta*nb_models))\nall_vowel_preds = np.empty((n_samples, n_tta*nb_models))\nall_consonant_preds = np.empty((n_samples, n_tta*nb_models))\nfor augment_nb in tqdm(range(n_tta)):\n    # Iterate over data.\n    root_preds = []\n    vowel_preds = []\n    cons_preds = []\n    start_index=0\n    end_index=0\n    for inputs in tqdm(dataloader):\n        inputs = inputs.to(device)\n        end_index = start_index + inputs.shape[0]\n        for model_nb, model in enumerate(models):\n            col_index = augment_nb*nb_models + model_nb\n            model.eval()\n            # forward\n            with torch.set_grad_enabled(False):\n                out_root, out_vowel, out_consonant = model(inputs)\n                _, preds_root = torch.max(out_root, 1)\n                _, preds_vowel = torch.max(out_vowel, 1)\n                _, preds_consonant = torch.max(out_consonant, 1)\n\n                all_root_preds[start_index:end_index, col_index] = preds_root.cpu().numpy()\n                all_vowel_preds[start_index:end_index, col_index] = preds_vowel.cpu().numpy()\n                all_consonant_preds[start_index:end_index, col_index] = preds_consonant.cpu().numpy()\n        start_index = end_index\n\nreturn all_root_preds, all_vowel_preds, all_consonant_preds\n</code></pre>\n\n<p>```</p>",
      "rawMarkdown": "You could do something like this in pytorch,  it does multiple model + TTA voting.\nAll you need to do then is select the mode for each row of the outputs.\n\nIf you want to average the probabilities, it is fairly easy to adapt the code by getting rid of the `max` part.\n\nHope it helps,\n\nCheers\n\n```\ndef predict_TTA_multiple_models(models, dataloader, n_samples, n_tta=10):\n    \"\"\"Voting TTA with different models\n    \n    Parameters\n    ----------\n    models : list\n        List of models in the voting\n    dataloader : torch.DataLoader\n        data loader with (random) augmentation\n    n_samples : int\n        number of images in total for prediction (len(dataset))\n    n_tta : int (default=10)\n        Number of test augmentation you are willing to perform\n    \n    Returns\n    -------\n    all_root_preds, all_vowel_preds, all_consonant_preds : np.arrays\n        arrays of size n_samples x n_tta*nb_models that you can use to perform the voting\n    \"\"\" \n    nb_models = len(models)\n    all_root_preds = np.empty((n_samples, n_tta*nb_models))\n    all_vowel_preds = np.empty((n_samples, n_tta*nb_models))\n    all_consonant_preds = np.empty((n_samples, n_tta*nb_models))\n    for augment_nb in tqdm(range(n_tta)):\n        # Iterate over data.\n        root_preds = []\n        vowel_preds = []\n        cons_preds = []\n        start_index=0\n        end_index=0\n        for inputs in tqdm(dataloader):\n            inputs = inputs.to(device)\n            end_index = start_index + inputs.shape[0]\n            for model_nb, model in enumerate(models):\n                col_index = augment_nb*nb_models + model_nb\n                model.eval()\n                # forward\n                with torch.set_grad_enabled(False):\n                    out_root, out_vowel, out_consonant = model(inputs)\n                    _, preds_root = torch.max(out_root, 1)\n                    _, preds_vowel = torch.max(out_vowel, 1)\n                    _, preds_consonant = torch.max(out_consonant, 1)\n                    \n                    all_root_preds[start_index:end_index, col_index] = preds_root.cpu().numpy()\n                    all_vowel_preds[start_index:end_index, col_index] = preds_vowel.cpu().numpy()\n                    all_consonant_preds[start_index:end_index, col_index] = preds_consonant.cpu().numpy()\n            start_index = end_index\n\n    return all_root_preds, all_vowel_preds, all_consonant_preds\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 723159,
      "author_name": "optimo",
      "author_url": "",
      "post_date": "01/19/2020 15:10:26",
      "content": "<p>You could do something like this in pytorch,  it does multiple model + TTA voting.\nAll you need to do then is select the mode for each row of the outputs.</p>\n\n<p>If you want to average the probabilities, it is fairly easy to adapt the code by getting rid of the <code>max</code> part.</p>\n\n<p>Hope it helps,</p>\n\n<p>Cheers</p>\n\n<p>```\ndef predict_TTA_multiple_models(models, dataloader, n_samples, n_tta=10):\n    \"\"\"Voting TTA with different models</p>\n\n<pre><code>Parameters\n----------\nmodels : list\n    List of models in the voting\ndataloader : torch.DataLoader\n    data loader with (random) augmentation\nn_samples : int\n    number of images in total for prediction (len(dataset))\nn_tta : int (default=10)\n    Number of test augmentation you are willing to perform\n\nReturns\n-------\nall_root_preds, all_vowel_preds, all_consonant_preds : np.arrays\n    arrays of size n_samples x n_tta*nb_models that you can use to perform the voting\n\"\"\" \nnb_models = len(models)\nall_root_preds = np.empty((n_samples, n_tta*nb_models))\nall_vowel_preds = np.empty((n_samples, n_tta*nb_models))\nall_consonant_preds = np.empty((n_samples, n_tta*nb_models))\nfor augment_nb in tqdm(range(n_tta)):\n    # Iterate over data.\n    root_preds = []\n    vowel_preds = []\n    cons_preds = []\n    start_index=0\n    end_index=0\n    for inputs in tqdm(dataloader):\n        inputs = inputs.to(device)\n        end_index = start_index + inputs.shape[0]\n        for model_nb, model in enumerate(models):\n            col_index = augment_nb*nb_models + model_nb\n            model.eval()\n            # forward\n            with torch.set_grad_enabled(False):\n                out_root, out_vowel, out_consonant = model(inputs)\n                _, preds_root = torch.max(out_root, 1)\n                _, preds_vowel = torch.max(out_vowel, 1)\n                _, preds_consonant = torch.max(out_consonant, 1)\n\n                all_root_preds[start_index:end_index, col_index] = preds_root.cpu().numpy()\n                all_vowel_preds[start_index:end_index, col_index] = preds_vowel.cpu().numpy()\n                all_consonant_preds[start_index:end_index, col_index] = preds_consonant.cpu().numpy()\n        start_index = end_index\n\nreturn all_root_preds, all_vowel_preds, all_consonant_preds\n</code></pre>\n\n<p>```</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "722784": "Can anyone please help with TTA implementation. I am not able to do so with multiout",
    "723159": "You could do something like this in pytorch,  it does multiple model + TTA voting.\nAll you need to do then is select the mode for each row of the outputs.\n\nIf you want to average the probabilities, it is fairly easy to adapt the code by getting rid of the `max` part.\n\nHope it helps,\n\nCheers\n\n```\ndef predict_TTA_multiple_models(models, dataloader, n_samples, n_tta=10):\n    \"\"\"Voting TTA with different models\n    \n    Parameters\n    ----------\n    models : list\n        List of models in the voting\n    dataloader : torch.DataLoader\n        data loader with (random) augmentation\n    n_samples : int\n        number of images in total for prediction (len(dataset))\n    n_tta : int (default=10)\n        Number of test augmentation you are willing to perform\n    \n    Returns\n    -------\n    all_root_preds, all_vowel_preds, all_consonant_preds : np.arrays\n        arrays of size n_samples x n_tta*nb_models that you can use to perform the voting\n    \"\"\" \n    nb_models = len(models)\n    all_root_preds = np.empty((n_samples, n_tta*nb_models))\n    all_vowel_preds = np.empty((n_samples, n_tta*nb_models))\n    all_consonant_preds = np.empty((n_samples, n_tta*nb_models))\n    for augment_nb in tqdm(range(n_tta)):\n        # Iterate over data.\n        root_preds = []\n        vowel_preds = []\n        cons_preds = []\n        start_index=0\n        end_index=0\n        for inputs in tqdm(dataloader):\n            inputs = inputs.to(device)\n            end_index = start_index + inputs.shape[0]\n            for model_nb, model in enumerate(models):\n                col_index = augment_nb*nb_models + model_nb\n                model.eval()\n                # forward\n                with torch.set_grad_enabled(False):\n                    out_root, out_vowel, out_consonant = model(inputs)\n                    _, preds_root = torch.max(out_root, 1)\n                    _, preds_vowel = torch.max(out_vowel, 1)\n                    _, preds_consonant = torch.max(out_consonant, 1)\n                    \n                    all_root_preds[start_index:end_index, col_index] = preds_root.cpu().numpy()\n                    all_vowel_preds[start_index:end_index, col_index] = preds_vowel.cpu().numpy()\n                    all_consonant_preds[start_index:end_index, col_index] = preds_consonant.cpu().numpy()\n            start_index = end_index\n\n    return all_root_preds, all_vowel_preds, all_consonant_preds\n```"
  },
  "source": "meta"
}