{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Quick intro\n\n## This notebook shall show how to easily ensemble various models and improve the robustness for the private LB.\n"},{"metadata":{},"cell_type":"markdown","source":"## A good way to get many models and do some averaging can be found here: [TabData:Find best Hyperparameter & ENSEMBLE](https://www.kaggle.com/ChristianDenich/6-82x-tabdata-find-best-hyperparameter-ensemble/edit/run/42823401). In fact, you can simply run the before mentioned notebook and upload the best models to do a submission with this notebook."},{"metadata":{},"cell_type":"markdown","source":"## Load all dependencies you need\n<span style=\"color:darkgreen ;font-family: Impact; font-size:13;\"> from  </span> coffee  <span style=\"color:darkgreen ;font-family: Impact; font-size:13;\"> import  </span> ***** "},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport random\nimport pandas as pd\nimport os\n\nfrom sklearn.metrics import mean_absolute_error\nimport tensorflow as tf\nimport tensorflow.keras.backend as K\nimport tensorflow.keras.layers as Layers\nimport tensorflow.keras.models as Models\nimport warnings\nwarnings.filterwarnings('ignore') #Ignore \"future\" warnings and Data-Frame-Slicing warnings.\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def seed_everything(seed): \n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    tf.random.set_seed(seed)\nseed_everything(2000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"ROOT = '../input/osic-pulmonary-fibrosis-progression'\n\ntrain_df = pd.read_csv(f'{ROOT}/train.csv')\nprint(f'Train data has {train_df.shape[0]} rows and {train_df.shape[1]} columnns and looks like this:')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Wrangling"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_unique_df = train_df.drop_duplicates(subset = ['Patient'], keep = 'first')\n#train_unique_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# CHECK FOR DUPLICATES & DEAL WITH THEM\n# keep = False: All duplicates will be shown\ndupRows_df = train_df[train_df.duplicated(subset = ['Patient', 'Weeks'], keep = False )]\n# dupRows_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_df.drop_duplicates(subset=['Patient','Weeks'], keep = False, inplace = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true,"_kg_hide-input":true},"cell_type":"code","source":"print(f'So there are {dupRows_df.shape[0]} (= {dupRows_df.shape[0] / train_df.shape[0] * 100:.2f}%) duplicates.')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"test_df = pd.read_csv(f'{ROOT}/test.csv')\nprint(f'Test data has {test_df.shape[0]} rows and {test_df.shape[1]} columnns, has no duplicates and looks like this:')\ntest_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Getting the format right\nIn this section we are going to do all the Data-Wrangling and pre-processing which we need later to create our folds and evaluate our models."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"## CHECK SUBMISSION FORMAT\nsub_df = pd.read_csv(f\"{ROOT}/sample_submission.csv\")\n\nprint(f\"The sample submission contains: {sub_df.shape[0]} rows and {sub_df.shape[1]} columns.\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# split Patient_Week Column and re-arrage columns\nsub_df[['Patient','Weeks']] = sub_df.Patient_Week.str.split(\"_\",expand = True)\nsub_df =  sub_df[['Patient','Weeks','Confidence', 'Patient_Week']]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sub_df = sub_df.merge(test_df.drop('Weeks', axis = 1), on = \"Patient\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# introduce a column to indicate the source (train/test) for the data\ntrain_df['Source'] = 'train'\nsub_df['Source'] = 'test'\n\ndata_df = train_df.append([sub_df])\ndata_df.reset_index(inplace = True)\n#data_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_baseline_week(df):\n    # make a copy to not change original df    \n    _df = df.copy()\n    # ensure all Weeks values are INT and not accidentaly saved as string\n    _df['Weeks'] = _df['Weeks'].astype(int)\n    # as test data is containing all weeks, \n    _df.loc[_df.Source == 'test','min_week'] = np.nan\n    _df[\"min_week\"] = _df.groupby('Patient')['Weeks'].transform('min')\n    _df['baselined_week'] = _df['Weeks'] - _df['min_week']\n    \n    return _df   ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"data_df = get_baseline_week(data_df)\n#data_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_baseline_FVC(df):\n    # same as above\n    _df = df.copy()\n    base = _df.loc[_df.Weeks == _df.min_week]\n    base = base[['Patient','FVC']].copy()\n    base.columns = ['Patient','base_FVC']\n    \n    # add a row which contains the cumulated sum of rows for each patient\n    base['nb'] = 1\n    base['nb'] = base.groupby('Patient')['nb'].transform('cumsum')\n    \n    # drop all except the first row for each patient (= unique rows!), containing the min_week\n    base = base[base.nb == 1]\n    base.drop('nb', axis = 1, inplace = True)\n    \n    # merge the rows containing the base_FVC on the original _df\n    _df = _df.merge(base, on = 'Patient', how = 'left')    \n    _df.drop(['min_week'], axis = 1)\n    \n    return _df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_df = get_baseline_FVC(data_df)\ndata_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now the data has the format we need to work with."},{"metadata":{},"cell_type":"markdown","source":"## Preparing the data for the Neural Network"},{"metadata":{},"cell_type":"markdown","source":"### Let's do the Data-Prep and all related legwork ourselfs, w/o sklearn. How to use sklearn, you can read here: \nhttps://www.kaggle.com/chrisden/6-82-quantile-reg-lr-schedulers-checkpoints  \n\n"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def own_MinMaxColumnScaler(df, columns):\n    \"\"\"Adds columns with scaled numeric values to range [0, 1]\n    using the formula X_scld = (X - X.min) / (X.max - X.min)\"\"\"\n    for col in columns:\n        new_col_name = col + '_scld'\n        col_min = df[col].min()\n        col_max = df[col].max()        \n        df[new_col_name] = (df[col] - col_min) / ( col_max - col_min )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def own_OneHotColumnCreator(df, columns):\n    \"\"\"OneHot Encodes categorical features. Adds a column for each unique value per column\"\"\"\n    for col in cat_attribs:\n        for value in df[col].unique():\n            df[value] = (df[col] == value).astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## APPLY DEFINED TRANSFORMATIONS\n# define which attributes shall not be transformed, are numeric or categorical\nno_transform_attribs = ['Patient', 'Weeks', 'min_week']\nnum_attribs = ['FVC', 'Percent', 'Age', 'baselined_week', 'base_FVC']\ncat_attribs = ['Sex', 'SmokingStatus']\n\nown_MinMaxColumnScaler(data_df, num_attribs)\nown_OneHotColumnCreator(data_df, cat_attribs)\n\ndata_df[data_df.Source != \"train\"].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get back original data split\ntrain_df = data_df.loc[data_df.Source == 'train']\nsub = data_df.loc[data_df.Source == 'test']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loss function & Model\nIn this section we are going to define the loss & a first model.\nFirst we are taking care of the loss. We are trying to minimize the following:\n\n![image.png](attachment:image.png)\n\nThe global minimum of this function is achieved for delta = 0 and sigma = 70, which [results in a loss of roughly -4.59.](https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/168469).\n\nGetting our model to predict the Confidence and FVC values (which is what we need!) is not working fine so far, as you can read [here](https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/167764).\nCurrently the way to go seems to be pinball loss. \n","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAVsAAACDCAYAAAAj1bgbAAAgAElEQVR4Ae1972vbTNfm+wfqX9AnE7i9BX+xeT6IQEWWioDeQFAhIhQHIroEFSJK1l26zr73um8fh+2jwo322eDsFkHXheBQMIQVBEHhWs7MSBrZcn7btZMplMj6MXPmGumaM+ecOfMvUP8UAgoBhYBCYO4I/Mvca1AVKAQUAgoBhQAU2aqXQCGgEFAILAABRbYLAFlVoRBQCCgEFNmqd0AhoBBQCCwAAUW2CwBZVaEQUAgoBBTZqndAIXArBBIMoxjjX7e6eaE3JecDDC7ShdapKrs7Aops746ZeuLZITBCf7sO+8/Rcrb8agC/2YJ/pgh3OTuIS6XIdpl7R8m2FAiMPprQd/oYz0Ga0UcDWtOBdxggmPx/PEBCdf4aITxwYNou2jsmDNtHeDEhTBygpTnoX06cVz+XBgFFtkvTFUqQpUTgZw+WZqLzYx7SpYj2NGha1X9daNJj9Ld0aJtdjIQJY/SnBU2z0Pspy0Rl6ai/HUDptzIuy3OsyHZ5+kJJsoQIxIcNaC+7mI8BYYTuuouQqa9F40efbLTeRlyrjQM0NA3WJ0mvHvdhaxrq7+LiIQBp5EHXbPSlW0s3qB+/FQFFtr8VflX5ciMQI2hoMD4M5yPmrwGCnX6ZyMkc0PQxEAQcv28wzdc7lUUYwCdt+I8AJbpNI7Q1Dc7JBHvLj6rj34aAItuFQp9g+KkNq1mrnDqWtJeFyrUslY0QvrFhGjXU7Q7iiwGC1xbct22YL0y0P4+QXITwdyy4Ow7MJj83OW0eRwEcw4SzR3bOAN3jAMEXmTATDA4d2LttOC9NtD9FCA8dOFTmRhthNj0/78LQNLhfJ2sQeP0aIzp0YLzQK/vTP7sjrmnMHF1BzqCFmaGSbDUX4ZVcB2nKGrT9SD6pjpcEAUW2C+uIEXpke1uz4H8KEZ34MHUN2t889KIIEYUVzfimFybivSoaomsbMNZv+99Gd4b9k6bBrXcxkr/anLyaPiLh8BkcCLvmywCxUNz4ubJTiKbgum6jJxxIyYnDytIPBqJ1KeJ3Rh5ZkH512fXGuxjxUYsd54PemQ9Na6AgPwkgFgGgQTccdE4ihMcuWqRtbncQUX+eDZHcMUwsft+Cthty8wGraozeJm93NdlaeTu5ZAnCHQ3aq15ZW5bEVoe/DwFFtgvBnj7wFjSJBKha7ugok0WlOJcD+Js1aPq0fa/y/kc5OUK4b6Oh6fCixYwCg0MDwVmKwQFpig3434qGDN4S6Vjo5l74TOuTHEXMI6+BiDP7l3zhZJprp1cRvO2CjFg0gCDUceTD3enmZD7+JBxReZ1ZqcJpJU336QqTcXJqnz1y098khKtPatF3JVuAD0AesqHlpmrV9cUhoMh2EVgzj3aZBKha/qFLZHGNLOxeyVGTfG2jprXQ+X7NQw+9dOZDn/J6P7TQm54XU2HdlwhDnFvvoDAGDODTzCA/lyLaJ5LWIWuBnLhnYUykqUHTPQwqtNCZZMtw0eB8lj1RgvxLct/U1uI618DlwYSuZQOKVmoTIGy2U2YERbYFost3pMh2AX2SaVdeyZQmpnyNCSdHpTycFGqHhcaGXymSZL4aJ48BlQmuUrjHPSk87TSdzlsnzhWmAADCS184r7gzSysRUBVJS+IKh5K23Zem7sX1WWTLnVaTxCjqn1FWUWrVkXgXKuJkr3WQVbw7SrOtwnc5zimyXUA/jD/bPC5Sno4Kbdf8OBFU9GuE/p4N56ALb6sF72tCMT1o59P5FIN3NoymnnvJ4yMTrRc1GK9duDs+2lsG2l/HQNyB2ayjZjhwd1z4ezaMvZAH54tAeXs/gLdpIvjGqS2NO3A22vD3LNTWNOj7UUF6lVjd1WZroXuNNp4Ke60lrdbKzuWmAJqyM1ODie45mWMctL/8k2u6r6QwLYExI+m4A+tthPQqRpewiMaA0FCt46wPRuhte4gypxOz2danbLbxO9Kgy1P19MxnJpf2X/kQwdBKz0N0DgP04vL5MpQxl32iTHZPVejXjJkSUGWzTTAgB+FRpOy4ZdAX/kuR7SIgP+/C1OrwTsUH94s7y/StXh6ozsXgtsAaEdxlH44mCPWbz5w+ffKSn/mo73bR3dGgM2fKAL4RoEPkI8wMTCNrdvDfDwwER2QK4KQE9pGS6YGmpzpqwhlD9zOt+Wcftl7jcl6FcDUddmmqPH+wuCZnlBYR8HOybVvYMsmEkAzgNV30x8KM8LdMExcOSYpR/XMEipelaT8f+DS03g8QvRVhVWLGQc4140BaFHBBCxom7ahZPKvknEpouayG1oFY8SXBxImZHKGZXNLF7DDT5qvIFsI+LA0izNY/Yf/nRQlNvmJWQAsn5MEqq1r9XRwCimwXhHVyGsBpUjiSC3vDgX8ST3urmRZTKzmGSLzSdD5NkFyGTNN1SesFmROG6L7SOGHSDJtiM9cCnCUJhscWO2YGCFF+cELkzx1f6WUE3zCYZsuf83nsJrt3zjbhKezH6G/rqG3LsacJwl0+MMgW0vHXNgv9sjYddL6J8IQkRseuw9h24Ww4CKIBeq/r0JstmOQUI7vszxBtw4Tz2oR9GCI8slFv2mjvWHAOo4lEM9w00JqKs01ZCJ/ZtNBm4WUuOtG4egaQjDCIOnAqiVQAQM4ximSYZe8VsxBjw0F714a57aP/vSKWdkac7TiO0H9rsEFnCnJ1YmEIKLJdGNQ3V8Q00qmIA2GvfRdieDritkWa/tJ9P1P+gQszAydfrvVxrZc7WPgxkJX/7z0Xmm7BP+EhSmM2bZafA9cAdR9RklSTyM3NeRJ3sBVk65Jp4j6tohnFZhEBUVVE+nOI4eV1poaqp8rn+Aoyp3IF2eCAZizl+9WvxSKgyHaxeF9bG7NN6kKzJJ31LID75xe2isna92C94TGYpIHqOz309g1uT2RaaINHJtCHrXNNFeCaWeOIfPhEpjqMwxhjCoeS6sHlEIPzCxajqbOwqRThrgZ9u4POrg1mvrhW8id8kZleymaNu7aW4mfLkQt3LeE293PTUKMqN8JlCLfpI66IuLhNyeqex0FAke3j4PhIpYzQf92C9a6H/nsX7kEfwysgvRggigYYCccN2R11Mkkccvsi01jX6jDfBPC3LbS/CIcPI4oa6httBAcOrL2QO0nSEXqvW3COQoTHHtz9HobkhzsLYBptBO8dONsWak0DpiD4R2rgShbDFhvcK8oAwI8OzCnb/BxgoBhjvT2VZ4HMTOEbA/5phdlhDmKoImcjoMh2NjYrcqVsKpCF5pryIhdCyLU/pWPKZ1tbXpsnrWYzFKEu+xunyHbZe+hG+bipwMzDl4oHmMNLWghRXFFHd0dA7NTwMLPq3au9xRNsp4ZzpbneAqrfeosi298K/0MrFzGUIul0eF6Ul5x1i2TUmVmhuKyOFAIKgQUjoMh2wYCr6hQCCoHniYAi2+fZ76rVCgGFwIIRUGS7YMBVdQoBhcDzRECR7fPsd9VqhYBCYMEIKLJdMOCqOoWAQuB5IqDI9nn2u2q1QkAhsGAEFNkuGHBVnUJAIfA8EVBk+zz7XbVaIaAQWDACimwXDLiqTiGgEHieCCiyfZ79rlqtEFAILBgBRbYLBlxVpxBQCDxPBBTZPs9+V61WCCgEFoyAItsFA66qUwgoBJ4nAopsn2e/q1YrBBQCC0ZAke2CAVfVKQQUAs8TAUW2z7PfVasVAgqBBSOgyHZOgNO+YYM8e36K8bcY4yXM8j+n5qtinwkC6c8Y8c8VfbF/xoiqtoSfU989AtmmiPZ1aH94GKwo5o+NbXrmwzB8DPKdSmhnWwu9i+mahsc2jPUWapoGTdPYJovGugH+vzivverhywcTRrPG7svvfR9LhcboGMV1/YWB9tdciOK+yxj9QxdWswa6xzBaaNkBBpdA8peH1m642tuXX0XwCMMXOsdKrws8Ba7ZeU2Ddxqjs2Ggtcbx17QaWuu0O3EBF+IOjPy6jrrRRngpXWeHKcZRF+1tA3Wdl9Fqmmh/GiK9GqK7ZaLzY/KZ1f9Nm41af4oNRleuOSP0tmqwPy1G/oeT7XkXpiCK1QU9e0vGCPcsGDt9jLNTd/17FaGtt/gW4/mzs8mW3zKAxzC00PuZP8QPaDO/pgbt7YD/Hvdhs3tdhGK33fITMfw/Gmh/qWpBguGxg5qmw3jTQ3xZjI7J9w4sXYeuabAq9jMr17Eav0Z/Woxs9QOBnST26JMNXSsGQNqxmAYwbdZA881HvdFGWAXrOIRn6NDWLATRCGm2Zfgvep8a0HUifQ9Rdl6SY9UP70+2KcZnHThrFe88gORbF65twtlrwzZMuMcxJtWG0VcfzoYNd8+Fadjwv06Q5q8RwgMHpu2ivWPCsH2EkwoPbfOuNya+1/n0ygPJNsXgoIGWYbCPVPvDh6xnzUfkOZaaEPAaWnvRPcmW46HvhRMvxg1ke96FQR/6egfDiubRxo2NwwzZAfyMmCdfHABEMNP1U6EJov0WNK0F92TipRR1Dg6IFHT4ZxVCrNwpvuswEaj7tRhU8mZQX2sO+pmGeuZzsn3V49u95zfSwQi9Tb1ylpD+6MLWNegbAeJJNqBH2XbyGrTN3j3fqZIgS/fjzmQ7DtFeN9BqtlDXaTZRDHhZ49JvPlpaA14kAL0awGtoaBwM8hkXDY40WHazb+CiB4sUhU/ZaDhGf0uHttnFSAxyfPCdJvfRRxM0c6z+KjKpHv73YWTLXiQH/Z98h1d6se3PWWMfLtzKlXDZh6PpaP81+XFfT7a5VrUfVTaZXpJi1jBC7xWf8k6RIo3Say76FV0Qvyei1dDKSXu6KvpwNG2Wxjx9/3KfieGLjzn/IEsC02zCQ67zio9V0/zinLg/+eKiVjXbYVqRBk2vxpw/zgdH40PVMFoSaCV/3Jls81bSN1FFtgn624RpuR/id6QI2OLdFnxTGsCIXDXkCl8coFEiXwBiVlh/lykuQhg2O6/Dl01HuZyPd/Agso0PG2gIwZMTh2sGq67dPgBb+iiryep6sh285eRZjMqkoTpof+Ej++BAg5fzcIJwh99f1thIk5thf8q0K72NqNL0wBvNyHa7P6GVPwCQ3/loRp7yR5sO4G90+OyLMHnVLbQZpukSrhODDZmF1uwKezufxdAAZn68Ticisl3MNPV3wP3oZJv1w4SmyRUBDc5JAggizU1rouH0nWhanZkEaDZIfeOdyqiIWeEfwcQMfITuuob6NYqIXMp9j+9PtkyLcwotil7kBicBBsh9JbrXcyniIxum0ULtZRvhjyF6ezacfQ+OUYd9OECSxOju2rB3yTHUgkPnsrqu6H4L1ksDdbuDOFNML0K0bROGXoP1IcbotAN3y4W3Z6HVdBCc5SUAENPWSlPAdWSbzQqMwoHCpk3S70xO8Ze/VJqk7QLpqYeGNGWSH6FBkV68bGCUr5WO0wRJkjW+dGXlfvCBr2yDZdPIWTZZVJlnUgzeNqpt2Oz9p/dd+gZmoJQmSWHHnXHPqp5+dLLNTGozyLb1YYj0rzZX7DI/hgAv+y7cr/8P0R7nokqynRxQs293Qxp859Ah9yZbptW+LWwoJFs2+miNyZGjWnLuic887zf/tY9nTMVoGrDdx/iHsH3qNnrnnDQKmVz0hX2Hn+MjICPJ/Rrck3HeibyDaDrTgP+NPjjeca2DCAmz/4ipfGmEFNOiSs3wGrLNtE6NvNwcA+YZ1z0MZjhU8jZlL1saw2/O8nZnZK5PjPLVffJUznL7swZtrcUjEYw6d/7N9JyL/pO0IWY7fNnBsKIfcjLfnL+tb5n7hN7FwsR1F0kzvCdstjNs5/I7Lx/LNWZka/35v4WJYoZmW2En5pqwZFaSC36k4/uRLbNVTYBEAqURPGYnm+GUeCShJ4sZfXLgfh5hxGyOGhzJbsyN4pKxndwdH43CEURhQpsdxL+EZppNtfPzgqwaPuL8o+PTDnJ6FBNIoRllBFgScjbZ5qO0pHENj4zZXnGCORvZBbGTPbYlOQ/KVWfRC5IzqHTDU/wh+keTZgdseir9nmp24VDjM7MYQbMF/6xa0+c2RE1yXE4VuIATCQbHLosGMucdQXIewttqQNM9RBIkT4VsOYFXcNoj9uK9yJa8d/pelHsGZXlGxyZX8dfnq5LLdfLj7GORvY3ZOdkOl42o8jkAYlqoT5KlmNaUwoeENlp2etyPbLOPVtYOaJSVf0+19ZvPoz9o2kNafcND5ridujfXnG8YteOARWFI3xGS8xija2y8U3U94MRdZjkzZzhZ/Vl4nDw7ILKVf2f3Sn+zviDione88TYqTE3SfXRYaFHFcDtxC4sA6W+b6J5PX3m0M6x/pQiS7x1m9nKFvX9mPb/GiO8Y0M9mCyWnFJ/NXvuuzhQg+w4nCO4GMwJ9c7myMfGtZn1ydzNCNiufkGWm7Pe7cHeyZXGk17xA7Dqfdk975e8n5O2eEt7nks1UnJNfkMzWJmmSVD4fJMhsUK4tmy7Kzig+Ck5qSfch20wDm3CgXCW41nSaE6gF61V1SFLRisyMMDG4FDcwUgh3WuW2M01QhyerMaVnlvdH/jGWTDrpjfbofHr6yoKltxHKJvmJ5mbvBdkQZ/6jAYzMWzNvePgF1lbdlWS9uZ1UK5s2v7yLQsTf1dZRub2Prtk+2EHGv6VrHWQVZs6l1GyJlKrjOIsXJwsz0m7Qbu+izdCKKutjuaOLGgFkGqg82olzsgbKQeXhWempD/MDhYGM0H2ZxbmO0Nv2hNc+04zlEU/cy15UujdbTSRG6tIHnkk4w4yQvVh3DnjPFkFo0KtCkrJqxd+sP+QBQ76FAvyNA8lhSBdPPRbSxAlnxFZAGc06TNuBs9lC7XWI0VkAa8NAY00Q03kPznqGBwARVG7vB/A2TQTfZL1ZluBxj7MP7c6LM6jNLIZZZzb8a6XKwr5Kg7v0BC1GMeQohgThGwP1tRqsnTbctx5cw0L3e4roLV8ZaG7acLYtNIwgN1mxwH3bQ7BvwTyM+WySOW5tePsujDUdmiDN0WcHZrOB2hspzjuJ0Xltwz0K0DZMdL+TjNznUCJO4UB2DgK4Lymck+4jx7MD842P9maNLYaZHHynyZZMGwGCo0gysUm45IczNFuI0K+JEDyutWbOSKFAyEoURHmZua8q9EsoKVWOYv7OyLO/27Yjb9CNB3fTbIVNtrXtIzgMZv/fs1BnL20d3uliPrBM05AJhZ8ra42800gToEB1sdJLdAKRcnLqwcwdf5m9VnL4TdzbksiOOdL+VrUwYQbZZh93JUFf13dCLnIEZkHd192e0pJUilM0yxEU6RjROwvm2xDj3B49RvSBPjgNWtOBd9hB9HcPxvsugoaG1vsI4b4BY+/f8HHLRv8LxTPymQ7DOw+1ooFKR22Xf/j0UdbmHFrDIcjikMv9fh08+TXxgepbsi0+vzp1MP7qokVRHjt9jKTXPIl7jLC68jT9Zw/2TgfBlpab4Ni7+K+7+I87XXR3aOAMEX20YJAPQdjm9TWXLw2m924tQMycobowMw3R+ZuGGoVfUvlGB12KPGmKdzC7l+y53wLUtBp2//O/Idi30dB0mG8CBJ+IwEmB0GEc8fhTMhc4J2OwhQNrfBl++tWFptuChAsopsg2M+HMWkySPzqLbAG+qKFezKrEogbZL1G9qEGXlt6KRQ1SeB/z31R+M0KpkgfOW7cjb9CNB3ciW64VchMB1wBucbw132lU1sL4sAad8hFINsbhUQu6pCXQvWncgf3ChGlbaOcrqWhZpYla04S13SlWAo37cPQanNLaaX5v66VVvpcm47eMs03/8piHfHI9vjG18ixr3eRfPup2JpcnTt5W+p0gPm7DIq885Qp4ZcHeCdCvXPZUMWVkcau1Us6ANElL01H6SPVs2swCxbkZIr2M4BuUb0BipJJsj/FjiK5NeR541AG9nyzvw7qJzkQM+8zakgG6h53pJZ0zHwDSixDBNs+twN6fVw7ax9F00qFfNL2P4GmcyGhFH8VLc7xoppKdzyrjMyh9P0KaJogODBiHMcb0jmWkJ0w9LuW/oPIpB8O6hkxj5YPfNEFOmh7SyIMuBsz0Rw/2i2KhEiPyzOyQkXgmoohAmrTZjuMI/bfGDL9DjA7lrZD6KYsY6UgmPLZcd8OAvdfmS3KPIkkh4AIwrd+gJb0u7A0H/smwbGMXMytjw0F714a57aMvD4B5O/j7XqzS5Beub0f+8K0P7kS2ty71ud4o7MHTtuoZmu2y4sQ+4lrJhss/3LInmplf8o87I+gY6ZUYeHQL/kmE6GyIsTQILmuz5y4XM2u10GFT+QFb4caIkWnUE34QQaTWuz6iaIChSBnHtOGM9Jij1ELvXMTxsllXC504ZXG9pXulxtGUWZeUIDaFXnPRjSJE8Yj7C1hZOhiRg6/O0in0ccKZMKXZinoGB7XVCTVkisG0v4aa8pjtUGQrvYQPPxSriiacb2D2JNnu+/Ca5llCrvlcjtDb4bMFMpHkWmteOXcKsiXabKDR4R0FMHdDXDANTMqVcTmUUk7mBTyrg3zmkwCUGa6xxm2jbHpLJgIZDUG2hcM2wfB0hH/SyihBlMMPBrS1AJ33BhsYWfnNAP2PFigagYU4SqQ6+tyGH/3Itd808tmsjZGtdB+lB43/L+WO0PmAe8WPnQ+0qKc8U60kW7JnN+VQSblhy3fMciOUwjiFjI/cDkW2j933zHEyaS9cLc02/RbAXLPQ3ncQnHKXfPI9wuBi0gxAZgQT9S0P/mtynNXQeulwc0E6Qu91C85RiPDYg7vfw/Aa7/5jd8MylsfMLC9aIIdhe9NhGijJSeQ2nVc1ZUu2W9sdhF+68HY99GgKfNGH88KG/8GDs23DWGvBpFWPVA4lcGkYMMmGTAVfDRC8pIxZPXT3XbhH5AQdo/+6DnOnDedNjydpITPPSxPepxD9ozbcQ5qypxgcmjD2AgRZ3xrmVDKeabJNmSPQF+/NMvZDSSb2vVYtCHr8diiyLSH/OD9YPtumJ9mPV4tsHwcFVUoZgczMck1ETfmBlfg1TbYrIbYQkvLZ1id8MvOTX5HtnLBl2sqPTJWjnRoGC1sgMKcmqWIfgoAwC0yGTj2kyGV4lnZqmJ7xLINkt5CBdmqI5xkFXZZBkW0ZD/VLITAHBEYI81DJrrSDxxyqUkUuLQKKbJe2a5RgCgGFwFNCQJHtU+pN1RaFgEJgaRFQZLu0XaMEUwgoBJ4SAopsn1JvqrYoBBQCS4uAItul7RolmEJAIfCUEFBk+5R6U7VFIaAQWFoEFNkubdcowRQCCoGnhIAi26fUm6otCgGFwNIioMh2abtGCaYQUAg8JQQU2T6l3lRtUQgoBJYWAUW2S9s1SjCFgELgKSGgyPYp9aZqi0JAIbC0CCiyXdquUYIpBBQCTwkBRbZPqTdVWxQCCoGlRUCR7dJ2zWoJlnyf2GxvtcRX0ioE5o6AItu5Q/wMKqD9x5oB4nw79GfQZtVEhcAdEVBke0fA1O3TCMSHDfzrfwoQ5AmyZxx/pD2w1D+FwPNEQJHt8+z3x2v1o+xAmiLa16H94WEwuafk40mqSlII/FYEFNn+VvhXv/L4fQvWn2wvV96YJEZ3x4a5Xoeu0y6uHUQ3bfN03oWpadA0rVzW6sOjWqAQyBFQZJtDoQ7ujMBVhPZaG9GVeDKN4Rs2eudCPU0G8JtEohZ6F7NKTzE4aKBlGNCJcP/w2bbcs+5W5xUCq4qAIttV7bklkHt0bML8WGi1yReXaaeNgwEyawBtdU0aq/Z2UC3xzx4szUH/Z4ygwbVb+/NNqnB1UeqsQmCZEVBku8y9s8yypRG8tTZCyeOVfHWZdqrvhIUj7MznZPuqh4KWi4aRc63xLmYnkhOH36u02wIgdfRkEFBk+2S6crENIY21dchJUq45TRIkmVoLINN29SrNlkLGSKvNFNl0AF9ot86JxOJyBepYIbCiCCiyXdGOm7vYV0P0jyNkPFiqj5GiRJKli/KPMXqbZBpoIZjmZTCt9m1hcqAnc7NDI1C2WxlKdbzyCCiyXfkunEMDLvpwmWOrDv/bdPk03c+m/tNXizPjzzZ0rQbnU4UBgULG9ArHGZkndG67db9KKnJRrDpSCKwkAopsV7Lb5ij0VQT/dYDwzzaPDtjqT2i35MiqIMlJkS56sPUWvK+VujFGH03oe1HuSJMfJ8cbc6qtdyvtvPK96lghsCoIKLJdlZ5auJwjdF+Shmmge15UTjbY1sTUv7gqji76cF4Y8E+F3TUdI/42LoiVQsZ0s1RuqQx2nWu37b+UdlvCRv1YWQQU2a5s181fcCJWFvuaa6BEwNeQJIl0NYBvSERL57750PejXGDSXPU9KWIhv1Ic0GIJpd0WeORHl0MML/Nf6mCFEFBku0KdtXhRs9hXm0UMpH+1UcuJt0qaEXpbOuqv2vClPAneVgPGhyF/QNhkW9v+9bkU9izU2aqyOrzTp6LdphidDTC6d6BFgv72f4C9MyP3hIR5+KOqf1bxXILR6QCjbOHMKjZByKzIdoU7bxGiZ/bTxmGPabWdaz5icogxbVQsvZWPs1CuPNqg4h75/tLxlN14ES0XdYxDtJs10S4PM5Zm3EKgFIN3BoyDByTjiQM0tidt6LeoesVvSc98tJo+BitOuM+PbK8SJPfotDTuwFm3EHx7KlrWLb9AKTpA33l+HzpHaYTeK7Ih359saVagPygNZYJwp1WKDhl99eFsmDDWNNSaDvyTp5tTmJmVtvvFYplbvr7LdNszI1sxLb5Hpw2PWtDIqfN9mbpvMbJw+2l1rOxiJPjdtWTxwvckWxaXrKP99d72AyAO0NosVuHRLMI4iJCIHMKjPynMTkPjJufl74byvvUzp+lqm5SeF9nSNEzTCvvhfTv+uT13OUD342AiBOw5gfAwsmXLkHUpYc+doUsR7dVQRGZktnQLvZ9ZYQP4LD75FmF52SMr9Vek4S1ccVMAAAy+SURBVPydJqUH4vWsyHb00YCm6fBOH4iaevyZIfAQsqXpvwZtNyxC3+6KHqWgfCnHHA/RYYtO5NlGJqP2ZN9vMsVotLx7RaMxloRsU8RHNkyjhdrLNsIfQ/T2bDj7HhyjDvtwgITypO7asHddWM0WHDo3+dKOI3R2bFhbLtqvTZh7IUZIEO4bMNYNtNZ47GatacDY6CCGVO9mB9FZD23bhvvaZs9SOH5yGsDZNGG8MNGuCNBPL0JWp73rwcvrnBRM/V5tBDIiE2YE4TSrvahDf9VBeOLD3fXQ3qyx97f3Q7LrpxHaD5pNkUZXmzZB/EqRyEkoIGm7pXSWCYaf2rByJx//BjIHpPWpetHJUvbXeReGpsErogiXUsxZQi0H2dLITV7WHxxMTS9yoube64aLvniJ+Ll6ab39iILttRb8KKPgEbrrGvwz0fQkhEsecMnuhazeU5GZijyelzGCLAfr/+rCZE4NXpY2kbkqr5MF7ycId7nXulJzvorgrXPSJ+K/+b9X5Imd1Xvq/IIQmCBbVmvmNNNgHY+E1ioIT175xlJIasiiMe4sMD1/C8caeezJRFZ2YvJQPG3Ngv8pRHTiwyRTw9889KIIURRjLI0Ld5Zt0Q+Ib7iUrH7RMjygvqUg29EnB+7nEUYi96kj5TMd/Un5UBvwchIFMnNATqQi038pUJ6Wi665RUYpCqwnB4KUqYrCmqjjsnr5WvwhejsO3OMYZ8c22l/GIFKmEVXO3cocFlSeSA8IJIgOLDgHIUZLvPFhptE8lb8PePfv8GgV2WbnXIR5dEt2TnKkiRSTlQMwkyDF8KSLKLe9ymLxxOry9yBfzY8paXtTg274GGS6Bs3a3pFT1y4lbuff081T8fRc5MdodiAipPPq5ncwQrhvo0GmvqhqFBjAJ4WpKoPc/IR6tJKXgmx5a8gJQFMc2eifnat6oYtz8fsGi4Osb7YRHHpwty1Y2z763/M3Lyfo6Zc+q2P2C0jZqcq7DQhjvdYoadeP1is3FjRE176Ndkz32OhKsbH/+Mc/8JT+Xw/V/XEql1tBorjluevI9tcI/V2+Uq6eD9pSzZSCsuHfsGsxEXIL+lYXw+J1B4RGXSgDvFyuqMjfmFRf6ZB/F7Tyj9NegvBNDdq8yfeMlKJZ8imyLXXR/X/E3Ju6Lo+k4txmr/CEsxyossMhe+lndRBJJJwUWkHQhZxFHRW5qYAszrQUTC6e0TxES6zFFm1URw9DIHvHJI31wWSbIjpwEHztoc2iCPgqPVlOGuSvnzLzhRL17V4+m0p+RGw5b5ZHuGzfFN/BrdJX0juuwzmRbLoUo16lcMpCP/CYDQYzCV2R7QPhFY+LqXopybQ4ly/1zPOd6iwMJj31YX74J/f2VgWcpwlSRoYDeDT9yOJrz3twsgD9ijrkBqWRx8wPZGKg+uxjmlQJsp2w4dJz6QW55Cr+3cNmK++CUFGiOrUwBB5Atrew2War9AxpiyGwXYu9a1dNjT7ZkIkWGKO/ZaF7AfDVfBNhYEKWkjmMMCQNmxzSB114W5SpLRGmM5EHg/Jd2AZauoEOizMnLddAfa0Ga6cN960H17B4DHrcgdmso2Y4cHdc+Hs2jL2QK0u/RggPHNj7AbxNM18gxBYMbbTh71morWksj0Ylp8+w2abzHgEe6T1bGjNCNhLLOUz5ufJUfXBApgYXYTJCb5OHvvCEKWXNIPneg0ubD5JTTbxkXEsgp4EB/0xMjti+WbPDwcr1FS8vC/Sf2L5lHPkwS3azR+olVcxvRuABZAs+0Ley3BBVLWG5fem9biMzVVIKyilSlJ5lS1i1FpwDOU+CCzObvTE/hrQI4Bd3lulbhRbMiyOC1lEjcwGbNeosDp2R9RrffHNwUId73IWr6XCJiH/2YO90EGxpeZpM9p3shvifBwaCIzIFCKJm314Lne9kltBR2+UJiMjJXSP/yc8+bL3G819ckRNbx8w96IRiVHBEZs7TUBqoJJyW6XBpyDY+rHEDf+5sAGjVlm4EJZsVjYL2CxOmbaF9kk38U8THLqx1B22y2W7acI9CKTtSisGhibphwdl0EEh7a1O92jUB57y+FowNuT6uDdAobWw48N66sF85aNPOBpVD8jJ1uZLlTghQmNd6CzWRy4HCBjtfK859qTjHEq/fLs6W+wU0TjSTuxZPCZyZscphXMzpKZkIWNhi04Sz58LeoOW8cb7iLC+SLfSplZYBA8JeK4iRbXX0tQ1NJyWH3n0KO4vgaZmZgbdR3+5jlCQYHlvQ1sROG6L84IS2q+eOr/QyYpnhaOk787cIUienc0MjYs6lKx1UxdnGR7Qrcw0uObKX/N/SkO3vwylFqgjy98H/DGrmG1kWWmtlk0VEjdYI0COt9jpNuLKA+51kYZQZieZFcHutfTxAHHMSGxzo0HdDjK/Ex8K0zIwY+eq11hGZ2MpEnZX/7z0Xmm7BP4kQnQ0xZkoVnzFQucThTJvWfURJUrEARGixJd9JLvBKHCiyXYluUkKuNAJiI8ti+lvVmmJKTBpkvglm1a2PeI5pizo3F1Cx6VkA97/8Vzadb+97gvQpfliH+6kHj2aa+UaeXNNlMb5rtB09lcBjjRuMeIlMdRiHMcZkrpPqweUQg/ML5m/RWSRGinBXg77dQWfXFmVJDWW5ERornW5Tka3Un+pQITAvBJhfQZriV9ZDyWY0rXLX4sr7H+XkCP3XLVjveui/d+Ee9DG8ApLvtOghyyLG7bqtlw4C4etgmu6LFnN2tTcddGKh8TIbbQ31jTaCAwcWW8VJLD5C73ULzlGI8NiDu99joWpE7qbRRvDegbNtgcw05pvpxPLkIynF0T9K2xdbiCLbxeKtanu2CPAwreu3FEow+NjFYOnNj3xFJTcblDuUa8rCtlu+dO9fpDkbT8DxrMj23q/AE3lwHCHYNlAXO9qWV5b5D0iW/UTwedRm0E4NPA72UYtddGEsBKt6lRdzeJWS5jxUOLFTQ2U85UPLXuzzimwXi/dS1cbDh3QYrzvoRyG6YjWTc0RTyAiDH0/gDV8qxJ+CMCOE+fY7XWl5MJCcdYutjr5kkUJPoc2P0wZFto+D4+qVwuIbNbRK27TwmNC6lD+iumEpRp8p8Y+Gqqlk9TMPP0sfs0s7/j6q5vRwuVQJCoHbIKDI9jYoPcF7yMHBcoPK9kGRDlA/uMVOW+xeaSr5vQNDp3jHeWrDIlToNvI9wT5TTVptBBTZrnb/3VN6kQrwlZyQmqJ2+E4Wt0oHSFnUdEcKUZrMr3pP0a57TBA8W8V03X3qmkJgCRFQZLuEnTJ/kcQKpFKqOp7Or2o1XfKtA8d20Tlsw9jospR7LGGImM6PPjswmw3UspCdyxBto47amgV3z4W378LY7GL465o19RS7+a0L13bgH7o8vzEDYoRwz4a9T+vva8Uy0PmDpGpQCDwqAopsHxXOVSlMBNBLiXSSU58nX2eJ0It2pN98tHQL3XOALW1eayNMKO5SQ42C0WmdvNFBl9JQimxN40+0XDqArfGEQQDP1uT+t+OZa+pTWkFFiU4oYp7uZ1ozz8mqCzmHH1rQsqWdhYjqSCGwEggosl2JbpqDkFe09ZCJ1mabbSFEiUWin5PrlsWa98lN9uTpPK2TvxqyXTFyZxml4vvLg5aZGUSokPP30Yw19T183te54+tXiuGflM2qj7F4jicmEbKIpZ1zQEQVqRCYKwKKbOcK76oXzjXSnESz5jB7rY3uaYyYlmhmmZ3iVKS0FLtpZHlJWUJosY6+ck393xE0NNR2uyzkLL4Qa+NZ4u1s/T03fVDmtkRKVpSJpP4qBJYdAUW2y95Dv1U+vlKoSHk3Qv+Nj//xmda5t+HtmyxDE0uF2QzQ/2iJaASR6WqHll1yW3BNJBDhaTMn19Rzh11RDwX/xxj/H3LYidSZPzowtBqCDwGMql0NfitOqnKFwM0IKLK9GaNnfQdbu/7SRfdTF96ui47Y3HJIGwaKbYeYXbdhwNzpg4eyk81VR71pwztsw9ruIBba6Kw19QnlAn7pofe1j84bV6TBpHX7ddgHHXivHdhGDa2XtrDrPutuUY1fQQQU2a5gpy29yCVTgSzt7DX18l3qWCHwFBFQZPsUe/U3t4mZCnQv33UgF+eaNfX5PepAIfBEEVBk+0Q79rc16zws1scfD6T92Gavqf9tsqqKFQILRECR7QLBVlUpBBQCzxcBRbbPt+9VyxUCCoEFIqDIdoFgq6oUAgqB54uAItvn2/eq5QoBhcACEVBku0CwVVUKAYXA80VAke3z7XvVcoWAQmCBCCiyXSDYqiqFgELg+SKgyPb59r1quUJAIbBABBTZLhBsVZVCQCHwfBH4/yPe5+FhkkCiAAAAAElFTkSuQmCC"}}},{"metadata":{"trusted":true},"cell_type":"code","source":"# create constants for the loss function\nC1, C2 = tf.constant(70, dtype='float32'), tf.constant(1000, dtype=\"float32\")\n\n# define competition metric\ndef score(y_true, y_pred):\n    \"\"\"Calculate the competition metric\"\"\"\n    tf.dtypes.cast(y_true, tf.float32)\n    tf.dtypes.cast(y_pred, tf.float32)\n    sigma = y_pred[:, 2] - y_pred[:, 0]\n    fvc_pred = y_pred[:, 1]\n    \n    sigma_clip = tf.maximum(sigma, C1)\n    delta = tf.abs(y_true[:, 0] - fvc_pred)\n    delta = tf.minimum(delta, C2)\n    sq2 = tf.sqrt( tf.dtypes.cast(2, dtype = tf.float32) )\n    metric = (delta / sigma_clip) * sq2 + tf.math.log(sigma_clip * sq2)\n    return K.mean(metric)\n\n# define pinball loss\ndef qloss(y_true, y_pred):\n    \"\"\"Calculate Pinball loss\"\"\"\n    # IMPORTANT: define quartiles, feel free to change here!\n    qs = [0.2, 0.50, 0.8]\n    q = tf.constant(np.array([qs]), dtype = tf.float32)\n    e = y_true - y_pred\n    v = tf.maximum(q * e, (q-1) * e)\n    return K.mean(v)\n\n# combine competition metric and pinball loss to a joint loss function\ndef mloss(_lambda):\n    \"\"\"Combine Score and qloss\"\"\"\n    def loss(y_true, y_pred):\n        return _lambda * qloss(y_true, y_pred) + (1 - _lambda) * score(y_true, y_pred)\n    return loss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"## GET TRAINING DATA AND TARGET VALUE\nseed_everything(1950)\n\n### Features: choose which features you want to use\nfeatures_list = ['baselined_week_scld', 'Percent_scld', 'Age_scld', 'base_FVC_scld', 'Male', 'Female', 'Ex-smoker', 'Never smoked', 'Currently smokes']\n\n# get target value\ny = train_df['FVC'].values.astype(float)\n\n# get training & test data\nX_train = train_df[features_list].values\nX_test = sub[features_list].values","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"import tensorflow_addons as tfa\n\ndef get_model():\n    \"Creates and returns a model\"\n   \n    inp = Layers.Input((len(features_list),), name = \"Patient\")\n    x = Layers.BatchNormalization()(inp)\n    x = tfa.layers.WeightNormalization(Layers.Dense(196, activation = \"elu\", name = \"d1\"))(x)\n    x = Layers.BatchNormalization()(x)\n    x = Layers.Dropout(0.3)(x)\n    x = tfa.layers.WeightNormalization(Layers.Dense(128, activation = \"elu\", name = \"d2\"))(x)\n    x = Layers.BatchNormalization()(x)\n    x = Layers.Dropout(0.2)(x)\n    # predicting the quantiles\n    p1 = Layers.Dense(3, activation = \"relu\", name = \"p1\")(x)\n    # quantile adjusting p1 predictions\n    p2 = Layers.Dense(3, activation = \"relu\", name = \"p2\")(x)\n    preds = Layers.Lambda(lambda x: x[0] + tf.cumsum(x[1], axis = 1), \n                     name = \"preds\")([p1, p2])\n    \n    model = Models.Model(inputs = inp, outputs = preds, name = \"NeuralNet\")\n    model.compile(loss = mloss(0.8), optimizer = tf.optimizers.Adam(), metrics = [score])\n        \n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"neuralNet = get_model()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Evaluation & Ensemble\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# create list with the all pathes to all models\nmodels_weights_list = []\nmodels_root = '../input/osic-basic-models-public'\n\nfor filename in os.listdir(models_root):\n    if filename.endswith('.h5'):\n        file_path = os.path.join(models_root, filename)\n        models_weights_list.append(file_path)\n        print(file_path)        \n    else:\n        continue","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# instantiate target arrays\ntrain_preds = np.zeros((X_train.shape[0], 3))\ntest_preds = np.zeros((X_test.shape[0], 3))\nval_scores = np.zeros((len(models_weights_list)))\n\n# Ensemble: Average predictions of all models\nfor counter, model_weights in enumerate(models_weights_list):\n    neuralNet.load_weights(model_weights)\n    \n    # predict and evaluate training-data\n    train_preds = neuralNet.predict(X_train, verbose = 0)        \n    val_scores[counter] = neuralNet.evaluate(X_train, y, verbose = 0)[1]    \n    print(f\"Based on training data, this model has a val_score of: {val_scores[counter]}\")\n        \n    # predict on test set and average the predictions over all folds\n    test_preds += neuralNet.predict(X_test, verbose = 0)\n\nprint(f\"\\nAverage val_score of all ensembled/averaged models: {np.mean(val_scores)}\")    \ntest_preds /= len(models_weights_list)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Prepare submission file\nIn the next section we are going to use the ```train_preds``` to calculate the optimized sigma, which is a measure for certainty or rather uncertainty. We can do that, as we have both: the model's estimate and the real data. We subtract the lower quartile from the upper quartile (defined in the loss function) and average it."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"## FIND OPTIMIZED STANDARD-DEVIATION\nsigma_opt = mean_absolute_error(y, train_preds[:,1])\nsigma_uncertain = train_preds[:,2] - train_preds[:,0]\nsigma_mean = np.mean(sigma_uncertain)\nprint(sigma_opt, sigma_mean)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## PREPARE SUBMISSION FILE WITH OUR PREDICTIONS\nsub['FVC1'] = test_preds[:, 1]\nsub['Confidence1'] = test_preds[:,2] - test_preds[:,0]\n\n# get rid of unused data and show some non-empty data\nsubmission = sub[['Patient_Week','FVC','Confidence','FVC1','Confidence1']].copy()\nsubmission.loc[~submission.FVC1.isnull()].head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.loc[~submission.FVC1.isnull(),'FVC'] = submission.loc[~submission.FVC1.isnull(),'FVC1']\n\nif sigma_mean < 70:\n    submission['Confidence'] = sigma_opt\nelse:\n    submission.loc[~submission.FVC1.isnull(),'Confidence'] = submission.loc[~submission.FVC1.isnull(),'Confidence1']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Okay, we made it! Let's finally check our stats and submit it!"},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.describe().T","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the following last step we overwrite our predictions with the known data from the orginal submission file to not waste known data."},{"metadata":{"trusted":true},"cell_type":"code","source":"org_test = pd.read_csv('../input/osic-pulmonary-fibrosis-progression/test.csv')\n\nfor i in range(len(org_test)):\n    submission.loc[submission['Patient_Week']==org_test.Patient[i]+'_'+str(org_test.Weeks[i]), 'FVC'] = org_test.FVC[i]\n    submission.loc[submission['Patient_Week']==org_test.Patient[i]+'_'+str(org_test.Weeks[i]), 'Confidence'] = 70","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission[[\"Patient_Week\",\"FVC\",\"Confidence\"]].to_csv(\"submission.csv\", index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <font color='blue'>Thanks a lot for reading, I hope you could gain as much insights from reading this as I got from writing it. If you liked it, an upvote is highly appreciated. If you are interested in more content like this, feel free to follow me! ;)</font>"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}