{"cells":[{"metadata":{},"cell_type":"markdown","source":"# About the competition\nThis competition is arranged by Open Source Imaging Consortium (OSIC) - a non-profit organization.\n\nPulmonary Fibrosis is an incurable lung disease. It occurs when lung tissue becomes damaged and scarred. This affects proper functioning of lungs and infact breathing.\n\nExpectation from the competiton is to predict patient's severity of decline in the lung function based on data provided - CT scan of patient's lungs & allied details like gender, smoking status, FVC. We need to determine lung function based on the output from spirometer, which measures volume of air inhaled and exhaled. The challenge is to use machine learning techniques to make prediction.\n\nIf the prediction outcome is successful, it will benefit patients and their families to better understand any decline in lung function in advance and try for better cure or improved health condition.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 1. Importing the packages","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport plotly as plty\nimport seaborn as sns\nimport plotly.graph_objs as go\nfrom plotly.offline import iplot\nfrom plotly.subplots import make_subplots\nimport plotly.io as pio\nimport os\n%matplotlib inline\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path = '../input/osic-pulmonary-fibrosis-progression/'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(f'{path}train.csv')\ndf_test = pd.read_csv(f'{path}test.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. Training Data","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"2.1 Metadata Information","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"df_train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.describe(include='all').T","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_tmp = df_train.groupby(['Patient', 'Sex'])['SmokingStatus'].unique().reset_index()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_tmp","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_tmp['SmokingStatus'] = df_tmp['SmokingStatus'].str[0]\ndf_tmp['Sex'] = df_tmp['Sex'].str[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_tmp['SmokingStatus'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_tmp['Sex'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize = (20,6), sharex=True)\nsns.countplot(x='SmokingStatus',data=df_tmp,ax=ax[0])\nsns.countplot(x='SmokingStatus',hue='Sex', data=df_tmp,ax=ax[1])\nax[0].title.set_text('Smoking Status')\nax[1].title.set_text('Smoking Status Vs Sex')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# What do we have in training dataset (metadata info excluding CT Scan)\n* we have 1549 data with no missing values.\n* 176 unique patient data is made available  along with data related to their age, gender, smoking status, FVC, weeks\n* Age of patients is between 49 and 88. Average age of the patient within the dataset is 67\n* We have 139 Male and 37 female patients\n* We have 118 Ex-Smoker, 49 Never-Smoked and 9 people who are smoking currently (active)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. Test Data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We just have 5 patient data available in test set","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Exploration of Data will continue","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}