{"cells":[{"metadata":{},"cell_type":"markdown","source":"\n\n# COVID-19: current situation\n\n- EDA including **the recent updates, recovering country analysis & sigmoid fitting convergence date estimation**.\n- `plotly` visualization is heavy used","attachments":{"images.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAfoAAABjCAMAAABE1TJEAAAAflBMVEUAAAD///+8vLyMjIz19fU/Pz/7+/uHh4fr6+vw8PB/f393d3fl5eWVlZWcnJxGRkajo6MzMzMoKCjm5ubJycna2tqqqqrDw8OwsLDPz8+3t7dqamoxMTFubm7e3t4fHx9SUlJfX18bGxs5OTkQEBBDQ0NOTk5iYmIVFRULCwuwtSMmAAARuElEQVR4nO2djXqiOhCGoyACgloFUayIpbXt/d/gMTMB8guhx3bXLd/TZ7dAhJA3mUwmiSVk1KhRo0aNGjVq1KhRo0aNGvXgWk1H/RplJx79ZNRv0tuI/rdq9svQu2HuhfvJ3Pnl8n4fep8sPHKafKvP9Ahyfxl6v1q6JPTIYkT/29BvyXxEj/pt6P0Rfa0R/a/ViP7XakT/R3S+6eWP5uCXod9W6+CvQP8WBX5w+JM5IL8MfUScvwP9yoXc/FkNRO8Wh5vSwu1L6Kc0YRHcBdm99K+iP183X/jUAPTBYV5Wm1dCPj82VblIjFi3++Pq+eWTkNczTeerKXZUedfDMpoiYQdeTo9S/rq33zXKs6TY+ha1zBL924nqC4Vprfui3wVukL8O/pg1ej9+lj66CQsttKOU7qymm9XPNsmFBPv62Wd6tOQTBJ/ym8xOcXYf9FNI/JWGZKu7osdy2Q/+nCV6P9R++pLKCZOzTbopl2etcnr9qbYX/pOSPNCTuUy7Gr8d+vPki2Vpr3uif2YtSG6ZvbJDvzc1gUpM5y9Mz1l6AjjIZ2lu9u/0etjc1hr9rb4szfDt0Dss8Tc2+3uin7GXe+tPKsoKfdvkT+tplmXTeMWORRNbNKW1Wexput3poz5zEYhgFdF2GHAjsB2Hr6C/NYO9esMh6GsL9T6kHIfpUVp9wDh/OjwqWN0zFxJO2U0qvuH5Tt0FRDxbOLMwMVrTq0/tPUzoGWTPP0yX82vzFuVWc083L7Y26N/rD3zjuPtB+nqPWZKTXJ7p+0poyYz8ayzZ8S0zGk9czfEucMZkm4Hrujk0ot8JbP1dyV6jOkwUFcTxbdDDwIN2T+7KlOR/60E8fOaw64ZifDtmDZk8a0o9x0tvXJ2Adk0MLjnc6rN1De3QUx0u7EXUvqQgsRV6Sv0AbxsPLk1bPca43oHzH8ZuuWEBeMhVezHCjuidOwUnSv29HOmaPfpmiPGiZNgSPTh5IXRF7hfK004PEc3DDu9V05QloeN2NZjwCIfhnOnAkIk2uQukp+2JIegnEdrp0pPOW6KnbxrMPiFj5V2LmtNDoK/gtNFnbsR8IqNxQJv/3KLO4MRalxbuteGCgIPQT7ZP2lvboX9mGYCKN9Wn+f96BPToNx5NQFuhLzg1J5hDgrn0iatuaH+UUg5EX5cpf+uUxJEVeqjkt2Hdhf7vVvcubqYHQO+W9OSHErNThI1e39EzfNAHf7TNfgkf0dx6u5EvDETP/LOYO5OSpRX6c8Syegb7Ne8pvep9vlzOjzPD1csiXi6d01WZmf8a+rfj7WnOorQZwb9cTw48WxtdBXWjT+GkRaPHnr4zho48Wlj45qGaEPqGM39mKHofLlacI2GL/tRkAG5daJLESZYdSvrbJWGGxZ1qzMP80JidYiddE9BfD1mWJKWalyTJkqw5DA+19+JL0ySQTkQcNy2nmJrCfN3o8RG93v0kglGlwV9nQn+Zi/xCddmojh4Uo9BTD0XPWivnm9qih49AbOiF/uZpInrQqd0KpkzVJzZ65U0OlTgHIqB/AaRy7WANqT49EyEE/MQinOGHd+9ioaYXolMneuzrOkLttdAfMoFgwgy03htWaCVe4NOz4mByMHolWGiJfkPTbvF3IKuJkTH0C2kE4fBpVmprWQrXBYMP/oUykoS81HXvTYlPrkVoHof+ojxcO7XSiT6Bc3IF1ggzqJmY54X1g2vOcPwuJ4MyEg3IYPQBvNOsPWGJHh7BIjnw2p6aBtCXSgzaa+PIXLVw/eZX3kiL6LEI5KYJT6h7HBY885MkZTfkksPjW/T1MKpIElYDPe0ywE70aG36x/STvok4zApk4NSewDKWLL4LXZPoNQxGj/b13EYb7dC/0qKqpyywK1InqwF9TgF4e8B9Qsvf9srM3ERL9ABKfJktdw/JzYM2I48k4a6sH8HJRGb8N+tANDL0GoceMuiyBLOpZ5qI6kR/1JzTqYD5uXlPKhcKihvOpeAhSLP24Fk+iwZkOHo0r0lzbIcebE0za5MJR2LJwqXaf/rExsGluZk2d37mDydCgEhCH7dv1AiCY94z94CW9nOe8knpxRb9RopGrQ6GmZ1O9PDgV1PxtuoIz/CCmrThAv+QPynsBs1M8vuHo8e3bf0IO/RAtXGgYGDjKXM4NXrOL7vCK/AWO90KAz4wvHl7LKHH5iAuCXPagsXJj+KDGAT5bNAvuA92qhM95suIUirojngOCo0n5xhjYQgdCi7ikEb7w9FjY22DkFboYTQYNFNgOLRfy6nYawpuXSqnlJaPnTAPjeRxfSJdJ6x3Z7XhQn8306RXW/SxkjuD+tFbjOoRfa9PEMvJArCJgscEPd5G+uBw9FiM7HJa5YUN+nn7Lu3HfTkVohccdsxaR9j3jV7nmq2MHso44A0MuKi113/pvj29+i3ojespWqHt6435YbPgawjYgQ1v8cEjkRftDUePnjH7xIGsrdBDK+fMNsTzlYGRLrrfZ2Sf6CtyK76UaB5YfJ7Xsi1X5oMkxrvTqxL6pTGx8Mz/jx6Z9qJXBwzYOLmhPdTzszwq/ip6dtkSPTRN3g/HuiATBfTSSOykS8gJXC82rUSloF9Lz4axhleyo5mreWYrerFFD3nRDEoV3cXgWw4Cl2oy6Nm5oT1Uoov8wa8afNbXW6KH1IKlxIiG5F7V0TxeUEZd6OmIZdvG3hX06Ci24eBV+xJNmfkm9vRii/4MHv6hf4nRHd28vlXwal9fV5pmvIfrthR/cTh6tI7MnNihP9OAmXvlT2FrkxbrfAX9GdB3tHoiuZTwYm1wmFnevX4VFpRc25tghXd7bX4nepxs6+ZJZRXHZb2maM2x/2vihWCnn5Ww/nD0WKVYZbRDD4ZS6lDB85ZGVQPRP5fHxSLsM/g4nC3qUMAr9e/5GFzdXva6xkwvcOjrSJa37J7i60Rfas7phEOp3oAvNOmZGK2B4UsTBwRzq4aGhqOHllpPNtuhh9LVIZWCYfboq+O68AO39mK7DD6p/KaECKuGvCt5rh0pN1HNPj3Px/Cf6uJ096WSuFUneuOUuiQcpB3lJVGSfHiQFLM/CI+Azk71GYajhxpVr+a2Qg805DkUXCqUC+ds0ZfyRqDOVo8Wqn7SrnmHWh9td3qQtwPSkzx6UrWWNTP3+Z3o8WUspm/w/j0bHpGy1KY9ePqJe95Z/eRg9D7YykvzYAv0cHclfgMPEGc/7NBf1QbTjR4cu+DcYgikrJxanoXY8iGPYhh43rJIDAtJutFjT/ze05ontSeUdCfSOPj12TOaKHAGNPHgweixywZ/J63y1AY9lFWwlYQlKLj9VugdZnP9NHfC0/EEfX2XwScEpmUx4AktQamGL2Ezc+sJF+GMtBz7edl0rK5hN1z3Uo2yzVSn0jbHZmHm5HqEBYC+OP1NtxpsMHokA+7Ogexs0HcuMxHmcGzQM9M2PbGAbr+Hz6p8u4ahGdTzWjQNh5+SoccyerpZquGmj+11o8/5F+kQruFT177zSgz3go++N6+jzN9PhqPHKUGsiZbou0emfBFZoGeTtu0wfWOBvgJDQX+DXXSGbV+zeuTbsVSjUVnXFO2XBXSjxwLe9KzBaDLSWUewtFQfDh3ZoM6gbhJoKHp8U+x/7NBX3WsN+DbWjx4nJwtuaNUb0mmKho7loc7qd7UTupQPsuS1/T0c6vffHLHl+7qrPYuxMargdJYLPBk/3uHoYaN/Uy/49Vq8LfVxzrpbDESPDaaqD2zQw7A62mkEGCNu0WM/epxyLrnrNq0ebQXNO7A1TtGq+ysBgGHr1Ss2AF1334OeIe1fqIPN/mK8jpOx2htB9So9fBdtLRuGPkWHvBnU26AHwKWu7PCVuDbYjz4Tjqis0KOxuJIZrf056RDuCG0O4ci46y7FUlDVt/sG3fJZr6eHgzTzQBDDItqpoILlsKT/aSvZIPQF5qSuQ1boL9T5lEdTqDNzFhv1owfDJfjgVgYfRyRLzLn8rTSCcHVLcwjlb0QftmUsqg+9Z9jBpijhsq+K+ZhapwF92V1AG+uL9tND0KcbMcNW6OE+6nJo7o1abP3oPe6NUTB47Wv12EVtcdefPi9Mn41HCIJCNKKHmf+voGe7ZkgVqZcmBd/IWTXXLdNymYdpGPiDpZ+Bu6fvvO3R+3OuFCmNG3cL9DStdjTV3KptxP3o4QPC7Al0Ab2tHmyzB5axe3/3Zgh6GLammgv9++tZc94oC+a9/Cz444zvSWnaKavupsV7Lh39ftI5M2WmngG1Q+9FMSuA0q/zvrdBL6x7VoSOXrPkqh99JBzVH+lv9Wjq6SXvKl0Rd/fASKrtg+DdW/SvYi0Aq6XEh4jVF6rUruRF7IendHTxwnUE9XaBDxFx8+Vb5i/Magryor9uhf6wbG5zrIdqlujBGhnXNKHXUNaH/eixi2gGXxtWbj43Ha9H38y7yF5Z6fK7e3DjYmsYoPBb3o4wsX9RlozWsvkapaQeaHzEWeq7bhClWZ0VIYLXlN4pO9C9B250aOeOOuZ0G3NkCKwY0ZfO/KbwdOFbxab1N+zQw4qEwBTqZnv3muncfvQYlfKQTdnkhlsLYNpuWb+/FIKha3Zch6393sQAM2i7DxF9dbtczNnh205blUBW36AVtdM/H8/VrNq0g06xha+biY7Xp7fratZWxaorwI/BwLoPG4Beo7NTdzhu5NmhXwrgVEHWg3rVvUU0j72rnya3lkJ/m0qrukzoWShTXl6FdsfdHvJ9Vn/3LGcGRPTYuwRRst8nW1agpe617L43zzMs+djIY7FU+5Bbb9o9q8eMlykaOAB93HoLU7I/WKGHutIxmjoKZW2B/lmqwjn2AW1bNqFnvo48qNd4QHznDYQ2/JEk/V5x2y9Kdefqvq1SF3RNNGUY6oYHwt0xnWllgA/GTUSv2Td+dVrTss39jORW6MHJ61rH+AF1o16VbTN9sxG+HSBk7bb1Joz76+Ed1f29juQ6B0KYF061TWEnVbyo1L+W9XfkTvwdv6f/vJqn+rG+V8z5/fybctkH/qb522q1Ms6eBZfb1ZkQMnDLVa1rWZbHcMdsK9OU5Ikl+rW/3fqdK9loiu2WWfwdTS65TRd6jg/0v8QMlhfE1GSVNEEbOJgVt/vpOmBYXbJVz2/CtF3A6K9Fh59mruCs4FMcNYndyDH9yQV79BO60XO6DI+n0Mmz7j33RZbHi+NxEU+T/s3536Ipmdqi/yad4vVu7Qz8yk3o8vSD+tki3uXT/doxLsrmtJov97fES6c0pxmE/lG0J9Psj6P/kqjv5A7+ttuv6d9DP60OU5I9JnoYZpq32dxX/xj6QxjtSfKw6MHLM367+J31j6HfkUP+wOhhT/1PPexfQh9ED44e5lcNE4j317+EfkkO+4dGDzN35U897V9CvybpQ6PXbf36Rv0gejdy0yqfRN/zB9Bu1v7B0c+EvVffr59Cn4ZpQnawZGYZhL2bcocoC2mZxSTdPTR63IBg8yU4d9KPoFcC6rGX9+/k4+DCMpEE/j1wS0YWxE9zzyFRTIr1Y6N/ciAs/mPuPfkJ9G7k6QLqDrP8UeBWt9/9SQVBzGpBf2+0jSbv18kzmZyqyYpMwupmECdO5d3M++1nToI58WMSLR8Z/dIvCrbDy/vWP6wo6ZvRH8IoJ1lCsgOZAod9QdbR7W0ph/kknLoUwu3nOCHvk3A3IeWEVJM5xDac+MaHcp8Ryr0klPvxVg3gZ0G8OXFjEsTEX5JoTYo9oJ+SQ0YS7SMDErtk7t0GUX8Tem7i27jv4jukos+md1K2IPGcxA5xbj8xmcckXJJwTRa3nx057clxTy45/SmnpMzINSOzhP5UCTnfHF368/KZbEjyTJKKJDOSrEh2JVlJpiXJLyR/J/sj2Z/IbkHWt5+QLMP+R77Xj/xL9NlMcEU/FLxnUtF/5x/0HKWKfXe7W2hWRHyrVPTf9ZcfRul1pn8vIQ7lNbjfrxH9r9WI/tdKRf/DPc6oP6V/KYY/aqhG9L9W/GhS95drR/2rSju+v2HUqFGjRo0aNWrUqFGjRo0aNWrUX6z/AAe7ThKvltOyAAAAAElFTkSuQmCC"}},"execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Table of Contents\n\n\n**[Load Data](#id_load)**
\n**[Worldwide trend](#id_ww)**
\n**[Country-wise growth](#id_country)**
\n**[Going into province](#id_province)**
\n\n**[Europe](#id_europe)**
\n**[Asia](#id_asia)**
\n**[Which country is recovering now?](#id_recover)**
\n","execution_count":null},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_kg_hide-input":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import gc\nimport os\nfrom pathlib import Path\nimport random\nimport sys\n\nfrom tqdm.notebook import tqdm \n#tqdm to print progress in a script I'm running in a Jupyter notebook. I can print all messages to the console via tqdm.write().\nimport numpy as np\nimport pandas as pd\n\n\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom IPython.core.display import display, HTML\n\n\n\n# --- plotly ---\nfrom plotly import tools, subplots\nimport plotly.offline as py\npy.init_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport plotly.express as px\nimport plotly.figure_factory as ff\nimport plotly.io as pio\npio.templates.default = \"xgridoff\" #giving light grid background template for all the visualization\n\n# --- models ---\nfrom sklearn import preprocessing\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- gc — Garbage Collector interface\n\nThis module provides an interface to the optional garbage collector. It provides the ability to disable the collector, tune the collection frequency, and set debugging options. It also provides access to unreachable objects that the collector found but cannot free. Since the collector supplements the reference counting already used in Python, you can disable the collector if you are sure your program does not create reference cycles. Automatic collection can be disabled by calling gc.disable(). To debug a leaking program call gc.set_debug(gc.DEBUG_LEAK). Notice that this includes gc.DEBUG_SAVEALL, causing garbage-collected objects to be saved in gc.garbage for inspection.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"- IPython.core.display.DisplayObject\n\n(Create an audio object.)\n\nWhen this object is returned by an input cell or passed to the display function, it will result in Audio controls being displayed in the frontend (only works in the notebook).","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"- XGBoost Documentation\n\nXGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"- What is CatBoost?\n\nCatBoost is a recently open-sourced machine learning algorithm from Yandex. It can easily integrate with deep learning frameworks like Google’s TensorFlow and Apple’s Core ML. It can work with diverse data types to help solve a wide range of problems that businesses face today. To top it up, it provides best-in-class accuracy.\n\nIt is especially powerful in two ways:\n\n It yields state-of-the-art results without extensive data training typically required by other machine learning methods, and\n Provides powerful out-of-the-box support for the more descriptive data formats that accompany many business problems.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n# Load Data\n\nDownload latest data from Johns Hopkins University github repository: [https://github.com/CSSEGISandData/COVID-19](https://github.com/CSSEGISandData/COVID-19)","execution_count":null},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"\nimport os\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n# filenames.sort()\n# for filename in filenames:\n# print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"scrolled":true,"trusted":true},"cell_type":"code","source":"%%time\nimport requests\n\nfor filename in ['time_series_covid19_confirmed_global.csv',\n 'time_series_covid19_deaths_global.csv',\n 'time_series_covid19_recovered_global.csv',\n 'time_series_covid19_confirmed_US.csv',\n 'time_series_covid19_deaths_US.csv']:\n print(f'Downloading {filename}')\n\n url = f'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/{filename}'\n#if check_update and False:\n myfile = requests.get(url)\n open(filename, 'wb').write(myfile.content)\n \n #importing the datasets\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 1 : ANALYSIS FOR GLOBAL CASES","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from datetime import datetime #importing datetime to convert dates , String into datetime\n\ndef _convert_date_str(df):\n try:\n df.columns = list(df.columns[:4]) + [datetime.strptime(d, \"%m/%d/%y\").date().strftime(\"%Y-%m-%d\") for d in df.columns[4:]]\n except:\n print('_convert_date_str failed with %y, try %Y')\n df.columns = list(df.columns[:4]) + [datetime.strptime(d, \"%m/%d/%Y\").date().strftime(\"%Y-%m-%d\") for d in df.columns[4:]]\n\n#converting function for all the date columns into datetime format with exception handeling \n\nconfirmed_global_df = pd.read_csv('time_series_covid19_confirmed_global.csv')\n_convert_date_str(confirmed_global_df)\n\ndeaths_global_df = pd.read_csv('time_series_covid19_deaths_global.csv')\n_convert_date_str(deaths_global_df)\n\nrecovered_global_df = pd.read_csv('time_series_covid19_recovered_global.csv')\n_convert_date_str(recovered_global_df)","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"#confirmed dataframe\nconfirmed_global_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Lat and Long_: Dot locations on the dashboard. All points (except for Australia) shown on the map are based on geographic centroids, and are not representative of a specific address, building or any location at a spatial scale finer than a province/state. Australian dots are located at the centroid of the largest city in each state.","execution_count":null},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"#death dataframe\ndeaths_global_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#recovered dataframe\nrecovered_global_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Filter out problematic data points (The West Bank and Gaza had a negative value, cruise ships were associated with Canada, etc.)\nremoved_states = \"Recovered|Grand Princess|Diamond Princess\"\nremoved_countries = \"US|The West Bank and Gaza\"\n\n#Renaming columns\nconfirmed_global_df.rename(columns={\"Province/State\": \"Province_State\", \"Country/Region\": \"Country_Region\"}, inplace=True)\ndeaths_global_df.rename(columns={\"Province/State\": \"Province_State\", \"Country/Region\": \"Country_Region\"}, inplace=True)\nrecovered_global_df.rename(columns={\"Province/State\": \"Province_State\", \"Country/Region\": \"Country_Region\"}, inplace=True)\n\n#confirmed_global_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Replacing NaN values with \"nan\"\n# ~ Stands for Opposite \n\nconfirmed_global_df = confirmed_global_df[~confirmed_global_df[\"Province_State\"].replace(np.nan, \"nan\").str.match(removed_states)]\ndeaths_global_df = deaths_global_df[~deaths_global_df[\"Province_State\"].replace(np.nan, \"nan\").str.match(removed_states)]\nrecovered_global_df = recovered_global_df[~recovered_global_df[\"Province_State\"].replace(np.nan, \"nan\").str.match(removed_states)]\n\nconfirmed_global_df = confirmed_global_df[~confirmed_global_df[\"Country_Region\"].replace(np.nan, \"nan\").str.match(removed_countries)]\ndeaths_global_df = deaths_global_df[~deaths_global_df[\"Country_Region\"].replace(np.nan, \"nan\").str.match(removed_countries)]\nrecovered_global_df = recovered_global_df[~recovered_global_df[\"Country_Region\"].replace(np.nan, \"nan\").str.match(removed_countries)]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"#df.melt ===>>> Unpivot a DataFrame from wide to long format, optionally leaving identifiers set.\n\nconfirmed_global_melt_df = confirmed_global_df.melt(\n id_vars=['Country_Region', 'Province_State', 'Lat', 'Long'], value_vars=confirmed_global_df.columns[4:], var_name='Date', value_name='ConfirmedCases')\ndeaths_global_melt_df = deaths_global_df.melt(\n id_vars=['Country_Region', 'Province_State', 'Lat', 'Long'], value_vars=confirmed_global_df.columns[4:], var_name='Date', value_name='Deaths')\nrecovered_global_melt_df = recovered_global_df.melt(\n id_vars=['Country_Region', 'Province_State', 'Lat', 'Long'], value_vars=confirmed_global_df.columns[4:], var_name='Date', value_name='Recovered')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#confirmed_global_melt_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"#merging confirm,death and recovered dateframe\n\ntrain = confirmed_global_melt_df.merge(deaths_global_melt_df, on=['Country_Region', 'Province_State', 'Lat', 'Long', 'Date'])\ntrain = train.merge(recovered_global_melt_df, on=['Country_Region', 'Province_State', 'Lat', 'Long', 'Date'])","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"train.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"Confirm=pd.DataFrame(train.groupby(['Date'])[\"ConfirmedCases\"].agg([\"sum\"]))\nConfirm.max()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"date= Confirm.index[Confirm[\"sum\"] == 5031588 ]\ndate","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" - 3582327 Confirm Cases Recorded Globally till Date 08,june,2020 ","execution_count":null},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"train.head(10)","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"cnf=pd.DataFrame(train.groupby(['Country_Region'])[\"ConfirmedCases\"].agg(\"max\"))\nsorted_cnf=cnf.sort_values(\"ConfirmedCases\",ascending=False).head(10)\nsorted_cnf\n# create a dataframe with Groupby 'Country_egion' with ConfirmCases's max values\n#and Sorted it in Decsending order\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"country=list(sorted_cnf.index[:10])\ncountry\n#we can take it as list but that will not be compatible with Run time Data","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"colors=[\"r\",\"r\",\"r\",\"r\",\"r\",\"orange\",\"orange\",\"orange\",\"yellow\",\"yellow\"]\nplt.figure(figsize=(12,8))\nsns.barplot(x=country,y=\"ConfirmedCases\",data=sorted_cnf,palette=colors)\nplt.title(\"Top 10 Countries with Maximum Confirm Cases (which have Most Confirm Cases)\",fontsize=14)\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Confirm Cases\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Russia,Brazil,United_Kingdom,Spain,India are Top 5 Countries with Most Confirm Cases\n- INDIA is now in 5th Country with Most Confirm Case","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dth=pd.DataFrame(train.groupby(['Country_Region'])[\"Deaths\"].agg(\"max\"))\ndth\nsorted_dth=dth.sort_values(\"Deaths\",ascending=False).head(10)\nsorted_dth\ncountry=list(sorted_dth.index[:10])\ncountry\n\ncolors=[\"k\",\"k\",\"k\",\"k\",\"k\",\"r\",\"r\",\"r\",\"r\",\"orange\"]\nplt.figure(figsize=(12,8))\nsns.barplot(x=country,y=\"Deaths\",data=sorted_dth,palette=colors)\nplt.title(\"Top 10 Countreis with Maximum Deaths (which have most confirm cases)\",fontsize=14)\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Deaths\")\nplt.show()\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n - United_Kingdom,Italy,France,Spain,Brazil are Top 5 Countries with Most Death Cases\n - India is in 10th Country with Most Confirm Case\n\n","execution_count":null},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"rcvd=pd.DataFrame(train.groupby(['Country_Region'])[\"Recovered\"].agg(\"max\"))\nrcvd\nsorted_rcvd=rcvd.sort_values(\"Recovered\",ascending=False).head(10)\nsorted_rcvd\ncountry=list(sorted_rcvd.index[:10])\ncountry\n\ncolors=[\"g\",\"g\",\"g\",\"g\",\"skyblue\",\"skyblue\",\"skyblue\",\"orange\",\"orange\",\"orange\"]\nplt.figure(figsize=(12,8))\nsns.barplot(x=country,y=\"Recovered\",data=sorted_rcvd,palette=colors)\nplt.title(\"Top 10 Countreis with Maximum Recoveries(which have most confirm cases)\",fontsize=14)\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Recoveries\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n- Germany,Russia,Italy,Brazil are Top 4 Countries with Most Recovery\n- INDIA is in 8th Country with Most Good Recovery Rate \n\n","execution_count":null},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"#Top Countries with Most Confirm Cases\nsorted_cnf","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"#Top Countries with Most Death Cases\nsorted_dth","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"#Top Countries with Most Recovery Cases\nsorted_rcvd","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"x=sorted_cnf.merge(sorted_rcvd, on='Country_Region')\ndf_rc=pd.DataFrame(x)\n#df_rc\ncountry_rc=list(df_rc.index)\ncountry_rc\ndf_rc[\"Recovery_Percentage\"]=(df_rc.Recovered/df_rc.ConfirmedCases)*100\ndf_rc\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"colors=[\"#e31212\",\"#f0ac0e\",\"#52ff80\",\"#91fa2f\",\"#f77625\",\"#008223\",\"#1df557\",\"#07b836\",\"#f0e91d\"]\nsns.barplot(x=country_rc,y=\"Recovery_Percentage\",data=df_rc,palette=colors)\nplt.title(\"Recovery_Percentage Of Most Confirm Cases Country\",fontsize=14)\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Recovery\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Recovery pecentage is quite high of Germany,Iran and Italy [70% +]","execution_count":null},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"y=sorted_cnf.merge(sorted_dth, on='Country_Region')\ndf_dt=pd.DataFrame(y)\ndf_dt\ncountry_dt=list(df_dt.index)\n#country_dt\ndf_dt[\"Death_Percentage\"]=(df_dt.Deaths/df_dt.ConfirmedCases)*100\ndf_dt","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"colors=[\"r\",\"k\",\"darkgray\",\"k\",\"k\",\"r\",\"r\"]\nsns.barplot(x=country_dt,y=\"Death_Percentage\",data=df_dt,palette=colors)\nplt.title(\"Death_Percentage Of Most Confirm Cases Countries\",fontsize=14)\nplt.xlabel(\"Country\")\nplt.ylabel(\"Death_Pct\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Death Percentage of France ,Italy,United Kingdom is very high [14% +]","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 2: ANALYZE US","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# --- US ---\n#importing data\nconfirmed_us_df = pd.read_csv('time_series_covid19_confirmed_US.csv')\ndeaths_us_df = pd.read_csv('time_series_covid19_deaths_US.csv')\nconfirmed_us_df.head(3)\ndeaths_us_df.head(3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n#permanently_dropping _columns _which_we_will_not_use\n\nconfirmed_us_df.drop(['UID', 'iso2', 'iso3', 'code3', 'FIPS', 'Admin2', 'Combined_Key'], inplace=True, axis=1)\ndeaths_us_df.drop(['UID', 'iso2', 'iso3', 'code3', 'FIPS', 'Admin2', 'Combined_Key', 'Population'], inplace=True, axis=1)\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"confirmed_us_df.head(3)\ndeaths_us_df.head(3)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"\n#renaming_columns_as_previous\n\nconfirmed_us_df.rename({'Long_': 'Long'}, axis=1, inplace=True)\ndeaths_us_df.rename({'Long_': 'Long'}, axis=1, inplace=True)\n\n#converting_datetime_in_string\n\n_convert_date_str(confirmed_us_df)\n_convert_date_str(deaths_us_df)\n\n# clean\nconfirmed_us_df = confirmed_us_df[~confirmed_us_df.Province_State.str.match(\"Diamond Princess|Grand Princess|Recovered|Northern Mariana Islands|American Samoa\")]\ndeaths_us_df = deaths_us_df[~deaths_us_df.Province_State.str.match(\"Diamond Princess|Grand Princess|Recovered|Northern Mariana Islands|American Samoa\")]\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# --- Aggregate by province state ---\n\n#confirmed_us_df.groupby(['Country_Region', 'Province_State']) & resetting_index\nconfirmed_us_df = confirmed_us_df.groupby(['Country_Region', 'Province_State']).sum().reset_index()\ndeaths_us_df = deaths_us_df.groupby(['Country_Region', 'Province_State']).sum().reset_index()\n\nconfirmed_us_df.head(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# remove lat,long columns\n\nconfirmed_us_df.drop(['Lat', 'Long'], inplace=True, axis=1)\ndeaths_us_df.drop(['Lat', 'Long'], inplace=True, axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\n#Unpivot a DataFrame from wide to long format, optionally leaving identifiers set.\n\nconfirmed_us_melt_df = confirmed_us_df.melt(\n id_vars=['Country_Region', 'Province_State'], value_vars=confirmed_us_df.columns[2:], var_name='Date', value_name='ConfirmedCases')\ndeaths_us_melt_df = deaths_us_df.melt(\n id_vars=['Country_Region', 'Province_State'], value_vars=deaths_us_df.columns[2:], var_name='Date', value_name='Deaths')\n\nconfirmed_us_melt_df\ndeaths_us_melt_df","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"#merging_both_dataframe\ntrain_us = confirmed_us_melt_df.merge(deaths_us_melt_df, on=['Country_Region', 'Province_State', 'Date'])\ntrain_us.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### The Data_set of US MERGING WITH GLOBLE DATA","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train = pd.concat([train, train_us], axis=0, sort=False)\n\ntrain_us.rename({'Country_Region': 'country', 'Province_State': 'province', 'Date': 'date', 'ConfirmedCases': 'confirmed', 'Deaths': 'fatalities'}, axis=1, inplace=True)\ntrain_us['country_province'] = train_us['country'].fillna('') + '/' + train_us['province'].fillna('')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"train","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"%%time\ndatadir = Path('/kaggle/input/covid19-global-forecasting-week-4')\n\n# Read in the data CSV files\n#train = pd.read_csv(datadir/'train.csv')\n#test = pd.read_csv(datadir/'test.csv')\n#submission = pd.read_csv(datadir/'submission.csv')\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"train","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#test","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#submission","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#Renaming Colums\n\ntrain.rename({'Country_Region': 'country', 'Province_State': 'province', 'Id': 'id', 'Date': 'date', 'ConfirmedCases': 'confirmed', 'Deaths': 'fatalities', 'Recovered': 'recovered'}, axis=1, inplace=True)\n\n#created new column with the help of \"country\" and \"province\" and fill NaN value to (\" \")\n\ntrain['country_province'] = train['country'].fillna('') + '/' + train['province'].fillna('')\n\n# test.rename({'Country_Region': 'country', 'Province_State': 'province', 'Id': 'id', 'Date': 'date', 'ConfirmedCases': 'confirmed', 'Fatalities': 'fatalities'}, axis=1, inplace=True)\n# test['country_province'] = test['country'].fillna('') + '/' + test['province'].fillna('')","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cnf2=pd.DataFrame(train.groupby(['country'])[\"confirmed\"].agg(\"max\"))\nsorted_cnf2=cnf2.sort_values('confirmed',ascending=False).head(5)\ncountry=list(sorted_cnf2.index[:5])\ncountry","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"colors=[\"#fa0707\",\"#db4116\",\"#db5b16\",\"#db7916\",\"#db9a16\"]\n#plt.figure(figsize=(12,8))\nsns.barplot(x=country,y=\"confirmed\",data=sorted_cnf2,palette=colors)\nplt.title(r\"Top 10 Countries with Maximum Confirm Cases(including US)\",fontsize=14 )\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Confirm Cases\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"###### here we can see the Confirm Cases in US is Less than Brazil & Russia","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dth2=pd.DataFrame(train.groupby(['country'])[\"fatalities\"].agg(\"max\"))\ndth2\nsorted_dth2=dth2.sort_values(\"fatalities\",ascending=False).head(3)\nsorted_dth2\ncountry=list(sorted_dth2.index[:3])\ncountry\n\ncolors=[\"k\",\"#33322f\",\"#9e9c98\"]\n#plt.figure(figsize=(12,8))\nsns.barplot(x=country,y=\"fatalities\",data=sorted_dth2,palette=colors)\nplt.title(\"Top 10 Countreis with Maximum Deaths (Including US)\",fontsize=14)\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Deaths\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"###### This is a little bit Positive sign For US that The US's Death_Rate is lesser than UK,Brazil and Italy .","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"rcvd2=pd.DataFrame(train.groupby(['country'])[\"recovered\"].agg(\"max\"))\nrcvd2\nsorted_rcvd2=rcvd2.sort_values(\"recovered\",ascending=False).head(20)\nsorted_rcvd2\ncountry=list(sorted_rcvd2.index[:20])\ncountry\n\n\nplt.figure(figsize=(18,12))\nsns.barplot(x=country,y=\"recovered\",data=sorted_rcvd2)\nplt.title(\"Top 10 Countreis with Maximum Recoveries(including US)\",fontsize=14)\nplt.xlabel(\"Countries\")\nplt.ylabel(\"Recoveries\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"###### Little bit shocking but US is not even in Top 20 In top Recovered Countries Where Brazil is in Top","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Worldwide trend","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ww_df = train.groupby('date')[['confirmed', 'fatalities']].sum().reset_index()\nww_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#ww_df = train.groupby('date')[['confirmed', 'fatalities']].sum().reset_index()\nww_df['new_case'] = ww_df['confirmed'] - ww_df['confirmed'].shift(1)\n\n\"\"\"Shift index by desired number of periods with an optional time `freq`.\n\nWhen `freq` is not passed, shift the index without realigning the data.\nIf `freq` is passed (in this case, the index must be date or datetime,\nor it will raise a `NotImplementedError`), the index will be\nincreased using the periods and the `freq`.\"\"\"\n\nww_df.head()\n\n#here new_case column will be increase as per date column","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#unpivoting the dataframe\nww_melt_df = pd.melt(ww_df, id_vars=['date'], value_vars=['confirmed', 'fatalities', 'new_case'])\nww_melt_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we see the confirmed cases in world wide, it just look like exponential growth curve. The number is increasing very rapidly especially recently. **the number almost doubled in last 1 week**...\n\nConfirmed cases reached 1M people, and 52K people already died on April 2.
\nConfirmed cases reached 3.3M people, and 238K people already died on May 1.\n\nConfirmed cases reached 7.2M people, and 409K people already died on june 8 .\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.line(ww_melt_df, x=\"date\", y=\"value\", color='variable', \n title=\"Worldwide Confirmed/Death Cases Over Time\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Moreover, when we check the growth in log-scale below figure, we can see that the speed of confirmed cases growth rate **slightly increases** when compared with the beginning of March and end of March.
\nIn spite of the Lockdown policy in Europe or US, the number is still increasing rapidly.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.line(ww_melt_df, x=\"date\", y=\"value\", color='variable',\n title=\"Worldwide Confirmed/Death Cases Over Time (Log scale)\",\n log_y=True)\n#log_y: boolean (default `False`)\n\"\"\"If `True`, the y-axis is log-scaled in cartesian coordinates.(comment)\"\"\"\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It looks like `fatalities` curve is just shifted the `confirmed` curve to below in log-scale, which means mortality rate is almost constant.\n\nIs it true? Let's see mortality rate in detail.
\nWe see that mortality rate is kept almost 3%, however it is slightly **increasing gradually to go over 7%** at the end of April.\n\nWhy? I will show you later that Europe & US has more seriously infected by Coronavirus recently, and mortality rate is high in these regions.
\nIt might be because when too many people get coronavirus, the country cannot provide enough medical treatment.","execution_count":null},{"metadata":{"_kg_hide-input":true,"scrolled":false,"trusted":true},"cell_type":"code","source":"ww_df['mortality'] = ww_df['fatalities'] / ww_df['confirmed']\n#creating New_column with the help of \"fatalities & confirmed\"\nfig = px.line(ww_df, x=\"date\", y=\"mortality\", \n title=\"Worldwide Mortality Rate Over Time\")\nfig.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- As we can see here that the Mortality rate was going upwards till 26 April but there is a decreasing in the Mortality rate after that. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n# Country-wise growth","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#creating New data frame with the help of previous one\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"country_df = train.groupby(['date', 'country'])[['confirmed', 'fatalities']].sum().reset_index()\ncountry_df.tail()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"What kind of country is in the dataset? How's the distribution of number of confirmed cases by country?","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"countries = country_df['country'].unique()\nprint(f'{len(countries)} countries are in dataset:\\n{countries}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- there are 177 Countries in this Report","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"target_date = country_df['date'].max()\nprint('Date: ', target_date)\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i in [1, 10, 100, 1000, 10000]:\n n_countries = len(country_df.query('(date == @target_date) & confirmed > @i'))\n print(f'{n_countries} countries have more than {i} confirmed cases')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- np.log10() ====>>> function helps user to calculate Base-10 logarithm of x where x belongs to all the input array elements. ","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"ax = sns.distplot(np.log10(country_df.query('date == \"2020-06-08\"')['confirmed'] + 1))\nax.set_xlim([0, 6])\nax.set_xticks(np.arange(7))\n_ = ax.set_xticklabels(['0', '10', '100', '1k', '10k', '100k'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is difficult to see all countries so let's check top countries.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"top_country_df = country_df.query('(date == @target_date) & (confirmed > 1000)').sort_values('confirmed', ascending=False)\ntop_country_melt_df = pd.melt(top_country_df, id_vars='country', value_vars=['confirmed', 'fatalities'])\ntop_country_melt_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now **US, Brazil and Russia and Even India** has more confirmed cases than China, and we can see many Europe countries in the top.\n\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.bar(top_country_melt_df.iloc[::-1],\n x='value', y='country', color='variable', barmode='group',\n title=f'Confirmed Cases/Deaths on {target_date}', text='value', height=1500, orientation='h')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's check these major country's growth by date.\n\nAs we can see, Coronavirus hit **China** at first but its trend is slowing down in March which is good news.
\nBad news is 2nd wave comes to **Europe (Italy, Spain, Germany, France, UK)** at March.
\nsadly 3rd wave now comes to **US, whose growth rate is much much faster than China, or even Europe**. Its main spread starts from middle of March and its speed is faster than Italy. Now US seems to be in the most serious situation in terms of both total number and spread speed.
\n\nBut more sadly 3rd wave now comes to **Brazil and India, whose growth rate is much much faster than China**.\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"top30_countries = top_country_df.sort_values('confirmed', ascending=False).iloc[:30]['country'].unique()\ntop30_countries_df = country_df[country_df['country'].isin(top30_countries)]\nfig = px.line(top30_countries_df,\n x='date', y='confirmed', color='country',\n title=f'Confirmed Cases for top 30 country as of {target_date}')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"raw","source":"In terms of number of Confirm cases , Brazil & US are serious situation now.
\nMany countries have more fatalities than China now, including US, Italy, Spain, France, UK, Iran Belgium, Germany, Brazil, Netherlands and even India too.\n\n**US's spread speed is the faster, US's fatality cases become top1 on Apr 10th.**","execution_count":null},{"metadata":{"_kg_hide-input":true,"scrolled":true,"trusted":true},"cell_type":"code","source":"top30_countries = top_country_df.sort_values('fatalities', ascending=False).iloc[:30]['country'].unique()\ntop30_countries_df = country_df[country_df['country'].isin(top30_countries)]\nfig = px.line(top30_countries_df,\n x='date', y='fatalities', color='country',\n title=f'Fatalities for top 30 country as of {target_date}')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- In terms of number of fatalities, Brazil & US are serious situation now","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"##### Now let's see mortality rate by country","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"top_country_df = country_df.query('(date == @target_date) & (confirmed > 100)')\ntop_country_df['mortality_rate'] = top_country_df['fatalities'] / top_country_df['confirmed']\ntop_country_df = top_country_df.sort_values('mortality_rate', ascending=False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.bar(top_country_df[:30].iloc[::-1],\n x='mortality_rate', y='country',\n title=f'Mortality rate HIGH: top 30 countries on {target_date}', text='mortality_rate', height=800, orientation='h')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Belgium, France and Italy is the most serious situation respectively , whose mortality rate is over 14% as of 2020/6/08.\n\nWe can also find countries from all over the world when we see top mortality rate countries.
\n\n\nSpain, Netherlands, India, and UK . It shows this coronavirus is really world wide pandemic.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"###### How about the countries whose mortality rate is low?\n\nBy investigating the difference between above & below countries, we might be able to figure out what is the cause which leads death.
\nBe careful that there may be a case that these country's mortality rate is low due to these country does not report/measure fatality cases properly.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.bar(top_country_df[-30:],\n x='mortality_rate', y='country',\n title=f'Mortality rate LOW: top 30 countries on {target_date}', text='mortality_rate', height=800, orientation='h')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see number of confirmed cases on map. Again we can see Europe, US, MiddleEast (Turkey, Iran) and Asia (China, Korea) are Not here.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"all_country_df = country_df.query('date == @target_date')\nall_country_df['confirmed_log1p'] = np.log10(all_country_df['confirmed'] + 1)\nall_country_df['fatalities_log1p'] = np.log10(all_country_df['fatalities'] + 1)\nall_country_df['mortality_rate'] = all_country_df['fatalities'] / all_country_df['confirmed']","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.choropleth(all_country_df, locations=\"country\", \n locationmode='country names', color=\"confirmed_log1p\", \n hover_name=\"country\", hover_data=[\"confirmed\", 'fatalities', 'mortality_rate'],\n range_color=[all_country_df['confirmed_log1p'].min(), all_country_df['confirmed_log1p'].max()], \n color_continuous_scale=\"peach\", \n title='Countries with Confirmed Cases')\n\n# I'd like to update colorbar to show raw values, but this does not work somehow...\n# Please let me know if you know how to do this!!\ntrace1 = list(fig.select_traces())[0]\ntrace1.colorbar = go.choropleth.ColorBar(\n tickvals=[0, 1, 2, 3, 4, 5],\n ticktext=['1', '10', '100', '1000','10000', '10000'])\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we see Confirm cases rate on map, we see **US,Brazil & Russia is high** . Also we notice MiddleEast (Iran, Iraq) is high.\n\nWhen we see tropical area, I wonder why Phillipines and Indonesia are high while other countries (Malaysia, Thai, Vietnam, as well as Australia) are low.\n\nFor Asian region, Korea's mortality rate is lower than China or India, I guess this is due to the fact that number of inspection is quite many in Korea.\n\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\nfig = px.choropleth(all_country_df, locations=\"country\", \n locationmode='country names', color=\"fatalities_log1p\", \n hover_name=\"country\", range_color=[0, 4],\n #hover_data=['confirmed', 'fatalities', 'mortality_rate'],\n color_continuous_scale=\"peach\", \n title='Countries with fatalities')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we can see **US, Brazil, France,Spain,Iran** with most numer of fatalities","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.choropleth(all_country_df, locations=\"country\", \n locationmode='country names', color=\"mortality_rate\", \n hover_name=\"country\", range_color=[0, 0.12], \n color_continuous_scale=\"peach\", \n title='Countries with mortality rate')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Daily NEW confirmed cases trend\n\nHow about **DAILY new cases** trend?
\nWe find from below figure:\n - China has finished its peak at Feb 14, new confirmed cases are surpressed now.\n - Europe&US spread starts on mid of March, after China slows down.\n - I feel effect of lock down policy in Europe (Italy, Spain, Germany, France) now comes on the figure,\n the number of new cases are not so increasing rapidly at the end of May.\n - Current US new confirmed cases are the worst speed, recording worst speed at more than 30k people/day at peak. \n\n- Daily new confirmed cases start to decrease from April 26 or April 10, I hope this trend will continue.\n\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"country_df['prev_confirmed'] = country_df.groupby('country')['confirmed'].shift(1)\ncountry_df['new_case'] = country_df['confirmed'] - country_df['prev_confirmed']\ncountry_df['new_case'].fillna(0, inplace=True)\ntop30_country_df = country_df[country_df['country'].isin(top30_countries)]\n\nfig = px.line(top30_country_df,\n x='date', y='new_case', color='country',\n title=f'DAILY NEW Confirmed cases world wide')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- **US, Brazil and INDIA** are top 3 countries now with daily confirm cases","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Geographical animation: spready by date\n\nYou can see animation how confirmed cases spread over time, you can see trend moving to China -> Europe -> US.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"country_df['date'] = country_df['date'].apply(str)\ncountry_df['confirmed_log1p'] = np.log1p(country_df['confirmed'])\ncountry_df['fatalities_log1p'] = np.log1p(country_df['fatalities'])\n\nfig = px.scatter_geo(country_df, locations=\"country\", locationmode='country names', \n color=\"confirmed\", size='confirmed', hover_name=\"country\", \n hover_data=['confirmed', 'fatalities'],\n range_color= [0, country_df['confirmed'].max()], \n projection=\"natural earth\", animation_frame=\"date\", \n title='COVID-19: Confirmed cases spread Over Time', color_continuous_scale=\"portland\")\n# fig.update(layout_coloraxis_showscale=False)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can see animation how confirmed cases spread over time, you can see trend moving to China -> Europe -> US. But Europe is worse than US for number of fatalities now.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.scatter_geo(country_df, locations=\"country\", locationmode='country names', \n color=\"fatalities\", size='fatalities', hover_name=\"country\", \n hover_data=['confirmed', 'fatalities'],\n range_color= [0, country_df['fatalities'].max()], \n projection=\"natural earth\", animation_frame=\"date\", \n title='COVID-19: Fatalities growth Over Time', color_continuous_scale=\"portland\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"New cases trend: it looks like China is almost converged now.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"country_df.loc[country_df['new_case'] < 0, 'new_case'] = 0.\nfig = px.scatter_geo(country_df, locations=\"country\", locationmode='country names', \n color=\"new_case\", size='new_case', hover_name=\"country\", \n hover_data=['confirmed', 'fatalities'],\n range_color= [0, country_df['new_case'].max()], \n projection=\"natural earth\", animation_frame=\"date\", \n title='COVID-19: Daily NEW cases over Time', color_continuous_scale=\"portland\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Europe","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Ref: https://www.kaggle.com/abhinand05/covid-19-digging-a-bit-deeper\neurope_country_list =list([\n 'Austria','Belgium','Bulgaria','Croatia','Cyprus','Czechia','Denmark','Estonia','Finland','France','Germany','Greece','Hungary','Ireland',\n 'Italy', 'Latvia','Luxembourg','Lithuania','Malta','Norway','Netherlands','Poland','Portugal','Romania','Slovakia','Slovenia',\n 'Spain', 'Sweden', 'United Kingdom', 'Iceland', 'Russia', 'Switzerland', 'Serbia', 'Ukraine', 'Belarus',\n 'Albania', 'Bosnia and Herzegovina', 'Kosovo', 'Moldova', 'Montenegro', 'North Macedonia'])\n\ncountry_df['date'] = pd.to_datetime(country_df['date'])\ntrain_europe = country_df[country_df['country'].isin(europe_country_list)]\n#train_europe['date_str'] = pd.to_datetime(train_europe['date'])\ntrain_europe_latest = train_europe.query('date == @target_date')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we look into the Europe, its Northern & Eastern areas are relatively better situation compared to Eastern & Southern areas.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.choropleth(train_europe_latest, locations=\"country\", \n locationmode='country names', color=\"confirmed\", \n hover_name=\"country\", range_color=[1, train_europe_latest['confirmed'].max()], \n color_continuous_scale='portland', \n title=f'European Countries with Confirmed Cases as of {target_date}', scope='europe', height=800)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Especially **Russia and UK** are in more serious situation.\n\nNumber of confirmed cases rapidly increasing in **Russia now (as of May 1)**, **Russia** is now potentially very dangerous situation.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_europe_march = train_europe.query('date >= \"2020-03-01\"')\nfig = px.line(train_europe_march,\n x='date', y='confirmed', color='country',\n title=f'Confirmed cases by country in Europe, as of {target_date}')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.line(train_europe_march,\n x='date', y='fatalities', color='country',\n title=f'Fatalities by country in Europe, as of {target_date}')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we check daily new cases in Europe, we notice:\n\n - **UK and Russia** daily growth are more than Italy now, These countries are potentially more dangerous now.\n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_europe_march['prev_confirmed'] = train_europe_march.groupby('country')['confirmed'].shift(1)\ntrain_europe_march['new_case'] = train_europe_march['confirmed'] - train_europe_march['prev_confirmed']\nfig = px.line(train_europe_march,\n x='date', y='new_case', color='country',\n title=f'DAILY NEW Confirmed cases by country in Europe')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Daily new Cases Increasing Rate is increasing unexpexctedly in **Russia and UK** ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n# Asia","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"In Asia, India & Iran have many confirmed cases, followed by South Korea & Turkey. ","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"country_latest = country_df.query('date == @target_date')\n\nfig = px.choropleth(country_latest, locations=\"country\", \n locationmode='country names', color=\"confirmed\", \n hover_name=\"country\", range_color=[1, 50000], \n color_continuous_scale='portland', \n title=f'Asian Countries with Confirmed Cases as of {target_date}', scope='asia', height=800)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"top_asian_country_df = country_df[country_df['country'].isin(['China', 'Indonesia', 'Iran', 'Japan', 'Korea, South', 'Malaysia', 'Philippines','India'])]\n\nfig = px.line(top_asian_country_df,\n x='date', y='new_case', color='country',\n title=f'DAILY NEW Confirmed cases world wide')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The coronavirus hit Asia in early phase, how is the situation now?\nChina & Korea is already in decreasing phase.\n\nUnlike China or Korea, daily new confirmed cases were kept increasing on April, especially in **India and Iran**. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n# Which country is recovering now?","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We saw that Coronavirus now hits Asia , Europe & US, in serious situation. How does it converge?\n\nWe can refer other country where confirmed cases is already decreasing.\nHere I defined `new_case_peak_to_now_ratio`, as a ratio of current new case and the max new case for each country.\nIf new confirmed case is biggest now, its ratio is 1.\nIts ratio is expected to be low value for the countries where the peak has already finished. ","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"max_confirmed = country_df.groupby('country')['new_case'].max().reset_index()\ncountry_latest = pd.merge(country_latest, max_confirmed.rename({'new_case': 'max_new_case'}, axis=1))\ncountry_latest['new_case_peak_to_now_ratio'] = country_latest['new_case'] / country_latest['max_new_case']","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"recovering_country = country_latest.query('new_case_peak_to_now_ratio < 0.5')\nmajor_recovering_country = recovering_country.query('confirmed > 100')\nmajor_recovering_country.sort_values(by=['confirmed'], inplace=True)\nmajor_recovering_country\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.bar(major_recovering_country.sort_values('new_case_peak_to_now_ratio', ascending=False),\n x='new_case_peak_to_now_ratio', y='country',\n title=f'Mortality rate LOW: top 30 countries on {target_date}', text='new_case_peak_to_now_ratio', height=1000, orientation='h')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see by map **China , Austrailia, France** currently decreasing in morality rate countries....so we can say these countries are recovering ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's see a recovering countries.\n\n## China\n\nWhen we check each state stats, we can see Hubei, the starting place, is extremely large number of confirmed cases.
\nOther states records actually few confirmed cases compared to Hubei.","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"china_df = train.query('country == \"China\"')\nchina_df['prev_confirmed'] = china_df.groupby('province')['confirmed'].shift(1)\nchina_df['new_case'] = china_df['confirmed'] - china_df['prev_confirmed']\nchina_df.loc[china_df['new_case'] < 0, 'new_case'] = 0.","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = px.line(china_df,\n x='date', y='new_case', color='province',\n title=f'DAILY NEW Confirmed cases in China by province')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## The situation of Hubei now?\n\nHubei record its new case peak on Feb 14. And finally, new case was not found on March 19.\n\nTo become no new case found, it took **about 2month after confirmed cases occured**, and **1 month after the peak has reached.**
\nThis term will be the reference for other country to how long we must lock-down the city.\n\nThe lockdown of Hubei's capital Wuhan will be lifted on April 8, a milestone in China's war against the epidemic.","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}