{"cells":[{"metadata":{"trusted":true,"_uuid":"7dfbe0ab8f5d2f64c1c4369bda4cfe37bdde46a0"},"cell_type":"code","source":"# Supress Warnings\n\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dc710b477814d0dd975bd3594aab316bbb45e002"},"cell_type":"code","source":"# Import the numpy and pandas packages\n\nimport numpy as np\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6f28843024cac604d0c398bea670e2e12700fa8e"},"cell_type":"markdown","source":"## Task 1: Reading and Inspection\n\n-  ### Subtask 1.1: Import and read\n\nImport and read the movie database. Store it in a variable called `movies`."},{"metadata":{"trusted":true,"_uuid":"496c9e4575b0fd2e7d1b8ec4380661f9f9e271b1"},"cell_type":"code","source":"movies =pd.read_csv('../input/movie_dataset.csv') # Write your code for importing the csv file here\nmovies","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8d12050d45a7d7379c21cb53d088f04a7ff6bde2"},"cell_type":"markdown","source":"-  ### Subtask 1.2: Inspect the dataframe\n\nInspect the dataframe's columns, shapes, variable types etc."},{"metadata":{"trusted":true,"_uuid":"af7ead1186c9ebb026c16f3fedfa8fc224fd5375"},"cell_type":"code","source":"# Write your code for inspection here\nprint(movies.shape)\nprint(type(movies.info()))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7498583960d1a2e2d3d80765f241fe56f65d331b"},"cell_type":"markdown","source":"## Task 2: Cleaning the Data\n\n-  ### Subtask 2.1: Inspect Null values\n\nFind out the number of Null values in all the columns and rows. Also, find the percentage of Null values in each column. Round off the percentages upto two decimal places."},{"metadata":{"trusted":true,"_uuid":"7fedd1c31c90a69700856866fc83902b54f74067"},"cell_type":"code","source":"# Write your code for column-wise null count here\nmovies.isnull().sum(axis=0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7f0a96b9f2e2a6e3ddd7dd719872c2ea371c0f0a"},"cell_type":"code","source":"# Write your code for row-wise null count here\nmovies.isnull().sum(axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"05f45135bcc6ed176e32bf95ab356f3ebfefc57a"},"cell_type":"code","source":"# Write your code for column-wise null percentages here\nround(100*(movies.isnull().sum()/len(movies.index)),2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"779f51a7a2d057a1ecfc1acece482e8e26231b91"},"cell_type":"markdown","source":"-  ### Subtask 2.2: Drop unecessary columns\n\nFor this assignment, you will mostly be analyzing the movies with respect to the ratings, gross collection, popularity of movies, etc. So many of the columns in this dataframe are not required. So it is advised to drop the following columns.\n-  color\n-  director_facebook_likes\n-  actor_1_facebook_likes\n-  actor_2_facebook_likes\n-  actor_3_facebook_likes\n-  actor_2_name\n-  cast_total_facebook_likes\n-  actor_3_name\n-  duration\n-  facenumber_in_poster\n-  content_rating\n-  country\n-  movie_imdb_link\n-  aspect_ratio\n-  plot_keywords"},{"metadata":{"trusted":true,"_uuid":"b576ace3cf4202e90ec09eed3c03628187417938"},"cell_type":"code","source":"# Write your code for dropping the columns here. It is advised to keep inspecting the dataframe after each set of operations \nmovies=movies.drop('color', axis=1)\nmovies=movies.drop('director_facebook_likes', axis=1)\nmovies=movies.drop('actor_1_facebook_likes', axis=1)\nmovies=movies.drop('actor_2_facebook_likes', axis=1)\nmovies=movies.drop('actor_3_facebook_likes', axis=1)\nmovies=movies.drop('actor_2_name', axis=1)\nmovies=movies.drop('cast_total_facebook_likes', axis=1)\nmovies=movies.drop('actor_3_name', axis=1)\nmovies=movies.drop('duration', axis=1)\nmovies=movies.drop('facenumber_in_poster', axis=1)\nmovies=movies.drop('content_rating', axis=1)\nmovies=movies.drop('country', axis=1)\nmovies=movies.drop('movie_imdb_link', axis=1)\nmovies=movies.drop('aspect_ratio', axis=1)\nmovies=movies.drop('plot_keywords', axis=1)\nround(100*(movies.isnull().sum()/len(movies.index)),2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"834a929325ef3c707206416d887c5224453b05f6"},"cell_type":"markdown","source":"-  ### Subtask 2.3: Drop unecessary rows using columns with high Null percentages\n\nNow, on inspection you might notice that some columns have large percentage (greater than 5%) of Null values. Drop all the rows which have Null values for such columns."},{"metadata":{"trusted":true,"_uuid":"eb5b65cc0eeb7e9763c580e6f914855858a3a8b0"},"cell_type":"code","source":"# Write your code for dropping the rows here\nmovies = movies[~np.isnan(movies['gross'])]\nmovies = movies[~np.isnan(movies['budget'])]\nround(100*(movies.isnull().sum()/len(movies.index)),2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"35a82ee94daec2779c304113fb013063b8229fe7"},"cell_type":"markdown","source":"-  ### Subtask 2.4: Drop unecessary rows\n\nSome of the rows might have greater than five NaN values. Such rows aren't of much use for the analysis and hence, should be removed."},{"metadata":{"trusted":true,"_uuid":"8da8c91f75206c8c8475d1db6c6b063c8138b44a"},"cell_type":"code","source":"# Write your code for dropping the rows here\nmovies = movies[movies.isnull().sum(axis=1) <= 5]\nround(100*(movies.isnull().sum()/len(movies.index)),2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"de42eb6e7766ec217df05ae81cbd84f1137b94ed"},"cell_type":"markdown","source":"-  ### Subtask 2.5: Fill NaN values\n\nYou might notice that the `language` column has some NaN values. Here, on inspection, you will see that it is safe to replace all the missing values with `'English'`."},{"metadata":{"trusted":true,"_uuid":"0c39b15b7401e1d3968dde320c63c205ef768b3a"},"cell_type":"code","source":"# Write your code for filling the NaN values in the 'language' column here\nmovies.loc[pd.isnull(movies['language']), ['language']] = 'English'\nround(100*(movies.isnull().sum()/len(movies.index)), 2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6368a1130e1a08cc234f91835a7b4b4331be12c6"},"cell_type":"markdown","source":"-  ### Subtask 2.6: Check the number of retained rows\n\nYou might notice that two of the columns viz. `num_critic_for_reviews` and `actor_1_name` have small percentages of NaN values left. You can let these columns as it is for now. Check the number and percentage of the rows retained after completing all the tasks above."},{"metadata":{"trusted":true,"_uuid":"3665291cb0cd3a67bca34d72c12cc8b84a23e094"},"cell_type":"code","source":"# Write your code for checking number of retained rows here\nprint(movies.index)\nround(100*(len(movies.index)/5043),2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8470bb21760bd7c95e3c810e88a68b610c085b48"},"cell_type":"markdown","source":"**Checkpoint 1:** You might have noticed that we still have around `77%` of the rows!"},{"metadata":{"_uuid":"c31bc051109b62c972c7845235bb6e52ec9449f8"},"cell_type":"markdown","source":"## Task 3: Data Analysis\n\n-  ### Subtask 3.1: Change the unit of columns\n\nConvert the unit of the `budget` and `gross` columns from `$` to `million $`."},{"metadata":{"trusted":true,"_uuid":"c93348ed1292b3ab8b2e637b0fc84fb614456bf2"},"cell_type":"code","source":"# Write your code for unit conversion here\nmovies[['gross','budget']].apply(lambda x:x/1000000)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"87c60a4e6fbf79bc017ef9c6c65a0593549ebf9b"},"cell_type":"markdown","source":"-  ### Subtask 3.2: Find the movies with highest profit\n\n    1. Create a new column called `profit` which contains the difference of the two columns: `gross` and `budget`.\n    2. Sort the dataframe using the `profit` column as reference.\n    3. Extract the top ten profiting movies in descending order and store them in a new dataframe - `top10`"},{"metadata":{"trusted":true,"_uuid":"46dedfd1d4f63f90c196bbaa8d6d986371060553"},"cell_type":"code","source":"# Write your code for creating the profit column here\nmovies['profit']=movies['gross'] - movies['budget']\nmovies","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"318f6696bb327ea2d8cca4351c4dd4367fad442a"},"cell_type":"code","source":"# Write your code for sorting the dataframe here\nmovies.sort_values(by='profit',ascending=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c7e87ef45a27779ceeb94e985d50fce35ee41d90"},"cell_type":"code","source":"top10=movies.sort_values(by=['profit'],ascending=False).head(10) # Write your code to get the top 10 profiting movies here\ntop10","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dca66a750b68f52dc712719906f198bfeccde13c"},"cell_type":"markdown","source":"-  ### Subtask 3.3: Drop duplicate values\n\nAfter you found out the top 10 profiting movies, you might have notice a duplicate value. So, it seems like the dataframe has duplicate values as well. Drop the duplicate values from the dataframe and repeat `Subtask 3.2`."},{"metadata":{"trusted":true,"_uuid":"8fb1a6d38402660ea5366a6ea76e3a854df00390"},"cell_type":"code","source":"# Write your code for dropping duplicate values here\nmovies.drop_duplicates()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"939e374073d89eaff9073cc7c8da80d029ff2736"},"cell_type":"code","source":"# Write code for repeating subtask 2 here\ntop10=movies.sort_values(by=['profit'],ascending=False).head(10)\ntop10","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b105868df0f670075ec104655d8963533d4732b8"},"cell_type":"markdown","source":"**Checkpoint 2:** You might spot two movies directed by `James Cameron` in the list."},{"metadata":{"_uuid":"8bfe2a7393cb839a5486d13a358fe09458918106"},"cell_type":"markdown","source":"-  ### Subtask 3.4: Find IMDb Top 250\n\n    1. Create a new dataframe `IMDb_Top_250` and store the top 250 movies with the highest IMDb Rating (corresponding to the column: `imdb_score`). Also make sure that for all of these movies, the `num_voted_users` is greater than 25,000.\nAlso add a `Rank` column containing the values 1 to 250 indicating the ranks of the corresponding films.\n    2. Extract all the movies in the `IMDb_Top_250` dataframe which are not in the English language and store them in a new dataframe named `Top_Foreign_Lang_Film`."},{"metadata":{"trusted":true,"_uuid":"e49d41f2c48e6bc8431ef1675566dce78c11540c"},"cell_type":"code","source":"# Write your code for extracting the top 250 movies as per the IMDb score here. Make sure that you store it in a new dataframe \n# and name that dataframe as 'IMDb_Top_250'\nIMDb_Top_250= movies.sort_values(by=['imdb_score'],ascending=False).head(250) # stored top 250 movies with highest imdb rating in new dataframe IMDb_top_250\nIMDb_Top_250.loc[(IMDb_Top_250.num_voted_users>25),:]   # checking for all these movies,num_voted_users is greater than 25000\nIMDb_Top_250['Rank']=range(1,251)      # Add a new column 'Rank' ranging from 1 to 251 indicate the ranks of the corresponding films  \nIMDb_Top_250","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ee393f96512ea9a15d698358c343111858a215fd"},"cell_type":"code","source":"Top_Foreign_Lang_Film =IMDb_Top_250.loc[(IMDb_Top_250.language != 'English'),:]       # Write your code to extract top foreign language films from 'IMDb_Top_250' here\nTop_Foreign_Lang_Film ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b28892b219bf436ee1883cb605da35c891390369"},"cell_type":"markdown","source":"**Checkpoint 3:** Can you spot `Veer-Zaara` in the dataframe?"},{"metadata":{"_uuid":"beed51a9b771bb2a462da115ee69368db887d0c8"},"cell_type":"markdown","source":"- ### Subtask 3.5: Find the best directors\n\n    1. Group the dataframe using the `director_name` column.\n    2. Find out the top 10 directors for whom the mean of `imdb_score` is the highest and store them in a new dataframe `top10director`. "},{"metadata":{"trusted":true,"_uuid":"10976a75e0abb18b1aacb63f980c4f93f3ae0bc9"},"cell_type":"code","source":"# Write your code for extracting the top 10 directors here\ntop10director=movies.groupby('director_name')['imdb_score'].mean().sort_values(ascending=False).head(10)\ntop10director","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5cf97605f0dc3aa58676b5e0b78dd7ac555ea0d1"},"cell_type":"markdown","source":"**Checkpoint 4:** No surprises that `Damien Chazelle` (director of Whiplash and La La Land) is in this list."},{"metadata":{"_uuid":"ef8536dedd907f7980db7466571393a746450025"},"cell_type":"markdown","source":"-  ### Subtask 3.6: Find popular genres\n\nYou might have noticed the `genres` column in the dataframe with all the genres of the movies seperated by a pipe (`|`). Out of all the movie genres, the first two are most significant for any film.\n\n1. Extract the first two genres from the `genres` column and store them in two new columns: `genre_1` and `genre_2`. Some of the movies might have only one genre. In such cases, extract the single genre into both the columns, i.e. for such movies the `genre_2` will be the same as `genre_1`.\n2. Group the dataframe using `genre_1` as the primary column and `genre_2` as the secondary column.\n3. Find out the 5 most popular combo of genres by finding the mean of the gross values using the `gross` column and store them in a new dataframe named `PopGenre`."},{"metadata":{"trusted":true,"_uuid":"5fd3d55c04658c61a476832f34259503e34ea947"},"cell_type":"code","source":"# Write your code for extracting the first two genres of each movie here\nfirst=movies['genres'].apply(lambda x: pd.Series(x.split('|')))\nmovies['genre_1']=first[0]\nmovies['genre_2']=first[1]\nmovies.loc[pd.isnull(movies['genre_2']), ['genre_2']] = movies['genre_1']\nprint(movies.genre_1)\nprint(movies.genre_2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"970f1efb4aee263a07898c9f49ce1e29777f4f48"},"cell_type":"code","source":"movies_by_segment =movies.groupby(['genre_1','genre_2']) # Write your code for grouping the dataframe here\nmovies_by_segment","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4b100a4ab9a328c5f660884662a11505ee7f824a"},"cell_type":"code","source":"PopGenre =movies_by_segment['gross'].mean().sort_values(ascending=False).head(5) # Write your code for getting the 5 most popular combo of genres here\nPopGenre","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e7b0bd78c5e05500c06885bc8bfa3e21c5670651"},"cell_type":"markdown","source":"**Checkpoint 5:** Well, as it turns out. `Family + Sci-Fi` is the most popular combo of genres out there!"},{"metadata":{"_uuid":"c136a411af866ee21702176c597796f3cb1ced56"},"cell_type":"markdown","source":"-  ### Subtask 3.7: Find the critic-favorite and audience-favorite actors\n\n    1. Create three new dataframes namely, `Meryl_Streep`, `Leo_Caprio`, and `Brad_Pitt` which contain the movies in which the actors: 'Meryl Streep', 'Leonardo DiCaprio', and 'Brad Pitt' are the lead actors. Use only the `actor_1_name` column for extraction. Also, make sure that you use the names 'Meryl Streep', 'Leonardo DiCaprio', and 'Brad Pitt' for the said extraction.\n    2. Append the rows of all these dataframes and store them in a new dataframe named `Combined`.\n    3. Group the combined dataframe using the `actor_1_name` column.\n    4. Find the mean of the `num_critic_for_reviews` and `num_user_for_review` and identify the actors which have the highest mean."},{"metadata":{"trusted":true,"_uuid":"6a87c6437f8cfdf877ded1e10dffc3854368dd2c"},"cell_type":"code","source":"# Write your code for creating three new dataframes here\nMeryl_Streep=movies.loc[(movies.actor_1_name=='Meryl Streep'),:].head(3891)  # Include all movies in which Meryl_Streep is the lead\nMeryl_Streep","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"71fc56efd1219a09c69414898ac2695b0f8cc39f"},"cell_type":"code","source":"Leo_Caprio=movies.loc[(movies.actor_1_name=='Leonardo DiCaprio'),:].head(3891) # Include all movies in which Leo_Caprio is the lead\nLeo_Caprio","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6faa06411470bdf0cefe847cdae960430cdb0663"},"cell_type":"code","source":"Brad_Pitt=movies.loc[(movies.actor_1_name=='Brad Pitt'),:].head(3891)  # Include all movies in which Brad_Pitt is the lead\nBrad_Pitt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f6d705a2baecc49b9fca8b47155ff5613b63f421"},"cell_type":"code","source":"# Write your code for combining the three dataframes here\nCombined=Meryl_Streep.append(Leo_Caprio).append(Brad_Pitt)\nCombined","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d946d07e465006cb35dababaacfc7fb48e1af1e6"},"cell_type":"code","source":"# Write your code for grouping the combined dataframe here\nactor_name=Combined.groupby('actor_1_name')   # grouping the Combined dataframe using actor_1_name and store it in new dataframe actor_name\nactor_name","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"acd1d8b498f3ce65dc66656c675cbdd12ef1b31a"},"cell_type":"code","source":"# Write the code for finding the mean of critic reviews and audience reviews here\ncritic_reviews=actor_name['num_critic_for_reviews'].mean().sort_values(ascending=False).head(49)\nprint(critic_reviews)\naudience_reviews=actor_name['num_user_for_reviews'].mean().sort_values(ascending=False).head(49)\nprint(audience_reviews)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"57221f943341c547ddae71f80bd86382e725c52c"},"cell_type":"markdown","source":"**Checkpoint 6:** `Leonardo` has aced both the lists!"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}