{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":9993,"databundleVersionId":868308,"sourceType":"competition"}],"dockerImageVersionId":29661,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Peking University/Baidu - Autonomous Driving\n## Can you predict vehicle angle in different settings?\n![](https://i.imgur.com/nV7p7ib.png)\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true}},{"cell_type":"markdown","source":"# Data Description\n## Much of the Text was taken from the official page [here](https://www.kaggle.com/c/pku-autonomous-driving/data)\n\nThis dataset contains photos of streets, taken from the roof of a car. We're attempting to predict the position and orientation of all un-masked cars in the test images. You should also provide a confidence score indicating how sure you are of your prediction.\n\nPose Information (train.csv)\nNote that rotation values are angles expressed in radians, relative to the camera.\n\n- The primary data is images of cars and related `pose` information. The pose information is formatted as strings, as follows:\n`model type, yaw, pitch, roll, x, y, z`\n\n- A concrete example with two cars in the photo:\n`5 0.5 0.5 0.5 0.0 0.0 0.0 32 0.25 0.25 0.25 0.5 0.4 0.7`\n\n- Submissions (per sample_submission.csv) are very similar, with the addition of a confidence score, and the removal of the model type. You are not required to predict the model type of the vehicle in question.\n\n`ID, PredictionString`\n`ID_1d7bc9b31,0.5 0.5 0.5 0.0 0.0 0.0 1.0` indicating that this prediction has a confidence score of 1.0.\n\nOther Data:\n- **Image Masks (test_masks.zip / train_masks.zip)**\nSome cars in the images are not of interest (too far away, etc.). Binary masks are provided to allow competitors to remove them from consideration.\n\n- **Car Models**\n3D models of all cars of interest are available for download as pickle files - they can be compared against cars in images, used as references for rotation, etc.","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false,"jupyter":{"outputs_hidden":true}}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pylab as plt\nimport seaborn as sns\nimport pickle\nimport cv2\nimport json\nfrom keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img\nimport matplotlib.pyplot as plt\nfrom mpl_toolkits.mplot3d import Axes3D\n\nsns.set_style(\"whitegrid\")\nmy_pal = sns.color_palette(n_colors=10)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"File descriptions\n- **train.csv** - pose information for all of the images in the training set.\n- **train_images.zip** - images in the training set.\n- **train_masks.zip** - ignore masks for the training set. (Not all images have a mask.)\n- **test_images.zip** - images in the test set.\n- **test_masks.zip** - ignore masks for the test set. (Not all images have a mask.)\n- **sample_submission.csv** - a sample submission file in the correct format\n- **ImageId** - a unique identifier for each image (and related mask, if one exists).\n- **PredictionString** - a collection of poses and confidence scores.\n- **car_models.zip** - 3D models of the unmasked cars in the training / test images. They can be used for pose estimation, etc.\n- **camera.zip** - camera intrinsic parameters.","metadata":{}},{"cell_type":"code","source":"# Look at the data folder\n!ls -GFlash --color ../input/pku-autonomous-driving/","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Train csv file\nThe train file contains `Pose Information`","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('../input/pku-autonomous-driving/train.csv')\ntrain.head()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print('Example Prediction String....')\nprint(train['PredictionString'].values[0])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Expanding out the prediction string for the first vehicle\nWe know the order of each value in the prediction string. We can expand it out for the first vehicle and see some statistics for this first vehicle position.","metadata":{}},{"cell_type":"code","source":"train_expanded = pd.concat([train, train['PredictionString'].str.split(' ', expand=True)], axis=1)\ntrain_expanded = train_expanded.rename(columns={0 : '1_model_type', 1 : '1_yaw', 2 : '1_pitch',\n                                                3 : '1_roll', 4 : '1_x', 5 : '1_y', 6 : '1_z'})\ntrain_expanded.drop('PredictionString', axis=1).head()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Training Set, First Car Stats\n\n- Model type (You are not required to predict the model type of the vehicle in question.)","metadata":{}},{"cell_type":"code","source":"train_expanded.groupby('1_model_type')['ImageId'] \\\n    .count() \\\n    .sort_values() \\\n    .plot(kind='barh',\n          figsize=(15, 8),\n          title='First Car, Count by Model Type',\n          color=my_pal[0])\nplt.show()","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_expanded['1_yaw'] = pd.to_numeric(train_expanded['1_yaw'])\ntrain_expanded['1_yaw'] \\\n    .dropna() \\\n    .plot(kind='hist',\n          figsize=(15, 3),\n          bins=100,\n          title='Distribution of First car YAW',\n          color=my_pal[1])\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_expanded['1_pitch'] = pd.to_numeric(train_expanded['1_pitch'])\ntrain_expanded['1_pitch'] \\\n    .dropna() \\\n    .plot(kind='hist',\n          figsize=(15, 3),\n          bins=100,\n          title='Distribution of First car pitch',\n          color=my_pal[2])\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_expanded['1_roll'] = pd.to_numeric(train_expanded['1_roll'])\ntrain_expanded['1_roll'] \\\n    .dropna() \\\n    .plot(kind='hist',\n          figsize=(15, 3),\n          bins=100,\n          title='Distribution of First car roll',\n          color=my_pal[3])\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## X, Y, and Z features","metadata":{}},{"cell_type":"code","source":"train_expanded['1_x'] = pd.to_numeric(train_expanded['1_x'])\ntrain_expanded['1_y'] = pd.to_numeric(train_expanded['1_y'])\ntrain_expanded['1_z'] = pd.to_numeric(train_expanded['1_z'])\ntrain_expanded['1_x'] \\\n    .dropna() \\\n    .plot(kind='hist',\n          figsize=(15, 3),\n          bins=100,\n          title='Distribution of x',\n          color=my_pal[0])\nplt.show()\ntrain_expanded['1_y'] \\\n    .dropna() \\\n    .plot(kind='hist',\n          figsize=(15, 3),\n          bins=100,\n          title='Distribution of y',\n          color=my_pal[1])\nplt.show()\ntrain_expanded['1_z'] \\\n    .dropna() \\\n    .plot(kind='hist',\n          figsize=(15, 3),\n          bins=100,\n          title='Distribution of z',\n          color=my_pal[2])\nplt.show()","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Sample submission","metadata":{}},{"cell_type":"code","source":"ss = pd.read_csv('../input/pku-autonomous-driving/sample_submission.csv')\nss.head()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Reading Images\n- Train and test images and masks are each in their own folder.\n- Lets look at the image with the most cars from the training set.\n- All are jpg files","metadata":{}},{"cell_type":"code","source":"# Lets look at the first few images on disk\n!ls -GFlash ../input/pku-autonomous-driving/train_images | head","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Image Example","metadata":{}},{"cell_type":"code","source":"plt.rcParams[\"axes.grid\"] = False\n\ntrain_ids = train['ImageId'].values\nimg_name = train.loc[2742]['ImageId']\nfig, ax = plt.subplots(figsize=(15, 15))\nimg = load_img('../input/pku-autonomous-driving/train_images/' + img_name + '.jpg')\nplt.imshow(img)\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Mask Example","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 15))\nmask = load_img('../input/pku-autonomous-driving/train_masks/' + img_name + '.jpg')\nplt.imshow(mask)\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Plotting Mask over the Images","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 15))\nplt.imshow(img)\nplt.imshow(mask, cmap=plt.cm.viridis, interpolation='none', alpha=0.5)\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Masks Next to Images","metadata":{}},{"cell_type":"code","source":"ids = train['ImageId'].values\nfig, axes = plt.subplots(4, 3, figsize=(18, 20))\nfor i in range(4):\n    img = load_img('../input/pku-autonomous-driving/train_images/' + ids[i] + '.jpg')\n    img_mask = load_img('../input/pku-autonomous-driving/train_masks/' + ids[i] + '.jpg')\n    #plt.subplot(1,2*(1+len(ids)),q*2-1)\n    ax=axes[i][0].imshow(img)\n    #plt.subplot(1,2*(1+len(ids)),q*2)\n    ax=axes[i][1].imshow(img_mask)\n    ax=axes[i][2].imshow(img)\n    ax=axes[i][2].imshow(img_mask, cmap=plt.cm.viridis, interpolation='none', alpha=0.4)\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Camera intrinsic parameters.","metadata":{}},{"cell_type":"code","source":"!cat ../input/pku-autonomous-driving/camera/camera_intrinsic.txt","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3D Car Models","metadata":{}},{"cell_type":"code","source":"!ls -GFlash ../input/pku-autonomous-driving/car_models/ | head","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Per the data description\n*3D models of all cars of interest are available for download as pickle files - they can be compared against cars in images, used as references for rotation, etc.*\n\nThe pickles were created in Python 2. For Python 3 users, the following code will load a given model:\n```\nwith open(model, \"rb\") as file:\n    pickle.load(file, encoding=\"latin1\")\n```\n\n**This doesn't appear to work on kaggle kernels, returns a `ModuleNotFoundError: No module named 'objloader'`**","metadata":{}},{"cell_type":"code","source":"# model = '../input/pku-autonomous-driving/car_models/aodi-Q7-SUV.pkl'\n# with open(model, \"rb\") as file:\n#     pickle.load(file, encoding=\"latin1\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3D Car Model JSON Files\nWe can however load the json files located in the `../input/pku-autonomous-driving/car_models_json/` directory","metadata":{}},{"cell_type":"code","source":"!ls -GFlash ../input/pku-autonomous-driving/car_models_json/ | head","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"with open('../input/pku-autonomous-driving/car_models_json/mazida-6-2015.json') as json_file:\n    car_model_data = json.load(json_file)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The File contains the car type, vertices and faces.","metadata":{}},{"cell_type":"code","source":"for keys in enumerate(car_model_data):\n    print(keys)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def plot_3d_car(model_json_file):\n    with open(f'../input/pku-autonomous-driving/car_models_json/{model_json_file}') as json_file:\n        car_model_data = json.load(json_file)\n\n    vertices = np.array(car_model_data['vertices'])\n    faces = np.array(car_model_data['faces']) - 1\n    car_type = car_model_data['car_type']\n    x, y, z = vertices[:,0], vertices[:,2], -vertices[:,1]\n    fig = plt.figure(figsize=(30, 10))\n    ax = plt.axes(projection='3d')\n    ax.plot_trisurf(x, y, faces, z,\n                    cmap='viridis', edgecolor='none')\n    ax.set_title(car_type)\n    ax.view_init(30, 0)\n    plt.show()\n    fig = plt.figure(figsize=(30, 10))\n    ax = plt.axes(projection='3d')\n    ax.plot_trisurf(x, y, faces, z,\n                    cmap='viridis', edgecolor='none')\n    ax.set_title(car_type)\n    ax.view_init(60, 0)\n    plt.show()\n    fig = plt.figure(figsize=(30, 10))\n    ax = plt.axes(projection='3d')\n    ax.plot_trisurf(x, y, faces, z,\n                    cmap='viridis', edgecolor='none')\n    ax.set_title(car_type)\n    ax.view_init(-20, 180)\n    plt.show()\n    return","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## MG GT\n![](https://i.ytimg.com/vi/tOG-EYjjyS0/maxresdefault.jpg)","metadata":{}},{"cell_type":"code","source":"plot_3d_car('MG-GT-2015.json')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Aodi Q7\n![](https://cars.usnews.com/static/images/Auto/izmo/i108385760/2019_audi_q7_angularfront.jpg)","metadata":{}},{"cell_type":"code","source":"plot_3d_car('aodi-Q7-SUV.json')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Mazia 6\n![](https://static.carsdn.co/cldstatic/wp-content/uploads/img-1360163241-1554498653476.jpg)","metadata":{}},{"cell_type":"code","source":"plot_3d_car('mazida-6-2015.json')","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}