{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport itertools\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The goal of the competition is to develop an algorithm to estimate the absolute pose of vehicles (6 degrees of freedom) from a single image in a real-world traffic environment\n\n**From the problem description:**\n- The primary data is images of cars and related `pose` information. The pose information is formatted as strings, as follows:\n`model type, yaw, pitch, roll, x, y, z`\n\n- A concrete example with two cars in the photo:\n`5 0.5 0.5 0.5 0.0 0.0 0.0 32 0.25 0.25 0.25 0.5 0.4 0.7`\n\n- Submissions (per sample_submission.csv) are very similar, with the addition of a confidence score, and the removal of the model type. You are not required to predict the model type of the vehicle in question.\n   \n  `ID, PredictionString`\n  `ID_1d7bc9b31,0.5 0.5 0.5 0.0 0.0 0.0 1.0` indicating that this prediction has a confidence score of 1.0.\n\n<font color='red'>** So, the goal is to predict one or more sets of **:</font> `yaw, pitch, roll, x, y, z, confidence` <font color='red'>**for each picture**:</font>"},{"metadata":{},"cell_type":"markdown","source":"## Yaw / Pitch / Roll..what ? \n\nLet us first try to understand what the 6 degrees of freedom mean. \n\n- To completely specify the position of a 3-D object, we need to specify how it is rotated with respect to X/Y/Z axis, in addition to the the position a reference point (say center of the object) in the object. \n- As illustrated in the figure below, roll/pitch/yaw correspond to rotation of an object around the X/Y/Z axis respectively. [Pic Credit Link](https://devforum.roblox.com/t/take-out-pitch-from-rotation-matrix-while-preserving-yaw-and-roll/95204) "},{"metadata":{},"cell_type":"markdown","source":"![title](https://camo.githubusercontent.com/2d9fda441f1b838bc7e682ca1f3a4f7ab46c9e53/687474703a2f2f646f632e616c6465626172616e2e636f6d2f322d312f5f696d616765732f726f6c6c50697463685961772e706e67)"},{"metadata":{},"cell_type":"markdown","source":"Another good picture for pitch / yaw / roll in the context of the cars is shown below from [this reference](https://carsexplained.wordpress.com/2017/02/21/fundamentals-of-car-science-pitch-and-roll/): \n\n![](https://carsexplained.files.wordpress.com/2017/01/post1-2figure2.jpg)"},{"metadata":{},"cell_type":"markdown","source":"A good visualization app for Yaw / Pitch / Roll can be found here: http://www.ctralie.com/Teaching/COMPSCI290/Materials/EulerAnglesViz/"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Reading the Files\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read the test and train data sets. \ndf_train = pd.read_csv('../input/pku-autonomous-driving/train.csv')\nprint(\"Shapes of training dataset:\");print(df_train.shape)\nprint(\"Training Data Sample\");display(df_train.head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since the prediction string may contain more than one car, let us try and convert the data to have one row for each car. So, one image can have multiple rows of data. \n\n## Converting the training predictions to one row per car (multiple rows per image)"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Number of cars in each picture can be calculated using number of spaces in prediction string\ndf_train['NumCars'] = [int((x.count(' ')+1)/7) for x in df_train['PredictionString']]\ndf_train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Now let us expand the dataframe so that we can separate out all the 6 degrees of freedom for the positions. "},{"metadata":{"trusted":true},"cell_type":"code","source":"image_id_expanded = [item for item, count in \n                     zip(df_train['ImageId'], df_train['NumCars']) for i in range(count)]\nprediction_strings_expanded = df_train['PredictionString'].str.split(' ',expand = True).values.reshape(-1,7).astype(float)\nprediction_strings_expanded = prediction_strings_expanded[~np.isnan(prediction_strings_expanded).all(axis=1)]\ndf_train_expanded = pd.DataFrame(\n    {\n        'ImageId': image_id_expanded,\n        'model_type': prediction_strings_expanded[:,0].astype(int),\n        'yaw':prediction_strings_expanded[:,1],\n        'pitch':prediction_strings_expanded[:,2],\n        'roll':prediction_strings_expanded[:,3],\n        'x':prediction_strings_expanded[:,4],\n        'y':prediction_strings_expanded[:,5],\n        'z':prediction_strings_expanded[:,6]\n    })\nprint(\"Shapes of exapanded training dataset:\");print(df_train_expanded.shape)\nprint(\"Expanded Training Data Sample\");display(df_train_expanded.head(10))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Let us take a look at the training data column distributions and any possible correlations. \n\nThe following observations can be made from the plot below:\n\n1. Position related columns (yaw / pitch / roll / x / y / z) are not correlated with each other as expected. \n2. X / Y / Z values are most concentrated near 0 (close to the camera), while there are a few points that are very far from the camera. \n3. Roll (rotation around X) and Pitch (roation around Y) are distributed between $\\pi$ and -$\\pi$ (Complete 360 degree rotations allowed around X and Y). \n4. Yaw (rotation around Z) seem to be more contrained, which makes sense as cars cannot be completely flipped up-side down on the road side :)"},{"metadata":{},"cell_type":"markdown","source":"As illustrated in the figure below"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_expanded.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.pairplot(df_train_expanded)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Exploring the images\n\nLet us start looking at the images, starting with the first image (ID_8a6e65317). From above tables, it appears it has 5 cars that are of interest in it."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 15))\nplt.imshow(plt.imread('../input/pku-autonomous-driving/train_images/ID_8a6e65317.jpg'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"From the above plot, we can see that there is one car facing our car / camer (white car on the left side), two cars to the right that are perpendicular to the direction of the road, and one car (red) ahead that is in the same direction as our car. Let us see what the training data says.  "},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_expanded.head(6)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Let us take a look at a second picture, that also has a mask associated with it to mark out the cars that are very far away. \n\nFrom the picture below, we can see 3 cars, which matches the count in the tabe below extracted for that image id (the bus, the car far ahead in our lane, and the another car far away on the opposite side of the road). \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 15))\nplt.imshow(plt.imread('../input/pku-autonomous-driving/train_images/ID_7c4a3e0aa.jpg'))\n\n# mask with a transparaent overlay\nplt.imshow(plt.imread('../input/pku-autonomous-driving/train_masks/ID_7c4a3e0aa.jpg'), alpha = 0.5)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_expanded[df_train_expanded['ImageId'] == 'ID_7c4a3e0aa']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Guessing the X/Y/Z directions from the pictures\nIt should be possibel to guess the X/Y/Z directions from the data. \n\nMy guess at this point is : \n\n- X direction approximately corresponds to the direction perpendicular to the road from left to right. Left side of the camera is negative X and right side of the camera is likely positive X. \n- Z direction seems to correspond to the direction of the road\n- Y direction seems to correspond to the vertical direction (perpendicular to the plane of the road)"},{"metadata":{},"cell_type":"markdown","source":"### Exploring more cases with only one car is tagged in the picture"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train[df_train['NumCars'] == 1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15, 15))\nplt.imshow(plt.imread('../input/pku-autonomous-driving/train_images/ID_6dd7f07a5.jpg'))\n\n# mask with a transparaent overlay\nplt.imshow(plt.imread('../input/pku-autonomous-driving/train_masks/ID_6dd7f07a5.jpg'), alpha = 0.4)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_expanded[df_train_expanded['ImageId'] == 'ID_6dd7f07a5']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Blind-guess submission"},{"metadata":{},"cell_type":"markdown","source":"#### Just for kicks let us do a blind guess submission, and see where it takes us. \n\nHere is the logic for the submission:\n\n* From the previous analysis, it looks like: \n    * median amount of cars in each picture is about 11, \n    * pitch is taking values that are either -pi or 0 or + pi, and \n    * roll is taking mostly -pi and +pi. \n\n* So, we generate a string that has most probable combinations of pitch and roll (3x2 = 6 combinations), and assign median values for all the other variables like yaw, x, y, z and assign a randomly chosen confidence factor of 0.8\n\n* With this we generate a submission file. \n\n* <font color='red'>** Since no one seems to have submitted any valid solution at all, this blind guess is currently leading the leader board (as of 10/24/2019) !! Yaay !!**</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"pitch_vals  = [-3.14, 0, 3.14 ]\nroll_vals = [-3.14, 3.14]\n\nblind_guess_string = ''\nfor r in itertools.product(pitch_vals, roll_vals): \n    blind_guess_string = blind_guess_string + (str(df_train_expanded.yaw.median()) + ' '+ \n                                                         str(r[0]) + ' '+ str(r[1]) +  ' '+ \n                                                         str(df_train_expanded.x.median()) + ' '+ \n                                                         str(df_train_expanded.y.median()) + ' '+ \n                                                         str(df_train_expanded.z.median()) + ' 0.8 '\n                                                        )\nprint(blind_guess_string)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_submission = pd.read_csv('../input/pku-autonomous-driving/sample_submission.csv')\ndf_submission['PredictionString'] = blind_guess_string\ndf_submission.to_csv('blind_guess_submission.csv',index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}