{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"!pip install mediapy -q","metadata":{"execution":{"iopub.status.busy":"2023-06-11T21:47:26.044974Z","iopub.execute_input":"2023-06-11T21:47:26.045379Z","iopub.status.idle":"2023-06-11T21:47:38.185946Z","shell.execute_reply.started":"2023-06-11T21:47:26.045336Z","shell.execute_reply":"2023-06-11T21:47:38.184544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport cv2\nimport pycolmap\n\nimport numpy as np\nimport mediapy as media\nimport pandas as pd\nimport plotly.express as px\nimport plotly.graph_objects as go\n\nfrom glob import glob\nfrom pathlib import Path\nfrom time import time","metadata":{"execution":{"iopub.status.busy":"2023-06-11T21:47:38.188360Z","iopub.execute_input":"2023-06-11T21:47:38.188860Z","iopub.status.idle":"2023-06-11T21:47:38.942333Z","shell.execute_reply.started":"2023-06-11T21:47:38.188818Z","shell.execute_reply":"2023-06-11T21:47:38.941445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset = 'heritage'\nscene = 'dioscuri'\nsrc = f'/kaggle/input/image-matching-challenge-2023/train/{dataset}/{scene}'","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:18:42.486838Z","iopub.execute_input":"2023-06-11T19:18:42.487216Z","iopub.status.idle":"2023-06-11T19:18:42.494682Z","shell.execute_reply.started":"2023-06-11T19:18:42.487186Z","shell.execute_reply":"2023-06-11T19:18:42.492838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# /kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/images/3DOM_FBK_IMG_1516.png\nl = glob(f'{src}/images/*')\n# print(l)","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:18:44.060737Z","iopub.execute_input":"2023-06-11T19:18:44.061105Z","iopub.status.idle":"2023-06-11T19:18:44.069077Z","shell.execute_reply.started":"2023-06-11T19:18:44.061077Z","shell.execute_reply":"2023-06-11T19:18:44.067896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"images = [cv2.cvtColor(cv2.imread(im), cv2.COLOR_BGR2RGB) for im in glob(f'{src}/images/*')]\nmedia.show_images(images, height=100, columns=7, titles=[i for i in range(len(images))])\n# Note that the image do get displayed in order of the list, because is how the code read\n# Noise images counting from 0 #: 11 man,maybe 23,24 man,25, 27 wall,32 crack, 42 crack,50 crack,58 crack,67,68,69 crack\n# 71 hand, 80 wall,82 column,86, 88, 89,91, 97 whatever, 99 hand again, 107 crack, 111 painted sun,113,116,126 crack,129 hand\n# 136. 138 whatever,139,140, 141 man,146 stands,148 crack,156 crack,158, 160,169. En total 172 imagenes contando desde 0 -> 173 BIEN\n# Total of 38 garbage images from 1 to 38.","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:18:45.363758Z","iopub.execute_input":"2023-06-11T19:18:45.364193Z","iopub.status.idle":"2023-06-11T19:18:53.004375Z","shell.execute_reply.started":"2023-06-11T19:18:45.364160Z","shell.execute_reply":"2023-06-11T19:18:53.002763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**It would awesome if I could just run a pre-trained image classifier to alert me about the images that do not relate to the rest of dataset, so I can delete them.**\nThis would be way more efficient for big datasets, when manual visual classification is not feasible.","metadata":{}},{"cell_type":"code","source":"img_0 = '/kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/images/IMG_0079.png'\nmedia.show_image(media.resize_image(cv2.cvtColor(cv2.imread(img_0), cv2.COLOR_BGR2RGB), (200,150)))","metadata":{"execution":{"iopub.status.busy":"2023-05-28T03:20:17.642530Z","iopub.execute_input":"2023-05-28T03:20:17.643080Z","iopub.status.idle":"2023-05-28T03:20:17.705380Z","shell.execute_reply.started":"2023-05-28T03:20:17.643030Z","shell.execute_reply":"2023-05-28T03:20:17.704018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_man = l[11]\nmedia.show_image(media.resize_image(cv2.cvtColor(cv2.imread(img_man), cv2.COLOR_BGR2RGB), (200,200)))","metadata":{"execution":{"iopub.status.busy":"2023-05-28T04:53:04.881389Z","iopub.execute_input":"2023-05-28T04:53:04.881776Z","iopub.status.idle":"2023-05-28T04:53:04.939068Z","shell.execute_reply.started":"2023-05-28T04:53:04.881747Z","shell.execute_reply":"2023-05-28T04:53:04.937899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"l_garbage = [l[11], l[23], l[24], l[25], l[27], l[32], l[42], l[50], l[58], l[67], l[68], l[69], l[71], l[80], l[82], l[86],\n             l[88], l[89], l[91],l[97], l[99], l[107], l[111], l[113], l[116], l[126], l[129], l[136], l[138],l[139], l[140],\n             l[141],l[146], l[148], l[156], l[158], l[160], l[169]]\n# Noise images counting from 0 #: 11 man,maybe 23,24 man,25, 27 wall,32 crack, 42 crack,50 crack,58 crack,67,68,69 crack\n# 71 hand, 80 wall,82 column,86, 88, 89,91, 97 whatever, 99 hand again, 107 crack, 111 painted sun,113,116,126 crack,129 hand\n# 136. 138 whatever,139,140, 141 man,146 stands,148 crack,156 crack,158, 160,169. En total 172 imagenes contando desde 0 -> 173 BIEN\n# Total of 38 garbage images from 1 to 38.\nprint(l_garbage)\nprint(f'Number of garbage images: {len(l_garbage)}')","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:18:53.006567Z","iopub.execute_input":"2023-06-11T19:18:53.007611Z","iopub.status.idle":"2023-06-11T19:18:53.017792Z","shell.execute_reply.started":"2023-06-11T19:18:53.007576Z","shell.execute_reply":"2023-06-11T19:18:53.016595Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"good_images = [im for im in glob(f'{src}/images/*') if im not in l_garbage]\ngood_images[:5]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:18:53.019031Z","iopub.execute_input":"2023-06-11T19:18:53.019362Z","iopub.status.idle":"2023-06-11T19:18:53.032462Z","shell.execute_reply.started":"2023-06-11T19:18:53.019335Z","shell.execute_reply":"2023-06-11T19:18:53.030800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(good_images))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:23:18.114907Z","iopub.execute_input":"2023-06-11T02:23:18.115710Z","iopub.status.idle":"2023-06-11T02:23:18.124093Z","shell.execute_reply.started":"2023-06-11T02:23:18.115680Z","shell.execute_reply":"2023-06-11T02:23:18.122821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"images2 = [cv2.cvtColor(cv2.imread(im), cv2.COLOR_BGR2RGB) for im in good_images]\nmedia.show_images(images2, height=100, columns=7,titles=[i for i in range(len(images2))])","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:18:54.193120Z","iopub.execute_input":"2023-06-11T19:18:54.193509Z","iopub.status.idle":"2023-06-11T19:19:00.336648Z","shell.execute_reply.started":"2023-06-11T19:18:54.193481Z","shell.execute_reply":"2023-06-11T19:19:00.335438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(images2), len(images))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:19:00.338584Z","iopub.execute_input":"2023-06-11T19:19:00.339707Z","iopub.status.idle":"2023-06-11T19:19:00.345564Z","shell.execute_reply.started":"2023-06-11T19:19:00.339667Z","shell.execute_reply":"2023-06-11T19:19:00.344210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## VGG19","metadata":{}},{"cell_type":"code","source":"pip install tensorflow keras","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:19:00.347121Z","iopub.execute_input":"2023-06-11T19:19:00.347585Z","iopub.status.idle":"2023-06-11T19:19:13.716463Z","shell.execute_reply.started":"2023-06-11T19:19:00.347555Z","shell.execute_reply":"2023-06-11T19:19:13.714763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nfrom tensorflow.keras.preprocessing.image import load_img, img_to_array\nfrom tensorflow.keras.applications.vgg19 import VGG19, preprocess_input\nfrom tensorflow.keras.models import Model","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:19:13.719023Z","iopub.execute_input":"2023-06-11T19:19:13.719560Z","iopub.status.idle":"2023-06-11T19:19:13.725996Z","shell.execute_reply.started":"2023-06-11T19:19:13.719522Z","shell.execute_reply":"2023-06-11T19:19:13.724966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load the pre-trained VGG19 model\nvgg19 = VGG19(weights='imagenet', include_top=True)\n\n# Create a new model that outputs the fc7 layer\nfc7_layer_model = Model(inputs=vgg19.input, outputs=vgg19.get_layer('fc2').output)\n\n# Define a function to load and preprocess the images\ndef load_and_preprocess_images(image_paths):\n    images = []\n    for path in image_paths:\n        # Load the image\n        img = load_img(path, target_size=(224, 224))\n        # Convert the image to a numpy array\n        img = img_to_array(img)\n        # Expand dimensions to match the input shape of VGG19 (batch size, height, width, channels)\n        img = np.expand_dims(img, axis=0)\n        # Preprocess the image (subtract mean RGB values)\n        img = preprocess_input(img)\n        # Add the preprocessed image to the list\n        images.append(img)\n    # Concatenate the images along the batch dimension\n    images = np.concatenate(images, axis=0)\n    return images\n\n# Assuming 'good_images' is a list of paths to images\n# Load and preprocess the images\nimages = load_and_preprocess_images(good_images)\n\n# Extract the fc7 layer activations or features\nfc7_features = fc7_layer_model.predict(images)\n\n# Print the shape of fc7_features\nprint(\"Shape of fc7_features:\", fc7_features.shape)\n\n# Print the model summary\nfc7_layer_model.summary()\n","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:20:02.731791Z","iopub.execute_input":"2023-06-11T19:20:02.732193Z","iopub.status.idle":"2023-06-11T19:22:33.917911Z","shell.execute_reply.started":"2023-06-11T19:20:02.732166Z","shell.execute_reply":"2023-06-11T19:22:33.916442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## LSA: Latent Semantic Analysis","metadata":{}},{"cell_type":"code","source":"from sklearn.decomposition import TruncatedSVD","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:33.920732Z","iopub.execute_input":"2023-06-11T19:22:33.921079Z","iopub.status.idle":"2023-06-11T19:22:33.926626Z","shell.execute_reply.started":"2023-06-11T19:22:33.921048Z","shell.execute_reply":"2023-06-11T19:22:33.925406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Perform Latent Semantic Analysis (LSA)\nlsa = TruncatedSVD(n_components=50)    # it went from 4096 to 50 \n# Maybe I could try changing this n_components to see if the performance of the SVR model improves \nlsa_features = lsa.fit_transform(fc7_features)\n\n# Print the shape of lsa_features\nprint(\"Shape of lsa_features:\", lsa_features.shape)","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:33.928664Z","iopub.execute_input":"2023-06-11T19:22:33.929017Z","iopub.status.idle":"2023-06-11T19:22:34.080974Z","shell.execute_reply.started":"2023-06-11T19:22:33.928987Z","shell.execute_reply":"2023-06-11T19:22:34.078121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#This file can be open with COLMAP according with the data competition description NOT USING CURRENTLY\ncameras_txt = '/kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/sfm/cameras.txt'\nimages_txt = '/kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/sfm/images.txt'\npoints3D_txt = '/kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/sfm/points3D.txt'\nscale_factor_txt = '/kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/sfm/scale_factor.txt'","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.087673Z","iopub.execute_input":"2023-06-11T19:22:34.088109Z","iopub.status.idle":"2023-06-11T19:22:34.106257Z","shell.execute_reply.started":"2023-06-11T19:22:34.088075Z","shell.execute_reply":"2023-06-11T19:22:34.105329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lsa_features[1]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:11:12.273911Z","iopub.execute_input":"2023-06-11T19:11:12.274324Z","iopub.status.idle":"2023-06-11T19:11:12.283390Z","shell.execute_reply.started":"2023-06-11T19:11:12.274280Z","shell.execute_reply":"2023-06-11T19:11:12.282159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So we have th 50th most important features from 136 images. That is going to be feed as X for the SVR","metadata":{}},{"cell_type":"markdown","source":"## Ordering traning labels data frame the same as our 'lsa_features'","metadata":{}},{"cell_type":"code","source":"train_labels = pd.read_csv(\"/kaggle/input/image-matching-challenge-2023/train/train_labels.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.107713Z","iopub.execute_input":"2023-06-11T19:22:34.108023Z","iopub.status.idle":"2023-06-11T19:22:34.152502Z","shell.execute_reply.started":"2023-06-11T19:22:34.107997Z","shell.execute_reply":"2023-06-11T19:22:34.150835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:27:57.533505Z","iopub.execute_input":"2023-06-11T02:27:57.533868Z","iopub.status.idle":"2023-06-11T02:27:57.563577Z","shell.execute_reply.started":"2023-06-11T02:27:57.533841Z","shell.execute_reply":"2023-06-11T02:27:57.562623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels.loc[(train_labels[\"dataset\"] == \"heritage\") & (train_labels[\"scene\"] == \"dioscuri\")]","metadata":{"execution":{"iopub.status.busy":"2023-06-05T01:01:31.511598Z","iopub.execute_input":"2023-06-05T01:01:31.512081Z","iopub.status.idle":"2023-06-05T01:01:31.534030Z","shell.execute_reply.started":"2023-06-05T01:01:31.512046Z","shell.execute_reply":"2023-06-05T01:01:31.532677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"l_garbage[:10]","metadata":{"execution":{"iopub.status.busy":"2023-06-05T01:02:21.680532Z","iopub.execute_input":"2023-06-05T01:02:21.681033Z","iopub.status.idle":"2023-06-05T01:02:21.689864Z","shell.execute_reply.started":"2023-06-05T01:02:21.680999Z","shell.execute_reply":"2023-06-05T01:02:21.688633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"substring_to_remove = \"/kaggle/input/image-matching-challenge-2023/train/\"\nbad_image_paths = [i.replace(substring_to_remove, \"\") for i in l_garbage]\nbad_image_paths[:10]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.155780Z","iopub.execute_input":"2023-06-11T19:22:34.156195Z","iopub.status.idle":"2023-06-11T19:22:34.183615Z","shell.execute_reply.started":"2023-06-11T19:22:34.156156Z","shell.execute_reply":"2023-06-11T19:22:34.181428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"good_train_labels = train_labels.loc[(train_labels[\"dataset\"] == \"heritage\") & (train_labels[\"scene\"] == \"dioscuri\") & (~train_labels[\"image_path\"].isin(bad_image_paths))]\ngood_train_labels","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.187294Z","iopub.execute_input":"2023-06-11T19:22:34.187763Z","iopub.status.idle":"2023-06-11T19:22:34.206635Z","shell.execute_reply.started":"2023-06-11T19:22:34.187721Z","shell.execute_reply":"2023-06-11T19:22:34.205396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# There is a quantity match of the good_train_labels and the good_images\nprint(len(good_images))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:28:55.452195Z","iopub.execute_input":"2023-06-11T02:28:55.452541Z","iopub.status.idle":"2023-06-11T02:28:55.457754Z","shell.execute_reply.started":"2023-06-11T02:28:55.452514Z","shell.execute_reply":"2023-06-11T02:28:55.456376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Remember that \"lsa_features\" have the same order than \"fc7_features\" and \"good_images\"\ngood_images[:5]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:30:19.812519Z","iopub.execute_input":"2023-06-11T02:30:19.812898Z","iopub.status.idle":"2023-06-11T02:30:19.818560Z","shell.execute_reply.started":"2023-06-11T02:30:19.812871Z","shell.execute_reply":"2023-06-11T02:30:19.817713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"substring_to_remove = \"/kaggle/input/image-matching-challenge-2023/train/\"\ngood_image_short_paths = [i.replace(substring_to_remove, \"\") for i in good_images]\ngood_image_short_paths[:5]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.208454Z","iopub.execute_input":"2023-06-11T19:22:34.208876Z","iopub.status.idle":"2023-06-11T19:22:34.218520Z","shell.execute_reply.started":"2023-06-11T19:22:34.208837Z","shell.execute_reply":"2023-06-11T19:22:34.217391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d = {'image_path': good_image_short_paths}\ndf = pd.DataFrame(data=d)\ndf[:5]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.220135Z","iopub.execute_input":"2023-06-11T19:22:34.220599Z","iopub.status.idle":"2023-06-11T19:22:34.233057Z","shell.execute_reply.started":"2023-06-11T19:22:34.220557Z","shell.execute_reply":"2023-06-11T19:22:34.231830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# \"ord_good_train_labels\" is a data frame ordered the same as the 'good_images' list, hence the same order than 'lsa_features'\nord_good_train_labels = good_train_labels.merge(df, how=\"right\", on=\"image_path\")\nord_good_train_labels","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.238125Z","iopub.execute_input":"2023-06-11T19:22:34.238628Z","iopub.status.idle":"2023-06-11T19:22:34.257968Z","shell.execute_reply.started":"2023-06-11T19:22:34.238596Z","shell.execute_reply":"2023-06-11T19:22:34.256861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" # ROTATION MATRIX LIST WE ARE GOING TO USE\nrotation_matrix_list =  list(ord_good_train_labels[\"rotation_matrix\"])\nrotation_matrix_list[:5]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.259360Z","iopub.execute_input":"2023-06-11T19:22:34.260376Z","iopub.status.idle":"2023-06-11T19:22:34.268494Z","shell.execute_reply.started":"2023-06-11T19:22:34.260334Z","shell.execute_reply":"2023-06-11T19:22:34.267363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rotation_matrix_list[1]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:37:47.847140Z","iopub.execute_input":"2023-06-11T02:37:47.847475Z","iopub.status.idle":"2023-06-11T02:37:47.852872Z","shell.execute_reply.started":"2023-06-11T02:37:47.847451Z","shell.execute_reply":"2023-06-11T02:37:47.851844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Number of elements a rotation matrix has per image on the ord_good_train_labels data frame\nn = rotation_matrix_list[1].split(\";\")\nprint(len(n))","metadata":{"execution":{"iopub.status.busy":"2023-06-05T02:49:58.125868Z","iopub.execute_input":"2023-06-05T02:49:58.126257Z","iopub.status.idle":"2023-06-05T02:49:58.131988Z","shell.execute_reply.started":"2023-06-05T02:49:58.126227Z","shell.execute_reply":"2023-06-05T02:49:58.130754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# TRASLATION VECTOR WE ARE GOING TO USE\ntranslation_vector_list = list(ord_good_train_labels[\"translation_vector\"])\ntranslation_vector_list[1]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:46:49.413760Z","iopub.execute_input":"2023-06-11T19:46:49.414214Z","iopub.status.idle":"2023-06-11T19:46:49.421289Z","shell.execute_reply.started":"2023-06-11T19:46:49.414179Z","shell.execute_reply":"2023-06-11T19:46:49.420385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This traslation vector does seems to be expressed in (x,y,z)","metadata":{}},{"cell_type":"markdown","source":"## SVR: Support Vector Regression","metadata":{}},{"cell_type":"markdown","source":"### Rotation matrix","metadata":{}},{"cell_type":"code","source":"import numpy as np\nfrom sklearn.svm import SVR\nfrom sklearn.multioutput import MultiOutputRegressor\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_squared_error\nfrom math import sqrt","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:22:34.280862Z","iopub.execute_input":"2023-06-11T19:22:34.281535Z","iopub.status.idle":"2023-06-11T19:22:34.289030Z","shell.execute_reply.started":"2023-06-11T19:22:34.281482Z","shell.execute_reply":"2023-06-11T19:22:34.288018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Knowing 'rotation_matrix_list' is a list of rotation matrices corresponding to 'lsa_features'\n# Split the data into training and testing sets\nX_train, X_test, y_train, y_test = train_test_split(lsa_features, rotation_matrix_list, test_size=0.2, random_state=42)\n\n# Convert the rotation matrices to numpy arrays\ny_train = np.array([list(map(float, matrix.split(';'))) for matrix in y_train])\ny_test = np.array([list(map(float, matrix.split(';'))) for matrix in y_test])\n\n# Create an instance of SVR for rotation matrix prediction\nsvr_rotation = SVR(kernel='rbf', verbose=True)\n\n# Create the Multioutput Regressor\nmt_svr_rotation = MultiOutputRegressor(svr_rotation)\n\n# Fit the SVR model to the training data\nmt_svr_rotation.fit(X_train, y_train)\n\n# Predict the rotation matrices for the test data\ny_pred_rotation = mt_svr_rotation.predict(X_test)","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:38:57.242705Z","iopub.execute_input":"2023-06-11T02:38:57.243095Z","iopub.status.idle":"2023-06-11T02:38:57.272789Z","shell.execute_reply.started":"2023-06-11T02:38:57.243063Z","shell.execute_reply":"2023-06-11T02:38:57.271520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(y_pred_rotation))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:43:50.356504Z","iopub.execute_input":"2023-06-11T02:43:50.356881Z","iopub.status.idle":"2023-06-11T02:43:50.361247Z","shell.execute_reply.started":"2023-06-11T02:43:50.356852Z","shell.execute_reply":"2023-06-11T02:43:50.360580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_rotation[:3]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:42:20.201289Z","iopub.execute_input":"2023-06-11T02:42:20.201653Z","iopub.status.idle":"2023-06-11T02:42:20.209351Z","shell.execute_reply.started":"2023-06-11T02:42:20.201626Z","shell.execute_reply":"2023-06-11T02:42:20.208131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_test[:3]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:42:28.661625Z","iopub.execute_input":"2023-06-11T02:42:28.662017Z","iopub.status.idle":"2023-06-11T02:42:28.670028Z","shell.execute_reply.started":"2023-06-11T02:42:28.661992Z","shell.execute_reply":"2023-06-11T02:42:28.668895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**The ground truth and the predicted rotation matrix are very similar!**","metadata":{}},{"cell_type":"code","source":"# Calculate the mean squared error (MSE) for rotation matrix prediction\nmse_rotation_one_1 = mean_squared_error(y_test[0], y_pred_rotation[0])\nprint(\"RMSE for rotation matrix prediction for the first image:\", sqrt(mse_rotation_one_1))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:46:24.598083Z","iopub.execute_input":"2023-06-11T02:46:24.598438Z","iopub.status.idle":"2023-06-11T02:46:24.604430Z","shell.execute_reply.started":"2023-06-11T02:46:24.598414Z","shell.execute_reply":"2023-06-11T02:46:24.603493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rmse_list = []\nfor im in range(len(y_test)):\n    rmse_list.append(mean_squared_error(y_test[im], y_pred_rotation[im]))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:52:03.463050Z","iopub.execute_input":"2023-06-11T02:52:03.463412Z","iopub.status.idle":"2023-06-11T02:52:03.475886Z","shell.execute_reply.started":"2023-06-11T02:52:03.463383Z","shell.execute_reply":"2023-06-11T02:52:03.474887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(rmse_list))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:52:07.287713Z","iopub.execute_input":"2023-06-11T02:52:07.288099Z","iopub.status.idle":"2023-06-11T02:52:07.292833Z","shell.execute_reply.started":"2023-06-11T02:52:07.288072Z","shell.execute_reply":"2023-06-11T02:52:07.292156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rmse_list[:3]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T02:52:19.698564Z","iopub.execute_input":"2023-06-11T02:52:19.698936Z","iopub.status.idle":"2023-06-11T02:52:19.705686Z","shell.execute_reply.started":"2023-06-11T02:52:19.698909Z","shell.execute_reply":"2023-06-11T02:52:19.704494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"string_y_test = [\";\".join([str(num) for num in row]) for row in y_test]\nstring_y_test[:2]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T03:44:59.147775Z","iopub.execute_input":"2023-06-11T03:44:59.148140Z","iopub.status.idle":"2023-06-11T03:44:59.155857Z","shell.execute_reply.started":"2023-06-11T03:44:59.148110Z","shell.execute_reply":"2023-06-11T03:44:59.154788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"string_y_pred_rotation = [\";\".join([str(num) for num in row]) for row in y_pred_rotation]\nstring_y_pred_rotation[:2]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T04:52:20.786888Z","iopub.execute_input":"2023-06-11T04:52:20.787243Z","iopub.status.idle":"2023-06-11T04:52:20.794503Z","shell.execute_reply.started":"2023-06-11T04:52:20.787217Z","shell.execute_reply":"2023-06-11T04:52:20.793092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dict_1 = {'rotation_matrix': string_y_test, 'rotation_matrix_pred': string_y_pred_rotation, 'rmse': rmse_list}\ndf_y_test = pd.DataFrame(data=dict_1)\ndf_y_test","metadata":{"execution":{"iopub.status.busy":"2023-06-11T04:53:42.457773Z","iopub.execute_input":"2023-06-11T04:53:42.458195Z","iopub.status.idle":"2023-06-11T04:53:42.470638Z","shell.execute_reply.started":"2023-06-11T04:53:42.458171Z","shell.execute_reply":"2023-06-11T04:53:42.469343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_y_test_data = ord_good_train_labels.merge(df_y_test, how=\"right\", on=\"rotation_matrix\")\ndf_y_test_rotation = df_y_test_data[['image_path','rotation_matrix','rotation_matrix_pred','rmse']]\ndf_y_test_rotation","metadata":{"execution":{"iopub.status.busy":"2023-06-11T04:55:06.156793Z","iopub.execute_input":"2023-06-11T04:55:06.157171Z","iopub.status.idle":"2023-06-11T04:55:06.178710Z","shell.execute_reply.started":"2023-06-11T04:55:06.157143Z","shell.execute_reply":"2023-06-11T04:55:06.177495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With this table we can see the name of the images that got the highest RMSE and maybe find out if there is too much noise on them so that the algorithm had problems with recognizing the most important features on the LSA, or maybe problems when fitting the Support Vector Regression model.","metadata":{}},{"cell_type":"code","source":"df_y_test_rotation.describe()","metadata":{"execution":{"iopub.status.busy":"2023-06-11T04:26:24.189855Z","iopub.execute_input":"2023-06-11T04:26:24.190238Z","iopub.status.idle":"2023-06-11T04:26:24.206969Z","shell.execute_reply.started":"2023-06-11T04:26:24.190208Z","shell.execute_reply":"2023-06-11T04:26:24.205486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"markdown","source":"### Translation vector","metadata":{}},{"cell_type":"code","source":"translation_vector_list[:3]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:06.618740Z","iopub.execute_input":"2023-06-11T19:47:06.619155Z","iopub.status.idle":"2023-06-11T19:47:06.625477Z","shell.execute_reply.started":"2023-06-11T19:47:06.619123Z","shell.execute_reply":"2023-06-11T19:47:06.624637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Knowing 'traslation_vector_list' is a list of traslation vectors corresponding to 'lsa_features'\n# Split the data into training and testing sets\nX_train_trans, X_test_trans, y_train_trans, y_test_trans = train_test_split(lsa_features, translation_vector_list, test_size=0.2, random_state=42)\n\n# Convert the traslation vectors to numpy arrays\ny_train_trans = np.array([list(map(float, vector.split(';'))) for vector in y_train_trans])\ny_test_trans = np.array([list(map(float, vector.split(';'))) for vector in y_test_trans])\n\n# Create an instance of SVR for traslation vector prediction\nsvr_translation = SVR(kernel='rbf', verbose=True)\n\n# Create the Multioutput Regressor\nmt_svr_translation = MultiOutputRegressor(svr_translation)\n\n# Fit the SVR model to the training data\nmt_svr_translation.fit(X_train_trans, y_train_trans)\n\n# Predict the traslation vectors for the test data\ny_pred_translation = mt_svr_translation.predict(X_test)","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:25.982305Z","iopub.execute_input":"2023-06-11T19:47:25.982740Z","iopub.status.idle":"2023-06-11T19:47:26.006514Z","shell.execute_reply.started":"2023-06-11T19:47:25.982711Z","shell.execute_reply":"2023-06-11T19:47:26.004580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(y_pred_translation))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:29.704608Z","iopub.execute_input":"2023-06-11T19:47:29.705591Z","iopub.status.idle":"2023-06-11T19:47:29.710922Z","shell.execute_reply.started":"2023-06-11T19:47:29.705558Z","shell.execute_reply":"2023-06-11T19:47:29.709517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_translation[:2]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:31.876549Z","iopub.execute_input":"2023-06-11T19:47:31.876949Z","iopub.status.idle":"2023-06-11T19:47:31.884657Z","shell.execute_reply.started":"2023-06-11T19:47:31.876907Z","shell.execute_reply":"2023-06-11T19:47:31.883405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_test_trans[:2]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:34.276920Z","iopub.execute_input":"2023-06-11T19:47:34.277294Z","iopub.status.idle":"2023-06-11T19:47:34.284958Z","shell.execute_reply.started":"2023-06-11T19:47:34.277266Z","shell.execute_reply":"2023-06-11T19:47:34.283786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Calculate the mean squared error (MSE) for rotation matrix prediction\nmse_translation_one = mean_squared_error(y_test_trans[0], y_pred_translation[0])\nprint(\"RMSE for translation vector prediction for the first image:\", sqrt(mse_translation_one))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:38.611198Z","iopub.execute_input":"2023-06-11T19:47:38.611579Z","iopub.status.idle":"2023-06-11T19:47:38.622709Z","shell.execute_reply.started":"2023-06-11T19:47:38.611551Z","shell.execute_reply":"2023-06-11T19:47:38.621370Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rmse_list_trans = []\nfor im in range(len(y_test_trans)):\n    rmse_list_trans.append(mean_squared_error(y_test_trans[im], y_pred_translation[im]))","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:44.553329Z","iopub.execute_input":"2023-06-11T19:47:44.553721Z","iopub.status.idle":"2023-06-11T19:47:44.566275Z","shell.execute_reply.started":"2023-06-11T19:47:44.553694Z","shell.execute_reply":"2023-06-11T19:47:44.565161Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rmse_list_trans[:3]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:47.624124Z","iopub.execute_input":"2023-06-11T19:47:47.625554Z","iopub.status.idle":"2023-06-11T19:47:47.632760Z","shell.execute_reply.started":"2023-06-11T19:47:47.625514Z","shell.execute_reply":"2023-06-11T19:47:47.631685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"string_y_test_trans = [\";\".join([str(num) for num in row]) for row in y_test_trans]\nstring_y_test_trans[:2]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:51.198809Z","iopub.execute_input":"2023-06-11T19:47:51.199187Z","iopub.status.idle":"2023-06-11T19:47:51.208207Z","shell.execute_reply.started":"2023-06-11T19:47:51.199160Z","shell.execute_reply":"2023-06-11T19:47:51.206673Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"string_y_pred_translation = [\";\".join([str(num) for num in row]) for row in y_pred_translation]\nstring_y_pred_translation[:2]","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:53.143174Z","iopub.execute_input":"2023-06-11T19:47:53.143567Z","iopub.status.idle":"2023-06-11T19:47:53.153988Z","shell.execute_reply.started":"2023-06-11T19:47:53.143538Z","shell.execute_reply":"2023-06-11T19:47:53.152930Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dict_2 = {'translation_vector': string_y_test_trans, 'translation_vector_pred': string_y_pred_translation, 'rmse_trans': rmse_list_trans}\ndf_y_test_trans = pd.DataFrame(data=dict_2)\ndf_y_test_trans","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:47:57.322712Z","iopub.execute_input":"2023-06-11T19:47:57.323101Z","iopub.status.idle":"2023-06-11T19:47:57.345251Z","shell.execute_reply.started":"2023-06-11T19:47:57.323072Z","shell.execute_reply":"2023-06-11T19:47:57.344122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_y_test_data_trans = ord_good_train_labels.merge(df_y_test_trans, how=\"right\", on=\"translation_vector\")\ndf_results_translation = df_y_test_data_trans[['image_path','translation_vector','translation_vector_pred','rmse_trans']]\ndf_results_translation","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:48:20.155239Z","iopub.execute_input":"2023-06-11T19:48:20.155639Z","iopub.status.idle":"2023-06-11T19:48:20.176833Z","shell.execute_reply.started":"2023-06-11T19:48:20.155601Z","shell.execute_reply":"2023-06-11T19:48:20.175770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_results_translation.describe()","metadata":{"execution":{"iopub.status.busy":"2023-06-11T19:48:37.148527Z","iopub.execute_input":"2023-06-11T19:48:37.149446Z","iopub.status.idle":"2023-06-11T19:48:37.164262Z","shell.execute_reply.started":"2023-06-11T19:48:37.149405Z","shell.execute_reply":"2023-06-11T19:48:37.163462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The SVR for the Translation vector does not perform as well as the SVR for the rotation matrix even though the are less data per image (x,y,z).","metadata":{}},{"cell_type":"markdown","source":"---","metadata":{}},{"cell_type":"code","source":"sample_submission = pd.read_csv(\"/kaggle/input/image-matching-challenge-2023/sample_submission.csv\")\nsample_submission","metadata":{"execution":{"iopub.status.busy":"2023-06-11T21:48:01.269846Z","iopub.execute_input":"2023-06-11T21:48:01.270209Z","iopub.status.idle":"2023-06-11T21:48:01.314105Z","shell.execute_reply.started":"2023-06-11T21:48:01.270181Z","shell.execute_reply":"2023-06-11T21:48:01.312893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here I got the question: ¿Should I train the SVR for each scene on each dataset so that the model gets good on them individually? or ¿should I run one single SVR for all the data and predict rotation and then do the same for translation?","metadata":{}},{"cell_type":"markdown","source":"## What I've learned so far?\n\n### Positive:\n* I have gained knowledge in computer vision problems and applications through my first hands-on project in this field. I utilized VGG19, LSA, and multi-target SVR for the first time.\n* I have learned how to use Chat-GPT to simplify certain parts of the machine learning process. It has been valuable in providing information about available methods, classes, and functions in both familiar and unfamiliar libraries. I have also realized that Chat-GPT is a tool and cannot complete the entire process, as it may occasionally generate errors, possibly due to my limited expertise in providing prompts.\n* This experience has enabled me to effectively manage three significant tasks in my life: studying, working, and conducting research. It has improved my agility in handling mental tasks.\n\n### Negative:\n* Although this project has been an incredible opportunity, it has also introduced additional stress that I needed to manage. Given my involvement in other activities, there were times when I felt burned out. I need to learn ways to relax within shorter periods of time and get used to higher levels of demanding workloads.\n* I acknowledge that I need to enhance my technical Python skills, particularly in areas such as list comprehension, as it proves highly beneficial in many cases.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}