{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.0.5"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This R environment comes with many helpful analytics packages installed\n# It is defined by the kaggle/rstats Docker image: https://github.com/kaggle/docker-rstats\n# For example, here's a helpful package to load\n\nlibrary(tidyverse) # metapackage of all tidyverse packages\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nlist.files(path = \"../input\")\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","execution":{"iopub.status.busy":"2023-05-14T11:10:33.443647Z","iopub.execute_input":"2023-05-14T11:10:33.445814Z","iopub.status.idle":"2023-05-14T11:10:34.642395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Goal of the Competition\n - The goal of this competition is to reconstruct accurate 3D maps. Last year's Image Matching Challenge focused on two-view matching. This year you will take one step further: your task will be to reconstruct the 3D scene from many different views.\n - Your work could be the key to unlocking mapping the world from assorted and noisy data sources, such as images uploaded by users to services like Google Maps.\n \n ### Dataset Description\n- Building a 3D model of a scene given an unstructured collection of images taken around it is a longstanding problem in computer vision research. Your challenge in this competition is to generate 3D reconstructions from image sets showing different types of scenes and accurately pose those images.\n\n### Files - sample_submission.csv A valid, randomly-generated sample submission with the following fields:\n\n - image_path: The image filename, including the path.\n- dataset: The unique identifier for the dataset.\n- scene: The unique identifier for the scene.\n- rotation_matrix: The first target column. A \n- matrix, flattened into a vector in row-major convection, with values separated by ;.\n- translation_vector: The second target column. A 3-D dimensional vector, with values separated by ;.\n- [train/test]/*/*/images A batch of images all taken near the same location. Some of training datasets may also contain a folder named images_full with additional images.\n\n- train/*/*/sfm A 3D reconstruction for this batch of images, which can be opened with colmap, the 3D structure-from-motion library bundled with this competition.\n\n- train/*/*/LICENSE.txt The license for this dataset.\n\n\n### train/train_labels.csv A list of images in these datasets, with ground truth.\n - dataset: The unique identifier for the dataset.\n- scene: The unique identifier for the scene.\n- image_path: The image filename, including the path.\n- rotation_matrix: The first target column. A \n- matrix, flattened into a vector in row-major convection, with values separated by ;.\n- translation_vector: The second target column. A 3-D dimensional vector, with values separated by ;.","metadata":{}},{"cell_type":"markdown","source":"### Source\n  - https://milospopovic.net/3d-maps-with-r/\n  - https://www.rayshader.com/\n  - https://krisbolton.com/2020/04/29/3D-maps-with-satellite-imagery-in-r.html\n  - https://www.r-bloggers.com/2023/04/fisheye-effect-with-r/\n  - https://www.r-bloggers.com/2018/11/image-segmentation-based-on-superpixels-and-clustering/\n  - https://plotly.com/r/displaying-images/\n  - https://arxiv.org/pdf/1612.01601.pdf\n  - https://search.r-project.org/CRAN/refmans/keras/html/application_vgg.html\n  - https://www.r-bloggers.com/2018/06/explaining-keras-image-classification-models-with-lime/\n  - https://www.r-bloggers.com/2016/01/color-quantization-in-r/\n  - https://www.r-bloggers.com/2020/06/visualizing-principle-components-for-images/\n  - https://www.r-bloggers.com/2020/04/predicting-images-using-convolutional-neural-network/\n  - https://imagefluency.com/articles/batch-processing.html\n  - https://imagefluency.com/articles/getting-started.html\n  - https://rpubs.com/Argaadya/image_conv","metadata":{}},{"cell_type":"code","source":"# devtools::install_github(\"tylermorganwall/rayshader\")\n#library(rayshader)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-14T11:10:34.644960Z","iopub.execute_input":"2023-05-14T11:10:34.676805Z","iopub.status.idle":"2023-05-14T11:10:34.687663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# # Import the packages needed:\n# packages <-   base::list(\n#   'stringi', 'dplyr', 'data.table', 'ggplot2','plyr','lubridate',\n#   'PerformanceAnalytics', 'svDialogs','plotly','corrplot','psych',\n#   'fastR','mosaic', 'DiagrammeR', 'rjson', 'jsonlite','cooltools')\n\n# # Function of packages checking:\n# pack <- function(pkg) {\n#   old_pkg <- packages[!(packages %in% installed.packages()[, \"Package\"])]\n#   for (i in old_pkg) {\n#       sapply(i, install.packages)\n#   }\n#   for (j in packages) {\n#       sapply(j, require, character.only = TRUE)\n#   }\n#   new_pkg <- packages[(packages %in% installed.packages()[, \"Package\"])]\n \n#     if (length(new_pkg) == length(packages)){\n#     print(\"All Packages are Loaded Successfully\")\n#   } else {\n#     print(\"Please check pack (Not All Packages are loaded)\")\n#   }\n#     }\n\n# # Execute the function:\n# pack(packages)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:10:34.690367Z","iopub.execute_input":"2023-05-14T11:10:34.691774Z","iopub.status.idle":"2023-05-14T11:10:34.703202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Install Packages\nmypack <- function(package){\n  new.package <- package[!(package %in% installed.packages()[, \"Package\"])]\n  if (length(new.package)) \n    install.packages(new.package, dependencies = TRUE)\n  sapply(package, require, character.only = TRUE)\n}","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:10:34.705762Z","iopub.execute_input":"2023-05-14T11:10:34.707145Z","iopub.status.idle":"2023-05-14T11:10:34.719996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#create a vector of most used R packages\npackages <- c(\"cooltools\",\"bayfoxr\")\n              ","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:10:34.722602Z","iopub.execute_input":"2023-05-14T11:10:34.724032Z","iopub.status.idle":"2023-05-14T11:10:34.747022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mypack(packages)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:10:34.749610Z","iopub.execute_input":"2023-05-14T11:10:34.750998Z","iopub.status.idle":"2023-05-14T11:12:05.105988Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Fisheye effect with R\n- Fisheye function below distorts a bitmap image with a fisheye effect or an anti-fisheye effect (rho > 0.5 or rho < 0.5)","metadata":{}},{"cell_type":"code","source":"fisheye_xy <- function(x, y, rho, stick) {\n  p <- c(x, y)\n  if(rho == 0.5) {\n    return(p)\n  }\n  m <- c(0.5, 0.5)\n  d <- p - m\n  r <- sqrt(c(crossprod(d)))\n  dnormalized <- d / r\n  mnorm <- sqrt(c(crossprod(m)))\n  power <- pi / mnorm * (rho - 0.5)\n  if(power > 0) {\n    bind <- if(stick == \"corners\") mnorm else m[2L]\n    uv <- m + dnormalized * tan(r*power) * bind / tan(bind*power)\n  } else {\n    bind <- m[2L]\n    uv <- m + dnormalized * atan(-10*r*power) * bind / atan(-10*bind*power)\n  }\n  uv\n}","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:05.113308Z","iopub.execute_input":"2023-05-14T11:12:05.114886Z","iopub.status.idle":"2023-05-14T11:12:05.127540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"library(imager)\nlibrary(cooltools)\n#' @param bitmapFile path to a bitmap file (jpg, png, ...)\n#' @param rho amount of effect; no effect if 0.5, fisheye if >0.5, \n#'   antifisheye if <0.5\n#' @param stick where to stick the image when rho>0.5, to the \n#'   corners or to the borders; if you stick to the corners, a \n#'   part of the image is lost\n#' @param bkg background color; it appears only if rho>0.5 and \n#'   stick=\"borders\" \nfisheye <- function(bitmapFile, rho, stick = \"corners\", bkg = \"black\") {\n  stopifnot(rho > 0, rho < 1)\n  stick <- match.arg(stick, c(\"borders\", \"corners\"))\n  # load the image\n  img <- load.image(bitmapFile)\n  dims <- dim(img)\n  nx <- dims[1L]\n  ny <- dims[2L]\n  nchannels <- dims[4L]\n  # if the image is gray, add colors\n  if(nchannels == 1L) {\n    img <- add.color(img)\n  } else if(nchannels != 3L) {\n    stop(\"Cannot process this image.\")\n  }\n  # fisheye matrix\n  PSI <- matrix(NA_complex_, nrow = nx, ncol = ny)\n  for(i in 1L:nx) {\n    x <- (i-1L) / (nx-1L)\n    for(j in 1L:ny) {\n      y <- (j-1L) / (ny-1L)\n      uv <- fisheye_xy(x, y, rho, stick)\n      PSI[i, j] <- complex(real = uv[1L], imaginary = uv[2L])\n    }\n  }\n  # take the r, g, b channels\n  r <- squeeze(R(img))\n  g <- squeeze(G(img))\n  b <- squeeze(B(img))\n  # interpolation\n  x_ <- seq(0, 1, length.out = nx)\n  y_ <- seq(0, 1, length.out = ny)\n  f_r <- approxfun2(x_, y_, r)\n  f_g <- approxfun2(x_, y_, g)\n  f_b <- approxfun2(x_, y_, b)\n  M_r <- f_r(Re(PSI), Im(PSI))\n  M_g <- f_g(Re(PSI), Im(PSI))\n  M_b <- f_b(Re(PSI), Im(PSI))\n  # set outside color\n  RGB <- col2rgb(bkg)[, 1L] / 255\n  M_r[is.na(M_r)] <- RGB[1L]\n  M_g[is.na(M_g)] <- RGB[2L]\n  M_b[is.na(M_b)] <- RGB[3L]\n  # convert to hex codes\n  rstr <- rgb(M_r, M_g, M_b)\n  dim(rstr) <- c(nx, ny)\n  # rotate\n  t(rstr)\n}","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-14T11:12:05.136998Z","iopub.execute_input":"2023-05-14T11:12:05.138506Z","iopub.status.idle":"2023-05-14T11:12:05.290221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transparent <- \"/kaggle/input/image-matching-challenge-2023/train/heritage/dioscuri/images/3DOM_FBK_IMG_1548.png\"\ngray        <- \"gray.png\"\ngray_color <- \"#aaaaaa\"\ncmd <- sprintf(\n  \"convert %s -background '%s' -alpha remove -alpha off %s\", \n  transparent, gray_color, gray\n)\nsystem(cmd)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:05.294262Z","iopub.execute_input":"2023-05-14T11:12:05.295989Z","iopub.status.idle":"2023-05-14T11:12:05.717582Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fisheye distortion of the image with rho=0.95\nimg <- fisheye(gray, rho = 0.95, stick = \"borders\", bkg = gray_color)\n# plot\nopar <- par(mar = c(0, 0, 0, 0))\nplot(c(-100, 100), c(-100, 100), type = \"n\", asp = 1, \n     xlab = NA, ylab = NA, axes = FALSE, xaxs = \"i\", yaxs = \"i\")\nrasterImage(img, -100, -100, 100, 100)\npar(opar)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:05.722379Z","iopub.execute_input":"2023-05-14T11:12:05.724431Z","iopub.status.idle":"2023-05-14T11:12:19.742802Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image segmentation based on Superpixels and Clustering\n- superpixels group pixels similar in color and other low-level properties. In this respect, superpixels address two problems inherent to the processing of digital images: firstly, pixels are merely a result of discretization; and secondly, the high number of pixels in large images prevents many algorithms from being computationally feasible.\n- superpixels() function of the OpenImageR package is based on the open source C++ code of the IMAGE AND VISUAL REPRESENTATION LAB IVRL and allows users to receive labels and output slic images.","metadata":{}},{"cell_type":"code","source":"library(OpenImageR)\nlibrary(arm)\nImage <- readImage('/kaggle/input/image-matching-challenge-2023/train/phototourism/british_museum/images/03077758_153556086.jpg')\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-14T11:12:19.745891Z","iopub.execute_input":"2023-05-14T11:12:19.747424Z","iopub.status.idle":"2023-05-14T11:12:21.432507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res_slic = superpixels(input_image = Image,\n                       method = \"slic\",\n                       superpixel = 200, \n                       compactness = 20,\n                       return_slic_data = TRUE,\n                       return_labels = TRUE, \n                       write_slic = \"\", \n                       verbose = TRUE)\nstr(res_slic)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:21.435473Z","iopub.execute_input":"2023-05-14T11:12:21.437568Z","iopub.status.idle":"2023-05-14T11:12:22.605128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res_slico = superpixels(input_image = Image,\n                        method = \"slico\",\n                        superpixel = 200, \n                        return_slic_data = TRUE,\n                        return_labels = TRUE, \n                        write_slic = \"\", \n                        verbose = TRUE)\nstr(res_slico)\n","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:22.607824Z","iopub.execute_input":"2023-05-14T11:12:22.609302Z","iopub.status.idle":"2023-05-14T11:12:23.348285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"par(mfrow=c(1,2), mar = c(0.2, 0.2, 0.2, 0.2))\nplot_slic = OpenImageR::NormalizeObject(res_slic$slic_data)\nplot_slic = grDevices::as.raster(plot_slic)\ngraphics::plot(plot_slic)\nplot_slico = OpenImageR::NormalizeObject(res_slico$slic_data)\nplot_slico = grDevices::as.raster(plot_slico)\ngraphics::plot(plot_slico)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:23.351090Z","iopub.execute_input":"2023-05-14T11:12:23.352528Z","iopub.status.idle":"2023-05-14T11:12:24.243515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"library(keras) \nlibrary(lime)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:24.246204Z","iopub.execute_input":"2023-05-14T11:12:24.247666Z","iopub.status.idle":"2023-05-14T11:12:24.446735Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load Imagenet model\nmodel <- application_vgg16(weights = \"imagenet\", include_top = TRUE)\nmodel","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:24.449243Z","iopub.execute_input":"2023-05-14T11:12:24.450661Z","iopub.status.idle":"2023-05-14T11:12:40.343120Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_superpixels(Image, n_superpixels = 35, weight = 10)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:40.346221Z","iopub.execute_input":"2023-05-14T11:12:40.347784Z","iopub.status.idle":"2023-05-14T11:12:47.359499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- VGG-19 is a convolutional neural network that is 19 layers deep.","metadata":{}},{"cell_type":"code","source":"model2 <- application_vgg19(weights = 'imagenet', include_top = FALSE)\n\nimg_path <- \"/kaggle/input/image-matching-challenge-2023/train/phototourism/buckingham_palace/images/00134044_3433575797.jpg\"\nimg <- image_load(img_path, target_size = c(224,224))\nx <- image_to_array(img)\nx <- array_reshape(x, c(1, dim(x)))\nx <- imagenet_preprocess_input(x)\n\nfeatures <- model2 %>% predict(x)\nplot(features)\n","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:47.362267Z","iopub.execute_input":"2023-05-14T11:12:47.363686Z","iopub.status.idle":"2023-05-14T11:12:51.228325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Color Quantization\n- Identify segments of an image based on RGB color values","metadata":{}},{"cell_type":"code","source":"library(\"grid\")\nlibrary(\"gridExtra\")\n\n### EX 1: show the full RGB image\ngrid.raster(Image)\n\n### EX 2: show the B channel in gray scale representing pixel intensity\ngrid.raster(Image[,,3])\n\n### EX 3: show the 3 channels in separate images\n# copy the image three times\nImage.R = Image\nImage.G = Image\nImage.B = Image\n\n# zero out the non-contributing channels for each image copy\nImage.R[,,2:3] = 0\nImage.G[,,1]=0\n\n# build the image grid\nimg1 = rasterGrob(Image.R)\nimg2 = rasterGrob(Image.G)\nimg3 = rasterGrob(Image.B)\ngrid.arrange(img1, img2, img3, nrow=1)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:51.231248Z","iopub.execute_input":"2023-05-14T11:12:51.232753Z","iopub.status.idle":"2023-05-14T11:12:53.829379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# reshape image into a data frame\ndf = data.frame(\n  red = matrix(Image[,,1], ncol=1),\n  green = matrix(Image[,,2], ncol=1),\n  blue = matrix(Image[,,3], ncol=1)\n)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:53.832657Z","iopub.execute_input":"2023-05-14T11:12:53.834237Z","iopub.status.idle":"2023-05-14T11:12:53.897206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### compute the k-means clustering\nK = kmeans(df,4)\ndf$label = K$cluster\n\n### Replace the color of each pixel in the image with the mean \n### R,G, and B values of the cluster in which the pixel resides:\n\n# get the coloring\ncolors = data.frame(\n  label = 1:nrow(K$centers), \n  R = K$centers[,\"red\"],\n  G = K$centers[,\"green\"],\n  B = K$centers[,\"blue\"]\n)\n\n# merge color codes on to df\ndf$order = 1:nrow(df)\ndf = merge(df, colors)\ndf = df[order(df$order),]\ndf$order = NULL","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:53.900077Z","iopub.execute_input":"2023-05-14T11:12:53.901627Z","iopub.status.idle":"2023-05-14T11:12:56.726113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get mean color channel values for each row of the df.\nR = matrix(df$R, nrow=dim(Image)[1])\nG = matrix(df$G, nrow=dim(Image)[1])\nB = matrix(df$B, nrow=dim(Image)[1])\n  \n# reconstitute the segmented image in the same shape as the input image\nImage.segmented = array(dim=dim(Image))\nImage.segmented[,,1] = R\nImage.segmented[,,2] = G\nImage.segmented[,,3] = B\n\n# View the result\ngrid.raster(Image.segmented)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:56.728910Z","iopub.execute_input":"2023-05-14T11:12:56.730394Z","iopub.status.idle":"2023-05-14T11:12:57.140838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"require(\"ggplot2\")\n\n# perform PCA and add the uv coordinates to the dataframe\nPCA = prcomp(df[,c(\"red\",\"green\",\"blue\")], center=TRUE, scale=TRUE)\ndf$u = PCA$x[,1]\ndf$v = PCA$x[,2]\n\n# Inspect the PCA\n# most of the cumulative proportion of variance in PC2 should be close to 1. \nsummary(PCA)\n\nggplot(df, aes(x=u, y=v, col=rgb(red,green,blue))) + \n  geom_point(size=2) + scale_color_identity()\n\n# segmented \nggplot(df, aes(x=u, y=v, col=rgb(R,G,B))) + \n  geom_point(size=2) + scale_color_identity()","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:12:57.143614Z","iopub.execute_input":"2023-05-14T11:12:57.145099Z","iopub.status.idle":"2023-05-14T11:14:41.698949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert image data to data frame\nimage_df = as.data.frame(Image,na.rm = False)\nhead(image_df)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:14:41.701846Z","iopub.execute_input":"2023-05-14T11:14:41.703420Z","iopub.status.idle":"2023-05-14T11:14:43.595227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_df <- image_df[lapply(image_df,length)>0]\n","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:14:43.598058Z","iopub.execute_input":"2023-05-14T11:14:43.599654Z","iopub.status.idle":"2023-05-14T11:14:43.614974Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Principle Component","metadata":{}},{"cell_type":"code","source":"library(stats)\n# pca analysis     \npca_model = prcomp(image_df)\n\n# plot the scree plot\nplot(pca_model)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:14:43.617936Z","iopub.execute_input":"2023-05-14T11:14:43.619531Z","iopub.status.idle":"2023-05-14T11:14:45.035639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualize the reconstructed image for each of the components\npar(mfrow= c(3,3))\nrecon_fun = function(comp){\n  recon = pca_model$x[, 1:comp] %*% t(pca_model$rotation[, 1:comp])\n  image(t(apply(recon, 2, rev)), col=grey(seq(0,1,length=256)), main = paste0(\"Principle Components = \", comp))\n}\n\n# run reconstruction for 1:17 alternating components\nlapply(seq(1,18, by = 2), recon_fun)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:14:45.038441Z","iopub.execute_input":"2023-05-14T11:14:45.039929Z","iopub.status.idle":"2023-05-14T11:15:30.324701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"library(imager)\nimage = load.image(\"/kaggle/input/image-matching-challenge-2023/train/heritage/wall/images/DSC_4928_acr.jpg\")\nimage_df = as.data.frame(image)\nimage_mat = matrix(image_df$value, nrow = 220, ncol = 282, byrow = TRUE)\n# visualize the image\nimage(t(apply(image_mat, 2, rev)), col=grey(seq(0,1,length=256)))","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:15:30.327558Z","iopub.execute_input":"2023-05-14T11:15:30.329091Z","iopub.status.idle":"2023-05-14T11:15:45.141085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pca analysis     \npca_model = prcomp(image_mat)\n\n# plot the scree plot\nplot(pca_model)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:15:45.143698Z","iopub.execute_input":"2023-05-14T11:15:45.145155Z","iopub.status.idle":"2023-05-14T11:15:45.310801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualize the reconstructed image of wall for each of the components\npar(mfrow= c(3,3))\nrecon_fun = function(comp){\n  recon = pca_model$x[, 1:comp] %*% t(pca_model$rotation[, 1:comp])\n  image(t(apply(recon, 2, rev)), col=grey(seq(0,1,length=256)), main = paste0(\"Principle Components = \", comp))\n}\n\n# run reconstruction for 1:17 alternating components\nlapply(seq(1,18, by = 2), recon_fun)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:15:45.313468Z","iopub.execute_input":"2023-05-14T11:15:45.314882Z","iopub.status.idle":"2023-05-14T11:15:47.182205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#  Read All jpg files\nssh <- suppressPackageStartupMessages\nssh(library(EBImage))\nssh(library(keras))\nssh(library(foreach))\n# Read all images of buckingam_palace and british_museum\nlibrary(jpeg)\nimages <- lapply(list.files(\"/kaggle/input/image-matching-challenge-2023/train/phototourism/british_museum/images/\", pattern = \"\\\\.jpg$\", full.names = TRUE), readJPEG)\nimages2 <- lapply(list.files(\"/kaggle/input/image-matching-challenge-2023/train/phototourism/buckingham_palace/images/\", pattern = \"\\\\.jpg$\", full.names = TRUE), readJPEG)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:15:47.185164Z","iopub.execute_input":"2023-05-14T11:15:47.186615Z","iopub.status.idle":"2023-05-14T11:16:24.828485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# images[[1]]\nplot(images2[[4]])","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:16:24.832617Z","iopub.execute_input":"2023-05-14T11:16:24.835179Z","iopub.status.idle":"2023-05-14T11:18:29.308151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Resize all the images to have the same size 150x150x3\nforeach(i=1:21) %do% {images[[i]] <- resize(images[[i]],150,150)}\nforeach(i=1:9) %do% {images2[[i]] <- resize(images2[[i]],150,150)}","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:18:29.312401Z","iopub.execute_input":"2023-05-14T11:18:29.314611Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"folder_list <- list.files(\"/kaggle/input/image-matching-challenge-2023/train/phototourism/buckingham_palace/images/\")\n\nhead(folder_list,2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"folder_path <- paste0(\"/kaggle/input/image-matching-challenge-2023/train/phototourism/buckingham_palace/images/\", folder_list, \"/\")\n\nhead(folder_path,5)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"library(magrittr)\nlibrary(purrr)\nfile_name <- map(folder_path, \n                 function(x) paste0(x, list.files(x))\n                 ) %>% \n  unlist()\n\n# first 6 file name\nhead(file_name)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:33:17.260133Z","iopub.execute_input":"2023-05-14T11:33:17.262014Z","iopub.status.idle":"2023-05-14T11:33:17.540663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cyprus <- list.files(path = \"../input/image-matching-challenge-2023/train/heritage/cyprus/images\")\ndioscuri <- list.files(path = \"../input/image-matching-challenge-2023/train/heritage/dioscuri\")\nwall <- list.files(path = \"../input/image-matching-challenge-2023/train/heritage/wall\")\nbritish_museum <- list.files(path = \"../input/image-matching-challenge-2023/train/phototourism/british_museum\")\npalace <- list.files(path = \"../input/image-matching-challenge-2023/train/phototourism/buckingham_palace\")\ngrand_palace <- list.files(path = \"../input/image-matching-challenge-2023/train/phototourism/grand_place_brussels\")\n\ncyprus_test <- list.files(path = \"../input/image-matching-challenge-2023/train/heritage/cyprus/images\")\ndioscuri_test <- list.files(path = \"../input/image-matching-challenge-2023/train/heritage/dioscuri\")\nwall_test <- list.files(path = \"../input/image-matching-challenge-2023/train/heritage/wall\")\nbritish_museum_test <- list.files(path = \"../input/image-matching-challenge-2023/train/phototourism/british_museum\")\npalace_test <- list.files(path = \"../input/image-matching-challenge-2023/train/phototourism/buckingham_palace\")\ngrand_palace_test <- list.files(path = \"../input/image-matching-challenge-2023/train/phototourism/grand_place_brussels\")\n\n","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:50:01.419625Z","iopub.execute_input":"2023-05-14T11:50:01.425903Z","iopub.status.idle":"2023-05-14T11:50:01.506888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"size = 150\nchannels = 3\n\ntrain <- c(\n    cyprus[1:1500], \n    dioscuri[1:1500],\n    wall[1:1500],\n    british_museum[1:1500],\n    palace[1:1500],\n    grand_palace[1:1500]\n)\ntrain <- sample(train)\nevaluation <- c(\n    cyprus[1501:1750], \n    dioscuri[1501:1750],\n    wall[1501:1750],\n    british_museum[1501:1750],\n    palace[1501:1750],\n    grand_palace[1501:1750]\n)\nevaluation <- sample(evaluation)\n\ntest <- c(cyprus_test, dioscuri_test, wall_test, british_museum_test, palace_test, grand_palace_test)\ntest <- sample(test)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:50:08.591861Z","iopub.execute_input":"2023-05-14T11:50:08.594362Z","iopub.status.idle":"2023-05-14T11:50:08.626179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_prep <- function(images, size, channels, path, list_img){\n\n  count<- length(images)\n  master_array <- array(NA, dim=c(count,size, size, channels))\n  \n  for (i in seq(length(images))) {\n    folder_list <- list(\"cyprus\", \"dioscuri\", \"wall\", \"british_museum\", \"palace\", \"grand_palace\")\n    for(j in 1:length(folder_list)) {\n        if(images[i] %in% list_img[[j]]) {\n            img_path <- paste0(path, folder_list[[j]], \"/\", images[i])\n            break\n        }\n    }\n    img <- image_load(path = img_path, target_size = c(size,size))\n    img_arr <- image_to_array(img)\n\n    img_arr <- array_reshape(img_arr, c(1, size, size, channels))\n    master_array[i,,,] <- img_arr\n  }\n  return(master_array)\n}\n\nlabel_prep <- function(images, list_img) {\n    y <- c()\n    for(i in seq(length(images))) {\n        folder_list <- list(\"cyprus\", \"dioscuri\", \"wall\", \"british_museum\", \"palace\", \"grand_palace\")\n        for(j in 1:length(folder_list)) {\n            if(images[i] %in% list_img[[j]]) {\n                y <- append(y, j-1)\n                break\n            }\n        }\n    }\n    return(y)\n}\n\nlist_img_train <- list(cyprus, dioscuri,wall,british_museum, palace, grand_palace)\nlist_img_test <- list(cyprus_test, dioscuri_test,wall_test,british_museum_test, palace_test, grand_palace_test)","metadata":{"execution":{"iopub.status.busy":"2023-05-14T11:57:12.308772Z","iopub.execute_input":"2023-05-14T11:57:12.310921Z","iopub.status.idle":"2023-05-14T11:57:12.335919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train <- data_prep(train, size, channels, \"../input/intel-image-classification/seg_train/seg_train/\", list_img_train)\nX_evaluation <- data_prep(evaluation, size, channels, \"../input/intel-image-classification/seg_train/seg_train/\", list_img_train)\nX_test <- data_prep(test, size, channels, \"../input/intel-image-classification/seg_test/seg_test/\", list_img_test)","metadata":{},"execution_count":null,"outputs":[]}]}