{"cells":[{"metadata":{"_uuid":"5e490ce47f3687790311710dd91119bb40417679","_cell_guid":"cde69d33-ceaa-45f8-8f66-810d405a2823"},"cell_type":"markdown","source":"## Read Image in R\n"},{"metadata":{"_uuid":"9c5e583d94e6a12b459009ed1190a7da658f5f8b","_cell_guid":"0796fd5f-9076-4bbb-b733-3bd178ce96c6"},"cell_type":"markdown","source":"### Load libraries"},{"metadata":{"_execution_state":"idle","_cell_guid":"25c2e0fe-c38c-46f2-b75f-706ac82e7bb3","_uuid":"05637b957be9b6cf676fa3ae4d1b9535486379ad","trusted":true,"_kg_hide-output":true,"scrolled":true},"cell_type":"code","source":"library(EBImage)\nlibrary(ggplot2)\nlibrary(tidyverse)\nlibrary(psych)","execution_count":1,"outputs":[]},{"metadata":{"_uuid":"377c70852b2af3d49865a30d40ef330804a8d1e5","_cell_guid":"e338c70f-f0b1-45eb-965b-2a4446732b4d"},"cell_type":"markdown","source":"### Load images\n\nFirst, get location of image files within the zip file."},{"metadata":{"scrolled":true,"_uuid":"2fcf725f20aed8a5b46eb250d474cdc7ace7e094","trusted":true,"_cell_guid":"4b1d2074-7f84-4aaf-81f0-9f47f871f44c"},"cell_type":"code","source":"temp <- unzip(\"../input/train_jpg.zip\", list = T)\nhead(temp, 2)","execution_count":7,"outputs":[]},{"metadata":{"_uuid":"0e90e44670c26a3c2bf6eaeaa8fa1d498ad76592","_cell_guid":"c693169a-e1f8-431d-8bb0-d57546a488e9"},"cell_type":"markdown","source":"Now we now the location of the image files inside the zip file, we can extract them using *unzip*, and then read them using the *readImage* function."},{"metadata":{"_uuid":"dfa72eb7f838a192dd0dfecb89f776338638fe5d","trusted":true,"_cell_guid":"d3d156eb-14b0-4d54-bc66-df08eb4f40a9"},"cell_type":"code","source":"image_loc <- temp[2, 1]\ntemp <- unzip(\"../input/train_jpg.zip\", files = image_loc)\nimage <- readImage(temp)","execution_count":11,"outputs":[]},{"metadata":{"_uuid":"84d0169d4d8b9c298fe0fe537aea4ced7165f0ab","_cell_guid":"99fc16a1-6b17-447e-abe3-02af3ba92247"},"cell_type":"markdown","source":"Then to display the image... It's an ELECTRIC DRILL! I just happen to be renovating my new house... I wonder if it's any good."},{"metadata":{"_uuid":"aab6635abb82c72941ec4892891a9557512125a1","trusted":true,"_cell_guid":"8fbdc7ef-ad16-42ab-8f25-9f86dbbe2e17"},"cell_type":"code","source":"display(image)","execution_count":12,"outputs":[]},{"metadata":{"_uuid":"572cb148b1f16e8d5b1757b888403bfc3223b9f6","_cell_guid":"dfcf6a99-616b-4a50-b2b5-fa4d53c36a10"},"cell_type":"markdown","source":"Next, to do some feature extraction, using the *tidyverse*!"},{"metadata":{"_uuid":"546c071eaaf88a90d3ff79080786c949261e0069","_cell_guid":"ad616758-da26-4226-bd8b-4755cd640f0b"},"cell_type":"markdown","source":"### Colours\n(JPG) Colour images are actually made of three images (Red, Green, and Blue) layered on top of each other. We can see this in the image data below.\n\nThe *dimensions* information tells us that the image is 485 pixels wide and 360 pixels tall. It also tells us there are 3 layers.\n\nThe matrix shows an extract of the first (red) layer."},{"metadata":{"_uuid":"b40884d073573b8f6772044c78f398dacac16144","trusted":true,"_cell_guid":"c3e92e44-d88a-467b-bb72-bbb143a3dc56"},"cell_type":"code","source":"print(image)","execution_count":13,"outputs":[]},{"metadata":{"_uuid":"3998e59bd52b534f02b85e5daf5b59eb35af7db6","_cell_guid":"3f70d8db-dada-44a7-8b7c-84c183c37aa2"},"cell_type":"markdown","source":"The image above is dominated by oranges and browns (black is not a colour, but an absence of colour; these are represented by really small values for the pixles representing the black box).\n\nColours in images are created through **additive mixing**. This contrasts with *subtractive mixing*, which is what most of us are used to by mixing physical substances like paint.\n\nWith additive mixing, oranges and browns are made of red and greens, with a little blue (in the case of browns). Hence, the image should have more intense reds, moderately intense greens, and low-intensity blues.\n\nAs a photographer, I'm used to seeing colours in an image represented by histograms, like the one below."},{"metadata":{"_uuid":"8b581a8c4bff85b9b157ce660433ec24dfe78711","trusted":false,"_cell_guid":"caec96eb-ef36-4e74-8b87-b4b7d43674dc"},"cell_type":"code","source":"hist(image)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d98a48aee338bdb1ac9d30254ba6926214166bca","_cell_guid":"d02f4e39-eb5c-4593-b2e2-50eb68ee8857"},"cell_type":"markdown","source":"This is kinda cool, but I think I can do a better job at visualising this than using base R graphics. \n\nTo do this, I'll need to extract each of the RGB channels, and convert the three matrices into a data frame."},{"metadata":{"_uuid":"7151d4dc1521776daa911f49f937c867d9639f0f","trusted":true,"_cell_guid":"0bc5910d-d87a-4b9d-966e-78edd36460cf"},"cell_type":"code","source":"## Extract each colour\nred   <- imageData(image)[, , 1]\ngreen <- imageData(image)[, , 2]\nblue  <- imageData(image)[, , 3]\n\n## Convert matrices into numeric vectors, and combine into data frame\nimage_data <- data.frame(red   = as.numeric(red),\n                         green = as.numeric(green),\n                         blue  = as.numeric(blue))\n\nhead(image_data, 2)","execution_count":14,"outputs":[]},{"metadata":{"_uuid":"efd230d29f9a6dff5dabd58cafa8bc4ba79174ff","_cell_guid":"f92beaa9-6a10-4b8e-975f-e6fd5962e411"},"cell_type":"markdown","source":"Now that the data is in a data frame, visualising a histogram with ggplot will be much easier."},{"metadata":{"scrolled":true,"_uuid":"0f7e6cd5ec8db6b8c9098f2fa48c3bb9ad8e02f3","trusted":true,"_cell_guid":"112a928d-754d-40ab-b33e-15607070702c"},"cell_type":"code","source":"ggplot(image_data) +\n    geom_histogram(aes(x = red), size = 0, fill = \"red\", alpha = 0.3, binwidth = 0.02) +\n    geom_histogram(aes(x = green), fill = \"green\", alpha = 0.3, binwidth = 0.02) +\n    geom_histogram(aes(x = blue), fill = \"blue\", alpha = 0.3, binwidth = 0.02) +\n    theme_minimal() +\n    theme(panel.grid.major = element_blank(),\n          panel.grid.minor = element_blank()) +\n    scale_x_continuous(labels = scales::percent) +\n    labs(x = \"Intensity\",\n         y = \"Count\",\n         title = \"Colour Histogram\")","execution_count":15,"outputs":[]},{"metadata":{"_uuid":"d612fdbada3ac5418087a1b36c2d73ba89cb5ea0"},"cell_type":"markdown","source":"### Colour Palette\n\nRGB colours are still not that meaningful, and contain too many levels.\n\nThese can be reduced by running k-means on the image data to extract a colour palette."},{"metadata":{"trusted":true,"_uuid":"4f3f0016d44bc873a85f755afb140c782e6b3dc0"},"cell_type":"code","source":"## Create data frame with r g b data for each pixel along the cartesian plane\nimage_rgb <- data.frame(\n  x = rep(1:dim(image)[2], dim(image)[1]),\n  y = rep(dim(image)[1]:1, each = dim(image)[2]),\n  red = as.vector(imageData(image)[, , 1]),\n  green = as.vector(imageData(image)[, , 2]),\n  blue = as.vector(imageData(image)[, , 3])\n)\n\n## Get clusters of colours by running k-means\nk_means <- kmeans(image_rgb[, c(\"red\", \"green\", \"blue\")],\n                  centers = 9,\n                  iter.max = 10)","execution_count":18,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"457426bbef5c0359bc625e7eceef88d345699e5d"},"cell_type":"code","source":"## Show the colour palette\nscales::show_col(rgb(k_means$centers))","execution_count":19,"outputs":[]},{"metadata":{"_uuid":"cded773a09745bff82528e0c6afa64e6d365fcd2"},"cell_type":"markdown","source":"This palette seems to match with the actual image. \n\nSo the HEX colours can be extracted as features."},{"metadata":{"trusted":true,"_uuid":"21790b918eb8fe4e5a3fa73605632f4814be763e"},"cell_type":"code","source":"colour_palette <- rgb(k_means$centers)","execution_count":20,"outputs":[]},{"metadata":{"_uuid":"774d30a86e1645c5c5c25a2bcb259acafbe63764","_cell_guid":"50dbc465-15bf-46cf-917f-b72cc254b8bc"},"cell_type":"markdown","source":"Visualising may be cool, but what we really want are data that is machine readable, such as measures of central tendency."},{"metadata":{"_uuid":"1231897ebee45a7ecd400f3cd56720cdf6aafefd","trusted":false,"_cell_guid":"fb84e2de-08c9-49ae-9668-0d372b7bf716"},"cell_type":"code","source":"describe(image_data$red)\ndescribe(image_data$green)\ndescribe(image_data$blue)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dc05ffa4463018ccd4f12f953f5a8eb82d57050f","_cell_guid":"934bb9f1-d058-4324-af93-b59e57bb6cfb"},"cell_type":"markdown","source":"### Saturation\n\nSaturation also plays a role in how attractive an image is. For most people, saturations levels greater than what the eye usually sees has a \"pop\" factor, and generally draws more attention than images that are less saturated.\n\nWe can calculate an image's *saturation* quite easily, using the rgb2hsv function in base R."},{"metadata":{"_uuid":"423852bd5492c3976589df4432395169a3c1bfe5","trusted":false,"_cell_guid":"aa4b8649-8dae-463a-b42b-d62e6191f472"},"cell_type":"markdown","source":"### Contrast\nContrast in an image can also make an image \"pop\". Our eyes are drawn to areas of an image that are brighter, and this is accentuated when the contrast in an image is higher. That is, brighter areas are very bright, and darker areas are very dark.\n\nTo do this, colour images are first turned to greyscale images, then a histogram can be generated to see if there are peaks at the higher end and lower ends.\n\nIn the histogram below \n\n"},{"metadata":{"trusted":true,"_uuid":"5a553cb7634bedd108e4bc3d40b20b17a398d7e5"},"cell_type":"code","source":"image_grey <- channel(image, \"gray\")\ndisplay(image_grey)","execution_count":23,"outputs":[]},{"metadata":{"_uuid":"9e08e3ec70c1c010474d1c595776c526866290a3"},"cell_type":"markdown","source":"Looking at the greyscale version of the image, we can see \n\n* some areas that are relatively bright underneath the blinds, but not overly bright\n* some areas that are relatively dark, due to the black colour of the drill box\n* some areas that are middle-of-the-road\n\nThis is reflected in the histogram below!"},{"metadata":{"trusted":true,"_uuid":"c363ef2b972e8388a5a79ba8bc286681673584bc"},"cell_type":"code","source":"hist(image_grey)","execution_count":24,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}