{"cells":[{"metadata":{},"cell_type":"markdown","source":"# iMet EDA\nThe Metropolitan Museum of Art in New York, also known as The Met , is the largest art museum in the United States. With 6,953,927 visitors in 2018. Including me is also attracted, I went to visit on 2019/05 with my wife and 3 kids. My children are very interested in Egyptian artifacts and have been in the exhibition area for a long time.\n\nIts permanent collection contains over two million works of which over 200K have been digitized with imagery.\n\nThe online cataloguing information is generated by Subject Matter Experts (SME) and includes a wide range of data. SME can also be indirect in describing finer-grained attributes from the museum-goer’s understanding. Adding fine-grained attributes to aid in the visual understanding of the museum objects will enable the ability to search for visually related objects.\n\nIn this study, we tried to extract the feature of image and analysis the feature by each attributes. Simple method (Random Forest) was performed and I hope it is useful for Machine Learningers.\n\n<img style=\"float: left;\" src=\"https://drive.google.com/uc?export=view&id=1xLUWwsOJNRU-n0tPG7WH7UJlgCOpQAph\" width=\"65%\">"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport pylab as plt\nimport seaborn as sns\nimport cv2\nimport os","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Import"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/train.csv')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Exploration\nWe explore the image by 3 view: BGR/RBG/HSV"},{"metadata":{"trusted":true},"cell_type":"code","source":"img_path='../input/train/'+train.id[5]+\".png\"\nimage=cv2.imread(img_path)\nimage_rgb=cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\nhsv_image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)\nh,s,v=np.average(hsv_image,axis=(0,1))\n\nplt.subplot(131),plt.imshow(image),plt.title('BGR')\nplt.subplot(132),plt.imshow(image_rgb),plt.title('RBG')\nplt.subplot(133),plt.imshow(hsv_image),plt.title('HSV')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Preprocess\nWe try to remove the background of image and let the object clear. First, we convert BGR image to HSV, we can use this to extract a colored object. In HSV, it is more easier to represent a color than RGB color-space. In our application, we will try to extract a background colored object. So here is the method:\n\n* Convert from BGR to HSV color-space\n* We threshold the HSV image for a range of color\n* Now extract the object alone, we can do whatever on that image we want."},{"metadata":{"trusted":true},"cell_type":"code","source":"image=cv2.imread(img_path)\nimg=cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n# Convert BGR to HSV\nhsv_image = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n# define range of blue color in HSV\nlower_blue=np.array([20,25,15])\nupper_blue=np.array([130,255,255])\n#Threshold the HSV impage to get only blue colors\nmask=cv2.inRange(hsv_image,lower_blue,upper_blue)\n# Bitwise-And mask and original image\nres = cv2.bitwise_and(img,img,mask=mask)\nplt.subplot(131),plt.imshow(img),plt.title('ORIGINAL')\nplt.subplot(132),plt.imshow(mask),plt.title('Mask')\nplt.subplot(133),plt.imshow(res),plt.title('Res')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's do the same thing in 5 image and check how the oject had been extract:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def mask(img_path):\n    image=cv2.imread(img_path)\n    img=cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    hsv_image = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n    lower_blue=np.array([20,25,15])\n    upper_blue=np.array([130,255,255])\n    mask=cv2.inRange(hsv_image,lower_blue,upper_blue)\n    res = cv2.bitwise_and(img,img,mask=mask)\n    plt.subplot(131),plt.imshow(img),plt.title('ORIGINAL')\n    plt.subplot(132),plt.imshow(mask),plt.title('Mask')\n    plt.subplot(133),plt.imshow(res),plt.title('Res')\n    plt.show()\n    h,s,v=np.average(res,axis=(0,1))\n    print(h, s, v)\n\nfor i in range(5):\n    img_path='../input/train/'+train.id[i]+\".png\"\n    mask(img_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Thanks for DHTT's kernel(https://www.kaggle.com/d5195295/hsv-analysis). After background remove, let's check the destribution of feature:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_feature_extracion(img_path):\n    image=cv2.imread(img_path)\n    img=cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    hsv_image = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n    lower_blue=np.array([20,25,15])\n    upper_blue=np.array([130,255,255])\n    mask=cv2.inRange(hsv_image,lower_blue,upper_blue)\n    res = cv2.bitwise_and(img,img,mask=mask)\n    h,s,v=np.average(hsv_image,axis=(0,1))\n    return h,s,v\n\nread_len=1000\nhsv_list=[]\nfor i in range(read_len):    \n    img_path='../input/train/'+train.id[i]+\".png\"    \n    hsv_list.append(image_feature_extracion(img_path))\n    \nimport seaborn as sns\ndf = pd.DataFrame(hsv_list, columns=[\"Hue\", \"y\",'Brightness(Values)'])\nsns.jointplot(x=\"Hue\", y=\"Brightness(Values)\", data=df)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We use 10000 images to extract the feature. All image used will be long time."},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_feature_extracion(img_path,ID):\n    image=cv2.imread(img_path)\n    img=cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    hsv_image = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n    lower_blue=np.array([20,25,15])\n    upper_blue=np.array([130,255,255])\n    mask=cv2.inRange(hsv_image,lower_blue,upper_blue)\n    res = cv2.bitwise_and(img,img,mask=mask)\n    h,s,v=np.average(hsv_image,axis=(0,1))\n    return ID,h,s,v\n\nread_len=10000 #109237\nhsv_list=[]\nfor i in range(read_len):    \n    img_path='../input/train/'+train.id[i]+\".png\"\n    ID=train.id[i]\n    hsv_list.append(image_feature_extracion(img_path,ID))\n\ndf = pd.DataFrame(hsv_list, columns=[\"ID\",\"Hue\", \"y\",'Brightness(Values)'])\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Preproces for Labels"},{"metadata":{"trusted":true},"cell_type":"code","source":"train['attribute_ids'].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train[\"attribute_ids\"] = train[\"attribute_ids\"].apply(lambda x:x.split(\" \"))\ntrain['attribute_ids'].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = pd.read_csv('../input/labels.csv')\nlabels.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_labels = []\nfor label in train['attribute_ids'][:10000].values:\n    zeros = np.zeros(labels.shape[0])\n    for label_i in label:\n        zeros[int(label_i)] = 1\n    train_labels.append(zeros)\n    \ntrain_labels = np.asarray(train_labels)\ntrain_labels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_labels.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"Y = train_labels\nfeatures = ['Hue','y','Brightness(Values)']\nX = df[features]\nprint(Y.shape,X.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Feature Analysis"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\nmodel = RandomForestClassifier()\nmodel.fit(X, Y)\nsns.set(style=\"darkgrid\")\nfig, ax = plt.subplots(figsize=(6,6))\ny_pos = np.arange(len(features))\nplt.barh(y_pos, model.feature_importances_, align='center', alpha=0.4)\nplt.yticks(y_pos, features)\nplt.xlabel('features')\nplt.title('feature_importances')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We analysis the feature by Random Forest. The brightness is the most important factor, followed by Saturation and Hue. The analysis results show that different luminance of target causing the observer to have different perceptions of the object."}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":4}