{"cells":[{"metadata":{},"cell_type":"markdown","source":"### INTRODUCTION\n\nSome time ago I came up with an idea to explore and show \"the timeline of color in arts\" through the means of data visualization. By \"the color timeline\" here, I mean how the usage of colors changed through the years. This particular dataset seems to be a good basis for such exploration.\n\nThe resulting visualization is at the end of this notebook. \n\nMy approach is the commonly used ML-based method of finding prevalent colors in an image - K-Means clustering of pixels based on their distance in 3D space represented by their R, G, B color components.\n\nIn general, I did the following steps:\n\n1. Subset the paintings from Wikiart data source and with the year of creation after 1500;\n2. Grouped the subset by half-decades and randomly sampled 50 paintings from each half-decade;\n3. Within every half-decade's subsample the paintings images were resized to a fixed dimension and combined into a single image as an array of pixel's RGB components;\n4. K-Means clustering (k=10) was applied over this array, producing clusters of 10 most prevalent colors. For each color cluster the pixels arranged  to this cluster (color) were counted giving the size of each cluster.\n5. The resulting clustering information (colors and size of color clusters for each half-decade) was used for creating final visualization of the colors timeline.\n\nImage processing code is mostly not my own. The references are at the bottom of the notebook."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom zipfile import ZipFile\nimport re\nimport cv2\nfrom sklearn.cluster import KMeans, MiniBatchKMeans\nfrom math import sqrt\nfrom collections import Counter\n\nimport altair as alt\nalt.renderers.enable('kaggle')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### PREPARE PAINTINGS INFORMATION FILE & SUBSET WIKIART PAINTINGS AFTER 1500"},{"metadata":{"trusted":true},"cell_type":"code","source":"# read paintings info data and parse year of creation\ndef parse_year(date):\n    '''\n    Helper to parse painting's year of creation\n    '''\n    if isinstance(date, str):\n        res = re.findall('([0-9]+)', date)\n        if res:\n            return res[0]\n        else:\n            return -1\n        \n    if pd.isnull(date):\n        return -1\n    \n    return date_val\n\ninfo = pd.read_csv('/kaggle/input/painter-by-numbers/all_data_info.csv')\ninfo['year'] = info.loc[:, 'date'].apply(parse_year).astype('int32')\n\n# Only paintings with year of creation after 1500\ninfo = info.loc[info.year>=1500]\n\n# Only paitings from wikiart\ninfo = info.loc[info.source=='wikiart']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"info.source.value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# add decades and half decades columns\ninfo['decade'] = info['year']//10*10\ninfo['half_decade'] = info['decade'] + ((info['year'] - info['decade']) // 5 * 5 )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# add folders to filenames\ninfo.loc[info.in_train==True, 'new_filename'] = 'train/' + info.loc[info.in_train==True, 'new_filename']\ninfo.loc[info.in_train==False, 'new_filename'] = 'test/' + info.loc[info.in_train==False, 'new_filename']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"info.head(3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"info.loc[:, ['half_decade', 'new_filename']].groupby('half_decade').count().describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### CREATE IMAGES SAMPLES FOR EACH HALF-DECADE"},{"metadata":{"trusted":true},"cell_type":"code","source":"def sample_data(info, sample_size, groupby='half_decade'):\n    '''\n    Sampling procedure\n    '''\n    def sample_func(x):\n        subsample_size = len(x) if len(x) < sample_size else sample_size\n        return list(x.sample(subsample_size, random_state=1))\n    \n    sample = info.groupby(groupby).agg({\n        'new_filename': sample_func\n    }).to_dict()\n    \n    return sample['new_filename']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_info = sample_data(info, sample_size=50, groupby='half_decade')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### DETECT 10 PREVALENT COLORS IN SAMPLES & CREATE VISUALIZATION DATA"},{"metadata":{"trusted":true},"cell_type":"code","source":"# LOAD AND RESIZE IMAGE\ndef load_img(filename):\n    zip_name = 'train.zip' if 'train/' in filename else 'test.zip'\n    \n    with ZipFile('/kaggle/input/painter-by-numbers/' + zip_name) as zip:\n        with zip.open(filename) as file:\n            #file.seek(0)\n            img_array = np.asarray(bytearray(file.read()), dtype='uint8')\n            img = cv2.imdecode(img_array, cv2.IMREAD_COLOR)\n            try:\n                img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n            except cv2.error as e:\n                print('Failed to load:', filename)\n                return None\n            \n    return img\n\ndef resize_img(img):\n    h, w, _ = img.shape\n    w_new = int(200 * w / max(w, h))\n    h_new = int(200 * h / max(w, h))\n    \n    return cv2.resize(img, (w_new, h_new), interpolation = cv2.INTER_AREA)\n\n\n# COLORS CONVERSION\ndef rgb_to_hex(rgb):\n    return \"#{:02x}{:02x}{:02x}\".format(int(rgb[0]), int(rgb[1]), int(rgb[2]))\n\ndef rgb_to_hsp(rgb):\n    \"\"\"\n    Calculates RGB color brightness\n    \"\"\"\n    return sqrt(0.299 * (rgb[0]**2) + 0.587 * (rgb[1]**2) + 0.114 * (rgb[2]**2))\n\n\n# PREVALENT COLORS DETECTION WITH K-Means\ndef detect_prevalent_colors(files, group_label, colors_num=10):\n    \"\"\"\n    Detect N-most prevalent colors in the list of given image files\n    \n    Params:\n    files: list of filenames\n    group_label: label of group as tuple of variable and string\n    colors_num: N most prevalent colors to detect \n    \"\"\"\n    data = []\n    for f in files:\n        img = load_img(f)\n        if img is not None:\n            mod_img = resize_img(img)\n            mod_img = mod_img.reshape(mod_img.shape[0] * mod_img.shape[1], 3).astype('float32')\n            data.append(mod_img)\n        \n    data = np.concatenate(data, axis=0)\n    \n    clustering_method = MiniBatchKMeans # KMeans or MiniBatchKMeans\n    \n    clf = clustering_method(n_clusters = colors_num)\n    labels = clf.fit_predict(data)\n    cluster_centers = clf.cluster_centers_\n    \n    counts = Counter(labels)\n    counts_sum = sum(counts.values())\n    \n    colors_data = [\n        {\n            'hex': rgb_to_hex( cluster_centers[i]),\n            'hsp': rgb_to_hsp(cluster_centers[i]),\n            'prop': (counts[i] / counts_sum),\n            group_label[0]: group_label[1]\n        } for i in counts.keys()\n    ]\n    \n    return colors_data\n\n\n# RUN\nviz_colors = []\nfor year, files in sample_info.items():\n    print('Processing year:', year)\n    \n    year_colors = detect_prevalent_colors(files, ('year', year), colors_num=10)\n    \n    # Sort colors by brightness\n    viz_colors.extend(\n        sorted(year_colors, key=lambda x: x['hsp'])\n    )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### PREPARE & PLOT COLORS TIMELINE VISUALIZATION"},{"metadata":{"trusted":true},"cell_type":"code","source":"viz_df = pd.DataFrame(viz_colors)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### VISUALIZE WITH COLOR PROPORTIONS"},{"metadata":{"trusted":true},"cell_type":"code","source":"alt.Chart(viz_df, title='Prevalent colors by half-decade').mark_bar(size=9).encode(\n    alt.X(\n        'year',\n        scale=alt.Scale(\n            domain=(1495, 2015),\n            nice=False\n        ),\n        axis=alt.Axis(\n            title='Year (half-decade)',\n            format='.4')\n    ),\n    alt.Y(\n        'sum(prop)',\n        scale=alt.Scale(domain=(0,1)),\n        axis=alt.Axis(\n            title='Prevalent color proportions'\n        )\n    ),\n    \n    color=alt.Color(\n        'hex',\n        scale=None,\n        legend=None\n    ),\n    order=alt.Order(\n        'hsp',\n        sort='ascending'\n    ),\n    tooltip=['year', 'hex', 'prop']\n).properties(\n    width=900,\n    height=500\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### VISUALIZE JUST WITH COLORS"},{"metadata":{"trusted":true},"cell_type":"code","source":"alt.Chart(viz_df, title='Prevalent colors by half-decade').mark_bar(size=9).encode(\n    alt.X(\n        'year',\n        scale=alt.Scale(\n            domain=(1495, 2015),\n            nice=False\n        ),\n        axis=alt.Axis(\n            title='Year (half-decade)',\n            format='.4')\n    ),\n    alt.Y(\n        'count()',\n        scale=alt.Scale(domain=(0,10)),\n        axis=alt.Axis(\n            title='Prevalent color'\n        )\n    ),\n    \n    color=alt.Color(\n        'hex',\n        scale=None,\n        legend=None\n    ),\n    order=alt.Order(\n        'hsp',\n        sort='ascending'\n    ),\n    tooltip=['year', 'hex']\n).properties(\n    width=900,\n    height=500\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### REFERENCES\n\n1. [Color Identification in Images: Machine Learning Application](https://towardsdatascience.com/color-identification-in-images-machine-learning-application-b26e770c4c71)\n2. [How to find the main colours in an image](https://www.alanzucconi.com/2015/05/24/how-to-find-the-main-colours-in-an-image/)\n3. [The incredibly challenging task of sorting colours](https://www.alanzucconi.com/2015/09/30/colour-sorting/)\n4. [HSP Color Model — Alternative to HSV (HSB) and HSL](http://alienryderflex.com/hsp.html)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}