{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# TPS_Jul_2022_Clustering🍪","metadata":{}},{"cell_type":"markdown","source":"* **Clustering** is the process of grouping the same group (called a **cluster**) that are more simmilar than others.\n1. Connectivity models : hierarchical clustering, HCA( based on distance connectivity)\n> * single linkage clustering : the minimum of object distances( min{d(a,b):a∈A, b∈B}..)\n> * complate linkage clustering: the maximum of object distances( max{d(a,b):a∈A, b∈B}.)\n> * UPGMA: Unweighted pair Group Method with Arithmetic Mean (avarage linkage *cluster* )\n> * WPGMA: Weighted pair Group Method with Arithmetic Mean (avarage linkage *cluste*r d(i∪j,k) = (d(i,k) + d(j,k))/2\n1. Centroid models : k_means algorithm (represents each cluster by a single mean vector)\n> * This results in a partitioning of the data space into Voronoi cells.\n> * Lloyd's algorithm(voronoi iteration or relaxation): It repeatedly finds the centroid of each set in the partition, then re-partitions of space the input according to which of these centroids is the nearest.\n> * It usually used in a Equclidean space. \n> * The centroid is the point that minimizes the average Squared Euclidean distance to the points in its cell. \n> >     Euclidean distance :  d(a,b) = √(a₁-b₁)²+(a₂-b₂)² \n> >     Squared Euclidean distance : d²(a,b) =  = (a₁-b₁)²+(a₂-b₂)²+.. \n> >     manhattan distance  d(a,b) = ∣a₁-b₁∣+∣a₂-b₂∣. \n> * The optimization problem is known to be NP-hard(computationally difficult).\n1. Distribution models : statistical distributions( the expecation-maximization algorithm)\n> * Gaussian mixture models : A probabilistic model that assumes all the data points are generated from a mixture of a finite number of gaussian distributions with unknown parameters.\n> * Bayesian Gaussian Mixture : a variant of the Gaussian Mixture model with variational inferefce algorithm.\n1. Density models : DBSCAN and OPTICS (connected dense regions in the data space)\n1. Groupe models : just provide the grouping infomation.\n1. Graph-based models : a clique (a subset of node in a graph)\n1. Neural models : unsupervised neural network(the self-organizing map, PCA:Principal Component Analysis or ICA: Independent Component Analysis)\n* Hard clustering: cluster or not\n* Soft clustering( fuzzy clustering)\n1. Strict partitioning clustering\n1. Strict partitioning with outliers\n1. Overlapping clustering\n1. Hierachical clustering\n1. Subspace clustering\n1. Elbow method : the percentage of explained variance as a function of the number of clusters.\n1. X-means clustering : a variation of k_means clustering. Keeping the best resulting splits, until a criterion as such as the Akaike information criterion(AIC) or Bayesian information criterion(BIC)\n1. Information criterion approach : If it is possible to make a likelihood function for clustering madel.\n1. Information-theoretic approach : Rate distortion theory ('jump' method)\n1. Silhouette method : -1 < s(i) < 1  It close to 1 is in an appropriate *cluster* and close to -1 is in the wrong *cluster*.\n                       s(i) = { 1 - a(i)/b(i) if a(i) < b(i),\n                                0,\n                                b(i)/a(i) - 1 if a(i) > b(i)} \n                     \n\n","metadata":{}},{"cell_type":"code","source":"pip install sklego","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-07-30T07:57:12.882522Z","iopub.execute_input":"2022-07-30T07:57:12.882972Z","iopub.status.idle":"2022-07-30T07:57:27.493673Z","shell.execute_reply.started":"2022-07-30T07:57:12.882879Z","shell.execute_reply":"2022-07-30T07:57:27.492380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport seaborn as sns\nsns.set()\nimport matplotlib.pyplot as plt\nimport plotly.express as px\n\nfrom yellowbrick.cluster import KElbowVisualizer\n\nfrom termcolor import colored, cprint\n\nfrom scipy.cluster.hierarchy import linkage\nfrom scipy import stats\nfrom scipy.stats import norm, shapiro, skew, kstest\n\nfrom sklearn.decomposition import PCA\nfrom sklearn.cluster import KMeans\nfrom sklearn.cluster import MiniBatchKMeans\nfrom sklearn.cluster import AgglomerativeClustering\nfrom sklearn.mixture import GaussianMixture\nfrom sklearn.mixture import BayesianGaussianMixture\nfrom sklearn.preprocessing import StandardScaler, RobustScaler, MaxAbsScaler, PowerTransformer\n\nfrom sklego.mixture import GMMClassifier\nfrom sklego.mixture import BayesianGMMClassifier","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:27.495824Z","iopub.execute_input":"2022-07-30T07:57:27.496196Z","iopub.status.idle":"2022-07-30T07:57:30.018521Z","shell.execute_reply.started":"2022-07-30T07:57:27.496163Z","shell.execute_reply":"2022-07-30T07:57:30.017285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = pd.read_csv('../input/tabular-playground-series-jul-2022/data.csv')\nsubmission = pd.read_csv('../input/tabular-playground-series-jul-2022/sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:30.021222Z","iopub.execute_input":"2022-07-30T07:57:30.021697Z","iopub.status.idle":"2022-07-30T07:57:31.391120Z","shell.execute_reply.started":"2022-07-30T07:57:30.021653Z","shell.execute_reply":"2022-07-30T07:57:31.390128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:31.394717Z","iopub.execute_input":"2022-07-30T07:57:31.395169Z","iopub.status.idle":"2022-07-30T07:57:31.428633Z","shell.execute_reply.started":"2022-07-30T07:57:31.395123Z","shell.execute_reply":"2022-07-30T07:57:31.427436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.head().T","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:31.430206Z","iopub.execute_input":"2022-07-30T07:57:31.430687Z","iopub.status.idle":"2022-07-30T07:57:31.460504Z","shell.execute_reply.started":"2022-07-30T07:57:31.430643Z","shell.execute_reply":"2022-07-30T07:57:31.459460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data.iloc[:, 1:].describe().T.style.background_gradient(cmap='gist_earth')","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:31.461606Z","iopub.execute_input":"2022-07-30T07:57:31.461963Z","iopub.status.idle":"2022-07-30T07:57:31.751224Z","shell.execute_reply.started":"2022-07-30T07:57:31.461934Z","shell.execute_reply":"2022-07-30T07:57:31.750126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* f_07 to f-13 and f_22 to f_28 are different from f_00 to f_06 and f_14 to f_21.","metadata":{}},{"cell_type":"code","source":"for col in data.columns[1:]:\n    stat, p_value = shapiro(data[col])\n    alpha = 0.05\n    if p_value > alpha:\n        result = colored('Accepted', 'green')\n    else:\n        result = colored('Rejected', 'red')    \n    print(col,result, p_value)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:31.752518Z","iopub.execute_input":"2022-07-30T07:57:31.752818Z","iopub.status.idle":"2022-07-30T07:57:32.016244Z","shell.execute_reply.started":"2022-07-30T07:57:31.752792Z","shell.execute_reply":"2022-07-30T07:57:32.015041Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **shapiro-wilk test** is a test of normality in frequentist statistics. Using for small number of data.","metadata":{}},{"cell_type":"code","source":"for col in data.columns[1:]:\n    stat, p_value = kstest(data[col],'norm')\n    alpha = 0.05\n    if p_value > alpha:\n        result = colored('Accepted', 'green')\n    else:\n        result = colored('Rejected', 'red' )    \n    print(col,result, p_value)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:32.017984Z","iopub.execute_input":"2022-07-30T07:57:32.018558Z","iopub.status.idle":"2022-07-30T07:57:32.432168Z","shell.execute_reply.started":"2022-07-30T07:57:32.018519Z","shell.execute_reply":"2022-07-30T07:57:32.430980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **Kolmogorov-Smirnov test**: is a nonparametric test of the equality of continuous.Using for a large number of data.","metadata":{}},{"cell_type":"code","source":"figure = plt.figure(figsize = (16,12))\nfor i in range(29):\n    feature_ = 'f_0{}'.format(i) if i < 10 else 'f_{}'.format(i) \n    plt.subplot(5, 6, i+1)\n    stats.probplot(data[feature_], dist = 'norm', plot=plt )   \n    plt.title(feature_)\nfigure.tight_layout(h_pad=1.0, w_pad=0.5)    \nplt.suptitle('Q-Q Plot', y = 1.05)\nplt.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-07-30T07:57:32.433719Z","iopub.execute_input":"2022-07-30T07:57:32.434855Z","iopub.status.idle":"2022-07-30T07:57:41.831052Z","shell.execute_reply.started":"2022-07-30T07:57:32.434805Z","shell.execute_reply":"2022-07-30T07:57:41.830275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **Q-Q plt**(quantile-quantile plot) is a probability plot, which is a graphical method for comparing two probability distributions by plotting their quantiles against each other.","metadata":{}},{"cell_type":"code","source":"data.drop('id', axis = 1, inplace = True)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:41.832280Z","iopub.execute_input":"2022-07-30T07:57:41.833209Z","iopub.status.idle":"2022-07-30T07:57:41.848969Z","shell.execute_reply.started":"2022-07-30T07:57:41.833171Z","shell.execute_reply":"2022-07-30T07:57:41.847755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"int_col = [x for (x,y) in data.dtypes.items() if y == 'int64']\nfloat_col = [x for (x,y) in data.dtypes.items() if y == 'float64']\ncol_q1 = float_col[:7]\ncol_q2 = int_col[:]\ncol_q3 = float_col[7:15]\ncol_q4 = float_col[15:]","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:41.853813Z","iopub.execute_input":"2022-07-30T07:57:41.854515Z","iopub.status.idle":"2022-07-30T07:57:41.863847Z","shell.execute_reply.started":"2022-07-30T07:57:41.854479Z","shell.execute_reply":"2022-07-30T07:57:41.862989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"col_q1,col_q2,col_q3,col_q4","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:41.865300Z","iopub.execute_input":"2022-07-30T07:57:41.865890Z","iopub.status.idle":"2022-07-30T07:57:41.879252Z","shell.execute_reply.started":"2022-07-30T07:57:41.865859Z","shell.execute_reply":"2022-07-30T07:57:41.878247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame(data, columns = data.columns)\ncorr = df.corr()\nmask = np.triu(corr)\n\nplt.figure(figsize=(15, 10))\ncmap = sns.color_palette(\"coolwarm\",100)\nsns.heatmap(corr, square = False, annot = True,fmt = '1.2f',linewidths = .5, mask = mask, robust = True, cmap = cmap, cbar = False)\nplt.show()\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T07:57:41.880820Z","iopub.execute_input":"2022-07-30T07:57:41.881473Z","iopub.status.idle":"2022-07-30T07:57:44.232860Z","shell.execute_reply.started":"2022-07-30T07:57:41.881439Z","shell.execute_reply":"2022-07-30T07:57:44.231867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame(data, columns = int_col)\ncorr_i = df.corr()\nmask = np.triu(corr_i)\ncmap = sns.color_palette(\"coolwarm\",100)\nsns.heatmap(corr_i,mask = mask, square = False,linewidths = .5, annot = True,  robust = False, cmap = cmap).set(title='integer_colums')\nplt.show()\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T07:57:44.234114Z","iopub.execute_input":"2022-07-30T07:57:44.234529Z","iopub.status.idle":"2022-07-30T07:57:44.654707Z","shell.execute_reply.started":"2022-07-30T07:57:44.234495Z","shell.execute_reply":"2022-07-30T07:57:44.653593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame(data, columns = float_col)\ncorr_f = df.corr()\nmask = np.triu(corr_f)\ncmap = sns.color_palette(\"coolwarm\",100)\nsns.heatmap(corr_f,mask = mask, square = False,linewidths = .5, annot = False,  robust = False, cmap = cmap).set(title='float_colums')\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T07:57:44.658102Z","iopub.execute_input":"2022-07-30T07:57:44.659046Z","iopub.status.idle":"2022-07-30T07:57:45.455277Z","shell.execute_reply.started":"2022-07-30T07:57:44.659008Z","shell.execute_reply":"2022-07-30T07:57:45.453037Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.DataFrame(data, columns = col_q4)\ncorr_q4 = df.corr()\nmask = np.triu(corr_q4)\ncmap = sns.color_palette(\"coolwarm\",100)\nsns.heatmap(corr_q4,mask = mask, square = False,linewidths = .5, annot = True,  robust = False, cmap = cmap)\nplt.show()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-30T07:57:45.456397Z","iopub.execute_input":"2022-07-30T07:57:45.456756Z","iopub.status.idle":"2022-07-30T07:57:45.880543Z","shell.execute_reply.started":"2022-07-30T07:57:45.456719Z","shell.execute_reply":"2022-07-30T07:57:45.877917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **seaborn.heatmap** plots the density and value distribution of data in terms of color shades and hues.","metadata":{}},{"cell_type":"code","source":"sns.clustermap(corr,figsize = (14,10),method='ward',metric = 'euclidean', annot = False,linewidths = .5, cmap = cmap)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:45.881567Z","iopub.execute_input":"2022-07-30T07:57:45.881876Z","iopub.status.idle":"2022-07-30T07:57:47.078554Z","shell.execute_reply.started":"2022-07-30T07:57:45.881849Z","shell.execute_reply":"2022-07-30T07:57:47.077732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.clustermap(corr_i,figsize = (5,5),method='ward', metric = 'euclidean', annot = True, linewidths = .5,cmap = cmap)\nplt.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-07-30T07:57:47.079804Z","iopub.execute_input":"2022-07-30T07:57:47.080723Z","iopub.status.idle":"2022-07-30T07:57:47.874087Z","shell.execute_reply.started":"2022-07-30T07:57:47.080690Z","shell.execute_reply":"2022-07-30T07:57:47.873027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.clustermap(corr_i,figsize = (5,5),method='average', metric = 'correlation', annot = True, linewidths = .5,cmap = cmap)\nplt.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-07-30T07:57:47.875564Z","iopub.execute_input":"2022-07-30T07:57:47.875885Z","iopub.status.idle":"2022-07-30T07:57:48.689368Z","shell.execute_reply.started":"2022-07-30T07:57:47.875856Z","shell.execute_reply":"2022-07-30T07:57:48.688125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.clustermap(corr_q4,figsize = (5,5),method='ward',metric = 'euclidean', annot = True,linewidths = .5, cmap = cmap)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:48.690702Z","iopub.execute_input":"2022-07-30T07:57:48.691645Z","iopub.status.idle":"2022-07-30T07:57:49.496397Z","shell.execute_reply.started":"2022-07-30T07:57:48.691608Z","shell.execute_reply":"2022-07-30T07:57:49.495307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.clustermap(corr_q4,figsize = (5,5),method='average',metric = 'correlation', annot = True,linewidths = .5, cmap = cmap)\nplt.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-07-30T07:57:49.497678Z","iopub.execute_input":"2022-07-30T07:57:49.498094Z","iopub.status.idle":"2022-07-30T07:57:50.329363Z","shell.execute_reply.started":"2022-07-30T07:57:49.498062Z","shell.execute_reply":"2022-07-30T07:57:50.328162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **seaborn.clustermap** plots the matrix dataset as a **hierarchically_clustered** heatmap.\n* The function clusters input the 2D data frames into row and columns, sorts the ciusters closer to each other, and visualizes them as a heatmap and dendrogram. \n* Linkage method to use for calculating clusters. Based on **Scipy.cluste.hierarchy.linkage**(a function that performs hierachical clustering)\n* Distance metric to use for the dara. Based on **scipy.spatial.distance.pdist**(pdist is more efficient for computong the distances between all pairs)","metadata":{}},{"cell_type":"markdown","source":"+++++++++++++++","metadata":{}},{"cell_type":"code","source":"cols = col_q2 + col_q4\ndata_1 = data[cols] ","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:00:08.412766Z","iopub.execute_input":"2022-07-30T08:00:08.413216Z","iopub.status.idle":"2022-07-30T08:00:08.422208Z","shell.execute_reply.started":"2022-07-30T08:00:08.413183Z","shell.execute_reply":"2022-07-30T08:00:08.421052Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#scaler = StandardScaler()\n#data_s = scaler.fit_transform(data)\n#scaler = RobustScaler()\n#data_r = scaler.fit_transform(data)\nscaler = MaxAbsScaler().fit(data_1)\ndata_ma = scaler.fit_transform(data_1)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:59:33.719536Z","iopub.execute_input":"2022-07-30T07:59:33.720540Z","iopub.status.idle":"2022-07-30T07:59:33.754246Z","shell.execute_reply.started":"2022-07-30T07:59:33.720498Z","shell.execute_reply":"2022-07-30T07:59:33.753435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"power = PowerTransformer().fit(data_ma)\ndata_p = power.transform(data_ma)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:00:19.725978Z","iopub.execute_input":"2022-07-30T08:00:19.726462Z","iopub.status.idle":"2022-07-30T08:00:21.487343Z","shell.execute_reply.started":"2022-07-30T08:00:19.726422Z","shell.execute_reply":"2022-07-30T08:00:21.486101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_test = pd.DataFrame(data_ma, columns = data_1.columns)\ndata_test","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:01:35.408185Z","iopub.execute_input":"2022-07-30T08:01:35.408643Z","iopub.status.idle":"2022-07-30T08:01:35.434270Z","shell.execute_reply.started":"2022-07-30T08:01:35.408608Z","shell.execute_reply":"2022-07-30T08:01:35.433314Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"seed = 1","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:01:39.115632Z","iopub.execute_input":"2022-07-30T08:01:39.116420Z","iopub.status.idle":"2022-07-30T08:01:39.121388Z","shell.execute_reply.started":"2022-07-30T08:01:39.116374Z","shell.execute_reply":"2022-07-30T08:01:39.120203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = MiniBatchKMeans(\n                        n_clusters = 8 ,# default = 8\n                        max_iter = 100, # default = 100\n                        batch_size = 1024, # default = 1024\n                        verbose = 0, # default = 0\n                        compute_labels = True, # default = True\n                        random_state = 1, # default = None\n                        tol = 0.0, # default = 0.0\n                        max_no_improvement = 10, # default = 10\n                        init_size = 3, # default = None\n                        n_init = 3, # default = 3\n                        reassignment_ratio = 0.01 # default = 0.001\n                       )\n","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:01:40.395553Z","iopub.execute_input":"2022-07-30T08:01:40.395963Z","iopub.status.idle":"2022-07-30T08:01:40.403161Z","shell.execute_reply.started":"2022-07-30T08:01:40.395930Z","shell.execute_reply":"2022-07-30T08:01:40.401913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"score : 0.20548 ( n_clusters : 8 default)","metadata":{}},{"cell_type":"code","source":"model_2 = KMeans( n_clusters = 7,# default = 8)\n                  max_iter = 300, # default = 300\n                  random_state = seed,\n                 # algorithm: default = 'lloyd' \n              )","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:50.516130Z","iopub.status.idle":"2022-07-30T07:57:50.516536Z","shell.execute_reply.started":"2022-07-30T07:57:50.516349Z","shell.execute_reply":"2022-07-30T07:57:50.516368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"score : 0.23251 ( n_clusters : 8 default)","metadata":{}},{"cell_type":"code","source":"model_3 = GaussianMixture(n_components = 7, # default = 1\n                          covariance_type = 'full', # defalut = 'full'\n                          max_iter = 100, # default = 100\n                          init_params = 'kmeans', #default = kmeans\n                          random_state = seed, # default = None\n                          verbose = 1, # default = 0\n                         )","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:50.518118Z","iopub.status.idle":"2022-07-30T07:57:50.518486Z","shell.execute_reply.started":"2022-07-30T07:57:50.518310Z","shell.execute_reply":"2022-07-30T07:57:50.518327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n![250px-Nonbayesian-gaussian-mixture.svg.png](attachment:4edf91ba-c460-4439-bfb6-90a015df8195.png)wiki\n* K = number of mixture components\n* N = number of observations\n* score : 0.48199 ( n_clusters : 8 default)","metadata":{},"attachments":{"4edf91ba-c460-4439-bfb6-90a015df8195.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAPoAAADPCAQAAAB2UgSoAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAAHdElNRQfhCxENLAsogkIZAAAX30lEQVR42u2deWAURRbGfzW5uMIhyCXIIUe4IT1BEcQAigiicgkqKJcouCqKHIoLKoIKLqyuICCLHCsK4RAERHEhqISFZMKN3MglKsiZkHOm9o8MkSBheqa7MzOmvvyRSaf7TVV9XVWv6tV7TyBRMAOf8ViwFDUU+JnNijNDsPFQMBU3FNhMV8WbIRQhLbjeUYVCODApKNIVFOkKinQFRbqCIl1Bka6gSFdQpCso0hUU6QqKdAVFuoIiXUGRrqBIV1CkKxDq6YZaEaVqeCvUsbdgCh9dS4R690TW8R2pinSPjVayIUk+jB8FcsZWJHCzd0+Ed2K1Il1XT5EbRIJuiV2pW4DlP8F83fc2oaMiXDfptrVJ4/UKtNeVBUn6Ucerem/VBirSlSKnSFdQpCso0hUU6QqKdAVFuoIiXUGRrqBIV1CkKyjSFRTpCop0BUW6giJdQZGuoEhXpJuPUNW0hY30+uzjbtW4gQp9p2G7R+s+7CjtQC0q8jWD+cTy8teOnqf3VlFLxT72ZhhuKpp6JXUFd7Kc2dzBs2RbWv7yoo/u11GxrZf05K0NIr2WKtmOncUMojo9OW9Z6aun65ieDjcWGTX2ueedtCTFOSBZapHkCGYjOUA9v9avMulssniVUgTJAkX6FQwiiws84Ncavo/kb4r0giMd7uMc2Yz0Yw2Lc4SLVFGkFxzpUJs9SP5DEb/VsSOSlYr0giQdSvIlko1U8Fst45B0UaT7RnpjrvivvkB3L54L4R0kx9EKrF41GcrzLOBRACpylp8prUj3hfRX+NT9aY9bPQujjs75cgAZpNCtgOo1h6pAKc5RHYBnkHyoSPeF9DU87e45WZQC4BZWMkzn03fyCy7eKRAzz3ga5izTaQOAje9x0lKR7i3pYVyiPgA9r0rrtdqLQbsKSUjiKF5AtWvOHiLcn+uSzk7CFenekX4HZxEATOMdoDgQyjFCCCVGZ3MWZzGSbVQrgLpV4AsqXfX3G0hGK9K9I30UG91q2X56oNEaaM4KqtCHsQzWKUUwEienLbfDVWAypShB1dwrEewhnajCTrp3du+7qUw42TyN5AhtmALEkk4j5lOL07pfs3c5xBy+ZgizTatJOLFUc49D+4mnApNYQAwtWMZx9z0ZPEM802lT2K0v+nt6KJfoynP0pyx1+BuN3DP6CP7tQzSXJvyEZIZJxy2as5vPWYGLRcygBzCOte6fiDx3foyknxre9ZJ+B8eu8yIcJYT7mU0FYr385nLEI1ljwuq5Ekfcuvo8hnq4txQn+Z3yhZl0bxZPsaz707UoNuJkDxF0c8/3+nGG+/iE+9hi2A73DlPZBcApPIU6vMAwbmKyGt71Ya5Xe3B6YdwOV4R09wYMrOL53Nfxddrl88QKpKmWP7X37jWM2uHKc9mtwFXkdyrmXp/Hg/k8cSuX+IkSinR/ojY/Ipnl88bJHkoBgo95/Kqr+26gLQxDMlGR7l8Ys8M1ZTx9GcftV12rSiKg0e+qdfofCCGJLKIV6f6F2Xa43kyiM42YQKd8loyZJBKiSPc3Bppoh5tFEg/d8I73kLyoSPc/zLPD7aMCExmGjXL53FGMQ6RSo/CRHmi+bAnYSWYkCw3a4apwkV9xUI425HeE+zLPUoypap0eCCjOEsN2uHr0BCJ5ifY3vG8Bkp6FracLJMvoGnClGsEEztKdDRZ/Uzl+xEk9zum49x5quj+t4iTQnupACgsoQhqf8Zjq6UbRg1TSTTeN/BkDkMzUdWcFWpDOl9SjKAB1Oc1L1FGKnJkw1w6X/6jyLS7a6rz7LFNyPz/vNvIo0k0efM2yw90ItUljn85T+VdIFwyngVqyWYMcf7j9FvvD/R3J616QLniZ29Q63UpY7w8Xzi4y3Ec+PZMewly25Jl0FOkWwHp/uNtx6vJtPctU3iCWy4xRpFuNHDvcx5YcYM7BNCSDdJCeSFngRTJppki3Glb7w5XkBOe5RaciZyOePbnKnyLdMljtD9cNyULdS7YaXOJdRXpBYKCl/nDLkPmetsnBOf6Z+3k42bRSpBcErPSHq8x5juZroOnOHFz8xD+oDlRlBZIzfEBpRbr1yPGHW0QxC2Q/h7xqz00vgtDDpZKHwwaBh0m8QA9iGK/bq0YvjrOXFzjFPq+eCguu5hMqvJpJCCIrWyh9g7aZY3gKG3P53mS53XmARaz28qnD6r0vGFhjh4tgL5fz7K0rBBTKscECO1wsLta5HSgUAhBX7HDmep3PRdJbNW4gw3w7XFl+5Qw3q6YNZJhvh3sCWQChyxUMwXw73DdI7lENG9gow1pT7XDVSeGA+xCkQsDCbDvcKCTjVbMGPnLscOac5A8lmSyaqkYNfLQ00Q4XQzabTfJtVbAUZtrhrE8RoGASzPCHuyLJ6hQBCqbBvLiUVqcIUDAVZvnDxSEDzslTIV+YY4ezNkWAgukwxw5nZYoABQtghh3OyhQBChZhEFmc5V4DEqxLEaBgGYzb4axKEaBgIYza4axKEaBgKXLscD/4bIdrjYt4dYwq2GDUDmdFigCFAoARO5wVKQIUCgRG7HC9kPwnuKvv7/mpjJ/OnJbmKaqylU/J9GH515CZ7kwSgQhHbmrUACU9ih/VwGMyJnpamIYGQCE/5d9+GuV6MZCLjGWHl0/2YDALmRGAhN/C/MB/K6OQTPDj9/tmhzM3RYC5exEyNz5GvrBRuBHHnfzCbC/tcE4GIJkdEOOkDyjspMN27HzHIFZ6ZYfbzgc04TmlQATj8J6DCD7x2g5nXooANbz7BRn042lqkOCFR0sQpwhQpF/BTB5AsMYLO9waPuN+E1IEqOHdz4Ojd3a4cpzmF8qo4T2YcYA7+ZaBrNNphzvDKCp4bmSFQO7pOWtwb+xw3qUICJCerki/Hryxw3mTIkAN7wGMWbQlhcW67HAHmEAdRqlGC/aeDt74w+lPEaB6eoDjBLEspQcJHv3hMhlAKP8OnrZUpOePFLozikYk0drDnZuZwR0MRNBcNVswD+9XoM8OV4qTnGc9G9Xw/ldAHC112OGy+YHixAZHEDJFumdsI8aDHe4B9vAIocCtwWBuVaTrwWnaM4f72JKPHW5/bqjwiNx8rIr0goNWrNnNWiWtlMliM+jHUGrmY4fbz30MJg2AxtcXUCuiUZmYqo3KBIKrROhfgmmbvYlsQwvqUoeInPdYc3JU7Hc55PoiCZvSTPmW99nL56xh9HVUJcl0fiCOKFqx+I/L0bVoI1rJerYoGQkuwtEQp+RedrI+c8POc/5prkA4Dfs2rxro2VGyn3icWwAXx9jHGZEqnRSjNLWoTTiQKpfKuVvXmZLOoDYriGIWz5JJKO/wcp7/liaOm3J27WMqOvvYnpQNAErJqqIMxRFkk8pvnJDpAoRTrpHzM5btzjSxPWuzPzhOw/qMZnbbaB4UNo6KSa51GT/sTsn7/9jQlGjZhodFH9FH2ybGJy0xTPwBWrKQgdSjG6MYymrWXfXf89zPXKo2s9lGuvqLiGLyLprQgNLXdi1xjF0khmzt6OpU9IT93fMfH8xQ63QdaF5Wm6W5NFf0cnusJ83EXld7X7usSe0HrZEJZc6xw/2ORPL1tf+sXCzqH/Y0TfZ0LZI/yZM3/Nklp8g2Lk3aD2jtC3KdHqSkR3fQftWk9t/oxnqfaFxem6Zla1naKFOmtHFI90+eQAfN6tt3afJB1woPdP/xc0hOkS1cmrTPiC1SUKQHo/YutHFiNeGyt6Ndsm43hR2/OYa47mAfb0evMqzbV6Z/7udJf0yR9kdDkkT9x/iX0O8UW4RHmCqaIAelbNZuVUu26yI2VJvNa8IREp38qbfPbk2iOXPF/cTHVDRUiExe4B9sB6AJw9yjz1D56U0RE0QvrxWlCrzBo9DYlqBFKdKvU95Lc+krv8qM3XLEl8cdlx19GUNT14bGRtyNz7CYl2lKdYaxi7HUA22YmHILE231faThUYZCZdv30bUU6dcu0CbzGMvFQztSfZfhGMcL1Alb3TLScHGOMplGtKN34wFMqirfFUa23tswUshyoWsbl1ekX62FP8EL4rvIXo4sY3IcHzAOLX2WScXapC0Jn1ZOvilKGhTUgsE4q0d8ZjUr+Uw/2q0MMEX+W0YJyrMNM42Tzu7x6cZlOcZqDXlEi3d8ZFxWy8iMRbawV0RZE+rYgcOsaauNdLztB9JlNTHGDPFZEzGLdCFmyKK2Tg5zsqtKZ/+QZkyMWZl43KiotDfEbX2pbVI1B7CbE69rixyHCpx0ADFJGvN1nmBm0ix7H9lazkjcYJa8beftz8pV8p9Gc7FHN7Y9f5vsbNp2dgTPMDrcNsVDHnerSOeUY6chms6bl7s3NvTS65yNeNXMqiet1r6QXe3Nk7YYGoBeJ2SwqZNwI+4koXO0luwo5IpcyqPUkB9sOmuy2DFIXjMiIKYBD9lNG9qvoCcQauGh6iAhXQ4hNeJfZkt17GS5fEAzkErX+Qy27qbXtgZNcHWxbummn/QYNrDsBqfAq7GMBGsirDWtze1yqen9HBAzETzh83oizPZoJVnPghq3RYaEdfc/6cVYShfSmcJcFvEJ4wAIYzGLmMdDHKULmwizpJA9EMKS2G01vuFnA8nu28iy7Sw5kXA7YYRb5gTt7Taxixd5kOXcxQ85azK+pCFvcsnKwV20Iz3yOw831ScTF1CSPWQC9clZz98wmX2cU1vNwJiaid6nvC9LmGwnsFtS42LUZc/tWph5uxxG5/RMyF19t8TGcGsp18JoQYLHLZlqPMUhptHE/SLfxz5Gew4oIteCs50PxVrCHtG6uKxuUa3r44pwNQmMnp4XralhfeZhUVMWJdnjbV+RxAhWMNf99yXuJd6z9JBNLkQzH4vWsIawShOuCYgokgJNe+9E7dwmNg8RDLsmqW0UiL1eShnETj2UQ+IJLsj6QAj1/hSBojRVb9h2JaxL11YeEBYNI76T3pmPaG9BiUJ4j0OM+mMdICsDx7yS0Z/NbNa7GuSUqEE1trOHn+lw1X/GcoZjfJHfikXYsNChpSwQHmCkdyWD/vSgl4d1rKD0DX8igSJ5rpQCKvE2x4mjB8VBRIJLv9YgeI8G7gMO+lDmUmk2Uo/pSJZyh/vqy7zOdpbwEJ+5c63aaIJGK+6hJmVyJsZilpFuA7IeCaw5fSvfANOZygZO5X/bsSo6Av6+yIvXuRpOd7pzmVkpWcWRabopH042L7EA/ZuYEcciieBxPmc26/mSFhzkEd5lD+24wDx68y+GcA9v0iLPc6lYmMdHACIksEjPObcynHuZQ4f8DxaHpXhQ9ErRlR15CAqlj3vYXU8cS/mtxAiJ0Ju6vhtPcoLmzMWOXhNsREVxskXWViCRx1jKWuYzgl/pyHlgALcwmPbcBqzi5FXTWyRkWko66YFF+pV3vS/f0T//KM5Vzp3tf0MJUXRlVR5nh2L04SIzmcFB96SbAiEldJZoMceBfuxkHMP1kl52f/EDu3M+r6A/0/k7R3mQo+4Fahe+IBbJhDz79HWJhjTLSC9KFU4EEOkRkLvztpGVTGYje00rkYu3eY+rtlzlKYGzqlcyjjKcaaxEhyFWq4RNxF/lJjGPVdRi+1W97ALtaEgaB67RAF1wxjLSQ7ks+SUwFDkbU+hOHIPc27AP4+RrxvCSaSVK51Xy7LKH7AdRx0spM4lnNnrGhyiQ+/Jc+Z3N1wysLnZcSzngEqknLSM9jXPCxK5kqKe7rlG6vuALqzdnzh0slYHn7ZOWtAbu4ThrSaciJ2nLYj5iuYe5s6lE7vGxaD8e1qSwxh3wMBKrsl4EgS/bwQz7ZnlXg3APjn5O4rEj3IEDJLN5H/Do1iDbkh2W4EOx1nFInr9kP+YxDpFv2AHo2176S5IOrnWiddE7PTTB//L89Yu++TC2yKXWcsvmiz4U6k2QD4iXki0ifSvictoWa9pT/5yexWMe7en34LSikCGLQT5uheRLnSkplvj6dMlvxYX10opy/co+5PLdmf4mPYHb6cLlG2jMXWhijb6ZuJutPGKCc8KfB/d+OPnM16fj0+Win8RBC2r8LRJhWfa3YDkuNZ2SGYPNlhrTVHQQyx2nDIiYDktMr20aK13iSIlvCjnpF+dyQr5kdl93vQ5yvBEJjmT5VYJ7F8c8fEmqzfV2fHYhJ/1ghniLCmlvmCkzugMPiWWOZGNSQsZK13TMnNhPEyfFwYy51rXmDbR32UtrbES0bGFmQZM+1vqL55ot2GrSsYKWkekfkipfNKxvJEZ/vPvptaZZmSUfkSHk33Zn+oV0mgdUpFOX7WnX/2wLtWjHBTPEpc/gNoY6jhmXlPVKRKeZlaNs5kQUWEEScn7y11Y2ZT6k35ZwuLQpWwyp5hU1cVv0MPEhC7SHjR8X1F7mUZY7PjCjXDvP2XtlbRgvJwrjweu2MUeKA+lDrO0/+ZAe5+QCAYfkqZqdvsyiHy4jcuxPyInszuxn1lSctDH6hVMfviHH67YAXx8HGC/lRdH12ihZhVSRu4KaA1nGE9piI0F5tCHyE36mo5mh+5KnyvEHxavyvAEZu3jNlZnh7Jy42+pWDDLS45yRj7GMLpe+8i1qjBYWPZmpHAq5y4zZPA/trzHukBghj/j4/BrGyPRU0SH5e+tbMegCDcWn1+whPyTWtc3e0ethvS7fixdlgqvlliPml8wxhsG/Zg+TK72eey4yiWm4TrhameeK/ZciHeKcyc/JXhSVq+xLmuvOddoy0j5e7qC5mCRit562pmSO6bTKPjaTl68x0N8ITtbwjOt7WJ7eJHlHwbSgUdI784xfVLqFtoZiqezq3K/NaebRD0SrFD02/ah8lb20ThphjauQW6XbUqQRkw+5hjOW7R57fBprGCSnkXqaxx0PF1x4YKOm1Q586p/+nnicbtFtGSOetD2p7ZALxX8jHX/euGxaO7QtD8t7RQiH5ciSn1i3tXkFGy8xLGa267Wtj2y1lZV3i6bUI+JPd51jBw4SZKbggvxn6BSfjLs+w+ixjy20MnQk1HAU6GYtbP3oQWkglX3s47RIIUtGchO3UY9yQBZfy/kll1pPeN7XLaS/6C2rQKisJKpSmqJEkE4Kv3FCnhEAYpP8T5H5G830BCyAKNA3c8HCU8C6sHUTm2o9V7IlbUWLnBOquYvv8+wjTqwLW2eFZ7vHbZYDvMLoGM3VNvuuE/WOVyP3DLtIl3vFLtc68d+kY/5pM19Jv5UhNKQsISxmF1P8u5VzMIN1OSG4tWK2ss4ShIWkuH53+H97yZWYSCLvQq2IyLJhJVzFxQXXxaTfkQQh+nKRZwhlKq0IZxHbfPbvCfwUXcEFy0J/d8TJCPeMHg50QNJdkR48pPuyZJvAJT4AynOOTKA0UFm1d/DAe9KL0gQH6cDdbg+SWOA71ZR/ZdLTOO0ONxJLPFCDPsxjW0DUJoQmhAPh+abN8wdKEUMxoBpRgZCgy7fh/Q3a0hSIIZHbWUMcgwKiccP5lm/5moqMJCtgKNfoRgYreZzGjGCoKTK78R7fMM39ChVnOktZwqvWVqQtn/IZp/ic6Rg7FGWmIteGBkANPskN4VjE75RHuJN4reZDYE5upAvjJfuCbJ7O/Ws8sdZq7znoyWgTpJivvfdCyx3sk/xuUCpPJQCO0TbP9Y2UMCj5Fd4hhSt5IJ50q9IWJ+65i/UBqKM8fVWIAyd2YydsTMBvnAKqUuEat6uWGD8dM5bDzMGHaBW+k97ImnBXhvSTl/mGPUAUUJUnqBIgJWtDEpeBWoQAN9ObOiZIzeAJYni+IEnv4O9d9z/hLQ5xHLiL+7DRmWyT1CYj6M5rQMccJ1R64AR6coq/myJ9GxN4mwYFR3pagFHeGwcX+Il9jGI6kSylE2v9Xqr2uOjE/6hGRQaxFAhnlYklG88O5lkTkdc6mKfI5Rw7r0Yb94tcmr2EEunn+kXQnppAFTpSMndpuZ9iGM3y84r7dz0uM7qgFLlAQ46Z8ijr3crbAyynI+X8XKoMvuEwcILVXDkocTcJNKWuSd/wI2MZi+bNI38d0q/FZn4nhSMBWLJdHKQkiabJm8wWhgQTNcrK5hsqsvyqsxC1SCmcw3thQk8mksG0XL39IE/hhQNZqGrBIMRCFl5zxatoGqqnF0IEQk/vTkNFhEkoHgykS5zUpKZiyzQ4rYnwpaCgEGz4P2dtFJtocKcIAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDE3LTExLTE3VDEzOjQ0OjExKzAwOjAwJFC8zQAAACV0RVh0ZGF0ZTptb2RpZnkAMjAxNy0xMS0xN1QxMzo0NDoxMSswMDowMFUNBHEAAAAASUVORK5CYII="}}},{"cell_type":"code","source":"model_4 = BayesianGaussianMixture(n_components = 7, # default = 1\n                          covariance_type = 'full', # defalut = 'full'\n                          max_iter = 200, # default = 100\n                          init_params = 'kmeans', #default = kmeans,'k-means++', 'random'\n                          random_state = seed, # default = None\n                          verbose = 1, # default = 0\n                          n_init = 3\n                                  \n                         )\n","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:01:45.172624Z","iopub.execute_input":"2022-07-30T08:01:45.173042Z","iopub.status.idle":"2022-07-30T08:01:45.179129Z","shell.execute_reply.started":"2022-07-30T08:01:45.173008Z","shell.execute_reply":"2022-07-30T08:01:45.177916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![300px-Bayesian-gaussian-mixture.svg.png](attachment:e6d122ef-cee9-4cd2-8da1-2c6b28e63f37.png)wiki\n* K = number of mixture components\n* N = number of observations\n1.  score : 0.47558 ( n_clusters : 7 + no scaler)\n1.  score : 0.51797 ( n_clusters : 7 + RobustScaler)\n1.  score : 0.59662 ( n_clusters : 7 + RobustScaler + PowerTransfomer)\n1.  score : 0.59799 ( n_clusters : 7 + MaxAbsScaler + PowerTransfomer)","metadata":{},"attachments":{"e6d122ef-cee9-4cd2-8da1-2c6b28e63f37.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAASwAAAEbCAQAAADKC3HrAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAAHdElNRQfhCxENLAsogkIZAAAqUElEQVR42u2deYCN1f/HX+feWcyYscUgZBuUsYw5d2goBomUlCKKUJKS6qd934tKWhUlkjUppbJmqRBmMJasWSpMFMZYZrvP+f3hNl/LLM+997l3rjvnPf/cufd5zvN5zvN+Puecz/ksoKGhoaGhoaGhoaGhoVHCEKXy0ho+hY1BQANCeQin7g4Nq3AT/wfAXPqWxOVD9BMIUuwkFIB0qunO0LAa1dlIVd0NGtYims+oq7tBw2pavUQloI1eFRaHe2nJELI0Zwp8krfQmbIAKB7kODPZTC4XsZhp/hfHXsAydRNTyQJ60ZL1AdV5KTyPnVWaRechnK+J4QeqkMTLrGArtTnFvxxhP/M5HggituAP16dF3BZwHfggKZpFBeA53nSphSNUDkwR/4/JrnfgBDUAuIHBDOT2gJAuhlziNI/Om00dpSIAYeRSxfVtFE2ICBwhv+EuAK5kKwC1+BKA12kVEPIt4KUA6alQ3mYNHV3/TaVmiUnSjjWuT23Z4vrUnmdpyou0CIzOsnGYZgA8zUcIanIHIwDozxsBIeFA9gTMkiOcLJf5sSnpJWhsTmKe69NMbnJ9WkM4UDn/F78T6Ww0pyK/A3ZuYB2JxFDftQo7wcUBsfapS5WSWUAXgJbsJh2A+3mLvBKTYxWnqEM4TzCPWQCU52KygX9oVTKvoe08pXqC7tRjCBtw0o51qHzBQgKAVm+ylG8DZlHRjqUAXEFz3ilBOQx60ZybmcR41zeVMc6YgQUAZtMfSVtCsNOO8kA/RgMwiBdLnFbvcRfQlcOEBwSxFtIbqMpK1zIncFCd/a5PGZQJhBnWP9Q+57sqzEMA44gvYVp9xMsuzXmA7gHw8MI4TnVaMDXgaAWwFxtQlk2BIEw8Owv4thVDGMR1JSzbO8zIH5RfY3YA9FYbDjGc2wswMgcCHqYXcDd3BIIwl9I7YG01Pc5Q6RW577zZof8RQ2MCGR3pQ1s0NDQ0NDQ0NDQ0NDSKggB5K++FhAk3DP+5OeKllNG+FCs2vOJBuxvLeGee0SF1rS8lcnRSM0LDzB9vO7kixveP74qDeZFuPLdsddtaP+0dhoAooyrVOVGprNlTskkrS6RvxaopMstVOc9WWxgOsRtXTIrPoMKoWMflm1I8dnLcL/ZudVGUrYHJY4+wo6wtDPxGLIDb0+Prmz0lncF+ES2Be0weOY8xfpHoJlqbPHJkvh+Lr3EZj5s88lde9eNQWPJmRo2ghCaWhiaWhiaWhiaWhoYmloYmloYmloaGJpaGJpaGJpaGhiaWhiaWhiaWhoYmloYmloYmloaGJpaGJpZGaYPL531jWfM5ro8Cyg/JvA7yq8kjd/ups7abfg8P+0McgYLDpntpe0kQa6Zb9VaOsO8O3vZ1kudUUgt4YLklWMHjqwK+O0AEFQoaCnyd3688P/B6Xs7WMq8SiAgBsdDdbFOHemb3ZRbXk+0rsZbmOAqUad9zeXWrDSjol7xtvu2o3DVhBUq0/6syP1csIMZS+ZZY0cyjFdWdPYRbYW/2gM+R/wqKuSWQV28uBwKsJ3L53O/XjGAJikeDc3b2OopZfs9LqokFYXyP4ulgnfYLxqCY5Od1pSaWnS9QjArmFaXgYxTj/ZruubQTy85UFO8Gu7HCzjSUXxNRl25iCcah+LQ0VPIO5RuUH4uQlGZiCT4ogclHiSGMH1A8qYnlc4xE8VVpquIdyRIUj2hi+RQvoZgXIIUT/Iay/IxhOuuQJpb7GI5iUSDUmPA3ypOCwSBNLJ9gGIrlRFEqUZlN5Pm8AEFpJNZADNYWuCFZShDDFnK4XhPLUvTDSRqVKNWoyS6y6aqJZRl6kMs2qlHqUZ99nCRZE8sSdCaLHQFRgDQA0JADHOcKTSyvcRWn+IM6FyYNfLE50JQlhNDR5ad3NZ0sbPsmYvgwoHpwOFv5wQftXkxPcplu0hk1s8TLlPoF8RzmEHEAvIjSfz7/+zvQKOCbDYL1XMsCFtOOrS4dZtUANoOmAVYlMJ2vLTcNxzEbGzeyweTx02heOogFK+nO9yx0FWI8yr+WzWgMy9qyCtkWS9SQGdjpxGo3eiXg4Lud8sV0pwoLA6OG+gWEWJZQjuvdoFVAwpcuGAvoQ236aa64gVospDI3s+xCvxHf+vZ8za1UAtO1jUo7qrKQGtzM3Av/VnztNDaTOcA0TS0TqMJiYrmdOcFwM773RkwD4vi+tO7Nm0YF5nEZdzM9OG7HP26uL5LEN0Ro9hSKcsynBfcyPlhuyD/E+pgX6cDs0ub/aBqRfEtLHuWj4LklfznmP8dIrmZ6afLYNo0wvqQdT/JmMN2U/yI+nmAMNzANu2bSWQhlFtfwIq/prnB3fqWoCYBgLIoJHpI5gjGM5Q9OMJZRARIC9QBjcbKdsYyloUct2JmO4m0v5fg+8PYK/UkssDEFxXsetvRZ/pbrKwFybzfmS7TJI6rb+BzFJ177mJR6YoGdGSje8qilWJwoFMcDxlFXkOYiVk+Pzv4QxUQLtK8mFhDGdyie9aituSgU4wLo7u5GodhPqAfnvoHiS0uWM5pYrtnSYhSPedBWdxSKxAC6uyiOeZhi4FUUsz0ipCZWIcSCSH5CMdyDFdRhP6fSLB5TUB54iD2LYoFlAaiaWPkozxoMBrvd2qSAW5b34DcPVpOKXyhrmQyaWGegAmtxcqvrv6YmW+tL+wC7v/KmDZtxLhvenRistNRP7fvS4ppsBkfpwlImkccXPMhj1MAoamEef4mtnrgos0p4RFhl20F2rfmzpLtOVrY1dFalbMaRCj3FcXapXalFe3KO4CS3cRvj2MC1ZAa3PinJLZaDdOInPqcTg4Aklhd0kKOF6qY6iJant7BPv+YGII+zgiXqm7Vb/C12YjXjRjrQlhgDAVQAFECe3MgS29w6S2Y6C9RsnQinOknsoLN/EsGXrCXG90PhM9Tir0J+rZufleAzBpz9U7OyoYO4izjgoPpZbFHbbP86T9htRBoxoqGKE1dSEUSKGps1aXOOf3pLdlVDxdXYOcWvrBPbOeA8Yc8iksqqjrpMtKY+sI9JIe+s+vu8edXbAGTSuNDe8HwodJRg+vsAm2OdxoP5tutTVDljmAmVD8uDUsk/HK8kNCuE/rbERMdomS6V/DNhsO83eeTVcp1U8njCJHl1bCF+GvISOVyuk0qelG/HVzjrBd6Sf6eTLX+d9eT9PFTmThZinL1Rk9habpJKbpK39ix2yzo2POEu+btUCasTmvnuJprFyBlSycOO55NMWP0dyQkLpZLpjj75X/23+bOOh6lnuXiaWIWgEe9ximNUBmzyCZkrj8phPU37QcSGO56SJ+Qp6aPUb7K93C+NhPGysvlzErrJ3VIljJeRgI0NOJlJGx/1sSZWEbiYD3gvLixhmlQJq1vWdfcyjkZyvVQJ45MtX47I/jJXHnK4nUOnTbScIlXC6hZVuJNvTRtUNLEsJhYQ6Wi+WCo5Tnq00ZEUIWdKJWdaSy35sDTkhlY1PTx7uDTklnJdfdzHmlhFITlEzpZKelPMyibHSCUnWDc5TrhLGgnLm3oRY+ToJ50ypU20JlaJEUu+I5X0thiBcHwulbSozoyjk8yTaWet7jwh51Cp5Nc+NexoYhXR/T2kSvjGe6OBDJU/y7yEDhYMgtVlukxvYUHaM8f7UslhpYtYAVLlQFYW49ibO6DIbR1TSM3N681h8Wkz77d4P6SKre+6/d7fXdTDpDFCXlKa9EmglM8YwUUM3njEiqbS9qnh1A59xssB7Dq6M3bNIiskWpolBlMmuKJwLoihsEVj6ZQzLbymcCyTp2R1r5YBm+XBphYmBpATpSF9ZXLQQ2EhPHgc1LMWNqjEs5ThIS9mRTfSmFHWaNDTsL+K8kYiPRS6jWYxojcWeymsWaZWcEes55HX95MRYmmu09Xb+YFebaI1sfyG0D6Eio8t14ITqFihm2fntqitrlTTVx2zWKRpRGR108TyH3ryd9RCy1udSa66wcNO6YlgitUClZlDruqiieUnxEXRUixcmmd1u6kZrMZDa5boSEb9FVZLtDyTDeJyTSw/ISKJUJb4ZEmwjOrxdTzpE3WF+KlAL1CvJ1rExkVpYvkFRhywzictbwG7BxkV4i8hSvlEIvYiytTUxPKPqaER6tQOn7S8A85xqmvISBNd0giUT6IXxQEQ1TSx/EOsGDI2H/dFy/YjkF/lT9CeGfzGLSa6pCrY9vlEhzrBotjngEeJJ0JT0b4KhMo+EQLlsJFAD26mAQC1io+PSQ2/jDI+kclmV6iOLNTE8gexwoSPImwapu/KzW3A49yIzHdayeKn4s4Lr29r4huZnCm2P7iXxzWx/IGTFoaan4VDgtCww7zKq9TmZgZxKXCIG4o7r/FgMZZIX0i07je5AYeeY/kHGZT3jRNcZgVQx1yrsVE0poPJQSgDlK/y0lfguCaWfybve4hoUd0nTdcHted/Yy5LuJrupowCiHo+ut1Ydmti+WeOtQ24zCdNXwr2bed8l2Zi7bYVlE8kkuWpJrZrYvkHa8HW2ie6sDWG8MDQuf4ou4RvJGoDRqomln94tZFDdPBFy6ojG1Z7VklwMfFJPshzqjqAWqyJ5aexUMznilaWp7Ro0Zz6zPNQs8zHlnuD5Xcq6MGWdXs1sfwEYwohuX0sv7F+IDx0fYn6jqPG7ZbPsNpQV0wBTSw/of5C/hJDe1pasaJVOe5Qa1I2eXb20iymi7aJ8Rar5gfJc07WxPIbZjrVKGJ/721lm3lDqcgILxp4E6fxpJUSJcaJG9W00jIQBkowxTjSxSvNLLPAy+o8Rtra2Z63kPo7U7k54UoLB/y3MGyvgCaWH5F6Uj1C7RDL4nTUW5Qz7vcu+NV4gkzxQaxFhfASbuFq3k3ZhoZlMBdiLxyLpDPxKkse4oAWyvGp9+047rMglwQA8XXkYbnXhxE6OndDUcOXTJfp7ufFOo9WstFJe57dilWmSPhGGvJWb5uJi5IpMqdFkg/7WAesFjEcHqAPFZ3zm8V4RatY8YPNaRxyTuYpr7e2VfhAtjNBXu0VrcLKfIkUw9etLF36xBY4oqQu4Rbqha6Q9b1YeS2hQtSNqik/8TIzvHXIWXk45CrS+dZxs6ctNCsbPpvOjEh5X8+JSmgodA2It8ps+VeiRzt1CdfJIzLTpV9CeBdFGnW8Fb9lQ7lb5iU86In+i68jU6SSI3ye9FzPsUxQ62p5WObKJ9xL+Cgj5ShpyL8S5BlfDiaHf7wvkdLiYpkilZzt7rZTwi3ysMxx3OeHPtbEMvme/yqV3Gg6eZpI6CF3S5Xww3lZja8gnVyPCtidhdhw+a405BE5LC7MJBkby/lSyb0+nbJrYuFmLF1yiHxAZkglf0roVrTmSi7j6CPXSyXTHbcXOODUJAXFWMK8vQ1HstwslfxDPlBccqMWSfIL6ZS5TT4Iu8FPfRyAxCrpkieFolXVvIcZQhQHmaUWqZ/XHTp3gLIl04kbKc9h3ubd1IxCGirLRG5mOTd52/kyVA0Qj1OPbH4Q821LVu84XULnf2aFMq1pz83E4mS6emXt63SkKb/7hVgBV/IkYIkFkFQpu4/oSysEcIjt/MNxbCJKVaERFQEny9Tn2V8WE5coeJRX2c+NpHh7M8khx7uqflxHGeAk2zjAcU6JaOMiUZdaCGA7U5yT1u8BmpHKYjprYgUcsVxL9piQ9iKJS6lLFaIwOM4hdrKV5c6l64+abORaphDOXVjiXZAUkdXa1pbGNKIqUZQhg2PsVttEim3xml1nHDiaB+nD9NJIrICcY/kEjdiKwQi/Wu7Kspt0KvqBWHryXoKoxEIUP1Dej9e8AcUYTazgJhbYGYFiG5f68ZqzcdJaEyu4iQVwGyfJ4Dq/Xa8WmWzwcSoQvQkdAJhCR04w23uzqUn8yXM05UE0AlBjjaQMAE1IsESmi/kVxdTTdaYtQSj9GMpTZ1aJPWMAXssJ6vqwj/VQ6OF67qDLLPItgyySKpwJKFKxqgzJE8QAV5FWoAEnkTx+0MQKNGIN4UvXm3/EleUqjmVc77Vkg8llP9akm/2StgBkUaPA399HcZMmVmARayoPACD5M/+7X7GgKhedOUwWAy27VwdbKTiMrRx/sd9nhg5NLI+wn9POMMOZ6Pommo2AoJzX0sWyGcVYS1Zt5fimiJlUTxTvaGIFDrEa4XRNsr9jKA25EujCGOpwK99bMEeK5msUSwucdrtHq7eoSsUiFgRzcHJ5aSFW4Jsb2mGjItCNeuykEyuBZHKpz1RG4301wUx68AJtWUmcW+fV43p60pOetAbK8RqzqMlDRey+3scpxgZADkWtsVwzrNcYRn8aEctQV6LFlQxjPLEWStmLE2Ryo8mjI/ictXzFSRYxlmHAy6SQQkoxa7/HUD6xaOmh0APsO28DJpoNwKMMoYuFlqjm7Da9ST2ZRwEYyni3rhHCOo5774evh0Lv0QjB1nO+a8IcYAW1iOCUZVdKI5FlPMb0YmN7rqQxbwCQ7iZJ8ribCN7Sw1jJa6zOfn0M5mJ73uC/ssDP53su1OcNk+njxqJMZULVQ6HPzQ3+RfGxPVO41aXtd9Ai/9tpXGGq/fLs4w+igp1YpW8TujiMowN5LChik3qeq0LPU7x9RnkpyWpT7WfwCLV4QXd0adNYUFxsj+BObuc+Wp3xXUMWIOjK05hJLf49eWfoOj0UlhpiQVlmovjFtCf5XbzCAKJ52tRWUywnWY09mImlh8KCcYJePE4SKSZLlCTTnErAy6ZMtjt5lUSG6G4ufRrrNK7lKKfoa+LIbYTyAA9Rl4tMtRzGZjIK8YTQGivo8T2t2MukYs2mDdlFLn8RQefiy9YBkMMQohmtu7h0aiwwF9sTSS0AHG6ZEcajLPK915P3CxK+iu2pxN/scSuH1yA+IYXZLg0aznekMJGWmlgXLnwT29Mf5WbS8ItQDMj/7/9c1Rc1sS5gJLGfPItjewSLyCXejTMqoujv+twn37tLT94vYKzEQQojLI3tUdxDHmM9egqD+J1fA7e7NLHMYz/tmEgffrEstgd2MJKWHsQePY/T5BaSHgovEFgZ2wMQzhaOmg4NOT0U3s+3/H7GClTPsYICVsf2tMNgihvE+oE6VOBPPtLEqhlk92RlbA/AJBRdTRPr9KqwCwZdNLGCDVbF9vxnRDjEDlOLgjNXhR+zj0p6VRhc8DS2p2D8yxPE4m4Zu4dQvKcn78EI92J7ioJgMdk0LuaoDoxAMYf+CKAlaSgm0kYPhcEHd2J7ikYjsvipmJyw1ann+hPARa7PVTSxghGVWYLiC7wv4/kKysO1piZWUMKquj0R7ORfYoKDWP4J+N50dqr9IMRJmrGLE+R5+TSi+JOTbp8XZdILLKiI9RuzS4VmvoiWlOU3dnjVioOarOGQ2+cd1Qv14IUVdXuqcpjtrsSYGhouuBvbUxDuQfGs7kqNsyF4DCd/moztKQg2lpPl10z0GhcIupqO7SkYTclhqR+qHGlccPC2bs/rKPrpbtQ4H97V7YlkF/9QWXejxvnwLrbnGhQf607UKBjexPZ8ieF9oXSNYIXnsT3VOcpWwnUXahQMz+v23I9y20crYFAaF7U2P991OGPoz3p68Iebcv5MM+L9Uq7cfRhBv//rNlag9J/Xf08X3cmlM519BjP9fs2atCeE5Wx366zLacKSgNNZ0dyi9VNBGmtriVzXk9gef5Urdw/1itdYmlj+fdPdj+250S/lyjWxLmhigeB5DHa6Fdvjj3LlmlgXOLHA/dgef5Qrt5xYOq7Q//iC1vzDLNOb1LpcudZYpuFebI/vy5XroTBIiOVubI+vy5VrYgUNseC/uj3mijv5tly5JlZQEQuuIJ1cU5vUvi1XrokVZMRyJ7bHl+XKNbGCjljuxPb4rly5JlYQEst8bE9tjpMWAPu72o51gUAxkm5E83MxsT17eYlm3Kc7TGssd2Amtue/cuVRSD0UamKZhZnYnpY4WcHeEi1ZrofCCwyH6cJIrmF1EbE9h/iLJC4J9OAwTazAgpPH6UstVhUS23M3m1zlC2I1sTTcwxQ6coLZBZpNZzIFA6DYfKUljAvGNbnlRXkNbNGqAtiOkGFsT80oaYmSyxxraI9xRosQcdx5wvg9bZ9lTa/EwVeMoDl3cuqcwXIwX/A51ShP/XNdlnva99Q2aquyIkJkO0+E/uncnZqriVUI4huEXEd7leSsLFxhIadfWLmfFSwOmbPqL79LVMF2na2DaptZV9gMBKCwYUNmksoS29w1ayy4yH7a8REDuJQb+AOIOINgi2jOF7Tjiv+I1dO+uwOdaL+7mQojXyInIs+xTS0Wi47O35nt714K4PCvuLCIvgxSSUAmK9VmsZNDKtNmM6JENdFQxdGKCAyxRI1NneVim9lVYSVP0wQlXCmG0Y0yONVakcY2sY8TnFIVKE99LqU1McA2JvCRJRp1MB9wiB6ksYibOXDGL2GMxUZ/SKxl3G/rZ1SFGKORrSbVCKcsJ8niEPvYof4SYMtQM5zvrPvNwlXh7zzDyxcisWyOu9RT1OIfpokZUauW5hU0FGW0sd+mbqIcW9Vza7/wNbESWxsjuJIc5orP1aICiSNkE3GT6kc9jqq3w19fecrrfujMNCLZQCIv8tw5T+7ZJp+GPyv6q9B6qoNoVciO0BFSWMxvKKW+Mp5Yv6NUEysx3jlOJLJbjDw6sTglHhcVfo94iKossA9ZvdtXxIqvYBslBnKSscaodfuLGwUc1xtPiUR2qXvXzve6MxryC1WADGL558yrJAy1vUq0g140KraRP5jFMqVy1cislzfn+INYAQjH7fKkPOl4PtZ03gIZ6XheZsuMhFtMEstNA2kLh9wplZwj3ahTKHvKv6Uh35He+qp3Ic8VIvr2mUsZOUeq64zv1T7Tf6vVQEMqR2pCrAXEKtZAGmAaKznk2FhxB+vtt6x2L7AT2Yrp1FHPr33Bao3l6K0+45QYlPKlexI1iwmdTCcxT92UetLjDongW9q6XGpycbABICHWvlDVuZ7+bq69FLP5THFMXZuy3NcaK6DsWHFhmV+KO5ganeQurSB1VU4CS8Tz8l1rXxZ5j5rCXqS7tIINB1O7qNdVFxa0ifb48qfoRDk68g57CeUT7JDQzLbCVvtR7nR7SS+4kREiMlr9KK/1+SQ5kCbsZSbSnfdS+y3N8uT0jUcyruFrhskXLaTVrbxPWu4VqZ4FuRtrHxOP0SZrplcDYjaLeZB6XMUunkmsZ18QdtELoo2HjV3KG7bKoeLLFm1LDbHkK/RhfOoDbpkOzsLO7KzeahFPJwywRqKEK5nIDqPzhoOet5HyOiPoLEZ7LYzBj/SuOMtYYIt52tbUi4Zq8LItOtQ+J75BqSCW4xoeY0G9u71LjrM5J6IHW8UHLSzY8JCVxVQynV3WHfKundQn+VoNdXS3opfqPkn9YaK5l61czHN2e1TIl0kRQU+s+ArqUw7k9pvp9Lal5Zn0Qtgm9bR725J6jxpqwPo9Xt+cChvEX2qc9DoQwtFb9O5kSf7IBgyyqWbZr/juiYaAo5N6MDRMuDHlzc0xJrphkDQjxsuqmrjBmyHnDA2xUb7Eq7vuOqMYtwdIvMroLSakzrFCopWHHfep2TzLQ9600iY6562KxiCLVEFXUoyU++Xk1LU+Ixa16BrmDDX9hjvJwbbcSiESYtUQvkv5xqr2skaV6c+LcpIXy3xhvMER52NWSZTyTcIiMVS+mXrAi7t6mOqDsW70GmxbZzjfNBnV6BGxgMf2xNc3e0o6gy0WQjyO7ZwNC6+wOSfhJTGZQbzrMdWvJZ5nvZ1dnQn7y8ZSNdRzh9420TkPXKKSLDSlVOM62zftE9utWRakc6ykSvTlB2tVcrkZ7OJ+LyxaD5CZ876VEq1Zxlpxl+czv+z+Rvnewlp7dg/shnrIN081AIiV05twMa6QHycynVaFntqF6Yws6IeleXxG/YRWnknUvIZor2ZuPGKxXp5MzC7PrUf9yxutLO75irQW6hpZPUiJpXrxj5pbyI8R9GYVA/mAsXzkSk4dzRjG8j6NmUdvqhVy5hQQPT0ctnpiF5Otvk/nV6C6eXZuy7rK0dZmt7zvrxaEKJ/kEy1xRz8ZyeV8VYyn4wRWsJXbmApAJnspw5ucKHJt+Lvc4enKXHQkk1+KPKQClVyfDnAKqEE4cJwi1rXr9so9IskzXh3sepFP4r2aEJ6b0/nM7e2g0VhGEuFqabGHnQL+c6DpxC5eKJpWp8dDmsdX8IRXXMnPxVC9Gt1I4TuuphwAN7KVx4pLAClWEZ/syas84+8hduULJ3c78ah2cWFBSCxbHJDqjvYmzFwybZWGLaShBzq0FuUpbimxlXfYzRo+ctWPP0U77mZJMRJto8ypah4RoOzFPqrne1koEWUaBQOxXjsnZ3AjVKh5v8bexPC9ScWzAwwz+2H3nD3KiEYg3POuuIMUVpo47gDk1QAECcScozocRaVgExG1fOTgVBtQzYKBWMPZxbv/C14S1Ti26pjJc3vyDLeaNiIcAGGmOnMya/iOrvl9UQ2UOyEaA1hDmqkjj4EoR1XWksofDDxjefYTa9jFK4U+Jbuv4lOrA6JmgEzeszmY6LGV1EYUwxjGCr7ka/aoKDJNnnkT73ELKQwyV8nPnn2Kf3pw/LwfqlL2LOnrIbiWa9nLTGaxWkWDYVYmwVs0YqLJg0MUqhev0pRn6MnH/Mu3QDhfkcRLtORJDjMKgMZckX/aQiHwWWH7jNNLkcAg1kkOdsfb3frWtGYg452RdrOhSQtYCbzAKBZhwrs9JyeLA20o2HFpbIGjwsP0Zdbxw1HYs0zS6kEWMYQ7GW+6t3tyhMF8yif8ylTak8KnJPMcLxLBYl7nD2by/Fn7EDchIMxHxEoPJGJFU2/srokeXvEnQoHtTGAmv4Ots+lg8dNa5HW68zntKNYPIiwT6s3a9eZ5P3xMubNqwbxMRyCDKUxnOUbZwWBEmZIogcc4wMu8xSL2Fn947vyQX6lNMzKBdK5hOQvZRBvG8yJwim6sYAr3kswinsz3S/tdKcjyEbGyAbU8QIgVQqU/d/3q4RUVCxnJ4v/8rmzHVDm3zncykHXcjwnXOTsVd/NrAQQNPevbf9nNa0z9z4BhO6YQ5mRK5QAwkuuZTLviHRTT9slcTuQP/VvoyBga8EZ+VcJ/6MhYkpjAvWcxycCEdcUzZAEiLUCI5RU6cNb7YewVkYnV1qQXeU4tkgHJRrYDR9nEaxxhrmupXxiD64GpCoGjWMcZVivnXhvUc+OO8ujPOoaZyg9aX2w54791nGsw/ZOuBd2Kyv3bR5Up0oGQvda36/9V4Tlq17YNjMuKOacJp+jFOpdZoC4j6ccJahWj2hoB20xItJqzjKFltoGpKB5bfu9tZTSvEV/cCW2iqa62e9Jp6uSfhm8exz5sf5telQe0xjoX60AlFWNaPHsncYXJNVhrVO569wVaeVjupbitl2SuJ4eavME4dnAZjfiNUSzitSKHnSQEHnlxODMPlT+cv4tkHZxsM4wUXzzWEidWSpr8l/a86pNBd5OHPqlL6CfLF5l9YSlLz/hvi8nU/h2gOOt8gWgeew3T03xQ1H4HWTYW++K5lrx3gyEWibayMPtfOdoWkbvuYtoWZuBJjKMRHga4i/nYRXcf3GsPdnoUSHYsfJ7IWeUDgX4FbPODk1gYUwgr1HVjGk0ofIV2EU2YVkirfcE2xcOF67dkGv2svk/ZigbCQ4lSM4w5q9Vxq3ueZYbYsGZzkBJLzCNdDC1EkkmMYVehp25kDLML+qFZWe4kbc16Dx/jSb4QHRLjLL7R+zHU5x6f/VmemGuxQCv412ZM8M1TdT3OY/ajmP3LtFiE1FwxmsscPaxsM/RuqogRXjTwJhhPWClRfANuETNTPS4bvvZ7fvvWsDJ/muILZTuS/YlviOWavL9ZpyR1VviHWY+o15K/W2qReVlW5im21J3p+XIndatjpurjeDdltWUdPUrhVeIfQ7yUMW1mMRUG3MGP7BG8sfm4z4hlpHnwblu6CbA80/G4+iTzCcsidd6gkujtXfCrepTr1JjkywtK+eY+HN1VN8akbPJq/TzDMfSrNskWuSIc41PDtrfsaF8pi0BJY2STy7ic9qm/eN9UQi8xQ01f26eIqYWpNEYJj4jXGZFqwYDYvEbIOvJy4rwNz3A0IbVOyJs2743wipdIUarr2nkenX4BpTEyuI1jTGtew9uGEuPEx+wy7vFepLWjWMij8gZv20kuEzKDStzmfdRPyiYe2m370Lv0FgB8QQqM9pBWF8qq0DWr+UP0o1rIvKYVvZpdXWLMw656rT9qBdlz+/IHUxOu9KaRnvbMabThqdQlVvRSyvtMXYS3AUQLmYrtZx736RBEwCDlBzWQuLCfPNdaifX4kaqi59pUayTacJCrOCbmJnTxtIXY8F1TuYEPU0da1k0DbPNn8rEXWmsRHyi1Jau7b3PAF08sP276rJ2shnBZyM+J8R5pq/bGr9SgV4qF5p7U3+lEhvjGcadHElUvv4hejEm1sBBcam72Tcyfw9t4Ynpw8hnvojbaOlgdjusesZoygBt4jSS/UWuc6kkVY6XjPvd0aVyY43kWguqcOtviIXqjsw071SdysruDtKMr62mtnkodiqWeCRtO0I3PljDc2O3mmX/zlJqFWGi/shg3JR+jFoMAiGSVPy8rL5VpUslfExNNP8JO8jepHMtMDqJuZ01uVlZOlEr+nTDQbO6F+DoJX0ol0xOv8lkvDZJZicYLapvJnMm71Gh1uVM65TMWTH+8LCvXzeXJXZ6K/iRW6taMljxDM2O1/N6RXLRBJDkkoZv8RS2gurqvbgcLq9mcoyNSB6jrOSE+3fVbwkAZWZxRIGG8fbu4kU/scWsW+ayXPlFN1PxvucP4/Kz07wXbrGYxyJhM3lrVMvUlDH88xcIfWwytOJ127GlOuWJH/IgWtcUTYgDh7GEGi/nl3FxXsrxqKzrRk2ocFx/mvOmGg4yHlSmSInLu4WGqk8nXzGfJubmukkMyEkV7cTMtUGqOemldiu97ydGVZ1UrQVPVSjTjkvMe5342kEKqcgqxzXh57VSLSOVVZYoBfMFdRHMRmTxngenEE3JdbLudfjQGnOxV28RhMoVNRVGFhqd7Ua0Rk+1TVv/rVrNe1NJJLnPsZtGXq7ADB8VWlS6OKieRVKABsYQB+5hmm+gbj4GCkXCluN12ixENYUYNUV1EEoqT4xxin3HSBiLLmG37PGWehZrKK2IN4SOaUxk7A+lTklO9xHpGB5XEpaI+lQgFsjnMNrGdX5w/Flt+xGJiudY0FcPaiXaqMQ2JIRIwOMxetos1zsXrNpTESyhDRUvVUTWzXSouMaIBxAm1X21ho31pyHILavq4TazCCXd3/ue0ooK//YvkMhYksLC2JrStWVkCDK3K+fgCJibvhVmpGvG/aJJIqrAnMLpsaVagPUSMDScCTSRfBEdYZcdqx38OI5WpyhY0NCwhVtv82ddw3ua47igNK4gl2Mnd2CjH08Dzups03EXBc6zLWMxO+pLLBPYFnMx1CTMViOprVGUYK/iBGpTh9wCQpz5tCGcSVxHFNtZ73d7pXG/7XJuS5bEBNk5iao1ZsMa6nFXsYxLTApBWMIgaASCFnXcx6M9D3M3uAJCnHi2ZBMxlD00sqX4quY3tfOWaFF3KSPbwgHe7MEMCWssuITIAzA2RXAyE8qCrWITg3hLtldNZ7R9gHhCXbyAagDfGkBjeJJu78v9/1ry5oWCN9WMA0yoSJycDQI6T7CeSoYx3DQ0hHClRed5FAYksADbnm4fy8M44uoEXGE39/Hv2cvK+I0BJ1ZT/YzyVuJf6ASBNRYbxYX40nFFY6Kxf0Y5lZw3Xk73eyBnJJia67xFh40JBdeYymrUc4Fl2soT7SpxWd/MW2QiGA315kf4lKk8iITSgChuADsQC13EP3nuuOumPZFiwEqsiP2GnM8uIZzELeJ/RVC5BeUJ5jQzeZzhzWE0kBxEl2pex/EQ1ruVvnERxKTuBUI5aItM2nmIEcQQlRqJoCpR1zf+Go85I/+r/ybuDGKALE1w5Tm1sJKoE+yeE2+lDA67gHvrwnzviXLyrOhDD7a67W0IKoTxsfvIecoEQqyv72Qi0cYXKtiLX0q1kd3Ha12oe/wVQJZNCHPv5s4TkyWOSa3b8v8jMi6nAYVqbzSdWBAwGsoFHyAm+ofCIKy1kMkuB+lzPe8U6TvoTTjJpWGK0Kkxp7OZa1ljS1h4e4blzih4EBW7iFPHAEiJozCYmeKFrV5SorruQ0PCMSbtgHmOCbyicxWGe5ASXMg4nQ1mmn7rPMZjbqERtHkEBiju5NVhvtTMvWtCK1ljewssoncDDlSzRT/XCQMgFJW3TQssYuYfa/o2UDDqEBxuxrEk4u4vyJWpzCgb8xiHdCRoaGhoaGhoaGhpBgf8Hf+RSkuZRc5EAAAAldEVYdGRhdGU6Y3JlYXRlADIwMTctMTEtMTdUMTM6NDQ6MTErMDA6MDAkULzNAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDE3LTExLTE3VDEzOjQ0OjExKzAwOjAwVQ0EcQAAAABJRU5ErkJggg=="}}},{"cell_type":"markdown","source":"+++++++++++++++++++++\n# sklean Agglomerative Clustering","metadata":{}},{"cell_type":"code","source":"'''\ndef plot_dendrogram(model, **kwargs):\n    count = np.zeros(model.children_.shape[0])\n    n_sample = len(model.labels_)\n    \n    for child_idx in marge:\n        if child_idx < n_samples:\n            current_count += 1\n        else:\n            current_count += counts[child_idx - n_samples]\n        counts[i] = current_count\n        \n    linkage_matrix = np.colum_stack([model.children_, model.distamces_, counts]).astype(float)\n    \n    dendrogram(linkage_matrix, **kwargs)\n    \n\nagglo_model = AgglomerativeClustering(distance_threshold = 0, n_clusters = None)\nmodel = agglo_model.fit(data_1)\n    \nplot_dendrogram(model, truncate_mode = 'level', p = 3)\nplt.title('Hierarchical Clustering Dendrogram')\npli.xlabel('Number of point in node(or index of poinr if no parenthesis).')\nplt.show()\n'''","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:50.521489Z","iopub.status.idle":"2022-07-30T07:57:50.521832Z","shell.execute_reply.started":"2022-07-30T07:57:50.521658Z","shell.execute_reply":"2022-07-30T07:57:50.521674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This model had a memory shortage problem and did not produce results.😔","metadata":{}},{"cell_type":"markdown","source":"+++++++++++++++++++++","metadata":{}},{"cell_type":"code","source":"elbow_m = KElbowVisualizer(model_2, k = (22) )\nelbow_m.fit(data)\nelbow_m.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:50.523670Z","iopub.status.idle":"2022-07-30T07:57:50.524021Z","shell.execute_reply.started":"2022-07-30T07:57:50.523848Z","shell.execute_reply":"2022-07-30T07:57:50.523865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* KElbowVisualize shows the percentage of explained variance as a function of the number of clusters.","metadata":{}},{"cell_type":"code","source":"model = model_4\nmodel.fit(data_p)\nprediction_1 = model.predict(data_p)\nprediction_1","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-07-30T08:01:50.001567Z","iopub.execute_input":"2022-07-30T08:01:50.001950Z","iopub.status.idle":"2022-07-30T08:03:49.825743Z","shell.execute_reply.started":"2022-07-30T08:01:50.001919Z","shell.execute_reply":"2022-07-30T08:03:49.824471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pl = sns.countplot(x=prediction_1)\npl.set_title(\"Distribution of clusters\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:08:03.229839Z","iopub.execute_input":"2022-07-30T08:08:03.230310Z","iopub.status.idle":"2022-07-30T08:08:03.490912Z","shell.execute_reply.started":"2022-07-30T08:08:03.230271Z","shell.execute_reply":"2022-07-30T08:08:03.487677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"+++++++++++++++++++++\n# sklego mixture BaysianGMMClassifier🎯\n\n","metadata":{}},{"cell_type":"markdown","source":"*  The **GMM(Gaussian Mixture model)** is the approximations for general probability destributions.\n*  This **sklgo's BaysianGMMClassifier model** can evaluate the likelihood scores to see if it is **deemed likely**.🎲🎲\n \n\nhttps://scikit-lego.netlify.app/\n\nhttps://scikit-lego.netlify.app/mixture-methods.html#classification\n\nhttps://scikit-lego.netlify.app/api/mixture.html","metadata":{}},{"cell_type":"code","source":"data_scaled = pd.DataFrame(PowerTransformer().fit_transform(data_1), columns=data_1.columns)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:12:26.197485Z","iopub.execute_input":"2022-07-30T08:12:26.197893Z","iopub.status.idle":"2022-07-30T08:12:27.932924Z","shell.execute_reply.started":"2022-07-30T08:12:26.197863Z","shell.execute_reply":"2022-07-30T08:12:27.931861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# If you want to use the resulting data from other notebooks.( Using + Add data)\n\n#pred_test = pd.read_csv(\"../input/tps-jul-2022-submission.csv\")\n#pred_test.drop('Id', axis = 1, inplace = True)\n\n#X = np.array(data_scaled)\n#y = np.ravel(pred_test)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:50.529458Z","iopub.status.idle":"2022-07-30T07:57:50.529823Z","shell.execute_reply.started":"2022-07-30T07:57:50.529644Z","shell.execute_reply":"2022-07-30T07:57:50.529661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = np.array(data_scaled)\ny = prediction_1\n\nX,y","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:12:54.936063Z","iopub.execute_input":"2022-07-30T08:12:54.936855Z","iopub.status.idle":"2022-07-30T08:12:54.946123Z","shell.execute_reply.started":"2022-07-30T08:12:54.936813Z","shell.execute_reply":"2022-07-30T08:12:54.945050Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_bgmc = BayesianGMMClassifier(n_components = 7, # default = 1\n                          covariance_type = 'full', # defalut = 'full'\n                          max_iter = 1000, # default = 100,\n                          tol =1e-3,\n                          init_params = 'kmeans', #default = kmeans,'k-means++', 'random'\n                          random_state = seed, # default = None\n                          verbose = 1, # default = 0\n                          n_init = 3\n                         \n                         )","metadata":{"_kg_hide-output":false,"execution":{"iopub.status.busy":"2022-07-30T08:12:57.195447Z","iopub.execute_input":"2022-07-30T08:12:57.195816Z","iopub.status.idle":"2022-07-30T08:12:57.201980Z","shell.execute_reply.started":"2022-07-30T08:12:57.195786Z","shell.execute_reply":"2022-07-30T08:12:57.200550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fold = 5\n\nfor i in range(fold):\n    \n    model = model_bgmc\n    model.fit(X,y)\n    prediction_2 = model.predict(X)\n\n    X = np.array(data_scaled)\n    y =  prediction_2 \n    print(prediction_2)\n    ","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-07-30T08:12:59.604393Z","iopub.execute_input":"2022-07-30T08:12:59.604839Z","iopub.status.idle":"2022-07-30T08:25:50.006866Z","shell.execute_reply.started":"2022-07-30T08:12:59.604802Z","shell.execute_reply":"2022-07-30T08:25:50.005515Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* y (test data) is the best submission data(score : 0.59799).\n* This results of method have been dramatically improved.👀\n* I tried to repeat the process to see if I could get even better results.\n* It all depends on how you choose the best results.","metadata":{}},{"cell_type":"markdown","source":"# Submission","metadata":{}},{"cell_type":"code","source":"submission['Predicted'] = prediction_2\nsubmission.to_csv('submission.csv', index = False)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:28:59.496030Z","iopub.execute_input":"2022-07-30T08:28:59.496481Z","iopub.status.idle":"2022-07-30T08:28:59.692970Z","shell.execute_reply.started":"2022-07-30T08:28:59.496444Z","shell.execute_reply":"2022-07-30T08:28:59.691943Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.head(15)","metadata":{"execution":{"iopub.status.busy":"2022-07-30T07:57:50.538714Z","iopub.status.idle":"2022-07-30T07:57:50.539472Z","shell.execute_reply.started":"2022-07-30T07:57:50.539263Z","shell.execute_reply":"2022-07-30T07:57:50.539287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ax = sns.countplot(x = submission['Predicted'])\nfor p in ax.patches:\n    ax.annotate('{:}'.format(p.get_height()), (p.get_x()+0.25, p.get_height()+0.01 ))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-30T08:26:38.626631Z","iopub.execute_input":"2022-07-30T08:26:38.627038Z","iopub.status.idle":"2022-07-30T08:26:38.887011Z","shell.execute_reply.started":"2022-07-30T08:26:38.627004Z","shell.execute_reply":"2022-07-30T08:26:38.885524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Summary and Conclusionm\n* There seem to be many different methods,so I wll try them.\n* I choose the feature(f_00 to f_06 and f_14 to f_21)that the results of **shapiro_wilk** test and **Kolmogorov-Smirnov test** are Accepted. but ** Kmeans**'s score for this featur is 0.00009🤔\n* In **Q-Q plot**, Integer feature(f_07 to f_13) show a different from the others and f_22 to f_28 are a slightly different.\n* The same results as above that feature are f_07 to f_13 and f_22 to f_28 show with **seaborn's heatmap**.\n* The cluster seems to be divided into 5 or 7 with **seaborn's clustermap**.\n* I tried to get a similar graph using **Sklean Agglomerative Clustering**, but could not do it this time.\n* **BayesianGaussianMixture model** is better than **GaussianMixture model**.\n* I changed various **seed**, but this time the random seed did not have much effect.\n* **Scikit lego's BaysianGMMClassifier** is the method that has changed the results unexpectedly this time.\n> [@ OUTATIME] https://www.kaggle.com/code/karlcini/bayesiangmmclassifier\n* This method depends on how you choose the best results.\n* There are still many methods(e.g. Using probability), I would like to do more research on clustering.\n* Thank you for reading! **GOOD LUCK!**\n* ~> in progress..~","metadata":{}}]}