{"cells":[{"metadata":{"_uuid":"73d45ee23393e6636351d06733a18dd83ef93d3c"},"cell_type":"markdown","source":"![](https://s7.wampi.ru/2018/08/11/cv_header2e2d4698438805b5b.png)\n# Here a computer vision technic  for TrackML problem is described. The main idea was to apply a sliding window (255 x 255) to a hits dataset preliminary converted to a spherical representation and then to feed the sliding window data to different neural networks architectures.**\n"},{"metadata":{"trusted":true,"_uuid":"9a2bfa9ae2395f30e49ce9926d1db44c18ff3b7b","collapsed":true},"cell_type":"code","source":"#  -------- Here we load required librations --------------------\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom mpl_toolkits import mplot3d\nimport seaborn as sns\n%matplotlib inline\nfrom trackml.dataset import load_event, load_dataset\nfrom trackml.randomize import shuffle_hits\nfrom trackml.score import score_event\n\n# ---- Loading one of the event for data exploration purposes--------\nevent_id = 'event000001002'\nhits, cells, particles, truth =  load_event('../input/train_1/'+event_id)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"28fc823c568e38f511f6eb16ab5efe4147f12dbd"},"cell_type":"code","source":"# Function for coordinate conversion\ndef cart2spherical(cart):\n    r = np.linalg.norm(cart, axis=0)\n    theta = np.degrees(np.arccos(cart[2] / r))\n    phi = np.degrees(np.arctan2(cart[1], cart[0]))\n    return np.vstack((r, theta, phi))\n\n# Convert to spherical coordinate.\nxyz = hits.loc[:, ['x', 'y', 'z']].values.transpose()\nrtp = cart2spherical(xyz).transpose()\nrtp_df = pd.DataFrame(rtp, columns=('r', 'theta', 'phi'))\n\nfig=plt.figure(figsize=(30,10))\nax = fig.add_subplot(131, facecolor='black')\nax.scatter(rtp_df.theta, rtp_df.phi, s=1, color=\"green\")\nax = fig.add_subplot(132, facecolor='black')\nax.scatter(rtp_df.theta, rtp_df.r, s=1, color=\"green\")\nax = fig.add_subplot(133, facecolor='black')\nax.scatter(rtp_df.phi, rtp_df.r, s=1, color=\"green\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d05f80112e12a2d37361ee17b051b8ebab2e84a7"},"cell_type":"markdown","source":"The sliding window data is then fed to neural networks engines with different architectures. Each sliding window data has a shape of 255 x 255 x 48 f values where 48 dimention represents respective hits of each sensing layer of the detector. The data points density in a particular layer is relatevely low, thus this should help the task of track detection to be more sutable from network performance perspective. The neural networks try to predict tracks (other data points) assuming that a seed track data point is located in a centre of a sliding window.  \n![](https://s7.wampi.ru/2018/08/11/sliding_window.jpg)"},{"metadata":{"_uuid":"3099e16806d12b3f278d4ac8ec0eb0c5bf8cb2e1"},"cell_type":"markdown","source":"The tests have been performed on different datasets in terms of simplisity (the initial event datasets were specially simplified keeping 1 % , 2 % , 5 %, 35% , 65 %, 100 % of initial tracks). Each sub-datasets has 100K of train / test samples and each sub-dataset consumed approximatelly 250 GB of a disk space. The technic of these subset generation is presented in  Appendix 2 below.\n\nFive different network architectures have been tested on different level of the sub-datasets data with different architectures /  different hits intensity coding / different loss function formats.\n\nSome testing results are shown in the table below. The score rating format here doesn't identical to  score format used for final submission  since it was desinged specially for sub-datasets scoring. Thus, the persantages shown below does not directly correspont with final submission scoring.\n\n![](https://s7.wampi.ru/2018/08/11/table.jpg)\n"},{"metadata":{"_uuid":"d0b427ff4c1df3ff07cff0c06c15079dd4763cfc"},"cell_type":"markdown","source":"The results were less promissing as was expected in the beginning of the experiments. Significant overfitting was observed on each experiment instead of the fact that 100K samples were fed on each experiment and convolution /  dropout layers were implemented. All tested networks architectures were not able to sufficiently solve the problem using direct network end-to-end approach.\n\nHowever, it is nessessary to note that only a  little subset of all possible network architectures, all possible input/output data replesentations have been tested. "},{"metadata":{"_uuid":"a4978895617f663115318d425eb0210cc1309201"},"cell_type":"markdown","source":"First network architecture is a simplest architecture with one convolution and one dense layer.\n\n![](https://s7.wampi.ru/2018/08/11/arch1.jpg)\n"},{"metadata":{"_uuid":"3d70584e4bfe2203714eeb6617ffdfceeb1ab7cf"},"cell_type":"markdown","source":"The second network archtecture implements addition hidden dense  layer with 1116 neurons.\n![](https://s7.wampi.ru/2018/08/11/arch2.jpg)"},{"metadata":{"_uuid":"29dfdca5a37abe28699e52c8ba48e1cf9ebe8c8a"},"cell_type":"markdown","source":"The fird network arhitecture has addition dropout layer.\n\n![](https://s7.wampi.ru/2018/08/11/arch3.jpg)"},{"metadata":{"_uuid":"086889f10f709bdd0298ee70235fe51b7f3fe68a"},"cell_type":"markdown","source":"The fourth network architecture has 3 hidden convolution layers.\n\n![](https://s7.wampi.ru/2018/08/11/arch4.jpg)"},{"metadata":{"_uuid":"556afebaef4a3180af66c31d09cae842de7a0473"},"cell_type":"markdown","source":"The fifth network architecture has 2 input data channels where the second channel is a transposed channel one matrix.\n\n![](https://s7.wampi.ru/2018/08/11/arch5.jpg)"},{"metadata":{"_uuid":"6eecd375d879f44466239bceddd511c636a365b1"},"cell_type":"markdown","source":"**CONCLUSIONS**\n\nThe results show that the network end-to-end approach most probably is not an optimal approach for this problem resolution.  It is recomended to play with  combination of technics: Clustering -> Neural Networks -> Calman Filtering etc. in different combinations.\n\nHowever,  in order to combine all these technics in a workable way, a super-comuter power may be required for achitecture optimisation.  Anyway, the task continues  to be challenging regardless of the maximum score ratings being currently achieved by different teams."},{"metadata":{"_uuid":"3257ff67f2c16763032125c679e6b1b31593f88a"},"cell_type":"markdown","source":"# APPENDIX 1  - COMPUTATION RESOURSES"},{"metadata":{"_uuid":"64bb3df03851cd7db717ada5055e6501a167a942"},"cell_type":"markdown","source":"The problem is extremely challenging in respect of computation resoursed. Total computation time for the described tests was approximatelly 14 days !  Special computation cell was constructed to prevent fire risk during the computation, see the picture below.\n\n![](https://s7.wampi.ru/2018/08/11/CC.png)"},{"metadata":{"_uuid":"b0e9897fd3868edf7175b82eca83757728e0b0df"},"cell_type":"markdown","source":"# APPENDIX 2 - INPUT DATA PREPARATION EXAMPLE"},{"metadata":{"_uuid":"7df8a30bec39b6f52d3932d76afc8c155ea0fff3"},"cell_type":"markdown","source":"The code below shows how train data is transformed into sub-dataset in order to be able to be passed to the networks."},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"2127e5a8ceeeb6f901e06b2523145a2bc05cd2fa"},"cell_type":"code","source":"# Function for coordinate conversion\ndef cart2spherical(cart):\n    r = np.linalg.norm(cart, axis=0)\n    theta = np.degrees(np.arccos(cart[2] / r))\n    phi = np.degrees(np.arctan2(cart[1], cart[0]))\n    return np.vstack((r, theta, phi))\n\n#=============================================================================================\n\n\n# hits, truth, cells - dataframe formats as was specified in the input library\n# density 1..100 percents. The initial density can be reduced into smaller subset os hits  \n# wsize - subnet neural network window size in  spherical coordinats. Default - 128/128\n\ndef CreateSpace(hits, truth, cells, density=100, wsize=(255,255),space_size=(2048,2048)):\n\n    \n    print(\"start\")\n    rcells=cells.copy()\n    rcells.drop_duplicates(subset=\"hit_id\", keep='first', inplace=True)\n    glvals = []\n    for idx, hts in rcells.iterrows():\n        current_hit_id=hts[\"hit_id\"]\n        gbl =  cells[cells.hit_id==current_hit_id]\n        gml=gbl.where(gbl!=1,0.4)\n        gvals = gml[\"value\"]\n        glvals.append(gvals.sum())\n    hits['values']=glvals    \n    print('Space creation...')\n        \n        \n#    for idx, hts in rcells.iterrows():\n#        current_hit_id=hts[\"hit_id\"]\n#        gbl =  cells[cells.hit_id==current_hit_id]\n#        rgbl =  cells[(cells.hit_id==current_hit_id)&(cells.value!=1)]\n        #gml=gbl.where(gbl!=1,0.08)\n        #gml=gbl\n        #vals = gbl[\"value\"]\n        #rvals = rgbl[\"value\"]\n        #gvals = gml[\"value\"]\n        \n    \n    # making a work copy of hits\n    ehits=hits.copy()\n    print(ehits.head())\n    \n    # window size determination\n    wsize_x,wsize_y = wsize\n    # Evenness check\n    if wsize_x % 2 == 0: return # Window has to have odd size in order to have a explisit centre\n    if wsize_y % 2 == 0: return\n    \n    #----- Add spherical coordinates ------------\n    xyz = ehits.loc[:, ['x', 'y', 'z']].values.transpose()\n    rtp = cart2spherical(xyz).transpose()\n    rtp_df = pd.DataFrame(rtp, columns=('r', 'theta', 'phi'))\n    ehits['r']  = rtp_df['r']\n    ehits['theta']  =rtp_df['theta']\n    ehits['phi']  =rtp_df['phi']\n    ehits['particle_id']=truth['particle_id']\n    \n    #  ---- Preparing volume/layers dataframe ---- \n    volume_list=[]\n    layer_list=[]\n    total_layers=0\n    for volume in range (1,19):\n        for layer in range (1,15):\n            local_df = ehits[(ehits.volume_id==volume)&(ehits.layer_id==layer)]\n            # finding a row cound in the filtered dataframe\n            count = local_df[\"r\"].count()\n            if count!=0:\n                total_layers=total_layers+1;\n                volume_list.append(volume)\n                layer_list.append(layer)\n    \n    vl_df = pd.DataFrame() \n    vl_df['volumes'] = volume_list\n    vl_df['layers'] = layer_list\n    \n    #  ----- NumPy memory allocation ----------------------\n    space_x, space_y = space_size # actual space representation size x\n    space_z = total_layers # actual space representation size x\n    # Now we need to add some space for padding,\n    # later x direction will be populated with zero padding\n    # y direction will be populated with circuit padding\n    full_space_x = space_x + wsize_x # full array x size with padding in place \n    full_space_y = space_y + wsize_y # full array y size with padding in place \n    a = np.zeros(shape=(full_space_x,full_space_y,space_z), dtype=np.int8)\n    \n    # Special technological array for easier focus window manipulation\n    a_tech = np.zeros(shape=(space_x,space_y), dtype=np.int8)     \n    \n    # Particle ID array (in order to define particle_id based)\n    ap = np.zeros(shape=(full_space_x,full_space_y), dtype=np.int64)\n    \n    #  ---- Here we reduce the number of hits based on 'density'\n    # Filter required number of trajectories\n    trajectories = truth.drop_duplicates('particle_id')\n    full_size = len(trajectories)\n    trajectories = trajectories.drop_duplicates('particle_id')\n    if density!=100:\n        trajectories = trajectories.sample(int(density*(full_size/100.0)))\n    trajectories = trajectories['particle_id']\n    trajectories = trajectories.values\n    # make addition coloumn with \"in play: flag\n    in_play_list=[]\n    for _, hit in ehits.iterrows():\n        part_id=hit['particle_id']\n        if part_id in trajectories:\n            in_play_list.append(True)\n        else:\n            in_play_list.append(False)\n    ehits['in_play']=in_play_list        \n    \n    collision_count=0\n    #  ---- Here we create 3d space representation  depending also on \"in play\" field --------\n    for z in range (0, space_z):\n        v = vl_df.iloc[z]['volumes']\n        l = vl_df.iloc[z]['layers']\n        l_hits=ehits[(ehits.volume_id==v)&(ehits.layer_id==l)&(ehits.in_play==True)]\n        # ----- Convertion hits into space array format --------\n        for i, hit in l_hits.iterrows():\n            x_idx = int (((hit['phi']+180) /360.0) *space_x)\n            y_idx = int ((hit['theta'] /180.0) *space_y)\n            z_idx = z\n            a[int(wsize_x/2) +x_idx , int(wsize_y/2) + y_idx,z_idx]=255 # intensity in this version is always maximum\n            \n            a_tech[x_idx,y_idx] = 255 # Technological array\n            \n            #Collision detection\n            #if ap[int(wsize_x/2) +x_idx , int(wsize_y/2) + y_idx] !=0: \n            #    if ap[int(wsize_x/2) +x_idx , int(wsize_y/2) + y_idx] != hit['particle_id']:\n            #        collision_count=collision_count+1\n                \n            # Adding a technological array with partice ids information    \n            ap[int(wsize_x/2) +x_idx , int(wsize_y/2) + y_idx] = hit['particle_id']\n            \n            # Adding space coordinated into common ehits dataframe\n            ehits.loc[i,'x_idx']=x_idx\n            ehits.loc[i,'y_idx']=y_idx\n            ehits.loc[i,'z_idx']=z_idx\n            \n    #print(collision_count)\n    \n    return ehits, a, a_tech, ap       \n\n#=============================================================================================\n\n\n# fsize - focus window size\n# max_i - maximum number of iterations (attempts to find a pivot point)\ndef GeneratePivot(ehits, a_tech, fsize =(25,25), space_size=(2048,2048), max_i=1000):\n    space_x, space_y = space_size # actual space representation size x\n    #  ----------- Pivot point genaration ------------------------------------------\n    # Here we generate a pivot point is 2D coorditates\n    random.seed()\n    found = False\n    for i in range(0,max_i):\n        if found == True: break\n        x_pivot = random.randint(0,space_x-1)\n        y_pivot = random.randint(0,space_y-1)\n        # Now we need to extend one pivot point to a focus window, because a probablility of not targeting a point is relatevelly high\n        # Usually a fucus window is small (<=30), otherwise this could be computantially expensive\n        # From otrher side the focus window can be very small because this will dramatically increase number of iteration on\n        # outer computation layer\n        fsize_x, fsize_y =  fsize\n        fx_start = x_pivot - int(fsize_x / 2)\n        fy_start = y_pivot - int(fsize_y / 2)\n        if fx_start < 0: fx_start=0\n        if fy_start < 0: fy_start=0\n        fx_end = x_pivot + int(fsize_x / 2)\n        fy_end = y_pivot + int(fsize_y / 2)\n        if fx_end > (space_x-1) : fx_end = space_x-1\n        if fy_end > (space_y-1) : fy_end = space_y-1\n        # Now we start iterating over a focus window trying to find non-black point\n        # checking if x_pivot and y_pivot correspond to any particle\n        for xf in range (fx_start,fx_end+1):\n            if found == True: break\n            for yf in range (fy_start,fy_end+1):\n                if a_tech[xf,yf]!=0: \n                    x_pivot = xf\n                    y_pivot = yf\n                    found = True\n                    break\n                \n    return  x_pivot, y_pivot, found    \n\n\n#=============================================================================================\n\ndef GenerateInputInstance(x_pivot, y_pivot, ehits, a, wsize=(255,255)):\n    \n    # Window size determination\n    wsize_x, wsize_y = wsize \n    # Evenness check\n    if wsize_x % 2 == 0: return # Window has to have odd size in order to have a explisit centre\n    if wsize_y % 2 == 0: return\n    # Define window corners\n    wstart_x = x_pivot -  int(wsize_x/2)\n    wstart_y = y_pivot -  int(wsize_y/2)\n    wend_x = x_pivot +  int(wsize_x/2) # inclusevely\n    wend_y = y_pivot +  int(wsize_y/2) # inclusevely\n    \n    # Correcting to coordinates with padding\n    wstart_x =  wstart_x + int(wsize_x/2)\n    wstart_y = wstart_y +  int(wsize_y/2)\n    wend_x = wend_x +  int(wsize_x/2) # inclusevely\n    wend_y = wend_y +  int(wsize_y/2) # inclusevely\n    \n    #print(wstart_x,wstart_y,wend_x,wend_y)\n    \n    # Resulting array\n    a_res = a[wstart_x:wend_x+1,wstart_y:wend_y+1]\n    at_res = np.zeros(shape=wsize, dtype=np.int8) \n    \n    #print('Start')\n    \n    # For testing purposes, will need to be deleted after valudation due to computation reasons \n#    for il in range (0,48):\n        #print(\"ZLayer - \"+str(il))\n#        for ix in range (0,wsize_x):\n#            for iy in range (0,wsize_y):\n#                if a_res[ix,iy,il]!=0:\n#                    print(ix,iy,il)\n#                    at_res[ix,iy] = a_res[ix,iy,il]\n    #print('End')\n    \n    return a_res\n\n\n#=============================================================================================\n\n\n# The function retrives a particle ID assotiated with a pivot point\n\ndef GetParticleID(x_pivot, y_pivot, ap, wsize=(255,255)):\n\n    # Window size determination\n    wsize_x, wsize_y = wsize \n    \n    # Note \n    # Here some collision effect is possible when several particles belong to same\n    # pivot point in 2d space, in this case only one particle_id is return\n    # later more sophisticated collision avoidance algorithm might need to be implemented\n    # most probably a priority based on layer ID will need to be implemented in CreateSpace function\n    particle_id = ap[int(wsize_x/2)+x_pivot,int(wsize_y/2)+y_pivot]\n    return particle_id  \n\n\n#=============================================================================================\n\n\n# This function generated a true label based on provided pivot point and true particle traectory information\n\n# The true label has the following format (NUMPY ARRAY INT8):\n\n# [0... (wsize_x-1), 0...(wsize_y-1), 0...(48-1)]\n\n# Overall size is wsize_x+wsize_y + 48 bytes\n\ndef GenerateLabel(x_pivot,y_pivot,ehits,ap,wsize):\n\n    # Window size determination\n    wsize_x, wsize_y = wsize \n\n    # Evenness check\n    if wsize_x % 2 == 0: return # Window has to have odd size in order to have a explisit centre\n    if wsize_y % 2 == 0: return\n\n    # Define window corners\n    wstart_x = x_pivot -  int(wsize_x/2)\n    wstart_y = y_pivot -  int(wsize_y/2)\n    \n\n    # Cet a particle ID in a centre point\n    pid = GetParticleID(x_pivot,y_pivot,ap,wsize)\n    thits = ehits[(ehits.particle_id==pid)&(ehits.in_play==True)]\n    \n    \n    label_array = np.zeros(shape=(wsize_x+wsize_y + 48),dtype=np.int8)\n\n    # ----- Convertion hits into space array format --------\n    for _, hit in thits.iterrows():\n                x_idx = hit['x_idx']\n                y_idx = hit['y_idx']\n                z_idx = hit['z_idx']    \n\n                #print(x_idx,y_idx,z_idx)\n                \n                # conver to local window coordinated\n                lx_idx = int(x_idx - wstart_x)\n                ly_idx = int(y_idx - wstart_y)\n                lz_idx = int(z_idx)\n\n                #print(lx_idx,ly_idx,lz_idx)\n\n                #print(x_idx,wstart_x, lx_idx)\n                \n                #Convertion to lable\n                if((lx_idx>=0) & (lx_idx<wsize_x)):\n                    label_array[lx_idx] = 1\n                if((ly_idx>=0) & (ly_idx<wsize_y)):\n                    label_array[wsize_x + ly_idx] = 1\n                label_array[wsize_x+wsize_y+lz_idx] = 1\n \n    #print(label_array.size)\n    return label_array\n\n#=============================================================================================\n\ndef IfPointInlabel(x,y,z,label,wsize):\n\n    # Window size determination\n    wsize_x, wsize_y = wsize \n    \n    # Local labels\n    xlabel = label[0: wsize_x ]\n    ylabel = label[wsize_x : wsize_x + wsize_y]\n    zlabel = label[wsize_x + wsize_y:wsize_x + wsize_y+48]\n    \n    res = ((xlabel[x]==1)&(ylabel[y]==1)&(zlabel[z]==1))\n    \n    return res\n\n#=============================================================================================\n\ndef IfPointInlabel(x,y,z,label,wsize):\n\n    # Window size determination\n    wsize_x, wsize_y = wsize \n    \n    # Local labels\n    xlabel = label[0: wsize_x ]\n    ylabel = label[wsize_x : wsize_x + wsize_y]\n    zlabel = label[wsize_x + wsize_y:wsize_x + wsize_y+48]\n    \n    res = ((xlabel[x]==1)&(ylabel[y]==1)&(zlabel[z]==1))\n    \n    return res\n\n#=============================================================================================\n\n\ndef SaveLabeledInstance(start_number,number, ehits, a, a_tech, ap):\n    for i in range(start_number,start_number+number):\n        x_pivot,y_pivot, found = GeneratePivot(ehits, a_tech)\n        InputInstance = GenerateInputInstance(x_pivot,y_pivot,ehits,a)\n        label = GenerateLabel(x_pivot,y_pivot,ehits=ehits,ap=ap,wsize=(255,255))\n        #np.save('INPUT/255_255_1P_NIP_NI/'+'input'+str(i), InputInstance)\n        #np.save('INPUT/255_255_1P_NIP_NI/'+'label'+str(i), label)\n        \n        np.save('input'+str(i), InputInstance)\n        np.save('label'+str(i), label)\n    None","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"569487978ffddc5157f67fd3aadba79dc14bf4ac"},"cell_type":"markdown","source":"For demonstration purposes only 1K samples input data is gerarated. In actual experiments 100K samples were used to each test"},{"metadata":{"trusted":true,"_uuid":"db4ccc8756f2eed8f9c5694473aa61b9aebaf6b8","collapsed":true},"cell_type":"code","source":"event_id = 'event000001000'\nhits, cells, particles, truth = load_event('../input/train_1/'+event_id)\nehits, a, a_tech, ap = CreateSpace(hits=hits, truth=truth, cells=cells, density=1, wsize=(255,255),space_size=(2048,2048))\n\nSaveLabeledInstance(0,1000, ehits, a, a_tech, ap)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7121591aeb3886e8d788be58c266d5678ec58d36"},"cell_type":"markdown","source":"# APPENDIX 3 - NETWORK CODE EXAMPLE (ARCHITECTURE 5)"},{"metadata":{"_uuid":"71c346a80ff16d0906baecd0009c0af8a1c0718e","trusted":true},"cell_type":"code","source":"import keras\nfrom keras import backend as K\nfrom keras.layers import Input, Dense, Flatten, Conv2D\nfrom keras.models import Model\nimport numpy as np\nimport gc\n#from keras.utils import plot_modell\nimport random\nimport matplotlib.pyplot as plt\nimport time\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"09f3b45c04e1281b63e50c50d20590ffdef38781"},"cell_type":"code","source":"ZIPPED = False\n\nBATCHSIZE = 5\n\n\ndef GetNextBatch(dir = None, minfn=0, maxfn=10000):\n    arraylist=[]\n    labellist=[]\n    for i in range(0,BATCHSIZE):\n        fidx = np.random.randint(minfn,maxfn)\n        \n        if ZIPPED==False:\n            cadr = np.load(dir+'input'+str(fidx)+'.npy').copy()\n            label = np.load(dir+'label'+str(fidx)+'.npy').copy()\n        if ZIPPED==True:\n            loaded = np.load(dir+'data_label_'+str(fidx)+'.npz')\n            cadr = loaded['data'].copy()\n            label = loaded['label'].copy()\n        \n        arraylist.append(cadr)\n        array = np.stack(arraylist, axis=0)\n\n        labellist.append(label)\n        labels = np.stack(labellist, axis=0)\n\n    return array,labels","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b175216047785d8126e2236e6b1ded63bcc14628"},"cell_type":"markdown","source":"The code below runs on significantly reducded dataset (only 1 K), thus is shown only for the network architecture refference"},{"metadata":{"trusted":true,"_uuid":"2895a467fe54a12583c8de02197cd315c1bc8f23"},"cell_type":"code","source":"from keras.layers import Dropout\n\nK.clear_session()\ngc.collect()\n# -----------------------------------------------\ndef Acc(y_true, y_pred):\n    y_pred=K.round(y_pred)\n    res = K.mean(K.square(y_pred - y_true), axis=-1)\n    res = K.mean(res, axis=-1)\n    res=1-res\n    res=res*100\n    return res\n# -----------------------------------------------\ndef LossFunction(y_true, y_pred):\n    res = K.mean(K.square(y_pred - y_true), axis=-1)\n    res = K.mean(res, axis=-1)\n    return res\n\n# -------------- Model definition -------------------------------------------------------------------------------------------\ninputs1 = Input(shape=(255,255,48))\ninputs2 = Input(shape=(48,255,255))\n\n# ----------\n\nx1 = Conv2D(filters = 32, kernel_size = (6,6), strides =(3,3),\n                 activation ='relu')(inputs1)\nx1 = Flatten()(x1)\n\n# ----------\n\nx2 = Conv2D(filters = 32, kernel_size = (6,6), strides =(3,3),\n                 activation ='relu')(inputs2)\nx2 = Flatten()(x2)\n\n# ----------\n\nx = keras.layers.concatenate([x1, x2])\n\nx = Dense(558, activation ='relu')(x) \n\npredictions = Dense(558, activation='sigmoid')(x)\n\n\n#  ----------- Model compilation -------------------------------------------------------------------------------------------- \n\nmodel = Model(inputs=[inputs1,inputs2], outputs=predictions)\n\nmodel.compile(optimizer='rmsprop',\n              loss=LossFunction,\n              metrics=[Acc])\n\n# ------------- Model summary print ------------------------------------------------------------------------------------------\nmodel.summary()\n\ntick=time.time()\n\n\n# ------------------ Test batch generation -----------------------------------------------------------------------------------\n#test_batch_input,test_batch_labels  =  GetNextBatch(dir = 'D:/KAGGLE/CERN/INPUT/255_255_100P_IP_I/',minfn=90000, maxfn = 100000)\n#test_batch_input1, test_batch_labels  =  GetNextBatch(dir = 'INPUT/255_255_100P_IP_I/',minfn=90000, maxfn = 100000)\ntest_batch_input1, test_batch_labels  =  GetNextBatch(dir = '',minfn=900, maxfn = 1000)\ntest_batch_input2 = []\nfor i in range(0,BATCHSIZE): test_batch_input2.append(test_batch_input1[i].transpose())\ntest_batch_input2 = np.stack(test_batch_input2, axis=0)\n\n# -------- Model saving and logging parameters -------------------------------------------------------------------------------\nms_counter = 0\ntestname = 'Test46_Old'\ntest_acc_list = []\ntest_true_acc_list = []\ntrain_acc_list = []\n\n\n# -----------------  Main calculation cycle ----------------------------------------------------------------------------------\nfor i in range(0,10):\n    #---- Printing -------------------------\n    print('Step'+str(i))\n    # ---- Saving --------------------------\n    ms_counter = ms_counter + 1\n    if ms_counter>=50:\n        ms_counter = 0\n        model.save(testname +'_model_'+str(i)+'.mdl')\n    # ---- Train batch genaration ----------\n#    batch_input,batch_labels  =  GetNextBatch(dir = 'D:/KAGGLE/CERN/INPUT/255_255_100P_IP_I/', minfn=10000, maxfn = 90000)\n\n    #batch_input1, batch_labels  =  GetNextBatch(dir = 'INPUT/255_255_100P_IP_I/', minfn=10000, maxfn = 90000)\n    batch_input1, batch_labels  =  GetNextBatch(dir = '', minfn=0, maxfn = 900)\n    \n    batch_input2 = []\n    for i in range(0,BATCHSIZE): batch_input2.append(batch_input1[i].transpose())\n    batch_input2 = np.stack(batch_input2, axis=0)\n    \n    process = model.fit([batch_input1,batch_input2], batch_labels, epochs = 5, \n                        validation_data=([test_batch_input1,test_batch_input2],test_batch_labels))\n    #                  batch_size=8) \n    \n    acc = process.history[\"Acc\"]\n    val_acc = process.history[\"val_Acc\"]\n    plt.plot(acc)\n    plt.plot(val_acc)\n    plt.show()\n    \n    # - Validation histogram calculation -\n    mean_train = np.mean(acc) \n    mean_test = np.mean(val_acc) \n    print('Mean train acc - ', mean_train)\n    print('Mean test acc - ', mean_test)\n    train_acc_list.append(mean_train)\n    test_acc_list.append(mean_test)\n    test_true_acc_list.append(acc[0])\n    \n    plt.plot(train_acc_list)\n    plt.plot(test_acc_list)\n    plt.plot(test_true_acc_list)\n    \n    plt.show()\n    \n    plt.plot(test_acc_list)\n    plt.show()    \n    \n    tack = time.time()\n    \n    print(\"Time from the test start, h: \", (tack-tick)/3600)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0dc021a8b57ad94a40d5a382d76454ea3b2e2a87"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"8b554c130c356ec874ef18dd313e0666b64230e5"},"cell_type":"markdown","source":""}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.5","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}