Is there any way to create multi-weight logloss loss function for keras, I am trying to create one but I can't make it work. \nHas anyone able to use it as a loss function and if not what are you guys using?
", "messages": [ { "id": "411096", "postDate": "10/27/2018 12:07:04", "content": "Is there any way to create multi-weight logloss loss function for keras, I am trying to create one but I can't make it work. \nHas anyone able to use it as a loss function and if not what are you guys using?
", "rawMarkdown": "Is there any way to create multi-weight logloss loss function for keras, I am trying to create one but I can't make it work. \nHas anyone able to use it as a loss function and if not what are you guys using?", "votes": null }, { "id": "411103", "postDate": "10/27/2018 12:25:28", "content": "def mywloss(y_true,y_pred): \n yc=tf.clip_by_value(y_pred,1e-15,1-1e-15)\n loss=-(tf.reduce_mean(tf.reduce_mean(y_true*tf.log(yc),axis=0)/wtable))\n return loss\nEdit:\n wtable - is a numpy 1d array with (the number of times class y_true occur in the data set)/(size of data set)
", "rawMarkdown": "def mywloss(y_true,y_pred): \n yc=tf.clip_by_value(y_pred,1e-15,1-1e-15)\n loss=-(tf.reduce_mean(tf.reduce_mean(y_true*tf.log(yc),axis=0)/wtable))\n return loss\n\nEdit:\n wtable - is a numpy 1d array with (the number of times class y_true occur in the data set)/(size of data set)", "votes": null }, { "id": "411104", "postDate": "10/27/2018 12:28:41", "content": "Thanks! that is way shorter than what I was trying to do.
", "rawMarkdown": "Thanks! that is way shorter than what I was trying to do.", "votes": null }, { "id": "411118", "postDate": "10/27/2018 13:01:04", "content": "You divide by wtable, hence these are not what people usually call weights. They are rather their inverse.
You can also just use sample weights.
", "rawMarkdown": "You can also just use sample weights.", "votes": null }, { "id": "411132", "postDate": "10/27/2018 13:32:50", "content": "That's what I use indeed.
", "rawMarkdown": "That's what I use indeed.", "votes": null }, { "id": "411145", "postDate": "10/27/2018 13:46:55", "content": "You are correct. thanks,\nedited.
", "rawMarkdown": "You are correct. thanks,\nedited.", "votes": null }, { "id": "417672", "postDate": "11/08/2018 16:04:02", "content": ">(the number of times class y_true occur in the data set)/(size of data set)
\n\nwhy do you divide by the 'size of data set', isn't that just a constant and 'N_j' for class j should be enough?
", "rawMarkdown": ">(the number of times class y_true occur in the data set)/(size of data set)\n\nwhy do you divide by the 'size of data set', isn't that just a constant and 'N_j' for class j should be enough?", "votes": null }, { "id": "417805", "postDate": "11/08/2018 19:53:48", "content": "When you do the calculation as a score on a full set, you don't need it. you do need to replace the reducemean with reducesum and it will take care of this constant. But when you calculate on a partial batch you must use reducemean, otherwise the result will depend on the size of the batch, hance you need to introduce this constant.
", "rawMarkdown": "When you do the calculation as a score on a full set, you don't need it. you do need to replace the reducemean with reducesum and it will take care of this constant. But when you calculate on a partial batch you must use reducemean, otherwise the result will depend on the size of the batch, hance you need to introduce this constant.", "votes": null }, { "id": "417952", "postDate": "11/09/2018 03:42:53", "content": "Have you run into issues with clipping blocking gradients while training? I tried a few simple networks and found that I am better off without the clipping in training loss.
", "rawMarkdown": "Have you run into issues with clipping blocking gradients while training? I tried a few simple networks and found that I am better off without the clipping in training loss.", "votes": null }, { "id": "417989", "postDate": "11/09/2018 04:47:47", "content": "I didn't. The clipping is for extreme valus, I don't see why it should cause issues. Actually, in later version I even clipped to 1e-4, because I didn't want one extreme value to degrade the full batch.
", "rawMarkdown": "I didn't. The clipping is for extreme valus, I don't see why it should cause issues. Actually, in later version I even clipped to 1e-4, because I didn't want one extreme value to degrade the full batch.", "votes": null }, { "id": "418369", "postDate": "11/09/2018 18:39:34", "content": "and you can also just use class weights.
", "rawMarkdown": "and you can also just use class weights.", "votes": null } ], "comments": [ { "id": 411103, "author_name": "yuval6967", "author_url": "", "post_date": "10/27/2018 12:25:28", "content": "def mywloss(y_true,y_pred): \n yc=tf.clip_by_value(y_pred,1e-15,1-1e-15)\n loss=-(tf.reduce_mean(tf.reduce_mean(y_true*tf.log(yc),axis=0)/wtable))\n return loss\nEdit:\n wtable - is a numpy 1d array with (the number of times class y_true occur in the data set)/(size of data set)
", "votes": null, "replies": [ { "id": 411104, "author_name": "satian", "author_url": "", "post_date": "10/27/2018 12:28:41", "content": "Thanks! that is way shorter than what I was trying to do.
", "votes": null, "replies": [] }, { "id": 411118, "author_name": "cpmpml", "author_url": "", "post_date": "10/27/2018 13:01:04", "content": "You divide by wtable, hence these are not what people usually call weights. They are rather their inverse.
You are correct. thanks,\nedited.
", "votes": null, "replies": [] }, { "id": 417672, "author_name": "samshipengs", "author_url": "", "post_date": "11/08/2018 16:04:02", "content": ">(the number of times class y_true occur in the data set)/(size of data set)
\n\nwhy do you divide by the 'size of data set', isn't that just a constant and 'N_j' for class j should be enough?
", "votes": null, "replies": [] }, { "id": 417805, "author_name": "yuval6967", "author_url": "", "post_date": "11/08/2018 19:53:48", "content": "When you do the calculation as a score on a full set, you don't need it. you do need to replace the reducemean with reducesum and it will take care of this constant. But when you calculate on a partial batch you must use reducemean, otherwise the result will depend on the size of the batch, hance you need to introduce this constant.
", "votes": null, "replies": [] }, { "id": 417952, "author_name": "mithrillion", "author_url": "", "post_date": "11/09/2018 03:42:53", "content": "Have you run into issues with clipping blocking gradients while training? I tried a few simple networks and found that I am better off without the clipping in training loss.
", "votes": null, "replies": [] }, { "id": 417989, "author_name": "yuval6967", "author_url": "", "post_date": "11/09/2018 04:47:47", "content": "I didn't. The clipping is for extreme valus, I don't see why it should cause issues. Actually, in later version I even clipped to 1e-4, because I didn't want one extreme value to degrade the full batch.
", "votes": null, "replies": [] } ] }, { "id": 411122, "author_name": "aerdem4", "author_url": "", "post_date": "10/27/2018 13:20:58", "content": "You can also just use sample weights.
", "votes": null, "replies": [ { "id": 411132, "author_name": "cpmpml", "author_url": "", "post_date": "10/27/2018 13:32:50", "content": "That's what I use indeed.
", "votes": null, "replies": [] }, { "id": 418369, "author_name": "mamasinkgs", "author_url": "", "post_date": "11/09/2018 18:39:34", "content": "and you can also just use class weights.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "411096": "Is there any way to create multi-weight logloss loss function for keras, I am trying to create one but I can't make it work. \nHas anyone able to use it as a loss function and if not what are you guys using?", "411103": "def mywloss(y_true,y_pred): \n yc=tf.clip_by_value(y_pred,1e-15,1-1e-15)\n loss=-(tf.reduce_mean(tf.reduce_mean(y_true*tf.log(yc),axis=0)/wtable))\n return loss\n\nEdit:\n wtable - is a numpy 1d array with (the number of times class y_true occur in the data set)/(size of data set)", "411104": "Thanks! that is way shorter than what I was trying to do.", "411118": "You divide by `wtable,` hence these are not what people usually call weights. They are rather their inverse.", "411122": "You can also just use sample weights.", "411132": "That's what I use indeed.", "411145": "You are correct. thanks,\nedited.", "417672": ">(the number of times class y_true occur in the data set)/(size of data set)\n\nwhy do you divide by the 'size of data set', isn't that just a constant and 'N_j' for class j should be enough?", "417805": "When you do the calculation as a score on a full set, you don't need it. you do need to replace the reducemean with reducesum and it will take care of this constant. But when you calculate on a partial batch you must use reducemean, otherwise the result will depend on the size of the batch, hance you need to introduce this constant.", "417952": "Have you run into issues with clipping blocking gradients while training? I tried a few simple networks and found that I am better off without the clipping in training loss.", "417989": "I didn't. The clipping is for extreme valus, I don't see why it should cause issues. Actually, in later version I even clipped to 1e-4, because I didn't want one extreme value to degrade the full batch.", "418369": "and you can also just use class weights." }, "source": "meta" }