{ "id": 24595, "title": "A war of FFMs", "url": "/competitions/outbrain-click-prediction/discussion/24595", "author_name": "", "post_date": "2016-10-20T15:00:36.780Z", "votes": 31, "comment_count": 31, "views": 4651, "content": "

Following qianqian's great post, keras fm, I'll also respond to some private emails and public discussions about how to handle big data.

\n\n

For people who has only small machine like me, the key is the willingness to go low level and customized classifiers. As you all know, FFM is the nuclear weapon for CTR. Currently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time). My ffm is largely based on Libffm which is a wonderful library. I managed to improve its out of core performance and add some tricks to improve both apk and speed.

\n\n

To me, It's all about scalability when we keep throwing more features to it. Solving bottlenecks one after another, It is the most fun thing I like to do in Kaggle.

", "messages": [ { "id": "140454", "postDate": "10/20/2016 15:00:36", "content": "

Following qianqian's great post, keras fm, I'll also respond to some private emails and public discussions about how to handle big data.

\n\n

For people who has only small machine like me, the key is the willingness to go low level and customized classifiers. As you all know, FFM is the nuclear weapon for CTR. Currently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time). My ffm is largely based on Libffm which is a wonderful library. I managed to improve its out of core performance and add some tricks to improve both apk and speed.

\n\n

To me, It's all about scalability when we keep throwing more features to it. Solving bottlenecks one after another, It is the most fun thing I like to do in Kaggle.

", "rawMarkdown": "Following qianqian's great post, [keras fm][1], I'll also respond to some private emails and public discussions about how to handle big data.\r\n\r\nFor people who has only small machine like me, the key is the willingness to go low level and customized classifiers. As you all know, FFM is the nuclear weapon for CTR. Currently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time). My ffm is largely based on Libffm which is a wonderful library. I managed to improve its out of core performance and add some tricks to improve both apk and speed. \r\n\r\nTo me, It's all about scalability when we keep throwing more features to it. Solving bottlenecks one after another, It is the most fun thing I like to do in Kaggle. \r\n\r\n\r\n [1]: https://www.kaggle.com/qqgeogor/outbrain-click-prediction/keras-based-fm", "votes": null }, { "id": "140458", "postDate": "10/20/2016 15:24:52", "content": "

thanks very much for your kindly sharing. I'm wondering how to solve FFM, like sgd? I think it is not easy to implemrnent the suitable solve? Thanks.

", "rawMarkdown": "thanks very much for your kindly sharing. I'm wondering how to solve FFM, like sgd? I think it is not easy to implemrnent the suitable solve? Thanks.", "votes": null }, { "id": "140463", "postDate": "10/20/2016 16:10:40", "content": "

@frankhu, it turned out adagrad is the best. I also tried sgd, momentum and remsprop, couldn't beat adagrad.

\n\n

Edit: adagrad is the default and only solver of libffm

", "rawMarkdown": "frankhu, it turned out adagrad is the best. I also tried sgd, momentum and remsprop, couldn't beat adagrad. \r\n\r\nEdit: adagrad is the default and only solver of libffm", "votes": null }, { "id": "140828", "postDate": "10/23/2016 14:57:21", "content": "

thanks very much for your kindly reply. It seems resonable.

", "rawMarkdown": "thanks very much for your kindly reply. It seems resonable.", "votes": null }, { "id": "142490", "postDate": "11/02/2016 15:39:08", "content": "

@rcarson hi, I'd like to ask for advice, is it reliable to use libfffm's '--on-disk' option for the huge data size this competition? or need some additional optimization by myself

", "rawMarkdown": "rcarson hi, I'd like to ask for advice, is it reliable to use libfffm's '--on-disk' option for the huge data size this competition? or need some additional optimization by myself", "votes": null }, { "id": "142651", "postDate": "11/03/2016 17:47:06", "content": "

@frankhu, I didn't try the default '--on-disk' option of ffm. Based on the code, you could expect same performance as in-core version but with a noticeable slowdown. Basically every time it reads a batch of data from the raw data to memory. Reading and parsing raw data file every time is slow. A better way is that you parse the whole raw data once in a batch by batch way and save the parsed batches in binaries, then it is much faster to reload each batch, which is what i'm doing actually.

", "rawMarkdown": "frankhu, I didn't try the default '--on-disk' option of ffm. Based on the code, you could expect same performance as in-core version but with a noticeable slowdown. Basically every time it reads a batch of data from the raw data to memory. Reading and parsing raw data file every time is slow. A better way is that you parse the whole raw data once in a batch by batch way and save the parsed batches in binaries, then it is much faster to reload each batch, which is what i'm doing actually.", "votes": null }, { "id": "142790", "postDate": "11/04/2016 15:27:37", "content": "

@rcarson thanks very much for your kindly reply, your idea is good and interesting :-)

", "rawMarkdown": "rcarson thanks very much for your kindly reply, your idea is good and interesting :-)", "votes": null }, { "id": "143649", "postDate": "11/09/2016 23:31:32", "content": "

I'm trying to use libffm and I have this stupid problem: ffm-predict input_path output_path produces a file with predictions - one per line. But the resulting file has slightly less lines then there are records in the input file. I suspect it's skipping records with field values that were not encountered in the training. The problem is, I have no way of joining the result back with the test set because there is no index column - there is only a prediction in each line.

\n\n

Is anyone else having this problem? Has anyone solved it? Is there a flag to tell ffm to default to predicting -1 or something instead of skipping the record?

\n\n

I know I can always find and remove the offending records before ffm-predict but this will bu super messy and tedious as it depends on my cross-val split and on feature set.

", "rawMarkdown": "I'm trying to use libffm and I have this stupid problem: `ffm-predict input_path output_path` produces a file with predictions - one per line. But the resulting file has slightly less lines then there are records in the input file. I suspect it's skipping records with field values that were not encountered in the training. The problem is, I have no way of joining the result back with the test set because there is no index column - there is only a prediction in each line. \r\n\r\nIs anyone else having this problem? Has anyone solved it? Is there a flag to tell ffm to default to predicting -1 or something instead of skipping the record?\r\n\r\nI know I can always find and remove the offending records before ffm-predict but this will bu super messy and tedious as it depends on my cross-val split and on feature set.", "votes": null }, { "id": "143805", "postDate": "11/10/2016 19:08:30", "content": "

I think ffm-predict doesn't skip rows due to unseen fields. For unseen fields, it should output 0.5, which you can try a 3-line dummy example. It is more likely that the input file is not correctly formatted.

", "rawMarkdown": "I think ffm-predict doesn't skip rows due to unseen fields. For unseen fields, it should output 0.5, which you can try a 3-line dummy example. It is more likely that the input file is not correctly formatted.", "votes": null }, { "id": "143922", "postDate": "11/11/2016 12:22:07", "content": "

Thank you very much for keras FM code!

\n\n

Can anyone explain why the dimensionality (shape) of "predict" function output depends on batch_size?\nSpecifically:

\n\n
\n

predictions = myNet.predict(X, batch_size=4)

\n \n

predictions.shape

\n
\n\n

(3072,)

\n\n
\n

predictions = myNet.predict(X, batch_size=3)

\n \n

predictions.shape

\n
\n\n

(2304,)

\n\n
\n

X.shape

\n
\n\n

(3, 768)

\n\n

Here is all my code:

\n\n

dataset = np.loadtxt("/myfolder/pima-indians-diabetes.data", delimiter=",")

\n\n

X = dataset[:,0:3]

\n\n

Y = dataset[:,8]

\n\n

X = np.transpose(X)

\n\n

Y = np.transpose(Y)

\n\n

myNet = KerasFM([ 1+round(max(X[0])), 1+round(max(X [ 1 ] )), 1+round(max(X[2])) ])

\n\n

predictions = myNet.predict(X, batch_size=4)

\n\n

predictions.shape

", "rawMarkdown": "Thank you very much for [keras FM code][1]!\r\n\r\nCan anyone explain why the dimensionality (shape) of \"predict\" function output depends on batch_size?\r\nSpecifically:\r\n> predictions = myNet.predict(X, batch_size=4)\r\n\r\n> predictions.shape\r\n\r\n(3072,)\r\n\r\n> predictions = myNet.predict(X, batch_size=3)\r\n\r\n> predictions.shape\r\n\r\n(2304,)\r\n\r\n> X.shape\r\n\r\n(3, 768)\r\n\r\nHere is all my code:\r\n\r\ndataset = np.loadtxt(\"/myfolder/pima-indians-diabetes.data\", delimiter=\",\")\r\n\r\nX = dataset[:,0:3]\r\n\r\nY = dataset[:,8]\r\n\r\nX = np.transpose(X)\r\n\r\nY = np.transpose(Y)\r\n\r\nmyNet = KerasFM([ 1+round(max(X[0])), 1+round(max(X [ 1 ] )), 1+round(max(X[2])) ])\r\n\r\npredictions = myNet.predict(X, batch_size=4)\r\n\r\npredictions.shape\r\n\r\n\r\n [1]: https://www.kaggle.com/qqgeogor/outbrain-click-prediction/keras-based-fm", "votes": null }, { "id": "143958", "postDate": "11/11/2016 15:17:59", "content": "

@Shakirov, keras ffm code is created by qianqian, you can ask him in that thread :)

", "rawMarkdown": "Shakirov, keras ffm code is created by qianqian, you can ask him in that thread :)", "votes": null }, { "id": "148968", "postDate": "12/07/2016 13:09:19", "content": "

I had a very basic/stupid question. In libfm, Rendle provides a script to convert \"user,item,rating\" kind of file to a format which libfm can directly ingest. Is there something similar to libffm?

\n\n

Thank you and Regards,\nSumit

", "rawMarkdown": "I had a very basic/stupid question. In libfm, Rendle provides a script to convert \"user,item,rating\" kind of file to a format which libfm can directly ingest. Is there something similar to libffm?\r\n\r\nThank you and Regards,\r\nSumit", "votes": null }, { "id": "153278", "postDate": "12/30/2016 18:57:57", "content": "

[quote=rcarson;140454]

\n\n

Currently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time).

\n\n

[/quote]

\n\n

This is awesome. This is my first time using FFM and it has been a little rough. I am only using 10% of the training data with 15 features and the FFM code is consuming all of my 16 GB memory allocation. If you don't mind me asking are you using all of the training data or a subset?

", "rawMarkdown": "[quote=rcarson;140454]\r\n\r\nCurrently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time).\r\n\r\n\r\n [1]: https://www.kaggle.com/qqgeogor/outbrain-click-prediction/keras-based-fm\r\n\r\n[/quote]\r\n\r\nThis is awesome. This is my first time using FFM and it has been a little rough. I am only using 10% of the training data with 15 features and the FFM code is consuming all of my 16 GB memory allocation. If you don't mind me asking are you using all of the training data or a subset?", "votes": null }, { "id": "153442", "postDate": "01/01/2017 13:43:40", "content": "

@RDizzl3, thank you for the kind reply. We can get 0.6937 lb now with 7 GB memory usage now. Trained with all training data. We will release the code after the contest finish.

", "rawMarkdown": "RDizzl3, thank you for the kind reply. We can get 0.6937 lb now with 7 GB memory usage now. Trained with all training data. We will release the code after the contest finish.", "votes": null }, { "id": "153446", "postDate": "01/01/2017 15:18:38", "content": "

Great, @rcarson! =)

\n\n

0.6937 lb = 1 model?

", "rawMarkdown": "Great, @rcarson! =)\r\n\r\n0.6937 lb = 1 model?", "votes": null }, { "id": "153462", "postDate": "01/01/2017 16:57:12", "content": "

@Eric, yes, one ffm model.

", "rawMarkdown": "Eric, yes, one ffm model.", "votes": null }, { "id": "153610", "postDate": "01/02/2017 14:29:28", "content": "

@rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?

", "rawMarkdown": "rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?", "votes": null }, { "id": "153774", "postDate": "01/03/2017 10:08:13", "content": "

@rcarson: how do you handle (encode) numerical features into you FFM ?

", "rawMarkdown": "rcarson: how do you handle (encode) numerical features into you FFM ?", "votes": null }, { "id": "153830", "postDate": "01/03/2017 16:28:27", "content": "

Is there any nice implementation of FFM in python?

", "rawMarkdown": "Is there any nice implementation of FFM in python?", "votes": null }, { "id": "154444", "postDate": "01/06/2017 08:00:31", "content": "

@rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong.

", "rawMarkdown": "rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong.", "votes": null }, { "id": "154446", "postDate": "01/06/2017 08:06:29", "content": "

@Sameh Faidi: Try this: Python Libffm

", "rawMarkdown": "Sameh Faidi: Try this: [Python Libffm][1]\r\n\r\n [1]: http://github.com/turi-code/python-libffm", "votes": null }, { "id": "154452", "postDate": "01/06/2017 09:00:39", "content": "

dont google ffm ...

", "rawMarkdown": "dont google ffm ...", "votes": null }, { "id": "154455", "postDate": "01/06/2017 09:18:57", "content": "

i am getting this error when i make on ubuntu

\n\n

make

\n\n
\n

g++ -O3 -std=c++11 -I ../sdk -shared -fPIC -march=native -fopenmp -DUSEOMP -c -o lib/ffm.o lib/ffm.cpp\n In file included from lib/ffm.cpp:10:0:\n /usr/lib/gcc/x86_64-linux-gnu/4.8/include/pmmintrin.h:31:3: error: #error \"SSE3 instruction set not enabled\"\n # error \"SSE3 instruction set not enabled\"\n ^\n lib/ffm.cpp: In function âffm::ffm_float ffm::{anonymous}::wTx(ffm::ffm_node*, ffm::ffm_node*, ffm::ffm_float, ffm::ffm_model&, ffm::ffm_float, ffm::ffm_float, ffm::ffm_float, bool)â:\n lib/ffm.cpp:138:34: error: â_mm_hadd_psâ was not declared in this scope\n XMMt = _mm_hadd_ps(XMMt, XMMt);\n ^\n make: * [lib/ffm.o] Error 1

\n
", "rawMarkdown": "i am getting this error when i make on ubuntu\r\n\r\n\r\n make\r\n\r\n\r\n\r\n> g++ -O3 -std=c++11 -I ../sdk -shared -fPIC -march=native -fopenmp -DUSEOMP -c -o lib/ffm.o lib/ffm.cpp\r\nIn file included from lib/ffm.cpp:10:0:\r\n/usr/lib/gcc/x86_64-linux-gnu/4.8/include/pmmintrin.h:31:3: error: #error \"SSE3 instruction set not enabled\"\r\n # error \"SSE3 instruction set not enabled\"\r\n ^\r\nlib/ffm.cpp: In function âffm::ffm_float ffm::{anonymous}::wTx(ffm::ffm_node*, ffm::ffm_node*, ffm::ffm_float, ffm::ffm_model&, ffm::ffm_float, ffm::ffm_float, ffm::ffm_float, bool)â:\r\nlib/ffm.cpp:138:34: error: â_mm_hadd_psâ was not declared in this scope\r\n XMMt = _mm_hadd_ps(XMMt, XMMt);\r\n ^\r\nmake: *** [lib/ffm.o] Error 1", "votes": null }, { "id": "154524", "postDate": "01/06/2017 17:14:54", "content": "

@Ashish Kulkarni I tried that one, it doesn't work.\n@Sameh Faidi Did you find the nice implementation of FFM in python?

", "rawMarkdown": "Ashish Kulkarni I tried that one, it doesn't work.\r\n@Sameh Faidi Did you find the nice implementation of FFM in python?", "votes": null }, { "id": "154556", "postDate": "01/06/2017 19:17:46", "content": "

Done some predictions using libffm and found that all the predictions on test data is in range 0.8.. but local validation scores are reasonable. Has somebody faced the same issue?\nOr I am making some mistake.. Help on sample data and its format will be highly appreciated

\n\n

Edit: Python has function gives different result on different sessions.\nhttp://stackoverflow.com/questions/27522626/hash-function-in-python-3-3-returns-different-results-between-sessions

\n\n

I have saved intermediate data in files and for different runs I got different hash. This also explains consistent 0.8 prediction for test data

\n\n

I hope this will help others and save their valuable time.

", "rawMarkdown": "Done some predictions using libffm and found that all the predictions on test data is in range 0.8.. but local validation scores are reasonable. Has somebody faced the same issue?\r\nOr I am making some mistake.. Help on sample data and its format will be highly appreciated\r\n\r\nEdit: Python has function gives different result on different sessions.\r\nhttp://stackoverflow.com/questions/27522626/hash-function-in-python-3-3-returns-different-results-between-sessions\r\n\r\nI have saved intermediate data in files and for different runs I got different hash. This also explains consistent 0.8 prediction for test data\r\n\r\nI hope this will help others and save their valuable time.", "votes": null }, { "id": "154578", "postDate": "01/06/2017 22:41:46", "content": "

[quote=Keerath Jaggi;153610]

\n\n

@rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?

\n\n

[/quote]

\n\n

I didn't include linear terms. Pairwise interaction only just like default ffm.

", "rawMarkdown": "[quote=Keerath Jaggi;153610]\r\n\r\n@rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?\r\n\r\n[/quote]\r\n\r\nI didn't include linear terms. Pairwise interaction only just like default ffm.", "votes": null }, { "id": "154579", "postDate": "01/06/2017 22:43:49", "content": "

[quote=Sameh Faidi;153774]

\n\n

@rcarson: how do you handle (encode) numerical features into you FFM ?

\n\n

[/quote]

\n\n

My ffm is particularly bad with numerical features comparing to default ffm. Still try to figure out why. But you can directly give numerical feature in \"value\" part of \"field:feature:value\" as specified in ffm's example.

", "rawMarkdown": "[quote=Sameh Faidi;153774]\r\n\r\n@rcarson: how do you handle (encode) numerical features into you FFM ?\r\n\r\n[/quote]\r\n\r\nMy ffm is particularly bad with numerical features comparing to default ffm. Still try to figure out why. But you can directly give numerical feature in \"value\" part of \"field:feature:value\" as specified in ffm's example.", "votes": null }, { "id": "154580", "postDate": "01/06/2017 22:46:49", "content": "

[quote=Ashish Kulkarni;154444]

\n\n

@rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong.

\n\n

[/quote]

\n\n

your machine is good. Please monitor your memory usage. if you are out of memory, turn off auto-stop.\nAlso start with a small k=4 to make sure everything goes through.

", "rawMarkdown": "[quote=Ashish Kulkarni;154444]\r\n\r\n@rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong. \r\n\r\n[/quote]\r\n\r\nyour machine is good. Please monitor your memory usage. if you are out of memory, turn off auto-stop.\r\nAlso start with a small k=4 to make sure everything goes through.", "votes": null }, { "id": "155363", "postDate": "01/10/2017 20:44:06", "content": "

Can FFM models make use of the unsupervised data in page_views or based solely on the supervised data in the clicks_train file?

", "rawMarkdown": "Can FFM models make use of the unsupervised data in page_views or based solely on the supervised data in the clicks_train file?", "votes": null }, { "id": "156308", "postDate": "01/15/2017 16:48:07", "content": "

after how many iteration your ffm model start to overfit? mine start overfitting after 3-4 iterations. any way to improve it?\nThanks!

", "rawMarkdown": "after how many iteration your ffm model start to overfit? mine start overfitting after 3-4 iterations. any way to improve it?\r\nThanks!", "votes": null }, { "id": "868778", "postDate": "05/31/2020 13:41:26", "content": "

but the big problem I met in use is how to transform the origin CSV data into FFM format data\ncould you pls help me?\nthx

", "rawMarkdown": "but the big problem I met in use is how to transform the origin CSV data into FFM format data\ncould you pls help me?\nthx", "votes": null }, { "id": "3083515", "postDate": "12/29/2024 15:31:17", "content": "

Outbrain click prediction helps optimize content recommendations for better engagement. At reelit com, we used similar tools to drive targeted traffic and improve campaign results!

", "rawMarkdown": "Outbrain click prediction helps optimize content recommendations for better engagement. At [reelit com](https://reelit.xyz/), we used similar tools to drive targeted traffic and improve campaign results!", "votes": null } ], "comments": [ { "id": 153278, "author_name": "rdizzl3", "author_url": "", "post_date": "12/30/2016 18:57:57", "content": "

[quote=rcarson;140454]

\n\n

Currently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time).

\n\n

[/quote]

\n\n

This is awesome. This is my first time using FFM and it has been a little rough. I am only using 10% of the training data with 15 features and the FFM code is consuming all of my 16 GB memory allocation. If you don't mind me asking are you using all of the training data or a subset?

", "votes": null, "replies": [] }, { "id": 3083515, "author_name": "harrywillson", "author_url": "", "post_date": "12/29/2024 15:31:17", "content": "

Outbrain click prediction helps optimize content recommendations for better engagement. At reelit com, we used similar tools to drive targeted traffic and improve campaign results!

", "votes": null, "replies": [] }, { "id": 140458, "author_name": "frank2236", "author_url": "", "post_date": "10/20/2016 15:24:52", "content": "

thanks very much for your kindly sharing. I'm wondering how to solve FFM, like sgd? I think it is not easy to implemrnent the suitable solve? Thanks.

", "votes": null, "replies": [] }, { "id": 140463, "author_name": "jiweiliu", "author_url": "", "post_date": "10/20/2016 16:10:40", "content": "

@frankhu, it turned out adagrad is the best. I also tried sgd, momentum and remsprop, couldn't beat adagrad.

\n\n

Edit: adagrad is the default and only solver of libffm

", "votes": null, "replies": [] }, { "id": 140828, "author_name": "frank2236", "author_url": "", "post_date": "10/23/2016 14:57:21", "content": "

thanks very much for your kindly reply. It seems resonable.

", "votes": null, "replies": [] }, { "id": 142490, "author_name": "frank2236", "author_url": "", "post_date": "11/02/2016 15:39:08", "content": "

@rcarson hi, I'd like to ask for advice, is it reliable to use libfffm's '--on-disk' option for the huge data size this competition? or need some additional optimization by myself

", "votes": null, "replies": [] }, { "id": 142651, "author_name": "jiweiliu", "author_url": "", "post_date": "11/03/2016 17:47:06", "content": "

@frankhu, I didn't try the default '--on-disk' option of ffm. Based on the code, you could expect same performance as in-core version but with a noticeable slowdown. Basically every time it reads a batch of data from the raw data to memory. Reading and parsing raw data file every time is slow. A better way is that you parse the whole raw data once in a batch by batch way and save the parsed batches in binaries, then it is much faster to reload each batch, which is what i'm doing actually.

", "votes": null, "replies": [] }, { "id": 142790, "author_name": "frank2236", "author_url": "", "post_date": "11/04/2016 15:27:37", "content": "

@rcarson thanks very much for your kindly reply, your idea is good and interesting :-)

", "votes": null, "replies": [] }, { "id": 143649, "author_name": "nadbor", "author_url": "", "post_date": "11/09/2016 23:31:32", "content": "

I'm trying to use libffm and I have this stupid problem: ffm-predict input_path output_path produces a file with predictions - one per line. But the resulting file has slightly less lines then there are records in the input file. I suspect it's skipping records with field values that were not encountered in the training. The problem is, I have no way of joining the result back with the test set because there is no index column - there is only a prediction in each line.

\n\n

Is anyone else having this problem? Has anyone solved it? Is there a flag to tell ffm to default to predicting -1 or something instead of skipping the record?

\n\n

I know I can always find and remove the offending records before ffm-predict but this will bu super messy and tedious as it depends on my cross-val split and on feature set.

", "votes": null, "replies": [] }, { "id": 143805, "author_name": "jiweiliu", "author_url": "", "post_date": "11/10/2016 19:08:30", "content": "

I think ffm-predict doesn't skip rows due to unseen fields. For unseen fields, it should output 0.5, which you can try a 3-line dummy example. It is more likely that the input file is not correctly formatted.

", "votes": null, "replies": [] }, { "id": 143922, "author_name": "shakirov", "author_url": "", "post_date": "11/11/2016 12:22:07", "content": "

Thank you very much for keras FM code!

\n\n

Can anyone explain why the dimensionality (shape) of "predict" function output depends on batch_size?\nSpecifically:

\n\n
\n

predictions = myNet.predict(X, batch_size=4)

\n \n

predictions.shape

\n
\n\n

(3072,)

\n\n
\n

predictions = myNet.predict(X, batch_size=3)

\n \n

predictions.shape

\n
\n\n

(2304,)

\n\n
\n

X.shape

\n
\n\n

(3, 768)

\n\n

Here is all my code:

\n\n

dataset = np.loadtxt("/myfolder/pima-indians-diabetes.data", delimiter=",")

\n\n

X = dataset[:,0:3]

\n\n

Y = dataset[:,8]

\n\n

X = np.transpose(X)

\n\n

Y = np.transpose(Y)

\n\n

myNet = KerasFM([ 1+round(max(X[0])), 1+round(max(X [ 1 ] )), 1+round(max(X[2])) ])

\n\n

predictions = myNet.predict(X, batch_size=4)

\n\n

predictions.shape

", "votes": null, "replies": [] }, { "id": 143958, "author_name": "jiweiliu", "author_url": "", "post_date": "11/11/2016 15:17:59", "content": "

@Shakirov, keras ffm code is created by qianqian, you can ask him in that thread :)

", "votes": null, "replies": [] }, { "id": 148968, "author_name": "sumitsidana", "author_url": "", "post_date": "12/07/2016 13:09:19", "content": "

I had a very basic/stupid question. In libfm, Rendle provides a script to convert \"user,item,rating\" kind of file to a format which libfm can directly ingest. Is there something similar to libffm?

\n\n

Thank you and Regards,\nSumit

", "votes": null, "replies": [] }, { "id": 153442, "author_name": "jiweiliu", "author_url": "", "post_date": "01/01/2017 13:43:40", "content": "

@RDizzl3, thank you for the kind reply. We can get 0.6937 lb now with 7 GB memory usage now. Trained with all training data. We will release the code after the contest finish.

", "votes": null, "replies": [] }, { "id": 153446, "author_name": "ericcouto", "author_url": "", "post_date": "01/01/2017 15:18:38", "content": "

Great, @rcarson! =)

\n\n

0.6937 lb = 1 model?

", "votes": null, "replies": [] }, { "id": 153462, "author_name": "jiweiliu", "author_url": "", "post_date": "01/01/2017 16:57:12", "content": "

@Eric, yes, one ffm model.

", "votes": null, "replies": [] }, { "id": 153610, "author_name": "keerath", "author_url": "", "post_date": "01/02/2017 14:29:28", "content": "

@rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?

", "votes": null, "replies": [ { "id": 154578, "author_name": "jiweiliu", "author_url": "", "post_date": "01/06/2017 22:41:46", "content": "

[quote=Keerath Jaggi;153610]

\n\n

@rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?

\n\n

[/quote]

\n\n

I didn't include linear terms. Pairwise interaction only just like default ffm.

", "votes": null, "replies": [] } ] }, { "id": 153774, "author_name": "samehif", "author_url": "", "post_date": "01/03/2017 10:08:13", "content": "

@rcarson: how do you handle (encode) numerical features into you FFM ?

", "votes": null, "replies": [ { "id": 154579, "author_name": "jiweiliu", "author_url": "", "post_date": "01/06/2017 22:43:49", "content": "

[quote=Sameh Faidi;153774]

\n\n

@rcarson: how do you handle (encode) numerical features into you FFM ?

\n\n

[/quote]

\n\n

My ffm is particularly bad with numerical features comparing to default ffm. Still try to figure out why. But you can directly give numerical feature in \"value\" part of \"field:feature:value\" as specified in ffm's example.

", "votes": null, "replies": [] } ] }, { "id": 153830, "author_name": "samehif", "author_url": "", "post_date": "01/03/2017 16:28:27", "content": "

Is there any nice implementation of FFM in python?

", "votes": null, "replies": [] }, { "id": 154444, "author_name": "ashishcool", "author_url": "", "post_date": "01/06/2017 08:00:31", "content": "

@rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong.

", "votes": null, "replies": [ { "id": 154580, "author_name": "jiweiliu", "author_url": "", "post_date": "01/06/2017 22:46:49", "content": "

[quote=Ashish Kulkarni;154444]

\n\n

@rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong.

\n\n

[/quote]

\n\n

your machine is good. Please monitor your memory usage. if you are out of memory, turn off auto-stop.\nAlso start with a small k=4 to make sure everything goes through.

", "votes": null, "replies": [] } ] }, { "id": 154446, "author_name": "ashishcool", "author_url": "", "post_date": "01/06/2017 08:06:29", "content": "

@Sameh Faidi: Try this: Python Libffm

", "votes": null, "replies": [] }, { "id": 154452, "author_name": "vigneshm", "author_url": "", "post_date": "01/06/2017 09:00:39", "content": "

dont google ffm ...

", "votes": null, "replies": [] }, { "id": 154455, "author_name": "samehif", "author_url": "", "post_date": "01/06/2017 09:18:57", "content": "

i am getting this error when i make on ubuntu

\n\n

make

\n\n
\n

g++ -O3 -std=c++11 -I ../sdk -shared -fPIC -march=native -fopenmp -DUSEOMP -c -o lib/ffm.o lib/ffm.cpp\n In file included from lib/ffm.cpp:10:0:\n /usr/lib/gcc/x86_64-linux-gnu/4.8/include/pmmintrin.h:31:3: error: #error \"SSE3 instruction set not enabled\"\n # error \"SSE3 instruction set not enabled\"\n ^\n lib/ffm.cpp: In function âffm::ffm_float ffm::{anonymous}::wTx(ffm::ffm_node*, ffm::ffm_node*, ffm::ffm_float, ffm::ffm_model&, ffm::ffm_float, ffm::ffm_float, ffm::ffm_float, bool)â:\n lib/ffm.cpp:138:34: error: â_mm_hadd_psâ was not declared in this scope\n XMMt = _mm_hadd_ps(XMMt, XMMt);\n ^\n make: * [lib/ffm.o] Error 1

\n
", "votes": null, "replies": [] }, { "id": 154524, "author_name": "beedata", "author_url": "", "post_date": "01/06/2017 17:14:54", "content": "

@Ashish Kulkarni I tried that one, it doesn't work.\n@Sameh Faidi Did you find the nice implementation of FFM in python?

", "votes": null, "replies": [] }, { "id": 154556, "author_name": "adityakumarsinha", "author_url": "", "post_date": "01/06/2017 19:17:46", "content": "

Done some predictions using libffm and found that all the predictions on test data is in range 0.8.. but local validation scores are reasonable. Has somebody faced the same issue?\nOr I am making some mistake.. Help on sample data and its format will be highly appreciated

\n\n

Edit: Python has function gives different result on different sessions.\nhttp://stackoverflow.com/questions/27522626/hash-function-in-python-3-3-returns-different-results-between-sessions

\n\n

I have saved intermediate data in files and for different runs I got different hash. This also explains consistent 0.8 prediction for test data

\n\n

I hope this will help others and save their valuable time.

", "votes": null, "replies": [] }, { "id": 155363, "author_name": "mpearce", "author_url": "", "post_date": "01/10/2017 20:44:06", "content": "

Can FFM models make use of the unsupervised data in page_views or based solely on the supervised data in the clicks_train file?

", "votes": null, "replies": [] }, { "id": 156308, "author_name": "srotemal", "author_url": "", "post_date": "01/15/2017 16:48:07", "content": "

after how many iteration your ffm model start to overfit? mine start overfitting after 3-4 iterations. any way to improve it?\nThanks!

", "votes": null, "replies": [] }, { "id": 868778, "author_name": "alimeituan", "author_url": "", "post_date": "05/31/2020 13:41:26", "content": "

but the big problem I met in use is how to transform the origin CSV data into FFM format data\ncould you pls help me?\nthx

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "140454": "Following qianqian's great post, [keras fm][1], I'll also respond to some private emails and public discussions about how to handle big data.\r\n\r\nFor people who has only small machine like me, the key is the willingness to go low level and customized classifiers. As you all know, FFM is the nuclear weapon for CTR. Currently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time). My ffm is largely based on Libffm which is a wonderful library. I managed to improve its out of core performance and add some tricks to improve both apk and speed. \r\n\r\nTo me, It's all about scalability when we keep throwing more features to it. Solving bottlenecks one after another, It is the most fun thing I like to do in Kaggle. \r\n\r\n\r\n [1]: https://www.kaggle.com/qqgeogor/outbrain-click-prediction/keras-based-fm", "140458": "thanks very much for your kindly sharing. I'm wondering how to solve FFM, like sgd? I think it is not easy to implemrnent the suitable solve? Thanks.", "140463": "frankhu, it turned out adagrad is the best. I also tried sgd, momentum and remsprop, couldn't beat adagrad. \r\n\r\nEdit: adagrad is the default and only solver of libffm", "140828": "thanks very much for your kindly reply. It seems resonable.", "142490": "rcarson hi, I'd like to ask for advice, is it reliable to use libfffm's '--on-disk' option for the huge data size this competition? or need some additional optimization by myself", "142651": "frankhu, I didn't try the default '--on-disk' option of ffm. Based on the code, you could expect same performance as in-core version but with a noticeable slowdown. Basically every time it reads a batch of data from the raw data to memory. Reading and parsing raw data file every time is slow. A better way is that you parse the whole raw data once in a batch by batch way and save the parsed batches in binaries, then it is much faster to reload each batch, which is what i'm doing actually.", "142790": "rcarson thanks very much for your kindly reply, your idea is good and interesting :-)", "143649": "I'm trying to use libffm and I have this stupid problem: `ffm-predict input_path output_path` produces a file with predictions - one per line. But the resulting file has slightly less lines then there are records in the input file. I suspect it's skipping records with field values that were not encountered in the training. The problem is, I have no way of joining the result back with the test set because there is no index column - there is only a prediction in each line. \r\n\r\nIs anyone else having this problem? Has anyone solved it? Is there a flag to tell ffm to default to predicting -1 or something instead of skipping the record?\r\n\r\nI know I can always find and remove the offending records before ffm-predict but this will bu super messy and tedious as it depends on my cross-val split and on feature set.", "143805": "I think ffm-predict doesn't skip rows due to unseen fields. For unseen fields, it should output 0.5, which you can try a 3-line dummy example. It is more likely that the input file is not correctly formatted.", "143922": "Thank you very much for [keras FM code][1]!\r\n\r\nCan anyone explain why the dimensionality (shape) of \"predict\" function output depends on batch_size?\r\nSpecifically:\r\n> predictions = myNet.predict(X, batch_size=4)\r\n\r\n> predictions.shape\r\n\r\n(3072,)\r\n\r\n> predictions = myNet.predict(X, batch_size=3)\r\n\r\n> predictions.shape\r\n\r\n(2304,)\r\n\r\n> X.shape\r\n\r\n(3, 768)\r\n\r\nHere is all my code:\r\n\r\ndataset = np.loadtxt(\"/myfolder/pima-indians-diabetes.data\", delimiter=\",\")\r\n\r\nX = dataset[:,0:3]\r\n\r\nY = dataset[:,8]\r\n\r\nX = np.transpose(X)\r\n\r\nY = np.transpose(Y)\r\n\r\nmyNet = KerasFM([ 1+round(max(X[0])), 1+round(max(X [ 1 ] )), 1+round(max(X[2])) ])\r\n\r\npredictions = myNet.predict(X, batch_size=4)\r\n\r\npredictions.shape\r\n\r\n\r\n [1]: https://www.kaggle.com/qqgeogor/outbrain-click-prediction/keras-based-fm", "143958": "Shakirov, keras ffm code is created by qianqian, you can ask him in that thread :)", "148968": "I had a very basic/stupid question. In libfm, Rendle provides a script to convert \"user,item,rating\" kind of file to a format which libfm can directly ingest. Is there something similar to libffm?\r\n\r\nThank you and Regards,\r\nSumit", "153278": "[quote=rcarson;140454]\r\n\r\nCurrently I'm developing my own FFM in c++. I can get 0.683 LB with 5 GB memory, 2 hours with a single FFM (excluding feature engineering time).\r\n\r\n\r\n [1]: https://www.kaggle.com/qqgeogor/outbrain-click-prediction/keras-based-fm\r\n\r\n[/quote]\r\n\r\nThis is awesome. This is my first time using FFM and it has been a little rough. I am only using 10% of the training data with 15 features and the FFM code is consuming all of my 16 GB memory allocation. If you don't mind me asking are you using all of the training data or a subset?", "153442": "RDizzl3, thank you for the kind reply. We can get 0.6937 lb now with 7 GB memory usage now. Trained with all training data. We will release the code after the contest finish.", "153446": "Great, @rcarson! =)\r\n\r\n0.6937 lb = 1 model?", "153462": "Eric, yes, one ffm model.", "153610": "rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?", "153774": "rcarson: how do you handle (encode) numerical features into you FFM ?", "153830": "Is there any nice implementation of FFM in python?", "154444": "rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong.", "154446": "Sameh Faidi: Try this: [Python Libffm][1]\r\n\r\n [1]: http://github.com/turi-code/python-libffm", "154452": "dont google ffm ...", "154455": "i am getting this error when i make on ubuntu\r\n\r\n\r\n make\r\n\r\n\r\n\r\n> g++ -O3 -std=c++11 -I ../sdk -shared -fPIC -march=native -fopenmp -DUSEOMP -c -o lib/ffm.o lib/ffm.cpp\r\nIn file included from lib/ffm.cpp:10:0:\r\n/usr/lib/gcc/x86_64-linux-gnu/4.8/include/pmmintrin.h:31:3: error: #error \"SSE3 instruction set not enabled\"\r\n # error \"SSE3 instruction set not enabled\"\r\n ^\r\nlib/ffm.cpp: In function âffm::ffm_float ffm::{anonymous}::wTx(ffm::ffm_node*, ffm::ffm_node*, ffm::ffm_float, ffm::ffm_model&, ffm::ffm_float, ffm::ffm_float, ffm::ffm_float, bool)â:\r\nlib/ffm.cpp:138:34: error: â_mm_hadd_psâ was not declared in this scope\r\n XMMt = _mm_hadd_ps(XMMt, XMMt);\r\n ^\r\nmake: *** [lib/ffm.o] Error 1", "154524": "Ashish Kulkarni I tried that one, it doesn't work.\r\n@Sameh Faidi Did you find the nice implementation of FFM in python?", "154556": "Done some predictions using libffm and found that all the predictions on test data is in range 0.8.. but local validation scores are reasonable. Has somebody faced the same issue?\r\nOr I am making some mistake.. Help on sample data and its format will be highly appreciated\r\n\r\nEdit: Python has function gives different result on different sessions.\r\nhttp://stackoverflow.com/questions/27522626/hash-function-in-python-3-3-returns-different-results-between-sessions\r\n\r\nI have saved intermediate data in files and for different runs I got different hash. This also explains consistent 0.8 prediction for test data\r\n\r\nI hope this will help others and save their valuable time.", "154578": "[quote=Keerath Jaggi;153610]\r\n\r\n@rcarson Does your ffm include linear terms (or is it only based on pair wise interactions) ?\r\n\r\n[/quote]\r\n\r\nI didn't include linear terms. Pairwise interaction only just like default ffm.", "154579": "[quote=Sameh Faidi;153774]\r\n\r\n@rcarson: how do you handle (encode) numerical features into you FFM ?\r\n\r\n[/quote]\r\n\r\nMy ffm is particularly bad with numerical features comparing to default ffm. Still try to figure out why. But you can directly give numerical feature in \"value\" part of \"field:feature:value\" as specified in ffm's example.", "154580": "[quote=Ashish Kulkarni;154444]\r\n\r\n@rcarson: Thanks a lot for sharing this information. I have a 32 GB RAM, 8 core PC. I used the entire training set on libffm. Even after 22 hours, not a single iteration was complete. Is it because of my machine's limitations or are there chances that I am doing something wrong. \r\n\r\n[/quote]\r\n\r\nyour machine is good. Please monitor your memory usage. if you are out of memory, turn off auto-stop.\r\nAlso start with a small k=4 to make sure everything goes through.", "155363": "Can FFM models make use of the unsupervised data in page_views or based solely on the supervised data in the clicks_train file?", "156308": "after how many iteration your ffm model start to overfit? mine start overfitting after 3-4 iterations. any way to improve it?\r\nThanks!", "868778": "but the big problem I met in use is how to transform the origin CSV data into FFM format data\ncould you pls help me?\nthx", "3083515": "Outbrain click prediction helps optimize content recommendations for better engagement. At [reelit com](https://reelit.xyz/), we used similar tools to drive targeted traffic and improve campaign results!" }, "source": "meta" }