{
  "id": 58880,
  "title": "SVD: the concept",
  "url": "/competitions/avito-demand-prediction/discussion/58880",
  "author_name": "",
  "post_date": "2018-06-15T02:25:08.782702600Z",
  "votes": null,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hi everybody!</p>\n\n<p>I was going through the concepts then came across the SVD and I started looking into it.</p>\n\n<p>I came on a conclusion, that SVD works like the PCA for test features and given a (m x n) feature matrix it converts it to the (m x p) feature matrix, where \"p\" is decided while using the SVD.</p>\n\n<p>Please let me know if I have missed anything or my reasoning is not clear.</p>\n\n<p>Thanks!:) </p>",
  "messages": [
    {
      "id": "343274",
      "postDate": "06/15/2018 02:25:08",
      "content": "<p>Hi everybody!</p>\n\n<p>I was going through the concepts then came across the SVD and I started looking into it.</p>\n\n<p>I came on a conclusion, that SVD works like the PCA for test features and given a (m x n) feature matrix it converts it to the (m x p) feature matrix, where \"p\" is decided while using the SVD.</p>\n\n<p>Please let me know if I have missed anything or my reasoning is not clear.</p>\n\n<p>Thanks!:) </p>",
      "rawMarkdown": "Hi everybody!\n\nI was going through the concepts then came across the SVD and I started looking into it.\n\nI came on a conclusion, that SVD works like the PCA for test features and given a (m x n) feature matrix it converts it to the (m x p) feature matrix, where \"p\" is decided while using the SVD.\n\nPlease let me know if I have missed anything or my reasoning is not clear.\n\nThanks!:)",
      "votes": null
    },
    {
      "id": "343368",
      "postDate": "06/15/2018 08:24:58",
      "content": "<p>SVD is simply the matrix factorization method used by PCA (as well as many other things). </p>",
      "rawMarkdown": "SVD is simply the matrix factorization method used by PCA (as well as many other things).",
      "votes": null
    },
    {
      "id": "343608",
      "postDate": "06/15/2018 17:09:15",
      "content": "<p>It will eat your memory in this case.</p>",
      "rawMarkdown": "It will eat your memory in this case.",
      "votes": null
    },
    {
      "id": "343611",
      "postDate": "06/15/2018 17:13:57",
      "content": "<p>please elaborate!</p>",
      "rawMarkdown": "please elaborate!",
      "votes": null
    },
    {
      "id": "343615",
      "postDate": "06/15/2018 17:27:32",
      "content": "<p>To clarify, you are doing SVD on text features from tf-idf, right? Also, sorry for my mistake, I think the main issue should the time complexity not memory</p>",
      "rawMarkdown": "To clarify, you are doing SVD on text features from tf-idf, right? Also, sorry for my mistake, I think the main issue should the time complexity not memory",
      "votes": null
    },
    {
      "id": "343620",
      "postDate": "06/15/2018 17:35:53",
      "content": "<p>yes, i was doing the SVD on text features from tf-idf and found that the columns(the number of features in vector space) was reduced</p>",
      "rawMarkdown": "yes, i was doing the SVD on text features from tf-idf and found that the columns(the number of features in vector space) was reduced",
      "votes": null
    },
    {
      "id": "343621",
      "postDate": "06/15/2018 17:37:34",
      "content": "<p>Also try Latent Dirichlet allocation, which should be better for dimensional reduce</p>",
      "rawMarkdown": "Also try Latent Dirichlet allocation, which should be better for dimensional reduce",
      "votes": null
    },
    {
      "id": "343626",
      "postDate": "06/15/2018 17:41:26",
      "content": "<p>Yes sure I will try it!</p>\n\n<p>Thanks for so enlightening discussion :)</p>",
      "rawMarkdown": "Yes sure I will try it!\n\nThanks for so enlightening discussion :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 343368,
      "author_name": "maw501",
      "author_url": "",
      "post_date": "06/15/2018 08:24:58",
      "content": "<p>SVD is simply the matrix factorization method used by PCA (as well as many other things). </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 343608,
      "author_name": "creatrol",
      "author_url": "",
      "post_date": "06/15/2018 17:09:15",
      "content": "<p>It will eat your memory in this case.</p>",
      "votes": null,
      "replies": [
        {
          "id": 343611,
          "author_name": "ashukr",
          "author_url": "",
          "post_date": "06/15/2018 17:13:57",
          "content": "<p>please elaborate!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 343615,
          "author_name": "creatrol",
          "author_url": "",
          "post_date": "06/15/2018 17:27:32",
          "content": "<p>To clarify, you are doing SVD on text features from tf-idf, right? Also, sorry for my mistake, I think the main issue should the time complexity not memory</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 343620,
          "author_name": "ashukr",
          "author_url": "",
          "post_date": "06/15/2018 17:35:53",
          "content": "<p>yes, i was doing the SVD on text features from tf-idf and found that the columns(the number of features in vector space) was reduced</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 343621,
          "author_name": "creatrol",
          "author_url": "",
          "post_date": "06/15/2018 17:37:34",
          "content": "<p>Also try Latent Dirichlet allocation, which should be better for dimensional reduce</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 343626,
          "author_name": "ashukr",
          "author_url": "",
          "post_date": "06/15/2018 17:41:26",
          "content": "<p>Yes sure I will try it!</p>\n\n<p>Thanks for so enlightening discussion :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "343274": "Hi everybody!\n\nI was going through the concepts then came across the SVD and I started looking into it.\n\nI came on a conclusion, that SVD works like the PCA for test features and given a (m x n) feature matrix it converts it to the (m x p) feature matrix, where \"p\" is decided while using the SVD.\n\nPlease let me know if I have missed anything or my reasoning is not clear.\n\nThanks!:)",
    "343368": "SVD is simply the matrix factorization method used by PCA (as well as many other things).",
    "343608": "It will eat your memory in this case.",
    "343611": "please elaborate!",
    "343615": "To clarify, you are doing SVD on text features from tf-idf, right? Also, sorry for my mistake, I think the main issue should the time complexity not memory",
    "343620": "yes, i was doing the SVD on text features from tf-idf and found that the columns(the number of features in vector space) was reduced",
    "343621": "Also try Latent Dirichlet allocation, which should be better for dimensional reduce",
    "343626": "Yes sure I will try it!\n\nThanks for so enlightening discussion :)"
  },
  "source": "meta"
}