Hi all,
\nI don't seem to be making much progress in the competition (last couple of ideas were a wash), so playing around with ideas as to why that might be. My current working theory is that hotel is just a really bad label to associate with an image - the content of the image doesn't relate well, or in a predictable way, to the hotel. The network must need to learn to identify features, then associate the fine detail in those features to the target, which seems hard.
\nAnyway, one of the major ways that the photos differ is in what they're looking at. Some will contain a view of the bathroom, with tiling, waterproof surfaces, etc, while others may contain a view of a bedroom, with carpets and painted or wallpapered walls, etc. These two things within a hotel don't necessarily relate to each other in an obvious way - given a particular set of wall decoration or curtains, the tiling chosen for the bathroom doesn't directly follow.
\nI feel that separating these might be important, so have flicked together a notebook in an attempt to do so, by identifying key features using YOLOv3:
\nhttps://www.kaggle.com/code/prubyg/hotel-id-room-type-classifier-public
\nAnyway, keen to hear other people's thoughts on training with different views, all of which are expected to have the same label :)
", "messages": [ { "id": "1769139", "postDate": "04/27/2022 01:24:56", "content": "Hi all,
\nI don't seem to be making much progress in the competition (last couple of ideas were a wash), so playing around with ideas as to why that might be. My current working theory is that hotel is just a really bad label to associate with an image - the content of the image doesn't relate well, or in a predictable way, to the hotel. The network must need to learn to identify features, then associate the fine detail in those features to the target, which seems hard.
\nAnyway, one of the major ways that the photos differ is in what they're looking at. Some will contain a view of the bathroom, with tiling, waterproof surfaces, etc, while others may contain a view of a bedroom, with carpets and painted or wallpapered walls, etc. These two things within a hotel don't necessarily relate to each other in an obvious way - given a particular set of wall decoration or curtains, the tiling chosen for the bathroom doesn't directly follow.
\nI feel that separating these might be important, so have flicked together a notebook in an attempt to do so, by identifying key features using YOLOv3:
\nhttps://www.kaggle.com/code/prubyg/hotel-id-room-type-classifier-public
\nAnyway, keen to hear other people's thoughts on training with different views, all of which are expected to have the same label :)
", "rawMarkdown": "Hi all,\n\nI don't seem to be making much progress in the competition (last couple of ideas were a wash), so playing around with ideas as to why that might be. My current working theory is that hotel is just a really bad label to associate with an image - the content of the image doesn't relate well, or in a predictable way, to the hotel. The network must need to learn to identify features, then associate the fine detail in those features to the target, which seems hard.\n\nAnyway, one of the major ways that the photos differ is in what they're looking at. Some will contain a view of the bathroom, with tiling, waterproof surfaces, etc, while others may contain a view of a bedroom, with carpets and painted or wallpapered walls, etc. These two things within a hotel don't necessarily relate to each other in an obvious way - given a particular set of wall decoration or curtains, the tiling chosen for the bathroom doesn't directly follow.\n\nI feel that separating these might be important, so have flicked together a notebook in an attempt to do so, by identifying key features using YOLOv3:\n\nhttps://www.kaggle.com/code/prubyg/hotel-id-room-type-classifier-public\n\nAnyway, keen to hear other people's thoughts on training with different views, all of which are expected to have the same label :)", "votes": null }, { "id": "1796432", "postDate": "05/20/2022 20:16:42", "content": "I don't know if your idea will help, but I think it's fun you posted it.
\nTo use it, I think you could add your encoding (bedroom, bathroom, other) as a one-hot input to your neural networks, and see if that helps your performance.
\nAlso, on this:
\n\n\nIt's interesting to note that some of the \"confused\" images we see (e.g. the bed it thinks is a fridge) are rotated wrong - correcting that may be a path to improvement.
\n
The winning entry from last year developed a model to rotate images, so I'd say your intuition is correct.
", "rawMarkdown": "I don't know if your idea will help, but I think it's fun you posted it.\n\nTo use it, I think you could add your encoding (bedroom, bathroom, other) as a one-hot input to your neural networks, and see if that helps your performance.\n\nAlso, on this:\n\n> It's interesting to note that some of the \"confused\" images we see (e.g. the bed it thinks is a fridge) are rotated wrong - correcting that may be a path to improvement.\n\nThe [winning entry from last year](https://www.kaggle.com/c/hotel-id-2021-fgvc8/discussion/242087) developed a model to rotate images, so I'd say your intuition is correct.", "votes": null }, { "id": "1796935", "postDate": "05/21/2022 10:37:56", "content": "Another way how to leverage the room type could be adding a second ArcFace module to have one for hotels and one for room types similar to some solutions in Happywhale competition (for example 19th Place - Single Model LB 860 Without Pseudo) where one is used for species and second ArcFace for individuals.
\nOr just use the room type in similarity search so you look only for similar images of the same room type (or assign weights to images for better search). Here you could check first if that's actually a problem by checking what your model considers a similar image (if there are some misclassifications caused by confusing different room types).
", "rawMarkdown": "Another way how to leverage the room type could be adding a second ArcFace module to have one for hotels and one for room types similar to some solutions in Happywhale competition (for example [19th Place - Single Model LB 860 Without Pseudo](https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/320298)) where one is used for species and second ArcFace for individuals.\n\nOr just use the room type in similarity search so you look only for similar images of the same room type (or assign weights to images for better search). Here you could check first if that's actually a problem by checking what your model considers a similar image (if there are some misclassifications caused by confusing different room types).", "votes": null } ], "comments": [ { "id": 1796432, "author_name": "dfrankow", "author_url": "", "post_date": "05/20/2022 20:16:42", "content": "I don't know if your idea will help, but I think it's fun you posted it.
\nTo use it, I think you could add your encoding (bedroom, bathroom, other) as a one-hot input to your neural networks, and see if that helps your performance.
\nAlso, on this:
\n\n\nIt's interesting to note that some of the \"confused\" images we see (e.g. the bed it thinks is a fridge) are rotated wrong - correcting that may be a path to improvement.
\n
The winning entry from last year developed a model to rotate images, so I'd say your intuition is correct.
", "votes": null, "replies": [ { "id": 1796935, "author_name": "michaln", "author_url": "", "post_date": "05/21/2022 10:37:56", "content": "Another way how to leverage the room type could be adding a second ArcFace module to have one for hotels and one for room types similar to some solutions in Happywhale competition (for example 19th Place - Single Model LB 860 Without Pseudo) where one is used for species and second ArcFace for individuals.
\nOr just use the room type in similarity search so you look only for similar images of the same room type (or assign weights to images for better search). Here you could check first if that's actually a problem by checking what your model considers a similar image (if there are some misclassifications caused by confusing different room types).
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1769139": "Hi all,\n\nI don't seem to be making much progress in the competition (last couple of ideas were a wash), so playing around with ideas as to why that might be. My current working theory is that hotel is just a really bad label to associate with an image - the content of the image doesn't relate well, or in a predictable way, to the hotel. The network must need to learn to identify features, then associate the fine detail in those features to the target, which seems hard.\n\nAnyway, one of the major ways that the photos differ is in what they're looking at. Some will contain a view of the bathroom, with tiling, waterproof surfaces, etc, while others may contain a view of a bedroom, with carpets and painted or wallpapered walls, etc. These two things within a hotel don't necessarily relate to each other in an obvious way - given a particular set of wall decoration or curtains, the tiling chosen for the bathroom doesn't directly follow.\n\nI feel that separating these might be important, so have flicked together a notebook in an attempt to do so, by identifying key features using YOLOv3:\n\nhttps://www.kaggle.com/code/prubyg/hotel-id-room-type-classifier-public\n\nAnyway, keen to hear other people's thoughts on training with different views, all of which are expected to have the same label :)", "1796432": "I don't know if your idea will help, but I think it's fun you posted it.\n\nTo use it, I think you could add your encoding (bedroom, bathroom, other) as a one-hot input to your neural networks, and see if that helps your performance.\n\nAlso, on this:\n\n> It's interesting to note that some of the \"confused\" images we see (e.g. the bed it thinks is a fridge) are rotated wrong - correcting that may be a path to improvement.\n\nThe [winning entry from last year](https://www.kaggle.com/c/hotel-id-2021-fgvc8/discussion/242087) developed a model to rotate images, so I'd say your intuition is correct.", "1796935": "Another way how to leverage the room type could be adding a second ArcFace module to have one for hotels and one for room types similar to some solutions in Happywhale competition (for example [19th Place - Single Model LB 860 Without Pseudo](https://www.kaggle.com/competitions/happy-whale-and-dolphin/discussion/320298)) where one is used for species and second ArcFace for individuals.\n\nOr just use the room type in similarity search so you look only for similar images of the same room type (or assign weights to images for better search). Here you could check first if that's actually a problem by checking what your model considers a similar image (if there are some misclassifications caused by confusing different room types)." }, "source": "meta" }