We would like to thank the organizers for creating such an engaging and challenging competition. This document outlines our comprehensive approach to solving the flight ranking problem.
\nOur solution began with the foundation of a public notebook (XGBoost Ranker), though by the competition's end, our implementation had evolved significantly beyond the original approach.
\nOur feature engineering process was methodical and multi-faceted, focusing on capturing the complexity of flight booking behavior and preferences.
\nRoute Analysis:
\nTemporal Features:
\nDistance Calculations:
\nAirport Characteristics:
\nHistorical Patterns:
\nContextual Analysis:
\nQuality Scoring:
\nPricing Analytics:
\nFollowing successful strategies employed by other competition winners, we implemented historical counters with multiple categorical feature combinations. Our approach involved:
\nTo ensure training stability and computational efficiency:
\nOur ensemble approach incorporated multiple ranking algorithms:
\nPrimary Models:
\nOur final prediction system employed a normalized score ensemble:
\nThe success of our solution stemmed from several critical discoveries:
\nPrice and Duration Features Were Paramount: As expected, flight price and duration emerged as the most important predictive features. We experimented with multiple variations of these features including:
\nThis comprehensive treatment of the core decision factors proved essential for model performance.
Historical Counters Provided Substantial Gains: Historical counters delivered significant performance improvements, contributing approximately 200 features to our final dataset. However, this represents only a fraction of the possible categorical combinations available in the data. There remains substantial opportunity for further score improvements by exploring additional feature combinations, though computational constraints limited our exploration.
Multi-Loss Function Ensemble Benefits: A key discovery was that combining models trained with different loss functions, even when using identical training data, yielded meaningful performance gains. This suggests that different loss functions capture complementary aspects of the ranking problem, making ensemble diversity valuable beyond just data variation.
Thanks for attention
", "messages": [ { "id": "3271368", "postDate": "08/18/2025 17:25:27", "content": "We would like to thank the organizers for creating such an engaging and challenging competition. This document outlines our comprehensive approach to solving the flight ranking problem.
\nOur solution began with the foundation of a public notebook (XGBoost Ranker), though by the competition's end, our implementation had evolved significantly beyond the original approach.
\nOur feature engineering process was methodical and multi-faceted, focusing on capturing the complexity of flight booking behavior and preferences.
\nRoute Analysis:
\nTemporal Features:
\nDistance Calculations:
\nAirport Characteristics:
\nHistorical Patterns:
\nContextual Analysis:
\nQuality Scoring:
\nPricing Analytics:
\nFollowing successful strategies employed by other competition winners, we implemented historical counters with multiple categorical feature combinations. Our approach involved:
\nTo ensure training stability and computational efficiency:
\nOur ensemble approach incorporated multiple ranking algorithms:
\nPrimary Models:
\nOur final prediction system employed a normalized score ensemble:
\nThe success of our solution stemmed from several critical discoveries:
\nPrice and Duration Features Were Paramount: As expected, flight price and duration emerged as the most important predictive features. We experimented with multiple variations of these features including:
\nThis comprehensive treatment of the core decision factors proved essential for model performance.
Historical Counters Provided Substantial Gains: Historical counters delivered significant performance improvements, contributing approximately 200 features to our final dataset. However, this represents only a fraction of the possible categorical combinations available in the data. There remains substantial opportunity for further score improvements by exploring additional feature combinations, though computational constraints limited our exploration.
Multi-Loss Function Ensemble Benefits: A key discovery was that combining models trained with different loss functions, even when using identical training data, yielded meaningful performance gains. This suggests that different loss functions capture complementary aspects of the ranking problem, making ensemble diversity valuable beyond just data variation.
Thanks for attention
", "rawMarkdown": "# Machine Learning Competition Solution\n\n## Overview\n\nWe would like to thank the organizers for creating such an engaging and challenging competition. This document outlines our comprehensive approach to solving the flight ranking problem.\n\nOur solution began with the foundation of a public notebook ([XGBoost Ranker](https://www.kaggle.com/code/ka1242/xgboost-ranker-with-polars)), though by the competition's end, our implementation had evolved significantly beyond the original approach.\n\n## Feature Engineering Strategy\n\nOur feature engineering process was methodical and multi-faceted, focusing on capturing the complexity of flight booking behavior and preferences.\n\n### 1. Core Flight Features\n\n**Route Analysis:**\n- Extracted segment counts and connection details for each flight leg\n- Constructed complete flight paths (origin → connections → destination)\n- Analyzed route structure and categorized connection types\n\n**Temporal Features:**\n- Converted departure and arrival times to minutes for consistent processing\n- Extracted time-based patterns including time of day, day of week, and seasonality\n- Identified holidays and weekends to capture booking pattern variations\n- Analyzed business-friendly timing preferences\n\n### 2. Geographic & Airport Features\n\n**Distance Calculations:**\n- Implemented Haversine formula for precise inter-airport distance measurements\n- Calculated elevation differences between airports\n- Identified intercontinental flights for specialized handling\n\n**Airport Characteristics:**\n- Classified airports as hub versus regional facilities\n- Analyzed route popularity and traffic patterns\n- Implemented continental grouping for geographic insights\n\n### 3. Advanced User Behavior Features\n\n**Historical Patterns:**\n- Analyzed user preferences across airlines, booking timing, and service classes\n- Incorporated loyalty program analysis\n- Evaluated corporate policy compliance patterns\n\n**Contextual Analysis:**\n- Calculated relative rankings within search sessions (price, duration, convenience)\n- Identified Pareto-optimal options in the choice set\n- Analyzed price-quality trade-off preferences\n\n### 4. Flight Quality & Convenience Metrics\n\n**Quality Scoring:**\n- Mapped aircraft quality characteristics (wide-body vs narrow-body)\n- Assessed schedule convenience (business hours vs red-eye flights)\n- Analyzed fare flexibility options\n- Evaluated connection time comfort levels\n\n**Pricing Analytics:**\n- Calculated price-per-kilometer and price-per-minute efficiency ratios\n- Measured deviation from group medians\n- Analyzed competitive positioning within search results\n\n### 5. Temporal Split & Historical Counters\n\nFollowing successful strategies employed by other competition winners, we implemented historical counters with multiple categorical feature combinations. Our approach involved:\n\n- Using 50% of the time-sorted training dataset as historical reference\n- Training on the remaining 50% to maintain temporal validity\n- Creating interaction features across multiple categorical dimensions\n\n### 6. Data Sampling Strategy\n\nTo ensure training stability and computational efficiency:\n\n- Limited group sizes to 64 samples maximum\n- Preserved all positive examples (selected=1) to maintain signal strength\n- Applied random sampling to negative examples within each group\n\n## Model Architecture\n\n### 7. Model Training\n\nOur ensemble approach incorporated multiple ranking algorithms:\n\n**Primary Models:**\n- **CatBoostRanker** with three different loss functions:\n - QueryCrossEntropy (alpha=1.0) - primary loss function\n - PairLogitPairwise - secondary approach\n - YetiRankPairwise - tertiary validation\n- **LightGBM Ranker** implemented by Farukcan\n\n### 8. Ensemble Strategy\n\nOur final prediction system employed a normalized score ensemble:\n\n- Normalized all individual ranker scores to ensure comparable scales\n- Combined normalized scores using optimized ratios\n- Selected ensemble weights primarily through leaderboard validation\n- Maintained simplicity in the ensemble approach for robust generalization\n\n## Key Insights\n\nThe success of our solution stemmed from several critical discoveries:\n\n1. **Price and Duration Features Were Paramount**: As expected, flight price and duration emerged as the most important predictive features. We experimented with multiple variations of these features including:\n - Absolute values (raw price and duration)\n - Relative measures (compared to group averages)\n - Ranked positions within search results\n \n This comprehensive treatment of the core decision factors proved essential for model performance.\n\n2. **Historical Counters Provided Substantial Gains**: Historical counters delivered significant performance improvements, contributing approximately 200 features to our final dataset. However, this represents only a fraction of the possible categorical combinations available in the data. There remains substantial opportunity for further score improvements by exploring additional feature combinations, though computational constraints limited our exploration.\n\n3. **Multi-Loss Function Ensemble Benefits**: A key discovery was that combining models trained with different loss functions, even when using identical training data, yielded meaningful performance gains. This suggests that different loss functions capture complementary aspects of the ranking problem, making ensemble diversity valuable beyond just data variation.\n\nThanks for attention", "votes": null }, { "id": "3271373", "postDate": "08/18/2025 17:44:06", "content": "Excellent work, so many valuable insights!
\nVery elaborate solution.
thank you for sharing your insights, is the code avaliable somewhere?
", "rawMarkdown": "thank you for sharing your insights, is the code avaliable somewhere?", "votes": null } ], "comments": [ { "id": 3271373, "author_name": "mikhailgolubchik", "author_url": "", "post_date": "08/18/2025 17:44:06", "content": "Excellent work, so many valuable insights!
\nVery elaborate solution.
thank you for sharing your insights, is the code avaliable somewhere?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3271368": "# Machine Learning Competition Solution\n\n## Overview\n\nWe would like to thank the organizers for creating such an engaging and challenging competition. This document outlines our comprehensive approach to solving the flight ranking problem.\n\nOur solution began with the foundation of a public notebook ([XGBoost Ranker](https://www.kaggle.com/code/ka1242/xgboost-ranker-with-polars)), though by the competition's end, our implementation had evolved significantly beyond the original approach.\n\n## Feature Engineering Strategy\n\nOur feature engineering process was methodical and multi-faceted, focusing on capturing the complexity of flight booking behavior and preferences.\n\n### 1. Core Flight Features\n\n**Route Analysis:**\n- Extracted segment counts and connection details for each flight leg\n- Constructed complete flight paths (origin → connections → destination)\n- Analyzed route structure and categorized connection types\n\n**Temporal Features:**\n- Converted departure and arrival times to minutes for consistent processing\n- Extracted time-based patterns including time of day, day of week, and seasonality\n- Identified holidays and weekends to capture booking pattern variations\n- Analyzed business-friendly timing preferences\n\n### 2. Geographic & Airport Features\n\n**Distance Calculations:**\n- Implemented Haversine formula for precise inter-airport distance measurements\n- Calculated elevation differences between airports\n- Identified intercontinental flights for specialized handling\n\n**Airport Characteristics:**\n- Classified airports as hub versus regional facilities\n- Analyzed route popularity and traffic patterns\n- Implemented continental grouping for geographic insights\n\n### 3. Advanced User Behavior Features\n\n**Historical Patterns:**\n- Analyzed user preferences across airlines, booking timing, and service classes\n- Incorporated loyalty program analysis\n- Evaluated corporate policy compliance patterns\n\n**Contextual Analysis:**\n- Calculated relative rankings within search sessions (price, duration, convenience)\n- Identified Pareto-optimal options in the choice set\n- Analyzed price-quality trade-off preferences\n\n### 4. Flight Quality & Convenience Metrics\n\n**Quality Scoring:**\n- Mapped aircraft quality characteristics (wide-body vs narrow-body)\n- Assessed schedule convenience (business hours vs red-eye flights)\n- Analyzed fare flexibility options\n- Evaluated connection time comfort levels\n\n**Pricing Analytics:**\n- Calculated price-per-kilometer and price-per-minute efficiency ratios\n- Measured deviation from group medians\n- Analyzed competitive positioning within search results\n\n### 5. Temporal Split & Historical Counters\n\nFollowing successful strategies employed by other competition winners, we implemented historical counters with multiple categorical feature combinations. Our approach involved:\n\n- Using 50% of the time-sorted training dataset as historical reference\n- Training on the remaining 50% to maintain temporal validity\n- Creating interaction features across multiple categorical dimensions\n\n### 6. Data Sampling Strategy\n\nTo ensure training stability and computational efficiency:\n\n- Limited group sizes to 64 samples maximum\n- Preserved all positive examples (selected=1) to maintain signal strength\n- Applied random sampling to negative examples within each group\n\n## Model Architecture\n\n### 7. Model Training\n\nOur ensemble approach incorporated multiple ranking algorithms:\n\n**Primary Models:**\n- **CatBoostRanker** with three different loss functions:\n - QueryCrossEntropy (alpha=1.0) - primary loss function\n - PairLogitPairwise - secondary approach\n - YetiRankPairwise - tertiary validation\n- **LightGBM Ranker** implemented by Farukcan\n\n### 8. Ensemble Strategy\n\nOur final prediction system employed a normalized score ensemble:\n\n- Normalized all individual ranker scores to ensure comparable scales\n- Combined normalized scores using optimized ratios\n- Selected ensemble weights primarily through leaderboard validation\n- Maintained simplicity in the ensemble approach for robust generalization\n\n## Key Insights\n\nThe success of our solution stemmed from several critical discoveries:\n\n1. **Price and Duration Features Were Paramount**: As expected, flight price and duration emerged as the most important predictive features. We experimented with multiple variations of these features including:\n - Absolute values (raw price and duration)\n - Relative measures (compared to group averages)\n - Ranked positions within search results\n \n This comprehensive treatment of the core decision factors proved essential for model performance.\n\n2. **Historical Counters Provided Substantial Gains**: Historical counters delivered significant performance improvements, contributing approximately 200 features to our final dataset. However, this represents only a fraction of the possible categorical combinations available in the data. There remains substantial opportunity for further score improvements by exploring additional feature combinations, though computational constraints limited our exploration.\n\n3. **Multi-Loss Function Ensemble Benefits**: A key discovery was that combining models trained with different loss functions, even when using identical training data, yielded meaningful performance gains. This suggests that different loss functions capture complementary aspects of the ranking problem, making ensemble diversity valuable beyond just data variation.\n\nThanks for attention", "3271373": "Excellent work, so many valuable insights!\nVery elaborate solution.", "3273648": "thank you for sharing your insights, is the code avaliable somewhere?" }, "source": "meta" }