{
  "id": 688612,
  "title": "Optuna-based Feature Selection for Anonymized Tabular Data",
  "url": "/competitions/drw-crypto-market-prediction/discussion/688612",
  "author_name": "ahmed Maalej",
  "post_date": "2026-04-06T14:15:40.262000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p>The main challenge in this competition was dealing with a large number of anonymized features (~800), where standard feature selection methods (correlation filtering, variance thresholding, univariate scoring) were not sufficient.</p>\n<p>Instead of ranking features individually, I treated feature selection as a wrapper optimization problem. Using Optuna, I searched over binary feature masks and evaluated each subset using 5-fold cross-validation with LightGBM, optimizing negative Pearson correlation with a small sparsity penalty.</p>\n<p>This reduced the feature space from ~600 to ~270 features and led to more stable validation performance.</p>\n<p>After feature selection, I trained an autoencoder on the selected features and used its latent representation (10 dimensions) as additional features.</p>\n<p>Final modeling used an ensemble of LightGBM, XGBoost, and MLP, combined via Ridge stacking.</p>",
  "messages": [
    {
      "id": 3436684,
      "postDate": "2026-04-06T14:15:40.263Z",
      "content": "<p>The main challenge in this competition was dealing with a large number of anonymized features (~800), where standard feature selection methods (correlation filtering, variance thresholding, univariate scoring) were not sufficient.</p>\n<p>Instead of ranking features individually, I treated feature selection as a wrapper optimization problem. Using Optuna, I searched over binary feature masks and evaluated each subset using 5-fold cross-validation with LightGBM, optimizing negative Pearson correlation with a small sparsity penalty.</p>\n<p>This reduced the feature space from ~600 to ~270 features and led to more stable validation performance.</p>\n<p>After feature selection, I trained an autoencoder on the selected features and used its latent representation (10 dimensions) as additional features.</p>\n<p>Final modeling used an ensemble of LightGBM, XGBoost, and MLP, combined via Ridge stacking.</p>",
      "rawMarkdown": "The main challenge in this competition was dealing with a large number of anonymized features (~800), where standard feature selection methods (correlation filtering, variance thresholding, univariate scoring) were not sufficient.\n\nInstead of ranking features individually, I treated feature selection as a wrapper optimization problem. Using Optuna, I searched over binary feature masks and evaluated each subset using 5-fold cross-validation with LightGBM, optimizing negative Pearson correlation with a small sparsity penalty.\n\nThis reduced the feature space from ~600 to ~270 features and led to more stable validation performance.\n\nAfter feature selection, I trained an autoencoder on the selected features and used its latent representation (10 dimensions) as additional features.\n\nFinal modeling used an ensemble of LightGBM, XGBoost, and MLP, combined via Ridge stacking.",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "3436684": "The main challenge in this competition was dealing with a large number of anonymized features (~800), where standard feature selection methods (correlation filtering, variance thresholding, univariate scoring) were not sufficient.\n\nInstead of ranking features individually, I treated feature selection as a wrapper optimization problem. Using Optuna, I searched over binary feature masks and evaluated each subset using 5-fold cross-validation with LightGBM, optimizing negative Pearson correlation with a small sparsity penalty.\n\nThis reduced the feature space from ~600 to ~270 features and led to more stable validation performance.\n\nAfter feature selection, I trained an autoencoder on the selected features and used its latent representation (10 dimensions) as additional features.\n\nFinal modeling used an ensemble of LightGBM, XGBoost, and MLP, combined via Ridge stacking."
  }
}