{
  "id": 418526,
  "title": "Submission Error",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/418526",
  "author_name": "Qurious",
  "post_date": "2023-06-21T06:27:20.739000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi fellow Kagglers,</p>\n<p>I need a little help with submitting, it seems that my submission format is wrong, here's how I submit :</p>\n<pre><code>` (test, sample_submission)  iter_test:\n    Augmenter = AttributesAugmenter(unique_events,click_events,unique_rooms, cols)\n    \n    test = preprocess_df(test)\n    gc.collect()\n    X = Augmenter.transform(test)\n     test \n    gc.collect()\n    prediction = loaded_model.predict(X)\n     X\n    gc.collect()\n    \n    \n    sample_submission[] = prediction[].values\n    \n    \n    \n    env.predict(sample_submission)\n</code></pre>\n<p>`</p>\n<p>And this is my modelwrapper class :</p>\n<pre><code>  (BaseEstimator, ClassifierMixin):\n     ():\n         base_model_class ==  : \n            self.base_model_class = RandomForestClassifier()\n         : \n            self.base_model_class = base_model_class\n        self.model_params = model_params  model_params  {}\n        self.optimize_for = optimize_for\n        self.use_grid_search = use_grid_search\n        self.models = {}\n         i  () : \n             self.model_params !=   self.use_grid_search: \n                self.models[(i+)] = GridSearchCV(self.base_model_class(), self.model_params, scoring=self.optimize_for, cv=, verbose=)\n             : \n                self.models[(i+)] = self.base_model_class(**self.model_params)\n\n     ():\n        \n        questions = np.unique(y[])\n        X.sort_index(inplace=)\n        y.sort_index(inplace=)\n         question  questions:\n            group = get_level_group_question(question)\n            X_arranged = X.xs(group, level=).fillna()\n            Y_arranged = y[y[] == (question)][[]]\n            X_arranged, Y_arranged = X_arranged.align(Y_arranged, join=, axis=)\n            Y_arranged = np.ravel(Y_arranged.values)\n            X_arranged = X_arranged.values\n            self.models[(question)].fit(X_arranged,Y_arranged)\n             self.use_grid_search:\n                (, (question), , self.models[(question)].best_params_)\n                (, (question), , self.models[(question)].best_score_)\n            y_pred = self.models[(question)].predict(X_arranged)\n            accuracy = accuracy_score(Y_arranged, y_pred)\n            f1 = f1_score(Y_arranged, y_pred)\n            (, (question), , accuracy)\n            (, (question), , f1)\n         self\n\n     ():\n        \n        X.sort_index(inplace=)\n        predictions = []\n        session_question_tuples = []\n        unique_level_groups = (X.index.get_level_values().unique())\n         question  [i+  i  ()]:\n            group = get_level_group_question(question)\n             group  unique_level_groups :\n                X_arranged = X.xs(group, level=).fillna()\n                session_ids = X_arranged.index.get_level_values()  \n                pred = self.models[(question)].predict(X_arranged.values)\n                \n                session_question_tuples.extend([  s_id  session_ids])\n                predictions.extend(pred)\n\n        \n        predictions_df = pd.DataFrame(predictions, columns=[], index=session_question_tuples)\n        predictions_df.index.name = \n\n         predictions_df\n\n\n     ():\n        \n        y_pred = self.predict(X)\n        y_formatted = self.format_y(y)\n\n        common_index = y_formatted.index.intersection(y_pred.index)\n\n        f1 = f1_score(y_formatted.loc[common_index, ], y_pred.loc[common_index, ])\n        ()\n         f1\n\n     ():\n        \n        y_formatted = y.copy()\n        y_formatted.reset_index(inplace=)\n        y_formatted[] = y_formatted[].astype() +  + y_formatted[].astype()\n        y_formatted.set_index(, inplace=)\n        y_formatted.drop([, ], axis=, inplace=)\n        y_formatted.rename(columns={: }, inplace=)\n\n         y_formatted\n</code></pre>\n<p>I just can't understand what is wrong with my submissions … any help would be greatly appreciated !</p>",
  "messages": [
    {
      "id": 2311385,
      "postDate": "2023-06-21T06:27:20.740Z",
      "content": "<p>Hi fellow Kagglers,</p>\n<p>I need a little help with submitting, it seems that my submission format is wrong, here's how I submit :</p>\n<pre><code>` (test, sample_submission)  iter_test:\n    Augmenter = AttributesAugmenter(unique_events,click_events,unique_rooms, cols)\n    \n    test = preprocess_df(test)\n    gc.collect()\n    X = Augmenter.transform(test)\n     test \n    gc.collect()\n    prediction = loaded_model.predict(X)\n     X\n    gc.collect()\n    \n    \n    sample_submission[] = prediction[].values\n    \n    \n    \n    env.predict(sample_submission)\n</code></pre>\n<p>`</p>\n<p>And this is my modelwrapper class :</p>\n<pre><code>  (BaseEstimator, ClassifierMixin):\n     ():\n         base_model_class ==  : \n            self.base_model_class = RandomForestClassifier()\n         : \n            self.base_model_class = base_model_class\n        self.model_params = model_params  model_params  {}\n        self.optimize_for = optimize_for\n        self.use_grid_search = use_grid_search\n        self.models = {}\n         i  () : \n             self.model_params !=   self.use_grid_search: \n                self.models[(i+)] = GridSearchCV(self.base_model_class(), self.model_params, scoring=self.optimize_for, cv=, verbose=)\n             : \n                self.models[(i+)] = self.base_model_class(**self.model_params)\n\n     ():\n        \n        questions = np.unique(y[])\n        X.sort_index(inplace=)\n        y.sort_index(inplace=)\n         question  questions:\n            group = get_level_group_question(question)\n            X_arranged = X.xs(group, level=).fillna()\n            Y_arranged = y[y[] == (question)][[]]\n            X_arranged, Y_arranged = X_arranged.align(Y_arranged, join=, axis=)\n            Y_arranged = np.ravel(Y_arranged.values)\n            X_arranged = X_arranged.values\n            self.models[(question)].fit(X_arranged,Y_arranged)\n             self.use_grid_search:\n                (, (question), , self.models[(question)].best_params_)\n                (, (question), , self.models[(question)].best_score_)\n            y_pred = self.models[(question)].predict(X_arranged)\n            accuracy = accuracy_score(Y_arranged, y_pred)\n            f1 = f1_score(Y_arranged, y_pred)\n            (, (question), , accuracy)\n            (, (question), , f1)\n         self\n\n     ():\n        \n        X.sort_index(inplace=)\n        predictions = []\n        session_question_tuples = []\n        unique_level_groups = (X.index.get_level_values().unique())\n         question  [i+  i  ()]:\n            group = get_level_group_question(question)\n             group  unique_level_groups :\n                X_arranged = X.xs(group, level=).fillna()\n                session_ids = X_arranged.index.get_level_values()  \n                pred = self.models[(question)].predict(X_arranged.values)\n                \n                session_question_tuples.extend([  s_id  session_ids])\n                predictions.extend(pred)\n\n        \n        predictions_df = pd.DataFrame(predictions, columns=[], index=session_question_tuples)\n        predictions_df.index.name = \n\n         predictions_df\n\n\n     ():\n        \n        y_pred = self.predict(X)\n        y_formatted = self.format_y(y)\n\n        common_index = y_formatted.index.intersection(y_pred.index)\n\n        f1 = f1_score(y_formatted.loc[common_index, ], y_pred.loc[common_index, ])\n        ()\n         f1\n\n     ():\n        \n        y_formatted = y.copy()\n        y_formatted.reset_index(inplace=)\n        y_formatted[] = y_formatted[].astype() +  + y_formatted[].astype()\n        y_formatted.set_index(, inplace=)\n        y_formatted.drop([, ], axis=, inplace=)\n        y_formatted.rename(columns={: }, inplace=)\n\n         y_formatted\n</code></pre>\n<p>I just can't understand what is wrong with my submissions … any help would be greatly appreciated !</p>",
      "rawMarkdown": "Hi fellow Kagglers,\n\nI need a little help with submitting, it seems that my submission format is wrong, here's how I submit :\n\n```python\n`for (test, sample_submission) in iter_test:\n    Augmenter = AttributesAugmenter(unique_events,click_events,unique_rooms, cols)\n    #print(test.columns)\n    test = preprocess_df(test)\n    gc.collect()\n    X = Augmenter.transform(test)\n    del test \n    gc.collect()\n    prediction = loaded_model.predict(X)\n    del X\n    gc.collect()\n    #print(sample_submission.head())\n    ## users make predictions here using the test data\n    sample_submission[\"correct\"] = prediction[\"correct\"].values\n    #print(sample_submission.head())\n    ## env.predict appends the session+level sample_submission to the overall\n    ## submission\n    env.predict(sample_submission)\n````\n\nAnd this is my modelwrapper class :\n\n```python\n class MultiModelWrapper(BaseEstimator, ClassifierMixin):\n    def __init__(self, base_model_class = None, model_params=None, optimize_for='f1', use_grid_search=True):\n        if base_model_class == None : \n            self.base_model_class = RandomForestClassifier()\n        else : \n            self.base_model_class = base_model_class\n        self.model_params = model_params if model_params else {}\n        self.optimize_for = optimize_for\n        self.use_grid_search = use_grid_search\n        self.models = {}\n        for i in range(18) : \n            if self.model_params != None and self.use_grid_search: \n                self.models[str(i+1)] = GridSearchCV(self.base_model_class(), self.model_params, scoring=self.optimize_for, cv=5, verbose=3)\n            else : \n                self.models[str(i+1)] = self.base_model_class(**self.model_params)\n\n    def fit(self, X, y):\n        # Assuming y is formatted properly\n        questions = np.unique(y['question'])\n        X.sort_index(inplace=True)\n        y.sort_index(inplace=True)\n        for question in questions:\n            group = get_level_group_question(question)\n            X_arranged = X.xs(group, level='level_group').fillna(0)\n            Y_arranged = y[y['question'] == int(question)][['correct']]\n            X_arranged, Y_arranged = X_arranged.align(Y_arranged, join='inner', axis=0)\n            Y_arranged = np.ravel(Y_arranged.values)\n            X_arranged = X_arranged.values\n            self.models[str(question)].fit(X_arranged,Y_arranged)\n            if self.use_grid_search:\n                print(\"Best parameters for question \", str(question), \": \", self.models[str(question)].best_params_)\n                print(\"Best score for question \", str(question), \": \", self.models[str(question)].best_score_)\n            y_pred = self.models[str(question)].predict(X_arranged)\n            accuracy = accuracy_score(Y_arranged, y_pred)\n            f1 = f1_score(Y_arranged, y_pred)\n            print(\"Accuracy for question \", str(question), \": \", accuracy)\n            print(\"F1 score for question \", str(question), \": \", f1)\n        return self\n\n    def predict(self, X):\n        \"\"\"\n        Predict method modified to output a DataFrame in the required competition format. \n        It iterates over each question, extracts the corresponding sessions, and predicts their correctness.\n        \"\"\"\n        X.sort_index(inplace=True)\n        predictions = []\n        session_question_tuples = []\n        unique_level_groups = list(X.index.get_level_values('level_group').unique())\n        for question in [i+1 for i in range(18)]:\n            group = get_level_group_question(question)\n            if group in unique_level_groups :\n                X_arranged = X.xs(group, level='level_group').fillna(0)\n                session_ids = X_arranged.index.get_level_values('session_id')  # extract session_ids from the subset\n                pred = self.models[str(question)].predict(X_arranged.values)\n                # Building list of strings \"{session_id}_q{question}\" for prediction\n                session_question_tuples.extend([f\"{s_id}_q{question}\" for s_id in session_ids])\n                predictions.extend(pred)\n\n        # Creating DataFrame from predictions with index as required by competition\n        predictions_df = pd.DataFrame(predictions, columns=['correct'], index=session_question_tuples)\n        predictions_df.index.name = 'session_id'\n\n        return predictions_df\n\n\n    def score(self, X, y):\n        \"\"\"\n        The score method now uses the f1_score directly on all predictions against all targets. \n        Before comparing, it filters the predictions and targets to only include rows that are present in both.\n        \"\"\"\n        y_pred = self.predict(X)\n        y_formatted = self.format_y(y)\n\n        common_index = y_formatted.index.intersection(y_pred.index)\n\n        f1 = f1_score(y_formatted.loc[common_index, 'correct'], y_pred.loc[common_index, 'correct'])\n        print(f\"Overall F1 Score: {f1}\")\n        return f1\n\n    def format_y(self, y):\n        \"\"\"\n        Format_y method added to transform the target DataFrame into the competition format.\n        \"\"\"\n        y_formatted = y.copy()\n        y_formatted.reset_index(inplace=True)\n        y_formatted['index'] = y_formatted['session_id'].astype(str) + \"_q\" + y_formatted['question'].astype(str)\n        y_formatted.set_index('index', inplace=True)\n        y_formatted.drop(['session_id', 'question'], axis=1, inplace=True)\n        y_formatted.rename(columns={'index': 'session_id'}, inplace=True)\n\n        return y_formatted\n```\n\nI just can't understand what is wrong with my submissions ... any help would be greatly appreciated !"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2311385": "Hi fellow Kagglers,\n\nI need a little help with submitting, it seems that my submission format is wrong, here's how I submit :\n\n```python\n`for (test, sample_submission) in iter_test:\n    Augmenter = AttributesAugmenter(unique_events,click_events,unique_rooms, cols)\n    #print(test.columns)\n    test = preprocess_df(test)\n    gc.collect()\n    X = Augmenter.transform(test)\n    del test \n    gc.collect()\n    prediction = loaded_model.predict(X)\n    del X\n    gc.collect()\n    #print(sample_submission.head())\n    ## users make predictions here using the test data\n    sample_submission[\"correct\"] = prediction[\"correct\"].values\n    #print(sample_submission.head())\n    ## env.predict appends the session+level sample_submission to the overall\n    ## submission\n    env.predict(sample_submission)\n````\n\nAnd this is my modelwrapper class :\n\n```python\n class MultiModelWrapper(BaseEstimator, ClassifierMixin):\n    def __init__(self, base_model_class = None, model_params=None, optimize_for='f1', use_grid_search=True):\n        if base_model_class == None : \n            self.base_model_class = RandomForestClassifier()\n        else : \n            self.base_model_class = base_model_class\n        self.model_params = model_params if model_params else {}\n        self.optimize_for = optimize_for\n        self.use_grid_search = use_grid_search\n        self.models = {}\n        for i in range(18) : \n            if self.model_params != None and self.use_grid_search: \n                self.models[str(i+1)] = GridSearchCV(self.base_model_class(), self.model_params, scoring=self.optimize_for, cv=5, verbose=3)\n            else : \n                self.models[str(i+1)] = self.base_model_class(**self.model_params)\n\n    def fit(self, X, y):\n        # Assuming y is formatted properly\n        questions = np.unique(y['question'])\n        X.sort_index(inplace=True)\n        y.sort_index(inplace=True)\n        for question in questions:\n            group = get_level_group_question(question)\n            X_arranged = X.xs(group, level='level_group').fillna(0)\n            Y_arranged = y[y['question'] == int(question)][['correct']]\n            X_arranged, Y_arranged = X_arranged.align(Y_arranged, join='inner', axis=0)\n            Y_arranged = np.ravel(Y_arranged.values)\n            X_arranged = X_arranged.values\n            self.models[str(question)].fit(X_arranged,Y_arranged)\n            if self.use_grid_search:\n                print(\"Best parameters for question \", str(question), \": \", self.models[str(question)].best_params_)\n                print(\"Best score for question \", str(question), \": \", self.models[str(question)].best_score_)\n            y_pred = self.models[str(question)].predict(X_arranged)\n            accuracy = accuracy_score(Y_arranged, y_pred)\n            f1 = f1_score(Y_arranged, y_pred)\n            print(\"Accuracy for question \", str(question), \": \", accuracy)\n            print(\"F1 score for question \", str(question), \": \", f1)\n        return self\n\n    def predict(self, X):\n        \"\"\"\n        Predict method modified to output a DataFrame in the required competition format. \n        It iterates over each question, extracts the corresponding sessions, and predicts their correctness.\n        \"\"\"\n        X.sort_index(inplace=True)\n        predictions = []\n        session_question_tuples = []\n        unique_level_groups = list(X.index.get_level_values('level_group').unique())\n        for question in [i+1 for i in range(18)]:\n            group = get_level_group_question(question)\n            if group in unique_level_groups :\n                X_arranged = X.xs(group, level='level_group').fillna(0)\n                session_ids = X_arranged.index.get_level_values('session_id')  # extract session_ids from the subset\n                pred = self.models[str(question)].predict(X_arranged.values)\n                # Building list of strings \"{session_id}_q{question}\" for prediction\n                session_question_tuples.extend([f\"{s_id}_q{question}\" for s_id in session_ids])\n                predictions.extend(pred)\n\n        # Creating DataFrame from predictions with index as required by competition\n        predictions_df = pd.DataFrame(predictions, columns=['correct'], index=session_question_tuples)\n        predictions_df.index.name = 'session_id'\n\n        return predictions_df\n\n\n    def score(self, X, y):\n        \"\"\"\n        The score method now uses the f1_score directly on all predictions against all targets. \n        Before comparing, it filters the predictions and targets to only include rows that are present in both.\n        \"\"\"\n        y_pred = self.predict(X)\n        y_formatted = self.format_y(y)\n\n        common_index = y_formatted.index.intersection(y_pred.index)\n\n        f1 = f1_score(y_formatted.loc[common_index, 'correct'], y_pred.loc[common_index, 'correct'])\n        print(f\"Overall F1 Score: {f1}\")\n        return f1\n\n    def format_y(self, y):\n        \"\"\"\n        Format_y method added to transform the target DataFrame into the competition format.\n        \"\"\"\n        y_formatted = y.copy()\n        y_formatted.reset_index(inplace=True)\n        y_formatted['index'] = y_formatted['session_id'].astype(str) + \"_q\" + y_formatted['question'].astype(str)\n        y_formatted.set_index('index', inplace=True)\n        y_formatted.drop(['session_id', 'question'], axis=1, inplace=True)\n        y_formatted.rename(columns={'index': 'session_id'}, inplace=True)\n\n        return y_formatted\n```\n\nI just can't understand what is wrong with my submissions ... any help would be greatly appreciated !"
  }
}