{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.14","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":84493,"databundleVersionId":9871156,"sourceType":"competition"}],"dockerImageVersionId":30786,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Starting Over in the Jane Street Competition**\n\nToday, I returned to one of the first [notebooks](https://www.kaggle.com/code/ravi20076/janestreet2024-baseline-train-v1) I studied in this competition. I started by passing a snippet of code to ChatGPT and asking for explanations. According to ChatGPT, the notebook uses the LightGBM regression model.\n\nTo better understand how this model works, I asked ChatGPT to explain the LightGBM regression model to me.\n\nChatGPT provided a clear explanation of the LightGBM algorithm, but I still wanted to delve deeper into the subject. I then found an interesting [article](https://towardsdatascience.com/visually-understand-xgboost-lightgbm-and-catboost-regularization-parameters-aa12abcd4c17) on Medium.\n\nTo better organize my ideas, I created a [mind map](https://www.dropbox.com/scl/fi/4p59kz290ybmmptd8gn1v/Screenshot_20241203_174909.jpg?rlkey=vp4q7t2qyigiyjqs2zjast2ij&st=lcy4fxbr&dl=0) based on this article.\n\nThe next step is to implement this methodology in the competition I am participating in. For this implementation, [I used ChatGPT](https://chatgpt.com/share/674f7013-748c-8008-9f9a-44f112d14909).\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\n# Initialize a list to hold samples from each file\nsamples = []\n# Load a sample from each file\nfor i in range(10):\n    file_path = f\"/kaggle/input/jane-street-real-time-market-data-forecasting/train.parquet/partition_id={i}/part-0.parquet\"\n    chunk = pd.read_parquet(file_path)\n    \n    # Take a sample of the data (adjust sample size as needed)\n    sample_chunk = chunk.sample(n=100000, random_state=42)  # For example, 100 rows\n    samples.append(sample_chunk)\n# Concatenate all samples into one DataFrame if needed\nsample_df = pd.concat(samples, ignore_index=True)\n","metadata":{"execution":{"iopub.status.busy":"2024-12-03T22:05:54.180823Z","iopub.execute_input":"2024-12-03T22:05:54.181636Z","iopub.status.idle":"2024-12-03T22:07:28.410531Z","shell.execute_reply.started":"2024-12-03T22:05:54.181519Z","shell.execute_reply":"2024-12-03T22:07:28.408880Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sample_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:07:28.413681Z","iopub.execute_input":"2024-12-03T22:07:28.414228Z","iopub.status.idle":"2024-12-03T22:07:28.468170Z","shell.execute_reply.started":"2024-12-03T22:07:28.414160Z","shell.execute_reply":"2024-12-03T22:07:28.466880Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Prepare data","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport tensorflow as tf\nfrom tensorflow.keras import layers, models\n# Separate features and responders\nfeatures = sample_df.filter(regex='^feature_')\nresponders = sample_df.filter(regex='^responder_')\n# Convert to numpy arrays for TensorFlow\nX = features.values  # Features for input\ny = responders.values  # Responders for output\nX = np.nan_to_num(X, nan=0.0, posinf=0.0, neginf=0.0)\ny = np.nan_to_num(y, nan=0.0, posinf=0.0, neginf=0.0)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:07:28.469627Z","iopub.execute_input":"2024-12-03T22:07:28.470139Z","iopub.status.idle":"2024-12-03T22:07:30.182591Z","shell.execute_reply.started":"2024-12-03T22:07:28.470086Z","shell.execute_reply":"2024-12-03T22:07:30.181365Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# LightGBM","metadata":{}},{"cell_type":"code","source":"import lightgbm as lgb\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_squared_error\n\n# Separate features and target\nfeatures = sample_df.filter(regex='^feature_')\ntarget = sample_df['responder_6']  # Only responder_6\n\n# Replace NaN, inf values\nfeatures = features.fillna(0).replace([np.inf, -np.inf], 0)\ntarget = target.fillna(0).replace([np.inf, -np.inf], 0)\n\n# Train-test split\nX_train, X_valid, y_train, y_valid = train_test_split(\n    features, target, test_size=0.2, random_state=42\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:07:30.184670Z","iopub.execute_input":"2024-12-03T22:07:30.185407Z","iopub.status.idle":"2024-12-03T22:07:32.355850Z","shell.execute_reply.started":"2024-12-03T22:07:30.185331Z","shell.execute_reply":"2024-12-03T22:07:32.354557Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_data = lgb.Dataset(X_train, label=y_train)\nvalid_data = lgb.Dataset(X_valid, label=y_valid, reference=train_data)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:07:32.359071Z","iopub.execute_input":"2024-12-03T22:07:32.359414Z","iopub.status.idle":"2024-12-03T22:07:32.374780Z","shell.execute_reply.started":"2024-12-03T22:07:32.359382Z","shell.execute_reply":"2024-12-03T22:07:32.373311Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"params = {\n    'objective': 'regression',\n    'metric': 'rmse',  # Root Mean Squared Error\n    'boosting_type': 'gbdt',\n    'learning_rate': 0.5,\n    'num_leaves': 31,  # Complexity control\n    'max_depth': -1,  # No limit\n    'min_data_in_leaf': 20,\n    'feature_fraction': 0.8,  # Random feature selection\n    'bagging_fraction': 0.8,  # Random sample selection\n    'bagging_freq': 5,\n    'alpha': 0.5,   # Quantile loss regularization\n    #'gamma': 0.1,   # Minimum loss reduction\n    'lambda_l1': 0.1,  # L1 regularization\n    'lambda_l2': 0.5,  # L2 regularization\n    'seed': 42\n}\n\nmodel = lgb.train(\n    params,\n    train_data,\n    valid_sets=[train_data, valid_data],\n    num_boost_round=1000#,\n    #early_stopping_rounds=50,\n    #verbose_eval=50\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:07:32.376301Z","iopub.execute_input":"2024-12-03T22:07:32.376688Z","iopub.status.idle":"2024-12-03T22:08:50.151778Z","shell.execute_reply.started":"2024-12-03T22:07:32.376654Z","shell.execute_reply":"2024-12-03T22:08:50.150455Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predict on validation set\ny_pred = model.predict(X_valid)\nrmse = mean_squared_error(y_valid, y_pred, squared=False)\nprint(f\"Validation RMSE: {rmse}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:08:50.153212Z","iopub.execute_input":"2024-12-03T22:08:50.153564Z","iopub.status.idle":"2024-12-03T22:08:54.214539Z","shell.execute_reply.started":"2024-12-03T22:08:50.153531Z","shell.execute_reply":"2024-12-03T22:08:54.213239Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(y_pred.shape)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:09:27.650533Z","iopub.execute_input":"2024-12-03T22:09:27.650961Z","iopub.status.idle":"2024-12-03T22:09:27.657486Z","shell.execute_reply.started":"2024-12-03T22:09:27.650928Z","shell.execute_reply":"2024-12-03T22:09:27.656014Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Submission\n\nSee [Jane Street RMF Demo Submission](https://www.kaggle.com/code/ryanholbrook/jane-street-rmf-demo-submission) for details","metadata":{}},{"cell_type":"code","source":"import os\nimport polars as pl\nimport kaggle_evaluation.jane_street_inference_server","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:08:54.227054Z","iopub.execute_input":"2024-12-03T22:08:54.227382Z","iopub.status.idle":"2024-12-03T22:08:54.760782Z","shell.execute_reply.started":"2024-12-03T22:08:54.227338Z","shell.execute_reply":"2024-12-03T22:08:54.759689Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import polars as pl\nimport numpy as np\n# Assuming `model` is your trained model\n# Assuming features required by the model are named 'feature_00', 'feature_01', etc.\ndef predict(test: pl.DataFrame, lags: pl.DataFrame | None) -> pl.DataFrame | pd.DataFrame:\n    \"\"\"Make a prediction.\"\"\"\n    global lags_\n    if lags is not None:\n        lags_ = lags\n    # Extract the features for the model input\n    feature_columns = [col for col in test.columns if col.startswith(\"feature_\")]\n    features = test.select(feature_columns).to_numpy()  # Convert to numpy array for model input\n    features = np.nan_to_num(features, nan=0.0, posinf=0.0, neginf=0.0)\n    # Generate predictions using the model\n    responder_6_predictions = model.predict(features)\n    # responder_6_predictions = model_predictions[:, 6]  # Assuming responder_6 is at index 6\n    # Create a new Polars DataFrame with row_id and responder_6 predictions\n    predictions = test.select(\"row_id\").with_columns(\n        pl.Series(\"responder_6\", responder_6_predictions)\n    )\n    print(predictions)\n    # Ensure the output format and length requirements\n    if isinstance(predictions, pl.DataFrame):\n        assert predictions.columns == ['row_id', 'responder_6']\n    elif isinstance(predictions, pd.DataFrame):\n        assert (predictions.columns == ['row_id', 'responder_6']).all()\n    else:\n        raise TypeError('The predict function must return a DataFrame')\n    \n    assert len(predictions) == len(test)\n    return predictions","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:12:16.641224Z","iopub.execute_input":"2024-12-03T22:12:16.641666Z","iopub.status.idle":"2024-12-03T22:12:16.651162Z","shell.execute_reply.started":"2024-12-03T22:12:16.641627Z","shell.execute_reply":"2024-12-03T22:12:16.649865Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"inference_server = kaggle_evaluation.jane_street_inference_server.JSInferenceServer(predict)\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n    inference_server.serve()\nelse:\n    inference_server.run_local_gateway(\n        (\n            '/kaggle/input/jane-street-real-time-market-data-forecasting/test.parquet',\n            '/kaggle/input/jane-street-real-time-market-data-forecasting/lags.parquet',\n        )\n    )","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-03T22:12:20.944223Z","iopub.execute_input":"2024-12-03T22:12:20.944645Z","iopub.status.idle":"2024-12-03T22:13:59.229075Z","shell.execute_reply.started":"2024-12-03T22:12:20.944609Z","shell.execute_reply":"2024-12-03T22:13:59.227351Z"}},"outputs":[],"execution_count":null}]}