{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# XGBoost Baseline - LB 0.678\nDalam buku catatan ini kami menyajikan baseline XGBoost. Kami melatih model GroupKFold untuk masing-masing dari 18 pertanyaan. Skor CV kami adalah 0,678. Kami menyimpulkan pengujian menggunakan salah satu model KFold kami. Kami dapat meningkatkan CV dan LB kami dengan merekayasa lebih banyak fitur untuk xgboost kami dan/atau mencoba model yang berbeda (seperti model ML dan/atau RNN dan/atau Transformer lainnya). Kami juga dapat meningkatkan LB kami dengan menggunakan lebih banyak model KFold ATAU melatih satu model menggunakan semua data (dan hyperparameter yang kami temukan dari validasi silang KFold kami).\n**UPDATE** Pada 20 Maret 2023, Kaggle menggandakan ukuran data train. Oleh karena itu kami memperbarui notebook ini untuk menghindari kesalahan memori. Kami mencapai ini dengan membaca data train dalam potongan dan rekayasa fitur dalam potongan. Perhatikan bahwa cara lain untuk menghindari kesalahan memori adalah dengan menggunakan dua notebook. Latih model dalam satu notebook yang memiliki RAM 32GB (dan simpan model), lalu kirimkan notebook RAM 8GB yang diperlukan (dengan model yang dimuat) sebagai notebook kedua. (Diskusi [di sini]).\n\n[1]: https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/386218","metadata":{"papermill":{"duration":0.005932,"end_time":"2023-02-07T00:59:58.147501","exception":false,"start_time":"2023-02-07T00:59:58.141569","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import pandas as pd, numpy as np, gc\nfrom sklearn.model_selection import KFold, GroupKFold\nfrom xgboost import XGBClassifier\nfrom sklearn.metrics import f1_score","metadata":{"papermill":{"duration":1.027875,"end_time":"2023-02-07T00:59:59.180261","exception":false,"start_time":"2023-02-07T00:59:58.152386","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:48:10.131018Z","iopub.execute_input":"2023-03-23T08:48:10.131543Z","iopub.status.idle":"2023-03-23T08:48:11.178035Z","shell.execute_reply.started":"2023-03-23T08:48:10.131442Z","shell.execute_reply":"2023-03-23T08:48:11.176498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Train Data and Labels\n\nPada 20 Maret 2023, Kaggle menggandakan ukuran data train (diskusi [di sini]). Data train sekarang 4,7GB! Untuk menghindari kesalahan memori, kita akan membaca data train sebagai 10 buah dan fitur insinyur setiap bagian sebelum membaca bagian berikutnya. Ini berfungsi karena rekayasa fitur mengecilkan ukuran setiap bagian.\n\n[1]: https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/396202","metadata":{"papermill":{"duration":0.004542,"end_time":"2023-02-07T00:59:59.189777","exception":false,"start_time":"2023-02-07T00:59:59.185235","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# READ USER ID ONLY\ntmp = pd.read_csv(\"/kaggle/input/predict-student-performance-from-game-play/train.csv\",usecols=[0])\ntmp = tmp.groupby('session_id').session_id.agg('count')\n\n# COMPUTE READS AND SKIPS\nPIECES = 10\nCHUNK = int( np.ceil(len(tmp)/PIECES) )\n\nreads = []\nskips = [0]\nfor k in range(PIECES):\n    a = k*CHUNK\n    b = (k+1)*CHUNK\n    if b>len(tmp): b=len(tmp)\n    r = tmp.iloc[a:b].sum()\n    reads.append(r)\n    skips.append(skips[-1]+r)\n    \nprint(f'To avoid memory error, we will read train in {PIECES} pieces of sizes:')\nprint(reads)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-23T08:48:11.180542Z","iopub.execute_input":"2023-03-23T08:48:11.180891Z","iopub.status.idle":"2023-03-23T08:49:44.013988Z","shell.execute_reply.started":"2023-03-23T08:48:11.180851Z","shell.execute_reply":"2023-03-23T08:49:44.012804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv', nrows=reads[0])\nprint('Train size of first piece:', train.shape )\ntrain.head()","metadata":{"papermill":{"duration":59.284316,"end_time":"2023-02-07T01:00:58.478743","exception":false,"start_time":"2023-02-07T00:59:59.194427","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:49:44.015731Z","iopub.execute_input":"2023-03-23T08:49:44.016445Z","iopub.status.idle":"2023-03-23T08:49:53.996908Z","shell.execute_reply.started":"2023-03-23T08:49:44.0164Z","shell.execute_reply":"2023-03-23T08:49:53.995527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"targets = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train_labels.csv')\ntargets['session'] = targets.session_id.apply(lambda x: int(x.split('_')[0]) )\ntargets['q'] = targets.session_id.apply(lambda x: int(x.split('_')[-1][1:]) )\nprint( targets.shape )\ntargets.head()","metadata":{"papermill":{"duration":0.598155,"end_time":"2023-02-07T01:00:59.082015","exception":false,"start_time":"2023-02-07T01:00:58.48386","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:49:53.999648Z","iopub.execute_input":"2023-03-23T08:49:54.000409Z","iopub.status.idle":"2023-03-23T08:49:55.259632Z","shell.execute_reply.started":"2023-03-23T08:49:54.000364Z","shell.execute_reply":"2023-03-23T08:49:55.25845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Feature Engineer\nKami membuat fitur agregat dasar. Coba buat lebih banyak fitur untuk meningkatkan CV dan LB! Ide untuk fitur EVENTS berasal dari [sini][1]\n\n[1]: https://www.kaggle.com/code/kimtaehun/lightgbm-baseline-with-aggregated-log-data","metadata":{"papermill":{"duration":0.005196,"end_time":"2023-02-07T01:00:59.092865","exception":false,"start_time":"2023-02-07T01:00:59.087669","status":"completed"},"tags":[]}},{"cell_type":"code","source":"CATS = ['event_name', 'fqid', 'room_fqid', 'text']\nNUMS = ['elapsed_time','level','page','room_coor_x', 'room_coor_y', \n        'screen_coor_x', 'screen_coor_y', 'hover_duration']\n\n# https://www.kaggle.com/code/kimtaehun/lightgbm-baseline-with-aggregated-log-data\nEVENTS = ['navigate_click','person_click','cutscene_click','object_click',\n          'map_hover','notification_click','map_click','observation_click',\n          'checkpoint']","metadata":{"papermill":{"duration":0.014685,"end_time":"2023-02-07T01:00:59.112856","exception":false,"start_time":"2023-02-07T01:00:59.098171","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:49:55.261022Z","iopub.execute_input":"2023-03-23T08:49:55.262038Z","iopub.status.idle":"2023-03-23T08:49:55.269195Z","shell.execute_reply.started":"2023-03-23T08:49:55.261981Z","shell.execute_reply":"2023-03-23T08:49:55.267504Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def feature_engineer(train):\n    \n    dfs = []\n    for c in CATS:\n        tmp = train.groupby(['session_id','level_group'])[c].agg('nunique')\n        tmp.name = tmp.name + '_nunique'\n        dfs.append(tmp)\n    for c in NUMS:\n        tmp = train.groupby(['session_id','level_group'])[c].agg('mean')\n        tmp.name = tmp.name + '_mean'\n        dfs.append(tmp)\n    for c in NUMS:\n        tmp = train.groupby(['session_id','level_group'])[c].agg('std')\n        tmp.name = tmp.name + '_std'\n        dfs.append(tmp)\n    for c in EVENTS: \n        train[c] = (train.event_name == c).astype('int8')\n    for c in EVENTS + ['elapsed_time']:\n        tmp = train.groupby(['session_id','level_group'])[c].agg('sum')\n        tmp.name = tmp.name + '_sum'\n        dfs.append(tmp)\n    train = train.drop(EVENTS,axis=1)\n        \n    df = pd.concat(dfs,axis=1)\n    df = df.fillna(-1)\n    df = df.reset_index()\n    df = df.set_index('session_id')\n    return df","metadata":{"papermill":{"duration":0.017716,"end_time":"2023-02-07T01:00:59.136021","exception":false,"start_time":"2023-02-07T01:00:59.118305","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:49:55.271479Z","iopub.execute_input":"2023-03-23T08:49:55.27195Z","iopub.status.idle":"2023-03-23T08:49:55.285019Z","shell.execute_reply.started":"2023-03-23T08:49:55.27191Z","shell.execute_reply":"2023-03-23T08:49:55.284016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\n# PROCESS TRAIN DATA IN PIECES\nall_pieces = []\nprint(f'Processing train as {PIECES} pieces to avoid memory error... ')\nfor k in range(PIECES):\n    print(k,', ',end='')\n    SKIPS = 0\n    if k>0: SKIPS = range(1,skips[k]+1)\n    train = pd.read_csv('/kaggle/input/predict-student-performance-from-game-play/train.csv',\n                        nrows=reads[k], skiprows=SKIPS)\n    df = feature_engineer(train)\n    all_pieces.append(df)\n    \n# CONCATENATE ALL PIECES\nprint('\\n')\ndel train; gc.collect()\ndf = pd.concat(all_pieces, axis=0)\nprint('Shape of all train data after feature engineering:', df.shape )\ndf.head()","metadata":{"papermill":{"duration":34.516494,"end_time":"2023-02-07T01:01:33.658043","exception":false,"start_time":"2023-02-07T01:00:59.141549","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:49:55.288686Z","iopub.execute_input":"2023-03-23T08:49:55.290213Z","iopub.status.idle":"2023-03-23T08:56:29.432575Z","shell.execute_reply.started":"2023-03-23T08:49:55.290161Z","shell.execute_reply":"2023-03-23T08:56:29.43164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train XGBoost Model\nKami melatih satu model untuk masing-masing 18 pertanyaan. Selanjutnya, kami menggunakan data dari 'level_groups = '0-4'' untuk melatih model pertanyaan 1-3, dan 'kelompok tingkat '5-12' untuk melatih pertanyaan 4 hingga 13 dan 'kelompok tingkat '13-22' untuk melatih pertanyaan 14 hingga 18. Karena ini adalah data yang kami dapatkan (untuk memprediksi pertanyaan terkait) dari API inferensi Kaggle selama inferensi pengujian. Kami dapat meningkatkan model kami dengan menyimpan data pengguna sebelumnya dari 'tingkat_grup' sebelumnya dan menggunakannya untuk memprediksi 'tingkat_grup' di masa depan.","metadata":{"papermill":{"duration":0.00565,"end_time":"2023-02-07T01:01:33.669525","exception":false,"start_time":"2023-02-07T01:01:33.663875","status":"completed"},"tags":[]}},{"cell_type":"code","source":"FEATURES = [c for c in df.columns if c != 'level_group']\nprint('We will train with', len(FEATURES) ,'features')\nALL_USERS = df.index.unique()\nprint('We will train with', len(ALL_USERS) ,'users info')","metadata":{"papermill":{"duration":0.014699,"end_time":"2023-02-07T01:01:33.689953","exception":false,"start_time":"2023-02-07T01:01:33.675254","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:56:29.434258Z","iopub.execute_input":"2023-03-23T08:56:29.434832Z","iopub.status.idle":"2023-03-23T08:56:29.446096Z","shell.execute_reply.started":"2023-03-23T08:56:29.434798Z","shell.execute_reply":"2023-03-23T08:56:29.445071Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gkf = GroupKFold(n_splits=5)\noof = pd.DataFrame(data=np.zeros((len(ALL_USERS),18)), index=ALL_USERS)\nmodels = {}\n\n# COMPUTE CV SCORE WITH 5 GROUP K FOLD\nfor i, (train_index, test_index) in enumerate(gkf.split(X=df, groups=df.index)):\n    print('#'*25)\n    print('### Fold',i+1)\n    print('#'*25)\n    \n    xgb_params = {\n    'objective' : 'binary:logistic',\n    'eval_metric':'logloss',\n    'learning_rate': 0.05,\n    'max_depth': 4,\n    'n_estimators': 1000,\n    'early_stopping_rounds': 50,\n    'tree_method':'hist',\n    'subsample':0.8,\n    'colsample_bytree': 0.4,\n    'use_label_encoder' : False}\n    \n    # ITERATE THRU QUESTIONS 1 THRU 18\n    for t in range(1,19):\n        \n        # USE THIS TRAIN DATA WITH THESE QUESTIONS\n        if t<=3: grp = '0-4'\n        elif t<=13: grp = '5-12'\n        elif t<=22: grp = '13-22'\n            \n        # TRAIN DATA\n        train_x = df.iloc[train_index]\n        train_x = train_x.loc[train_x.level_group == grp]\n        train_users = train_x.index.values\n        train_y = targets.loc[targets.q==t].set_index('session').loc[train_users]\n        \n        # VALID DATA\n        valid_x = df.iloc[test_index]\n        valid_x = valid_x.loc[valid_x.level_group == grp]\n        valid_users = valid_x.index.values\n        valid_y = targets.loc[targets.q==t].set_index('session').loc[valid_users]\n        \n        # TRAIN MODEL        \n        clf =  XGBClassifier(**xgb_params)\n        clf.fit(train_x[FEATURES].astype('float32'), train_y['correct'],\n                eval_set=[ (valid_x[FEATURES].astype('float32'), valid_y['correct']) ],\n                verbose=0)\n        print(f'{t}({clf.best_ntree_limit}), ',end='')\n        \n        # SAVE MODEL, PREDICT VALID OOF\n        models[f'{grp}_{t}'] = clf\n        oof.loc[valid_users, t-1] = clf.predict_proba(valid_x[FEATURES].astype('float32'))[:,1]\n        \n    print()","metadata":{"papermill":{"duration":69.877213,"end_time":"2023-02-07T01:02:43.57299","exception":false,"start_time":"2023-02-07T01:01:33.695777","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:56:29.447673Z","iopub.execute_input":"2023-03-23T08:56:29.448619Z","iopub.status.idle":"2023-03-23T08:59:07.59677Z","shell.execute_reply.started":"2023-03-23T08:56:29.448578Z","shell.execute_reply":"2023-03-23T08:59:07.595909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Compute CV Score\nKita perlu mengubah probabilitas prediksi menjadi '1s' dan '0s'. Metrik kompetisi adalah Skor F1 yang merupakan rata-rata harmonik presisi dan daya ingat. Mari kita temukan ambang optimal untuk 'p > threshold', kapan harus memprediksi '1' dan kapan harus memprediksi '0' untuk memaksimalkan Skor F1.","metadata":{"papermill":{"duration":0.011241,"end_time":"2023-02-07T01:02:43.59638","exception":false,"start_time":"2023-02-07T01:02:43.585139","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# PUT TRUE LABELS INTO DATAFRAME WITH 18 COLUMNS\ntrue = oof.copy()\nfor k in range(18):\n    # GET TRUE LABELS\n    tmp = targets.loc[targets.q == k+1].set_index('session').loc[ALL_USERS]\n    true[k] = tmp.correct.values","metadata":{"execution":{"iopub.status.busy":"2023-03-23T08:59:07.602772Z","iopub.execute_input":"2023-03-23T08:59:07.604992Z","iopub.status.idle":"2023-03-23T08:59:07.731923Z","shell.execute_reply.started":"2023-03-23T08:59:07.60495Z","shell.execute_reply":"2023-03-23T08:59:07.730562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# FIND BEST THRESHOLD TO CONVERT PROBS INTO 1s AND 0s\nscores = []; thresholds = []\nbest_score = 0; best_threshold = 0\n\nfor threshold in np.arange(0.4,0.81,0.01):\n    print(f'{threshold:.02f}, ',end='')\n    preds = (oof.values.reshape((-1))>threshold).astype('int')\n    m = f1_score(true.values.reshape((-1)), preds, average='macro')   \n    scores.append(m)\n    thresholds.append(threshold)\n    if m>best_score:\n        best_score = m\n        best_threshold = threshold","metadata":{"execution":{"iopub.status.busy":"2023-03-23T08:59:07.733269Z","iopub.execute_input":"2023-03-23T08:59:07.733721Z","iopub.status.idle":"2023-03-23T08:59:16.328836Z","shell.execute_reply.started":"2023-03-23T08:59:07.733683Z","shell.execute_reply":"2023-03-23T08:59:16.327603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# PLOT THRESHOLD VS. F1_SCORE\nplt.figure(figsize=(20,5))\nplt.plot(thresholds,scores,'-o',color='blue')\nplt.scatter([best_threshold], [best_score], color='blue', s=300, alpha=1)\nplt.xlabel('Threshold',size=14)\nplt.ylabel('Validation F1 Score',size=14)\nplt.title(f'Threshold vs. F1_Score with Best F1_Score = {best_score:.3f} at Best Threshold = {best_threshold:.3}',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-23T08:59:16.33063Z","iopub.execute_input":"2023-03-23T08:59:16.330998Z","iopub.status.idle":"2023-03-23T08:59:16.604113Z","shell.execute_reply.started":"2023-03-23T08:59:16.330966Z","shell.execute_reply":"2023-03-23T08:59:16.602855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('When using optimal threshold...')\nfor k in range(18):\n        \n    # COMPUTE F1 SCORE PER QUESTION\n    m = f1_score(true[k].values, (oof[k].values>best_threshold).astype('int'), average='macro')\n    print(f'Q{k}: F1 =',m)\n    \n# COMPUTE F1 SCORE OVERALL\nm = f1_score(true.values.reshape((-1)), (oof.values.reshape((-1))>best_threshold).astype('int'), average='macro')\nprint('==> Overall F1 =',m)","metadata":{"papermill":{"duration":0.771134,"end_time":"2023-02-07T01:02:44.378465","exception":false,"start_time":"2023-02-07T01:02:43.607331","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:59:16.605503Z","iopub.execute_input":"2023-03-23T08:59:16.605815Z","iopub.status.idle":"2023-03-23T08:59:16.998494Z","shell.execute_reply.started":"2023-03-23T08:59:16.605788Z","shell.execute_reply":"2023-03-23T08:59:16.997436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Infer Test Data","metadata":{"papermill":{"duration":0.011075,"end_time":"2023-02-07T01:02:44.400918","exception":false,"start_time":"2023-02-07T01:02:44.389843","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# IMPORT KAGGLE API\nimport jo_wilder\nenv = jo_wilder.make_env()\niter_test = env.iter_test()\n\n# CLEAR MEMORY\nimport gc\ndel targets, df, oof, true\n_ = gc.collect()","metadata":{"papermill":{"duration":0.052132,"end_time":"2023-02-07T01:02:44.464739","exception":false,"start_time":"2023-02-07T01:02:44.412607","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:59:16.999799Z","iopub.execute_input":"2023-03-23T08:59:17.000377Z","iopub.status.idle":"2023-03-23T08:59:17.154231Z","shell.execute_reply.started":"2023-03-23T08:59:17.000342Z","shell.execute_reply":"2023-03-23T08:59:17.153175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"limits = {'0-4':(1,4), '5-12':(4,14), '13-22':(14,19)}\n\nfor (test, sample_submission) in iter_test:\n    \n    # FEATURE ENGINEER TEST DATA\n    df = feature_engineer(test)\n    \n    # INFER TEST DATA\n    grp = test.level_group.values[0]\n    a,b = limits[grp]\n    for t in range(a,b):\n        clf = models[f'{grp}_{t}']\n        p = clf.predict_proba(df[FEATURES].astype('float32'))[0,1]\n        mask = sample_submission.session_id.str.contains(f'q{t}')\n        sample_submission.loc[mask,'correct'] = int( p > best_threshold )\n    \n    env.predict(sample_submission)","metadata":{"papermill":{"duration":1.002014,"end_time":"2023-02-07T01:02:45.47927","exception":false,"start_time":"2023-02-07T01:02:44.477256","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:59:17.155611Z","iopub.execute_input":"2023-03-23T08:59:17.156143Z","iopub.status.idle":"2023-03-23T08:59:17.309336Z","shell.execute_reply.started":"2023-03-23T08:59:17.156093Z","shell.execute_reply":"2023-03-23T08:59:17.308236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA submission.csv","metadata":{"papermill":{"duration":0.011427,"end_time":"2023-02-07T01:02:45.502331","exception":false,"start_time":"2023-02-07T01:02:45.490904","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df = pd.read_csv('submission.csv')\nprint( df.shape )\ndf.head()","metadata":{"papermill":{"duration":0.027432,"end_time":"2023-02-07T01:02:45.541022","exception":false,"start_time":"2023-02-07T01:02:45.51359","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:59:17.310635Z","iopub.execute_input":"2023-03-23T08:59:17.311062Z","iopub.status.idle":"2023-03-23T08:59:17.332147Z","shell.execute_reply.started":"2023-03-23T08:59:17.31103Z","shell.execute_reply":"2023-03-23T08:59:17.331138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df.correct.mean())","metadata":{"papermill":{"duration":0.020233,"end_time":"2023-02-07T01:02:45.57314","exception":false,"start_time":"2023-02-07T01:02:45.552907","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-03-23T08:59:17.333396Z","iopub.execute_input":"2023-03-23T08:59:17.334471Z","iopub.status.idle":"2023-03-23T08:59:17.340116Z","shell.execute_reply.started":"2023-03-23T08:59:17.33443Z","shell.execute_reply":"2023-03-23T08:59:17.339113Z"},"trusted":true},"execution_count":null,"outputs":[]}]}