{"cells":[{"cell_type":"markdown","id":"c3f0842b","metadata":{},"source":"# LANL Earthquake Prediction: End-to-End Tutorial\n\nThis notebook documents a full production-style pipeline for the Kaggle LANL Earthquake Prediction task: data validation, leakage diagnostics, feature engineering, leakage-aware validation, ensemble modeling, and submission generation."},{"cell_type":"markdown","id":"d7cfc0d1","metadata":{},"source":"## 1. Competition Objective and Metric\n\n- **Goal**: predict `time_to_failure` for each test segment from acoustic signals.\n- **Official metric**: Mean Absolute Error (MAE).\n- **Key risk**: leakage if segments from the same earthquake event appear in both train and validation."},{"cell_type":"code","execution_count":1,"id":"4b4afba8","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:11.880905Z","iopub.status.busy":"2026-06-12T13:14:11.880766Z","iopub.status.idle":"2026-06-12T13:14:14.116251Z","shell.execute_reply":"2026-06-12T13:14:14.115594Z"}},"outputs":[],"source":"from pathlib import Path\nimport json\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nPROJECT_ROOT = Path.cwd().resolve().parent\nARTIFACTS = PROJECT_ROOT / 'artifacts'\nREPORTS = ARTIFACTS / 'reports'\nFEATURES = ARTIFACTS / 'features'\nSUBMISSIONS = ARTIFACTS / 'submissions'\n\nprint('Project root:', PROJECT_ROOT)"},{"cell_type":"markdown","id":"721abe5f","metadata":{},"source":"## 2. Data + Leakage Diagnostics\nThe following report is created from real pipeline execution and summarizes segment/event structure."},{"cell_type":"code","execution_count":2,"id":"ab545835","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:14.118113Z","iopub.status.busy":"2026-06-12T13:14:14.117867Z","iopub.status.idle":"2026-06-12T13:14:14.124833Z","shell.execute_reply":"2026-06-12T13:14:14.124149Z"}},"outputs":[],"source":"eda_report_path = REPORTS / 'eda_report.json'\neda = json.loads(eda_report_path.read_text())\neda"},{"cell_type":"markdown","id":"e2e1925c","metadata":{},"source":"## 3. Engineered Feature Matrix\nWe use robust statistical + spectral features per 150,000-row segment."},{"cell_type":"code","execution_count":3,"id":"5f28c4be","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:14.126219Z","iopub.status.busy":"2026-06-12T13:14:14.126076Z","iopub.status.idle":"2026-06-12T13:14:14.195248Z","shell.execute_reply":"2026-06-12T13:14:14.194632Z"}},"outputs":[],"source":"train_features = pd.read_parquet(FEATURES / 'train_features.parquet')\ntest_features = pd.read_parquet(FEATURES / 'test_features.parquet')\nprint('Train shape:', train_features.shape)\nprint('Test shape:', test_features.shape)\ntrain_features.head()"},{"cell_type":"code","execution_count":4,"id":"d97276ce","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:14.196734Z","iopub.status.busy":"2026-06-12T13:14:14.196584Z","iopub.status.idle":"2026-06-12T13:14:14.551629Z","shell.execute_reply":"2026-06-12T13:14:14.551023Z"}},"outputs":[],"source":"plt.figure(figsize=(10, 4))\nsns.histplot(train_features['target'], bins=80, kde=True)\nplt.title('Target Distribution: time_to_failure')\nplt.show()"},{"cell_type":"markdown","id":"c9ea370b","metadata":{},"source":"## 4. Cross-Validation Results\nWe train CatBoost + LightGBM + XGBoost with purged group-aware folds (event IDs)."},{"cell_type":"code","execution_count":5,"id":"9150ff62","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:14.553524Z","iopub.status.busy":"2026-06-12T13:14:14.553312Z","iopub.status.idle":"2026-06-12T13:14:14.56279Z","shell.execute_reply":"2026-06-12T13:14:14.5623Z"}},"outputs":[],"source":"cv = pd.read_csv(REPORTS / 'cv_metrics.csv')\ncv"},{"cell_type":"code","execution_count":6,"id":"cfbbc495","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:14.564522Z","iopub.status.busy":"2026-06-12T13:14:14.564373Z","iopub.status.idle":"2026-06-12T13:14:14.574729Z","shell.execute_reply":"2026-06-12T13:14:14.574142Z"}},"outputs":[],"source":"cv.groupby('model', as_index=False)['mae'].agg(['mean', 'std'])"},{"cell_type":"markdown","id":"8fc3373c","metadata":{},"source":"## 5. Submission File\nThe pipeline generates a real `submission.csv` using weighted ensemble averaging."},{"cell_type":"code","execution_count":7,"id":"d095377f","metadata":{"execution":{"iopub.execute_input":"2026-06-12T13:14:14.576136Z","iopub.status.busy":"2026-06-12T13:14:14.575998Z","iopub.status.idle":"2026-06-12T13:14:14.585738Z","shell.execute_reply":"2026-06-12T13:14:14.585119Z"}},"outputs":[],"source":"submission_files = sorted(SUBMISSIONS.glob('submission_*.csv'))\nlatest_submission = submission_files[-1]\nsubmission_df = pd.read_csv(latest_submission)\nprint('Latest submission:', latest_submission.name)\nsubmission_df.head()"},{"cell_type":"markdown","id":"9b111418","metadata":{},"source":"## 6. Reproducibility Commands\nRun from project root:\n\n```bash\nuv sync\nuv run lanl-eq run-all\n```"}],"metadata":{"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.12.10"}},"nbformat":4,"nbformat_minor":5}