{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":117682,"databundleVersionId":14443416,"sourceType":"competition"},{"sourceId":281390877,"sourceType":"kernelVersion"}],"dockerImageVersionId":31192,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Interactive vs. Static Visualization for EDA\n\nInteractive visualization offers significant advantages over static images for Exploratory Data Analysis (EDA), especially when dealing with complex, multi-dimensional data like 3D TIFF images and their masks. Here's why:\n\n1.  **Deeper Insight**: Static images provide only a snapshot, limiting the ability to explore different angles, depths, or feature combinations. Interactive tools allow for dynamic manipulation, enabling users to uncover hidden patterns, anomalies, and relationships that might be missed in fixed views.\n2.  **Contextual Understanding**: By browsing through different slices and images, users gain a better understanding of the spatial context of features within the 3D structure. This is crucial for tasks like surface detection, where the surrounding layers influence interpretation.\n3.  **Efficiency in Exploration**: Instead of generating numerous static plots for each slice or image, an interactive viewer allows for on-the-fly exploration, drastically reducing the time and effort required to analyze the dataset.\n4.  **Validation and Quality Control**: For tasks involving annotation masks, interactive browsing is invaluable for validating the accuracy and quality of annotations across different slices and images. Users can quickly identify inconsistencies or errors in labeling.\n5.  **Engagement and Intuition**: Interactive tools foster a more engaging and intuitive exploration process, making it easier for researchers and analysts to build a strong understanding of their data.\n\n### How to Interact with the Viewer\n\nThe interactive viewer above allows you to:\n\n*   **Select Image**: Use the `Select Image:` dropdown to choose a different 3D TIFF image to view. The viewer will automatically load the new image and its corresponding mask.\n*   **Select Slice**: Use the `Select Slice:` slider to navigate through the vertical layers (z-axis) of the currently selected 3D image. As you move the slider, the corresponding 2D slices of both the image and its mask will update in real-time.\n\nSimply change the values in the dropdown or slider, and the visualization will update automatically to reflect your selection. No further code execution is needed to interact with the displayed widgets.","metadata":{}},{"cell_type":"code","source":"%matplotlib inline\n\n#!pip install -q tifffile imagecodecs ipywidgets\n!pip install --no-index --find-links=\"/kaggle/input/surface-detect-package-scraper\" -q tifffile imagecodecs ipywidgets","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-05T21:57:05.915068Z","iopub.execute_input":"2025-12-05T21:57:05.915451Z","iopub.status.idle":"2025-12-05T21:57:11.348938Z","shell.execute_reply.started":"2025-12-05T21:57:05.915419Z","shell.execute_reply":"2025-12-05T21:57:11.347398Z"},"_kg_hide-output":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport imagecodecs # Explicitly import imagecodecs\nimport tifffile\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport ipywidgets as widgets\nfrom IPython.display import display\nimport pandas as pd\n\nPATH_DATASET = \"/kaggle/input/vesuvius-challenge-surface-detection\"","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-12-05T21:57:11.351287Z","iopub.execute_input":"2025-12-05T21:57:11.351623Z","iopub.status.idle":"2025-12-05T21:57:11.944238Z","shell.execute_reply.started":"2025-12-05T21:57:11.351584Z","shell.execute_reply":"2025-12-05T21:57:11.943236Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Load sample","metadata":{}},{"cell_type":"code","source":"def load_image_and_mask(image_id):\n    \"\"\"Loads a 3D TIFF image and its corresponding annotation mask.\n\n    Args:\n        image_id (str): The unique identifier for the image.\n\n    Returns:\n        tuple: A tuple containing the 3D numpy array for the image and the mask.\n    \"\"\"\n    image_path = os.path.join(PATH_DATASET, 'train_images', f'{image_id}.tif')\n    mask_path = os.path.join(PATH_DATASET, 'train_labels', f'{image_id}.tif')\n\n    image = tifffile.imread(image_path)\n    mask = tifffile.imread(mask_path)\n\n    return image, mask\n\n# Test the function with a sample image_id\nsample_image_id = '602831951'\nsample_image, sample_mask = load_image_and_mask(sample_image_id)\n\nprint(f\"Loaded image '{sample_image_id}.tif'\"\n      f\" ({sample_image.dtype}) with shape: {sample_image.shape} and value range: {sample_image.min()} -> {sample_image.max()}\")\nprint(f\"Loaded mask for '{sample_image_id}.tif'\"\n      f\" ({sample_mask.dtype}) with shape: {sample_mask.shape} and labels: {np.unique(sample_mask)}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-05T21:57:29.556691Z","iopub.execute_input":"2025-12-05T21:57:29.557024Z","iopub.status.idle":"2025-12-05T21:57:30.613972Z","shell.execute_reply.started":"2025-12-05T21:57:29.556987Z","shell.execute_reply":"2025-12-05T21:57:30.613356Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig, axes = plt.subplots(2, 1, figsize=(10, 6))\n\n# Plot histogram for image volume\naxes[0].hist(sample_image.ravel(), bins=50, color='blue', alpha=0.7)\naxes[0].set_title('Histogram of Image Volume Pixel Intensities')\naxes[0].set_xlabel('Pixel Intensity')\naxes[0].set_ylabel('Frequency')\naxes[0].grid(True)\n\n# Plot bin count for mask\nunique_values, counts = np.unique(sample_mask, return_counts=True)\naxes[1].bar(unique_values.astype(str), counts, color='green', alpha=0.7)\naxes[1].set_title('Mask Label Counts')\naxes[1].set_xlabel('Mask Label')\naxes[1].set_ylabel('Count')\naxes[1].grid(True)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-12-05T20:18:20.187825Z","iopub.execute_input":"2025-12-05T20:18:20.188528Z","iopub.status.idle":"2025-12-05T20:18:21.893462Z","shell.execute_reply.started":"2025-12-05T20:18:20.188498Z","shell.execute_reply":"2025-12-05T20:18:21.892490Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_images_path = os.path.join(PATH_DATASET, 'train_images')\n\n# Get all file names in the train_images directory\nall_files = os.listdir(train_images_path)\n\n# Filter for TIFF files and extract image IDs\nimage_ids = sorted([os.path.splitext(f)[0] for f in all_files if f.endswith('.tif')])\n\nprint(f\"Found {len(image_ids)} image IDs. First 5: {image_ids[:5]}\")\nprint(f\"Last 5: {image_ids[-5:]}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-05T20:18:37.264002Z","iopub.execute_input":"2025-12-05T20:18:37.264330Z","iopub.status.idle":"2025-12-05T20:18:37.289297Z","shell.execute_reply.started":"2025-12-05T20:18:37.264307Z","shell.execute_reply":"2025-12-05T20:18:37.288135Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3D viewer","metadata":{}},{"cell_type":"code","source":"from ipywidgets import interact, IntSlider, fixed, FloatSlider\nfrom matplotlib.patches import PathPatch, Rectangle\nfrom matplotlib.path import Path\nfrom matplotlib.colors import ListedColormap, BoundaryNorm # Added imports\nimport matplotlib.patches as mpatches # Added import\n\nfrom datetime import datetime\n# Helper functions from the template\n# Note: _draw_line and _set_axes_labels are defined but not used in the template's show_volume\ndef _draw_line(ax, coords, clr='g'):\n    line = Path(coords, [Path.MOVETO, Path.LINETO])\n    pp = PathPatch(line, linewidth=3, edgecolor=clr, facecolor='none')\n    ax.add_patch(pp)\n\ndef _set_axes_labels(ax, axes_x, axes_y):\n    ax.set_xlabel(axes_x)\n    ax.set_ylabel(axes_y)\n    ax.set_aspect('equal', 'box')\n\n_rec_prop = dict(linewidth=5, facecolor='none')\n\ndef show_volume(vol, mask_vol, z, y, x, mask_alpha, fig_size=(8, 8)): # Increased fig_size\n    start = datetime.now()\n    fig = plt.figure(figsize=fig_size)\n    # Create subplots using fig.add_subplot\n    ax00 = fig.add_subplot(2, 2, 1) # Top-left: Axial\n    ax01 = fig.add_subplot(2, 2, 2) # Top-right: Sagittal\n    ax10 = fig.add_subplot(2, 2, 3) # Bottom-left: Coronal\n    ax11 = fig.add_subplot(2, 2, 4, projection='3d') # Bottom-right: 3D view\n    v_z, v_y, v_x = vol.shape\n\n    # --- Custom Colormap and Legend Setup for Mask ---\n    # Define colors for mask labels: 0=Background, 1=Foreground, 2=Unlabeled\n    # Using distinct colors for visualization:\n    # 'lightgray' for background (visible in legend, subtle on image)\n    # 'yellow' for foreground\n    # 'cyan' for unlabeled\n    colors = ['lightgray', 'yellow', 'cyan'] # This order maps to 0, 1, 2\n    mask_cmap = ListedColormap(colors)\n    # Define boundaries for the colormap. Values are centered in the bins.\n    # e.g., 0 is between -0.5 and 0.5, 1 is between 0.5 and 1.5, 2 is between 1.5 and 2.5\n    bounds = [-0.5, 0.5, 1.5, 2.5]\n    mask_norm = BoundaryNorm(bounds, mask_cmap.N)\n\n    # Legend handles\n    legend_patches = [\n        mpatches.Patch(color='lightgray', label='0: Background'),\n        mpatches.Patch(color='yellow', label='1: Foreground'),\n        mpatches.Patch(color='cyan', label='2: Unlabeled')\n    ]\n    # --- End Custom Colormap and Legend Setup ---\n\n    # Axial slice (XY plane - view along Z-axis)\n    ax00.imshow(vol[z, :, :], cmap=\"gray\")\n    # Overlay mask with adjustable alpha, using custom colormap and normalization\n    ax00.imshow(mask_vol[z, :, :], cmap=mask_cmap, norm=mask_norm, alpha=mask_alpha, interpolation='nearest')\n    ax00.add_patch(Rectangle((-1, -1), v_x, v_y, edgecolor='r', **_rec_prop))\n    #_draw_line(ax00, [(x, 0), (x, v_y)], \"g\") # Green line for sagittal cut\n    #_draw_line(ax00, [(0, y), (v_x, y)], \"b\") # Blue line for coronal cut\n    ax00.set_title(f\"Axial (XY) Slice - Z: {z}\")\n    # REMOVED: ax00.legend(handles=legend_patches, loc='upper left', bbox_to_anchor=(1.05, 1), borderaxespad=0.)\n\n    # Sagittal slice (YZ plane - view along X-axis, transposed for common orientation)\n    ax01.imshow(vol[:, :, x].T, cmap=\"gray\")\n    # Overlay mask with adjustable alpha, using custom colormap and normalization\n    ax01.imshow(mask_vol[:, :, x].T, cmap=mask_cmap, norm=mask_norm, alpha=mask_alpha, interpolation='nearest')\n    ax01.add_patch(Rectangle((-1, -1), v_z, v_y, edgecolor='g', **_rec_prop))\n    # Red line for axial cut (across Z-axis of transposed image)\n    #_draw_line(ax01, [(z, 0), (z, v_y)], \"r\")\n    # Blue line for coronal cut (across Y-axis of transposed image)\n    #_draw_line(ax01, [(0, y), (v_z, y)], \"b\")\n    ax01.set_title(f\"Sagittal (YZ) Slice - X: {x}\")\n\n    # Coronal slice (XZ plane - view along Y-axis)\n    ax10.imshow(vol[:, y, :], cmap=\"gray\")\n    # Overlay mask with adjustable alpha, using custom colormap and normalization\n    ax10.imshow(mask_vol[:, y, :], cmap=mask_cmap, norm=mask_norm, alpha=mask_alpha, interpolation='nearest')\n    ax10.add_patch(Rectangle((-1, -1), v_x, v_z, edgecolor='b', **_rec_prop))\n    #_draw_line(ax10, [(0, z), (v_x, z)], \"r\") # Red line for axial cut\n    #_draw_line(ax10, [(x, 0), (x, v_z)], \"g\") # Green line for sagittal cut\n    ax10.set_title(f\"Coronal (XZ) Slice - Y: {y}\")\n\n    # --- 3D Plotting for ax11 ---\n    # Define coordinates for planes\n    if False: \n        Y_grid, X_grid = np.meshgrid(np.arange(v_y), np.arange(v_x))\n        Z_grid_cor, X_grid_cor = np.meshgrid(np.arange(v_z), np.arange(v_x))\n        Z_grid_sag, Y_grid_sag = np.meshgrid(np.arange(v_z), np.arange(v_y))\n    \n        # Plot Axial Plane (XY) at current z\n        ax11.plot_surface(X_grid, Y_grid, np.full_like(X_grid, z), color='r', alpha=0.2)\n        # Plot Coronal Plane (XZ) at current y\n        ax11.plot_surface(X_grid_cor, np.full_like(X_grid_cor, y), Z_grid_cor, color='b', alpha=0.2)\n        # Plot Sagittal Plane (YZ) at current x\n        ax11.plot_surface(np.full_like(Y_grid_sag, x), Y_grid_sag, Z_grid_sag, color='g', alpha=0.2)\n    \n        # Set 3D Axis Labels\n        ax11.set_xlabel('X (width)')\n        ax11.set_ylabel('Y (height)')\n        ax11.set_zlabel('Z (depth)')\n    \n        # Set 3D Axis Limits\n        ax11.set_xlim([0, v_x])\n        ax11.set_ylim([0, v_y])\n        ax11.set_zlim([0, v_z])\n\n        # Set View Angle\n        ax11.view_init(elev=20, azim=45)\n        # Add legend to ax11, positioned at the top center in a single row\n        ax11.legend(handles=legend_patches, loc='upper center', bbox_to_anchor=(0.5, 1.0), ncol=len(legend_patches), fancybox=True, shadow=True)\n        # --- End 3D Plotting ---\n    end = datetime.now()\n    fig.tight_layout()\n    plt.show()\n    print(end-start)","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-12-05T21:05:53.975406Z","iopub.execute_input":"2025-12-05T21:05:53.976429Z","iopub.status.idle":"2025-12-05T21:05:53.995606Z","shell.execute_reply.started":"2025-12-05T21:05:53.976384Z","shell.execute_reply":"2025-12-05T21:05:53.994570Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def interactive_show(volume, mask_vol):\n    vol_z, vol_y, vol_x = volume.shape\n    interact(\n        lambda x, y, z, mask_alpha: plt.show(show_volume(volume, mask_vol, z, y, x, mask_alpha)),\n        z=IntSlider(min=0, max=vol_z - 1, step=1, value=vol_z // 2, description='Z-slice'),\n        y=IntSlider(min=0, max=vol_y - 1, step=1, value=vol_y // 2, description='Y-slice'),\n        x=IntSlider(min=0, max=vol_x - 1, step=1, value=vol_x // 2, description='X-slice'),\n        mask_alpha=FloatSlider(min=0.0, max=1.0, step=0.05, value=0.3, description='Mask Alpha') # New transparency slider\n    )\n\nrun_type = os.environ.get('KAGGLE_KERNEL_RUN_TYPE', 'Unknown')\n# Make sure sample_image and sample_mask are defined from previous cells\n# if 'sample_image' in globals() and 'sample_mask' in globals():\nif run_type == \"Interactive\":\n    interactive_show(sample_image, sample_mask)\n    print(\"Interactive 3D viewer created and displayed.\")\nelse:\n    print(f\"Running and '{run_type}'...\")\n    vol_z, vol_y, vol_x = sample_image.shape\n    show_volume(sample_image, sample_mask, z=vol_z // 2, y=vol_y // 2, x=vol_x // 2, mask_alpha=0.5)","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-12-05T21:05:54.375765Z","iopub.execute_input":"2025-12-05T21:05:54.376156Z","iopub.status.idle":"2025-12-05T21:05:55.313068Z","shell.execute_reply.started":"2025-12-05T21:05:54.376129Z","shell.execute_reply":"2025-12-05T21:05:55.311975Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3D analysis","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport plotly.graph_objects as go\n\ndef show_volume_3d(volume):\n    # Ensure float\n    if volume.dtype == bool:\n        volume = volume.astype(float)\n    volume = np.asarray(volume)\n\n    nx, ny, nz = volume.shape\n    vmin = float(volume.min())\n    vmax = float(volume.max())\n\n    # 3D grid of coordinates\n    X, Y, Z = np.mgrid[0:nx, 0:ny, 0:nz]\n\n    # Flatten for go.Volume\n    x = X.flatten()\n    y = Y.flatten()\n    z = Z.flatten()\n    values = volume.flatten().astype(float)\n\n    # Opacity: low values -> transparent, high values -> opaque\n    # (normalized on [vmin, vmax])\n    opacityscale = [\n        [0.0, 1],\n        #[0.3, 0.55],\n        [0.3, 1],\n        [1.0, 1.00],\n    ]\n\n    fig = go.Figure(data=go.Volume(\n        x=x,\n        y=y,\n        z=z,\n        value=values,\n        colorscale='Greys',\n        cmin=vmin,\n        cmax=vmax,\n        isomin=vmin,\n        isomax=vmax,\n        surface_count=20,  # more surfaces = smoother but slower\n        opacity=0.5,       # global opacity multiplier\n        opacityscale=opacityscale,\n        caps=dict(x_show=False, y_show=False, z_show=False),\n    ))\n\n    fig.update_layout(\n        title='3D Volume Rendering (low values = transparent)',\n        scene=dict(\n            xaxis_title='X',\n            yaxis_title='Y',\n            zaxis_title='Z',\n            aspectmode='data',\n            camera=dict(eye=dict(x=2, y=2, z=1.5))\n        ),\n        width=800,\n        height=800\n    )\n\n    return fig\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-05T22:04:43.053297Z","iopub.execute_input":"2025-12-05T22:04:43.053648Z","iopub.status.idle":"2025-12-05T22:04:43.064124Z","shell.execute_reply.started":"2025-12-05T22:04:43.053620Z","shell.execute_reply":"2025-12-05T22:04:43.062896Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Load the data print(\"Loading data...\") \nimage_name = os.path.join(PATH_DATASET, 'train_images', sample_image_id + \".tif\") \nvolume = tifffile.imread(image_name)[10:100,10:100,10:20] \nprint(f\"Volume shape: {volume.shape}\") \n# Create and display the visualization \nprint(\"\\nCreating visualization...\") \nfig = show_volume_3d(volume) \nfig.show() \nprint(\"\\nVisualization controls:\") \nprint(\"- Rotate: Click and drag\") \nprint(\"- Zoom: Scroll wheel\") \nprint(\"- Pan: Right-click and drag\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-05T22:05:11.051724Z","iopub.execute_input":"2025-12-05T22:05:11.052540Z","iopub.status.idle":"2025-12-05T22:05:11.264445Z","shell.execute_reply.started":"2025-12-05T22:05:11.052471Z","shell.execute_reply":"2025-12-05T22:05:11.262872Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}