{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":49349,"databundleVersionId":5447706,"sourceType":"competition"}],"dockerImageVersionId":31234,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# **Instance Segmentation for Human Contour Removal**","metadata":{}},{"cell_type":"markdown","source":"This code performs an automated **Instance Segmentation** pipeline to identify and \"redact\" (black out) people from images. Below is a detailed examination of each stage in the pipeline.\n\n---\n\n## 1. Pipeline Architecture\n\nThe process follows a standard computer vision workflow: **Detection → Masking → Refinement → Visualization**.\n\n### Step A: Model Loading & Inference\n\nThe code utilizes **YOLOv8-seg**, which is an instance segmentation model.\n\n* **Targeting:** It filters specifically for `classes=[0]`, which corresponds to the \"person\" category in the COCO dataset.\n* **Confidence:** A threshold of `0.3` is used to balance between missing people and creating false positives.\n\n### Step B: Mask Generation and Scaling\n\nSince segmentation models often perform inference on a smaller internal resolution (e.g., 640x640), the resulting masks must be scaled back to the original image size using `cv2.resize`.\n\n### Step C: Morphological Refinement\n\nTo ensure the \"blacked-out\" area is clean and professional, the pipeline applies two morphological operations:\n\n1. **Closing (`MORPH_CLOSE`):** Fills small holes inside the person’s mask (e.g., gaps between an arm and a torso).\n2. **Opening (`MORPH_OPEN`):** Removes small noise/pixels outside the main silhouette.\n\n### Step D: Contour Approximation\n\nInstead of using raw, jagged pixel masks, the code finds the **contours** of the person.\n\n* **Smoothing:** It uses `cv2.approxPolyDP` to simplify the shape. This makes the removal look more like a geometric silhouette rather than a pixelated blob.\n* **Filtering:** It ignores any detected area smaller than 100 pixels (`cv2.contourArea(contour) > 100`) to avoid blacking out tiny background artifacts.\n\n---\n\n## 2. Component Analysis\n\n| Component | Responsibility | Technical Key |\n| --- | --- | --- |\n| **Ultralytics YOLO** | Identifies person boundaries at a pixel level. | `model.predict()` |\n| **OpenCV (cv2)** | Handles image manipulation, resizing, and drawing. | `cv2.drawContours` |\n| **NumPy** | Manages the image arrays and mask logic. | `combined_mask == 255` |\n| **Matplotlib** | Provides the side-by-side visual comparison. | `plt.subplots(1, 2)` |\n\n---\n\n## 3. Potential Strengths & Limitations\n\n* **Strength: Contour Smoothing.** By using `approxPolyDP`, the \"holes\" left in the image look cleaner and more intentional.\n* **Strength: GUI Integration.** The `interactive_processing` function makes the script accessible to non-programmers via file dialogs.\n* **Limitation: Ghosting/Halos.** Because the code simply sets pixels to `[0, 0, 0]`, there might be a thin \"halo\" of the original person's colors around the edges if the segmentation isn't 100% tight.\n* **Limitation: Static Filling.** This \"removes\" people by covering them. To truly remove them and see what's behind, you would need an **Inpainting** model (like LaMa or Stable Diffusion Inpainting).\n\n---\n\n## 4. Summary of `remove_people_with_contours_and_display` logic\n\n1. **Initialize** a blank black-and-white mask.\n2. **Iterate** through every person detected by YOLO.\n3. **Draw** their smoothed silhouette onto that mask in white (255).\n4. **Apply** that mask to the original image: wherever the mask is white, turn the original pixels black.\n5. **Output** the result.\n\n","metadata":{}},{"cell_type":"code","source":"!pip install ultralytics","metadata":{"trusted":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import cv2\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom ultralytics import YOLO\nfrom pathlib import Path\n\n\ndef remove_people_with_contours_and_display(image_path, output_path=None, model_size='n', show=True):\n    \"\"\"\n    Fills person silhouettes with black based on detected contours and displays \n    the original vs. processed image.\n    \n    Args:\n        image_path: Path to the input image.\n        output_path: Path to save the output image (None to skip saving).\n        model_size: YOLOv8 model size ('n', 's', 'm', 'l', 'x').\n        show: If True, displays the result using matplotlib.\n    \"\"\"\n    \n    # Load the YOLOv8 segmentation model\n    model = YOLO(f'yolov8{model_size}-seg.pt')\n    \n    # Load the image\n    image = cv2.imread(image_path)\n    if image is None:\n        print(f\"Failed to load image: {image_path}\")\n        return None\n    \n    # Convert original image to RGB for display purposes\n    image_rgb = cv2.cvtColor(image.copy(), cv2.COLOR_BGR2RGB)\n    \n    # Create a copy for processing\n    processed_image = image.copy()\n    height, width = image.shape[:2]\n    \n    # Run segmentation (class 0 is 'person' in COCO)\n    results = model(image, classes=[0], conf=0.3)\n    \n    # Initialize an empty mask\n    combined_mask = np.zeros((height, width), dtype=np.uint8)\n    \n    # Track number of detected people\n    person_count = 0\n    \n    for result in results:\n        if hasattr(result, 'masks') and result.masks is not None:\n            masks = result.masks.data.cpu().numpy()\n            person_count = len(masks)\n            \n            for seg_mask in masks:\n                # Resize mask to match original image dimensions\n                seg_mask_resized = cv2.resize(\n                    seg_mask, \n                    (width, height)\n                )\n                \n                # Binarize the mask\n                _, binary_mask = cv2.threshold(\n                    seg_mask_resized, \n                    (0.3), \n                    255, \n                    cv2.THRESH_BINARY\n                )\n                binary_mask = binary_mask.astype(np.uint8)\n                \n                # Remove noise using morphological operations\n                kernel = np.ones((3, 3), np.uint8)\n                binary_mask = cv2.morphologyEx(binary_mask, cv2.MORPH_CLOSE, kernel)\n                binary_mask = cv2.morphologyEx(binary_mask, cv2.MORPH_OPEN, kernel)\n                \n                # Contour detection and filling\n                contours, _ = cv2.findContours(\n                    binary_mask, \n                    cv2.RETR_EXTERNAL, \n                    cv2.CHAIN_APPROX_SIMPLE\n                )\n                \n                for contour in contours:\n                    if cv2.contourArea(contour) > 100:\n                        # Smooth the contours\n                        epsilon = 0.005 * cv2.arcLength(contour, True)\n                        approx = cv2.approxPolyDP(contour, epsilon, True)\n                        \n                        # Add to the combined mask\n                        cv2.drawContours(combined_mask, [approx], -1, 255, -1)\n    \n    # Fill the detected person regions with black\n    processed_image[combined_mask == 255] = [0, 0, 0]\n    \n    # Convert processed image to RGB for display\n    processed_rgb = cv2.cvtColor(processed_image, cv2.COLOR_BGR2RGB)\n    \n    # Save the result if a path is provided\n    if output_path:\n        cv2.imwrite(output_path, processed_image)\n        print(f\"Processed image saved to: {output_path}\")\n    \n    # Display results\n    if show:\n        display_comparison(image_rgb, processed_rgb, person_count, image_path)\n    \n    return processed_image\n\n\ndef display_comparison(original, processed, person_count, image_path):\n    \"\"\"\n    Displays the original and processed images side-by-side.\n    \"\"\"\n    # Resize images for display if they are too large\n    max_height = 800\n    if original.shape[0] > max_height:\n        scale = max_height / original.shape[0]\n        new_width = int(original.shape[1] * scale)\n        original = cv2.resize(original, (new_width, max_height))\n        processed = cv2.resize(processed, (new_width, max_height))\n    \n    fig, axes = plt.subplots(1, 2, figsize=(15, 8))\n    \n    # Original Image\n    axes[0].imshow(original)\n    axes[0].set_title('Original Image', fontsize=14, fontweight='bold')\n    axes[0].axis('off')\n    \n    # Processed Image\n    axes[1].imshow(processed)\n    axes[1].set_title(f'Processed (Detected: {person_count})', fontsize=14, fontweight='bold')\n    axes[1].axis('off')\n    \n    # Main Title\n    image_name = Path(image_path).name\n    plt.suptitle(f'Person Removal Processing: {image_name}', fontsize=16, fontweight='bold')\n    \n    plt.tight_layout()\n    plt.show()\n    \n    # Display info in console\n    print(f\"=== Processing Info ===\")\n    print(f\"Image: {image_name}\")\n    print(f\"Dimensions: {original.shape[1]} x {original.shape[0]}\")\n    print(f\"People detected: {person_count}\")\n    print(\"=\" * 25)\n\n\ndef process_multiple_images(image_paths, output_dir='output'):\n    \"\"\"\n    Processes multiple images in a batch.\n    \"\"\"\n    # Create output directory\n    Path(output_dir).mkdir(exist_ok=True)\n    \n    for img_path in image_paths:\n        if Path(img_path).exists():\n            print(f\"\\nProcessing: {img_path}\")\n            output_path = Path(output_dir) / f\"processed_{Path(img_path).name}\"\n            remove_people_with_contours_and_display(\n                str(img_path), \n                str(output_path),\n                show=True\n            )\n        else:\n            print(f\"File not found: {img_path}\")\n\n\ndef interactive_processing():\n    \"\"\"\n    Interactive GUI-based image processing.\n    \"\"\"\n    import tkinter as tk\n    from tkinter import filedialog, simpledialog\n    \n    # Create hidden Tkinter root window\n    root = tk.Tk()\n    root.withdraw()\n    \n    # Select image file\n    image_path = filedialog.askopenfilename(\n        title=\"Select an image to process\",\n        filetypes=[\n            (\"Image files\", \"*.jpg *.jpeg *.png *.bmp *.tiff\"),\n            (\"All files\", \"*.*\")\n        ]\n    )\n    \n    if not image_path:\n        print(\"No image selected.\")\n        return\n    \n    # Select model size\n    model_size = simpledialog.askstring(\n        \"Model Size\",\n        \"Select model size (n, s, m, l, x):\\n\" +\n        \"n: Nano, s: Small, m: Medium, l: Large, x: X-Large\",\n        initialvalue=\"n\"\n    )\n    \n    if model_size not in ['n', 's', 'm', 'l', 'x']:\n        model_size = 'n'\n        print(f\"Invalid model size. Using default 'n'.\")\n    \n    # Ask to save\n    save_option = simpledialog.askstring(\n        \"Save Option\",\n        \"Would you like to save the processed image? (y/n):\",\n        initialvalue=\"y\"\n    )\n    \n    output_path = None\n    if save_option and save_option.lower() == 'y':\n        output_path = filedialog.asksaveasfilename(\n            title=\"Specify save location\",\n            defaultextension=\".jpg\",\n            filetypes=[\n                (\"JPEG Image\", \"*.jpg\"),\n                (\"PNG Image\", \"*.png\"),\n                (\"All files\", \"*.*\")\n            ]\n        )\n    \n    # Execute processing\n    processed_image = remove_people_with_contours_and_display(\n        image_path,\n        output_path,\n        model_size=model_size,\n        show=True\n    )\n    \n    return processed_image\n\n\n# Example Usage\nif __name__ == \"__main__\":\n    # Update this path to your specific image location\n    sample_image = \"/kaggle/input/image-matching-challenge-2023/train/phototourism/grand_place_brussels/images/00460368_4162644685.jpg\" \n\n    if Path(sample_image).exists():\n        remove_people_with_contours_and_display(\n            sample_image,\n            output_path=\"output_processed.jpg\",\n            model_size='n',\n            show=True\n        )\n    else:\n        print(\"Sample image path does not exist. Please check the path in the code.\")\n    \n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-14T15:10:13.716888Z","iopub.execute_input":"2026-01-14T15:10:13.717715Z","iopub.status.idle":"2026-01-14T15:10:13.735741Z","shell.execute_reply.started":"2026-01-14T15:10:13.717691Z","shell.execute_reply":"2026-01-14T15:10:13.734328Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}