{ "id": 236157, "title": "Semi-supervised-learning", "url": "/competitions/hubmap-kidney-segmentation/discussion/236157", "author_name": "", "post_date": "2021-05-03T05:35:07.286964200Z", "votes": 3, "comment_count": 2, "views": 0, "content": "

I read ClassMix and cutmix-semisup-seg and tried them in my own code, but I didn't get comparative results against simple 0.5 threshold pseudo label method.
\nfor detalis, I used strong augmentations in supervised dataloader and non-augmentations in semi loader, and in function below, add filp or cutimix augmentations in semi branch.
\nIs there anyone know what mistakes I made? Thanks for reply.
\n`def train_one_epoch(fold, epoch, model, ema_model, criterion, u_criterion, optimizer, labeled_dataloader, unlabeled_dataloader, device, scheduler=None):
\n model.train()
\n ema_model.train()

\n
unlabeled_dataloader_iter = iter(unlabeled_dataloader)\nL_l_running_loss = None\nL_u_running_loss = None\npbar = tqdm(labeled_dataloader)\nfor step, (imgs, labels) in enumerate(pbar):\n    imgs = imgs.to(device)\n    labels = labels.to(device)\n    mixup = args.mixup and (random.random() < args.mixup)\n    if mixup:\n        lam = np.random.beta(args.alpha, args.alpha)\n        index = torch.randperm(imgs.size(0)).cuda()\n        imgs = lam * imgs + (1-lam) * imgs[index]\n\n    try:\n        imgs_u = next(unlabeled_dataloader_iter)\n    except:\n        unlabeled_dataloader_iter = iter(unlabeled_dataloader)\n        imgs_u = next(unlabeled_dataloader_iter)\n\n    imgs_u = imgs_u.to(device)\n\n    with autocast():\n        preds = model(imgs)\n        if not mixup:\n            L_l = criterion(preds, labels)\n        else:\n            L_l = lam * criterion(preds, labels) + \\\n                (1-lam)*criterion(preds, labels[index])\n\n        imgs_u_w = imgs_u  # weak transform\n        with torch.no_grad():\n            logits_u_w = ema_model(imgs_u_w).sigmoid()\n\n\n\n\n        # std aug\n        flips = [[-1],[-2],[-2,-1]]\n        flip = np.random.choice(flips)\n        imgs_u_s = torch.flip(imgs_u, flip)\n        logits_u_w = torch.flip(logits_u_w, flip)\n\n        # cutmix\n        if args.cutmix:\n            lam = np.random.beta(5, 5)\n            index = torch.randperm(imgs_u.size(0)).cuda()\n            x1, y1, x2, y2 = rand_box(imgs_u.size(), lam)\n            imgs_u_w[:, :, x1:x2, y1:y2] = imgs_u_w[index, :, x1:x2, y1:y2]\n            logits_u_w[:, :, x1:x2, y1:y2] = logits_u_w[index, :, x1:x2, y1:y2] \n            imgs_u_s = imgs_u_w\n            logits_u_s = logits_u_w\n\n        hard_label = args.hard\n        if hard_label:\n            pseudo_label = (logits_u_w > 0.5).float()\n        else:\n            pseudo_label = logits_u_w\n\n        logits_u_s = model(imgs_u_s)\n\n        consistency_weight = args.consistence_weight\n\n\n        confidence_num = torch.sum(logits_u_w.ge(0.968).long() == 1).item(\n        ) + torch.sum(logits_u_w.le(1-0.968).long() == 1).item()\n        unlabeled_weight = confidence_num / np.size(np.array(pseudo_label.cpu()))\n\n        # bce loss\n        if args.u_loss == 'bce':\n            pixelWiseWeight = unlabeled_weight * torch.ones_like(logits_u_w)\n            # pixelWiseWeight = torch.ones_like(logits_u_w)\n            L_u = consistency_weight * u_criterion(logits_u_s, pseudo_label, pixelWiseWeight) \n        else:\n            # mse loss\n            L_u = consistency_weight * unlabeled_weight * u_criterion(logits_u_s, pseudo_label)\n\n        loss = L_l + L_u\n\n\n\n\n\n    # fitlog.add_loss(loss.item(), step+epoch*len(pbar), name='train loss')  # 图画的太丑了\n    scaler.scale(loss).backward()\n    if ((step + 1) % args.accum_iter == 0) or ((step+1) == len(dataloader)):\n        scaler.step(optimizer)\n        scaler.update()\n        optimizer.zero_grad()\n    ema_model = update_ema_variables(\n        ema_model=ema_model, model=model, alpha_teacher=0.99, iteration=len(labeled_dataloader)*epoch+step)\n    if L_l_running_loss:\n        L_l_running_loss = 0.99 * L_l_running_loss + L_l.item() * 0.01\n        L_u_running_loss = 0.99 * L_u_running_loss + L_u.item() * 0.01\n    else:\n        L_l_running_loss = L_l.item()\n        L_u_running_loss = L_u.item()\n\n    if ((step + 1) % args.verbose == 0) or ((step + 1) == len(train_loader)):\n        description = f'epoch {epoch} L_l: {L_l_running_loss:.4f} L_u: {L_u_running_loss:.4f}'\n        pbar.set_description(description)\n\nscheduler.step()\n
\n

`

\n
\n

我阅读了这两篇论文及其代码，并在自己的数据集上尝试了半监督训练，但是效果没有简单的使用伪标签好。我在有监督部分数据加载使用了强增强，无监督部分仅仅使用resize和归一化，并在无监督训练的分支里加入了flip或者cutmix增强。请问我的代码有什么问题吗？

", "messages": [ { "id": "1291532", "postDate": "05/03/2021 05:35:07", "content": "

I read ClassMix and cutmix-semisup-seg and tried them in my own code, but I didn't get comparative results against simple 0.5 threshold pseudo label method.
\nfor detalis, I used strong augmentations in supervised dataloader and non-augmentations in semi loader, and in function below, add filp or cutimix augmentations in semi branch.
\nIs there anyone know what mistakes I made? Thanks for reply.
\n`def train_one_epoch(fold, epoch, model, ema_model, criterion, u_criterion, optimizer, labeled_dataloader, unlabeled_dataloader, device, scheduler=None):
\n model.train()
\n ema_model.train()

\n
unlabeled_dataloader_iter = iter(unlabeled_dataloader)\nL_l_running_loss = None\nL_u_running_loss = None\npbar = tqdm(labeled_dataloader)\nfor step, (imgs, labels) in enumerate(pbar):\n    imgs = imgs.to(device)\n    labels = labels.to(device)\n    mixup = args.mixup and (random.random() < args.mixup)\n    if mixup:\n        lam = np.random.beta(args.alpha, args.alpha)\n        index = torch.randperm(imgs.size(0)).cuda()\n        imgs = lam * imgs + (1-lam) * imgs[index]\n\n    try:\n        imgs_u = next(unlabeled_dataloader_iter)\n    except:\n        unlabeled_dataloader_iter = iter(unlabeled_dataloader)\n        imgs_u = next(unlabeled_dataloader_iter)\n\n    imgs_u = imgs_u.to(device)\n\n    with autocast():\n        preds = model(imgs)\n        if not mixup:\n            L_l = criterion(preds, labels)\n        else:\n            L_l = lam * criterion(preds, labels) + \\\n                (1-lam)*criterion(preds, labels[index])\n\n        imgs_u_w = imgs_u  # weak transform\n        with torch.no_grad():\n            logits_u_w = ema_model(imgs_u_w).sigmoid()\n\n\n\n\n        # std aug\n        flips = [[-1],[-2],[-2,-1]]\n        flip = np.random.choice(flips)\n        imgs_u_s = torch.flip(imgs_u, flip)\n        logits_u_w = torch.flip(logits_u_w, flip)\n\n        # cutmix\n        if args.cutmix:\n            lam = np.random.beta(5, 5)\n            index = torch.randperm(imgs_u.size(0)).cuda()\n            x1, y1, x2, y2 = rand_box(imgs_u.size(), lam)\n            imgs_u_w[:, :, x1:x2, y1:y2] = imgs_u_w[index, :, x1:x2, y1:y2]\n            logits_u_w[:, :, x1:x2, y1:y2] = logits_u_w[index, :, x1:x2, y1:y2] \n            imgs_u_s = imgs_u_w\n            logits_u_s = logits_u_w\n\n        hard_label = args.hard\n        if hard_label:\n            pseudo_label = (logits_u_w > 0.5).float()\n        else:\n            pseudo_label = logits_u_w\n\n        logits_u_s = model(imgs_u_s)\n\n        consistency_weight = args.consistence_weight\n\n\n        confidence_num = torch.sum(logits_u_w.ge(0.968).long() == 1).item(\n        ) + torch.sum(logits_u_w.le(1-0.968).long() == 1).item()\n        unlabeled_weight = confidence_num / np.size(np.array(pseudo_label.cpu()))\n\n        # bce loss\n        if args.u_loss == 'bce':\n            pixelWiseWeight = unlabeled_weight * torch.ones_like(logits_u_w)\n            # pixelWiseWeight = torch.ones_like(logits_u_w)\n            L_u = consistency_weight * u_criterion(logits_u_s, pseudo_label, pixelWiseWeight) \n        else:\n            # mse loss\n            L_u = consistency_weight * unlabeled_weight * u_criterion(logits_u_s, pseudo_label)\n\n        loss = L_l + L_u\n\n\n\n\n\n    # fitlog.add_loss(loss.item(), step+epoch*len(pbar), name='train loss')  # 图画的太丑了\n    scaler.scale(loss).backward()\n    if ((step + 1) % args.accum_iter == 0) or ((step+1) == len(dataloader)):\n        scaler.step(optimizer)\n        scaler.update()\n        optimizer.zero_grad()\n    ema_model = update_ema_variables(\n        ema_model=ema_model, model=model, alpha_teacher=0.99, iteration=len(labeled_dataloader)*epoch+step)\n    if L_l_running_loss:\n        L_l_running_loss = 0.99 * L_l_running_loss + L_l.item() * 0.01\n        L_u_running_loss = 0.99 * L_u_running_loss + L_u.item() * 0.01\n    else:\n        L_l_running_loss = L_l.item()\n        L_u_running_loss = L_u.item()\n\n    if ((step + 1) % args.verbose == 0) or ((step + 1) == len(train_loader)):\n        description = f'epoch {epoch} L_l: {L_l_running_loss:.4f} L_u: {L_u_running_loss:.4f}'\n        pbar.set_description(description)\n\nscheduler.step()\n
\n

`

\n
\n

我阅读了这两篇论文及其代码，并在自己的数据集上尝试了半监督训练，但是效果没有简单的使用伪标签好。我在有监督部分数据加载使用了强增强，无监督部分仅仅使用resize和归一化，并在无监督训练的分支里加入了flip或者cutmix增强。请问我的代码有什么问题吗？

", "rawMarkdown": "I read [ClassMix](https://github.com/WilhelmT/ClassMix) and [cutmix-semisup-seg](https://github.com/Britefury/cutmix-semisup-seg) and tried them in my own code, but I didn't get comparative results against simple 0.5 threshold pseudo label method.\nfor detalis, I used strong augmentations in supervised dataloader and non-augmentations in semi loader, and in function below, add filp or cutimix augmentations in semi branch.\nIs there anyone know what mistakes I made? Thanks for reply.\n`def train_one_epoch(fold, epoch, model, ema_model, criterion, u_criterion, optimizer, labeled_dataloader, unlabeled_dataloader, device, scheduler=None):\n model.train()\n ema_model.train()\n\n unlabeled_dataloader_iter = iter(unlabeled_dataloader)\n L_l_running_loss = None\n L_u_running_loss = None\n pbar = tqdm(labeled_dataloader)\n for step, (imgs, labels) in enumerate(pbar):\n imgs = imgs.to(device)\n labels = labels.to(device)\n mixup = args.mixup and (random.random() < args.mixup)\n if mixup:\n lam = np.random.beta(args.alpha, args.alpha)\n index = torch.randperm(imgs.size(0)).cuda()\n imgs = lam * imgs + (1-lam) * imgs[index]\n\n try:\n imgs_u = next(unlabeled_dataloader_iter)\n except:\n unlabeled_dataloader_iter = iter(unlabeled_dataloader)\n imgs_u = next(unlabeled_dataloader_iter)\n\n imgs_u = imgs_u.to(device)\n\n with autocast():\n preds = model(imgs)\n if not mixup:\n L_l = criterion(preds, labels)\n else:\n L_l = lam * criterion(preds, labels) + \\\n (1-lam)*criterion(preds, labels[index])\n\n imgs_u_w = imgs_u # weak transform\n with torch.no_grad():\n logits_u_w = ema_model(imgs_u_w).sigmoid()\n \n\n\n\n # std aug\n flips = [[-1],[-2],[-2,-1]]\n flip = np.random.choice(flips)\n imgs_u_s = torch.flip(imgs_u, flip)\n logits_u_w = torch.flip(logits_u_w, flip)\n\n # cutmix\n if args.cutmix:\n lam = np.random.beta(5, 5)\n index = torch.randperm(imgs_u.size(0)).cuda()\n x1, y1, x2, y2 = rand_box(imgs_u.size(), lam)\n imgs_u_w[:, :, x1:x2, y1:y2] = imgs_u_w[index, :, x1:x2, y1:y2]\n logits_u_w[:, :, x1:x2, y1:y2] = logits_u_w[index, :, x1:x2, y1:y2] \n imgs_u_s = imgs_u_w\n logits_u_s = logits_u_w\n\n hard_label = args.hard\n if hard_label:\n pseudo_label = (logits_u_w > 0.5).float()\n else:\n pseudo_label = logits_u_w\n\n logits_u_s = model(imgs_u_s)\n\n consistency_weight = args.consistence_weight\n\n \n confidence_num = torch.sum(logits_u_w.ge(0.968).long() == 1).item(\n ) + torch.sum(logits_u_w.le(1-0.968).long() == 1).item()\n unlabeled_weight = confidence_num / np.size(np.array(pseudo_label.cpu()))\n\n # bce loss\n if args.u_loss == 'bce':\n pixelWiseWeight = unlabeled_weight * torch.ones_like(logits_u_w)\n # pixelWiseWeight = torch.ones_like(logits_u_w)\n L_u = consistency_weight * u_criterion(logits_u_s, pseudo_label, pixelWiseWeight) \n else:\n # mse loss\n L_u = consistency_weight * unlabeled_weight * u_criterion(logits_u_s, pseudo_label)\n\n loss = L_l + L_u\n\n\n\n\n\n # fitlog.add_loss(loss.item(), step+epoch*len(pbar), name='train loss') # 图画的太丑了\n scaler.scale(loss).backward()\n if ((step + 1) % args.accum_iter == 0) or ((step+1) == len(dataloader)):\n scaler.step(optimizer)\n scaler.update()\n optimizer.zero_grad()\n ema_model = update_ema_variables(\n ema_model=ema_model, model=model, alpha_teacher=0.99, iteration=len(labeled_dataloader)*epoch+step)\n if L_l_running_loss:\n L_l_running_loss = 0.99 * L_l_running_loss + L_l.item() * 0.01\n L_u_running_loss = 0.99 * L_u_running_loss + L_u.item() * 0.01\n else:\n L_l_running_loss = L_l.item()\n L_u_running_loss = L_u.item()\n\n if ((step + 1) % args.verbose == 0) or ((step + 1) == len(train_loader)):\n description = f'epoch {epoch} L_l: {L_l_running_loss:.4f} L_u: {L_u_running_loss:.4f}'\n pbar.set_description(description)\n\n scheduler.step()\n`\n\n------------------------------------------------\n我阅读了这两篇论文及其代码，并在自己的数据集上尝试了半监督训练，但是效果没有简单的使用伪标签好。我在有监督部分数据加载使用了强增强，无监督部分仅仅使用resize和归一化，并在无监督训练的分支里加入了flip或者cutmix增强。请问我的代码有什么问题吗？", "votes": null }, { "id": "1299945", "postDate": "05/10/2021 07:07:00", "content": "

this is a good work

", "rawMarkdown": "this is a good work", "votes": null }, { "id": "3523112", "postDate": "09/10/2026 15:57:06", "content": "

What if every image didn’t need a manual mask? Semi-supervised learning offers an intriguing possibility.

\n

https://mlguidance.blogspot.com/2026/09/what-are-models-of-semi-supervised.html

", "rawMarkdown": "**What if every image didn’t need a manual mask?** Semi-supervised learning offers an intriguing possibility.\n\nhttps://mlguidance.blogspot.com/2026/09/what-are-models-of-semi-supervised.html", "votes": null } ], "comments": [ { "id": 1299945, "author_name": "camelcv2", "author_url": "", "post_date": "05/10/2021 07:07:00", "content": "

this is a good work

", "votes": null, "replies": [] }, { "id": 3523112, "author_name": "kaustubh994", "author_url": "", "post_date": "09/10/2026 15:57:06", "content": "

What if every image didn’t need a manual mask? Semi-supervised learning offers an intriguing possibility.

\n

https://mlguidance.blogspot.com/2026/09/what-are-models-of-semi-supervised.html

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1291532": "I read [ClassMix](https://github.com/WilhelmT/ClassMix) and [cutmix-semisup-seg](https://github.com/Britefury/cutmix-semisup-seg) and tried them in my own code, but I didn't get comparative results against simple 0.5 threshold pseudo label method.\nfor detalis, I used strong augmentations in supervised dataloader and non-augmentations in semi loader, and in function below, add filp or cutimix augmentations in semi branch.\nIs there anyone know what mistakes I made? Thanks for reply.\n`def train_one_epoch(fold, epoch, model, ema_model, criterion, u_criterion, optimizer, labeled_dataloader, unlabeled_dataloader, device, scheduler=None):\n model.train()\n ema_model.train()\n\n unlabeled_dataloader_iter = iter(unlabeled_dataloader)\n L_l_running_loss = None\n L_u_running_loss = None\n pbar = tqdm(labeled_dataloader)\n for step, (imgs, labels) in enumerate(pbar):\n imgs = imgs.to(device)\n labels = labels.to(device)\n mixup = args.mixup and (random.random() < args.mixup)\n if mixup:\n lam = np.random.beta(args.alpha, args.alpha)\n index = torch.randperm(imgs.size(0)).cuda()\n imgs = lam * imgs + (1-lam) * imgs[index]\n\n try:\n imgs_u = next(unlabeled_dataloader_iter)\n except:\n unlabeled_dataloader_iter = iter(unlabeled_dataloader)\n imgs_u = next(unlabeled_dataloader_iter)\n\n imgs_u = imgs_u.to(device)\n\n with autocast():\n preds = model(imgs)\n if not mixup:\n L_l = criterion(preds, labels)\n else:\n L_l = lam * criterion(preds, labels) + \\\n (1-lam)*criterion(preds, labels[index])\n\n imgs_u_w = imgs_u # weak transform\n with torch.no_grad():\n logits_u_w = ema_model(imgs_u_w).sigmoid()\n \n\n\n\n # std aug\n flips = [[-1],[-2],[-2,-1]]\n flip = np.random.choice(flips)\n imgs_u_s = torch.flip(imgs_u, flip)\n logits_u_w = torch.flip(logits_u_w, flip)\n\n # cutmix\n if args.cutmix:\n lam = np.random.beta(5, 5)\n index = torch.randperm(imgs_u.size(0)).cuda()\n x1, y1, x2, y2 = rand_box(imgs_u.size(), lam)\n imgs_u_w[:, :, x1:x2, y1:y2] = imgs_u_w[index, :, x1:x2, y1:y2]\n logits_u_w[:, :, x1:x2, y1:y2] = logits_u_w[index, :, x1:x2, y1:y2] \n imgs_u_s = imgs_u_w\n logits_u_s = logits_u_w\n\n hard_label = args.hard\n if hard_label:\n pseudo_label = (logits_u_w > 0.5).float()\n else:\n pseudo_label = logits_u_w\n\n logits_u_s = model(imgs_u_s)\n\n consistency_weight = args.consistence_weight\n\n \n confidence_num = torch.sum(logits_u_w.ge(0.968).long() == 1).item(\n ) + torch.sum(logits_u_w.le(1-0.968).long() == 1).item()\n unlabeled_weight = confidence_num / np.size(np.array(pseudo_label.cpu()))\n\n # bce loss\n if args.u_loss == 'bce':\n pixelWiseWeight = unlabeled_weight * torch.ones_like(logits_u_w)\n # pixelWiseWeight = torch.ones_like(logits_u_w)\n L_u = consistency_weight * u_criterion(logits_u_s, pseudo_label, pixelWiseWeight) \n else:\n # mse loss\n L_u = consistency_weight * unlabeled_weight * u_criterion(logits_u_s, pseudo_label)\n\n loss = L_l + L_u\n\n\n\n\n\n # fitlog.add_loss(loss.item(), step+epoch*len(pbar), name='train loss') # 图画的太丑了\n scaler.scale(loss).backward()\n if ((step + 1) % args.accum_iter == 0) or ((step+1) == len(dataloader)):\n scaler.step(optimizer)\n scaler.update()\n optimizer.zero_grad()\n ema_model = update_ema_variables(\n ema_model=ema_model, model=model, alpha_teacher=0.99, iteration=len(labeled_dataloader)*epoch+step)\n if L_l_running_loss:\n L_l_running_loss = 0.99 * L_l_running_loss + L_l.item() * 0.01\n L_u_running_loss = 0.99 * L_u_running_loss + L_u.item() * 0.01\n else:\n L_l_running_loss = L_l.item()\n L_u_running_loss = L_u.item()\n\n if ((step + 1) % args.verbose == 0) or ((step + 1) == len(train_loader)):\n description = f'epoch {epoch} L_l: {L_l_running_loss:.4f} L_u: {L_u_running_loss:.4f}'\n pbar.set_description(description)\n\n scheduler.step()\n`\n\n------------------------------------------------\n我阅读了这两篇论文及其代码，并在自己的数据集上尝试了半监督训练，但是效果没有简单的使用伪标签好。我在有监督部分数据加载使用了强增强，无监督部分仅仅使用resize和归一化，并在无监督训练的分支里加入了flip或者cutmix增强。请问我的代码有什么问题吗？", "1299945": "this is a good work", "3523112": "**What if every image didn’t need a manual mask?** Semi-supervised learning offers an intriguing possibility.\n\nhttps://mlguidance.blogspot.com/2026/09/what-are-models-of-semi-supervised.html" }, "source": "meta" }