{
  "id": 425175,
  "title": "let's hack mmdet3.0 and write our own training loop, etc",
  "url": "/competitions/hubmap-hacking-the-human-vasculature/discussion/425175",
  "author_name": "hengck23",
  "post_date": "2023-07-17T14:07:38.927000",
  "votes": 20,
  "comment_count": 10,
  "views": 0,
  "content": "<p>I just want to use mmdet3.0 model and dataset.</p>\n<p>I want to write my own training loop, logging etc.</p>\n<p>Here is how it works:</p>\n<p>step.1 create model from cfg</p>\n<pre><code>=\n = Config.fromfile(cfg_file)\n = init_detector(cfg)\n</code></pre>\n<p>step.2 create dataset from cfg</p>\n<pre><code>from mmdet import DATASETS\ndataset = DATASETS(cfg.dataset)\n\n you want to get one dataset record\nr0= dataset   is (inputs=..., data_samples=...)\nr1= dataset\n\n\n you want to make one bach  training\nbatch=(\n    inputs=, r1, ... ],\n    data_samples=, r1, ...],\n)\n</code></pre>\n<p>step.3 compute loss</p>\n<pre><code>\n\n\n\n = model.data_preprocessor(batch, training=)\n = model._run_forward(data, mode=)\n</code></pre>\n<p>if you print loss, it will be</p>\n<pre><code>{: [tensor(0.5263, =, =&lt;MulBackward0&gt;),\n  tensor(0.1252, =, =&lt;MulBackward0&gt;),\n  tensor(0.0357, =, =&lt;MulBackward0&gt;),\n  tensor(0.0116, =, =&lt;MulBackward0&gt;),\n  tensor(0.0025, =, =&lt;MulBackward0&gt;)],\n : [tensor(0., =, =&lt;MulBackward0&gt;),\n  tensor(0., =, =&lt;MulBackward0&gt;),\n  tensor(0., =, =&lt;MulBackward0&gt;),\n  tensor(0.0037, =, =&lt;MulBackward0&gt;),\n  tensor(0., =, =&lt;MulBackward0&gt;)],\n : tensor(0.7018, =, =&lt;MulBackward0&gt;),\n : tensor([32.2266], =),\n : tensor(0.0002, =, =&lt;MulBackward0&gt;),\n : tensor([0.7019], =, =&lt;MulBackward0&gt;)}\n</code></pre>",
  "messages": [
    {
      "id": 2348283,
      "postDate": "2023-07-17T14:07:38.927Z",
      "content": "<p>I just want to use mmdet3.0 model and dataset.</p>\n<p>I want to write my own training loop, logging etc.</p>\n<p>Here is how it works:</p>\n<p>step.1 create model from cfg</p>\n<pre><code>=\n = Config.fromfile(cfg_file)\n = init_detector(cfg)\n</code></pre>\n<p>step.2 create dataset from cfg</p>\n<pre><code>from mmdet import DATASETS\ndataset = DATASETS(cfg.dataset)\n\n you want to get one dataset record\nr0= dataset   is (inputs=..., data_samples=...)\nr1= dataset\n\n\n you want to make one bach  training\nbatch=(\n    inputs=, r1, ... ],\n    data_samples=, r1, ...],\n)\n</code></pre>\n<p>step.3 compute loss</p>\n<pre><code>\n\n\n\n = model.data_preprocessor(batch, training=)\n = model._run_forward(data, mode=)\n</code></pre>\n<p>if you print loss, it will be</p>\n<pre><code>{: [tensor(0.5263, =, =&lt;MulBackward0&gt;),\n  tensor(0.1252, =, =&lt;MulBackward0&gt;),\n  tensor(0.0357, =, =&lt;MulBackward0&gt;),\n  tensor(0.0116, =, =&lt;MulBackward0&gt;),\n  tensor(0.0025, =, =&lt;MulBackward0&gt;)],\n : [tensor(0., =, =&lt;MulBackward0&gt;),\n  tensor(0., =, =&lt;MulBackward0&gt;),\n  tensor(0., =, =&lt;MulBackward0&gt;),\n  tensor(0.0037, =, =&lt;MulBackward0&gt;),\n  tensor(0., =, =&lt;MulBackward0&gt;)],\n : tensor(0.7018, =, =&lt;MulBackward0&gt;),\n : tensor([32.2266], =),\n : tensor(0.0002, =, =&lt;MulBackward0&gt;),\n : tensor([0.7019], =, =&lt;MulBackward0&gt;)}\n</code></pre>",
      "rawMarkdown": "I just want to use mmdet3.0 model and dataset.\n\nI want to write my own training loop, logging etc.\n\nHere is how it works:\n\nstep.1 create model from cfg\n\n```\n\ncfg_file='my_config.py'\ncfg = Config.fromfile(cfg_file)\nmodel = init_detector(cfg)\n\n```\n\nstep.2 create dataset from cfg\n```\n\nfrom mmdet.registry import DATASETS\ndataset = DATASETS.build(cfg.train_dataloader.dataset)\n\n#if you want to get one dataset record\nr0= dataset[0]  #format is dict(inputs=..., data_samples=...)\nr1= dataset[1]\n\n\n#if you want to make one bach for training\nbatch=dict(\n\tinputs=[r0['inputs'], r1['inputs'], ... ],\n\tdata_samples=[r0['data_samples'], r1['data_samples'], ...],\n)\n\n```\nstep.3 compute loss\n\n```\n# this is the function, but it is broken into 2 step:\n#model.forward(data, mode='loss') \n\n\ndata = model.data_preprocessor(batch, training=True)\nloss = model._run_forward(data, mode='loss')\n\n```\n\nif you print loss, it will be\n\n```\n{'loss_rpn_cls': [tensor(0.5263, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.1252, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0357, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0116, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0025, device='cuda:0', grad_fn=<MulBackward0>)],\n 'loss_rpn_bbox': [tensor(0., device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0., device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0., device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0037, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0., device='cuda:0', grad_fn=<MulBackward0>)],\n 'loss_cls': tensor(0.7018, device='cuda:0', grad_fn=<MulBackward0>),\n 'acc': tensor([32.2266], device='cuda:0'),\n 'loss_bbox': tensor(0.0002, device='cuda:0', grad_fn=<MulBackward0>),\n 'loss_mask': tensor([0.7019], device='cuda:0', grad_fn=<MulBackward0>)}\n\n```\n",
      "votes": 19
    },
    {
      "id": 2348702,
      "postDate": "2023-07-17T21:44:57.103Z",
      "content": "<p>i confirm the code is working by using the fanous balloon toy dataset and the mmdet instance segmentation tutorial:</p>\n<pre><code>cfg_file = \ncfg = Config.fromfile(cfg_file)\n\ntrain_dataset = DATASETS.build(cfg.train_dataloader.dataset)\n\nnet = init_detector(cfg)\nmy_load_and_fix_state_dict(net, cfg[])\nnet.cuda()\nnet.train()\n\nscaler = torch.cuda.amp.GradScaler(=)\noptimizer = torch.optim.SGD(filter(lambda p: p.requires_grad, net.parameters()), =0.0025)\n\n\n\n=len(train_dataset)*12\niter_log = len(train_dataset)\n\n iteration  range(max_num_iteration):\n    =4\n    index = np.random.choice(batch_size,batch_size)\n    r = [\n        train_dataset[i]  i  index\n    ]\n    batch = dict(\n        inputs=[r[b][].cuda()  b  range(batch_size)],\n        data_samples=[r[b][]  b  range(batch_size)],\n    )\n\n    with torch.cuda.amp.autocast(=):\n        data = net.data_preprocessor(batch, =)\n        loss = net._run_forward(data, =)\n\n        loss_rpn_cls  = sum(loss[])\n        loss_rpn_bbox = sum(loss[])\n        loss_cls = loss[]\n        loss_bbox = loss[]\n        loss_mask = loss[]\n\n    optimizer.zero_grad()\n    scaler.scale(\n        loss_rpn_cls + loss_rpn_bbox + loss_cls + loss_bbox + loss_mask\n    ).backward()\n    scaler.(optimizer)\n    scaler.update()\n\n    acc = loss[]\n\n    text = \n    text +=f\n    text +=f\n    text +=f\n    text +=f\n    text +=f\n    text +=f\n    (, text, end =, =)\n     (iteration%==0):\n        ()\n</code></pre>\n<pre><code>   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n</code></pre>",
      "rawMarkdown": "i confirm the code is working by using the fanous balloon toy dataset and the mmdet instance segmentation tutorial:\n\n```\ncfg_file = 'my_config.py'\ncfg = Config.fromfile(cfg_file)\n\ntrain_dataset = DATASETS.build(cfg.train_dataloader.dataset)\n\nnet = init_detector(cfg)\nmy_load_and_fix_state_dict(net, cfg['load_from'])\nnet.cuda()\nnet.train()\n\nscaler = torch.cuda.amp.GradScaler(enabled=True)\noptimizer = torch.optim.SGD(filter(lambda p: p.requires_grad, net.parameters()), lr=0.0025)\n\n#---\n\nmax_num_iteration=len(train_dataset)*12\niter_log = len(train_dataset)\n\nfor iteration in range(max_num_iteration):\n\tbatch_size=4\n\tindex = np.random.choice(batch_size,batch_size)\n\tr = [\n\t\ttrain_dataset[i] for i in index\n\t]\n\tbatch = dict(\n\t\tinputs=[r[b]['inputs'].cuda() for b in range(batch_size)],\n\t\tdata_samples=[r[b]['data_samples'] for b in range(batch_size)],\n\t)\n\n\twith torch.cuda.amp.autocast(enabled=True):\n\t\tdata = net.data_preprocessor(batch, training=True)\n\t\tloss = net._run_forward(data, mode='loss')\n\n\t\tloss_rpn_cls  = sum(loss['loss_rpn_cls'])\n\t\tloss_rpn_bbox = sum(loss['loss_rpn_bbox'])\n\t\tloss_cls = loss['loss_cls']\n\t\tloss_bbox = loss['loss_bbox']\n\t\tloss_mask = loss['loss_mask']\n\n\toptimizer.zero_grad()\n\tscaler.scale(\n\t\tloss_rpn_cls + loss_rpn_bbox + loss_cls + loss_bbox + loss_mask\n\t).backward()\n\tscaler.step(optimizer)\n\tscaler.update()\n\n\tacc = loss['acc']\n\n\ttext = ''\n\ttext +=f' loss_rpn_cls {loss_rpn_cls.item():0.3f}'\n\ttext +=f' loss_rpn_bbox {loss_rpn_bbox.item():0.3f}'\n\ttext +=f' loss_cls {loss_cls.item():0.3f}'\n\ttext +=f' loss_bbox {loss_bbox.item():0.3f}'\n\ttext +=f' loss_mask {loss_mask.item():0.3f}'\n\ttext +=f' : acc {acc.item():0.1f}'\n\tprint('\\r', text, end ='', flush=True)\n\tif (iteration%iter_log==0):\n\t\tprint('')\n\n```\n\n```\n  loss_rpn_cls 0.098 loss_rpn_bbox 0.018 loss_cls 0.709 loss_bbox 0.350 loss_mask 0.642 : acc 53.8\n  loss_rpn_cls 0.004 loss_rpn_bbox 0.001 loss_cls 0.044 loss_bbox 0.084 loss_mask 0.068 : acc 98.6\n  loss_rpn_cls 0.003 loss_rpn_bbox 0.006 loss_cls 0.061 loss_bbox 0.109 loss_mask 0.096 : acc 97.3\n  loss_rpn_cls 0.002 loss_rpn_bbox 0.005 loss_cls 0.044 loss_bbox 0.060 loss_mask 0.078 : acc 98.3\n  loss_rpn_cls 0.000 loss_rpn_bbox 0.008 loss_cls 0.062 loss_bbox 0.100 loss_mask 0.086 : acc 97.4\n  loss_rpn_cls 0.000 loss_rpn_bbox 0.001 loss_cls 0.012 loss_bbox 0.017 loss_mask 0.033 : acc 99.6\n\n```",
      "votes": 3,
      "replies": [
        {
          "id": 2351744,
          "postDate": "2023-07-20T11:05:20.993Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 2349405,
      "postDate": "2023-07-18T11:44:23.807Z",
      "content": "<p>hack the coco evalutaion to output iou=0.6</p>\n<pre><code>#use our  \n = (  #   \n    #='',  #             \n    ='',\n</code></pre>\n<pre><code> mmdet.registry  METRICS\n mmdet.evaluation.metrics.coco_metric  CocoMetric\n\n@METRICS.register_module()\n (CocoMetric):\n    def compute_metrics Dict[str, float]:\n\n               ...\n        \n         iou  [, ]:\n               ...  IoU  @,   index= \n</code></pre>\n<pre><code>refer  /pycocotools/cocoeval.py\ncoco results are stor  coco. dictionary\ndict_keys([, , , , , ])\ncoco_eval.[].shape\nOut[]: (, , , , ) = [T, R, K, A, M],\n\n\n   \n    \n    \n    \n    \n    \n</code></pre>\n<p>hence T is 10, and for iou=0.6, it is index 2</p>",
      "rawMarkdown": "hack the coco evalutaion to output iou=0.6\n\n```\n#use our override class\nval_evaluator = dict(  # Validation evaluator config\n    #type='CocoMetric',  #  coco metric  to evaluate AR, AP, and mAP for detection and instance segmentation\n\ttype='MyCocoMetric',\n\n\n```\n\n```\n\nfrom mmdet.registry import METRICS\nfrom mmdet.evaluation.metrics.coco_metric import CocoMetric\n\n@METRICS.register_module()\nclass MyCocoMetric(CocoMetric):\n\tdef compute_metrics(self, results: list) -> Dict[str, float]:\n\n               ...\n\t\t# indexes of IoU  @50 and @75 \n\t\tfor iou in [0, 5]:\n               ... for IoU  @60, it is index=2 ###<----modify here\n\n```\n\n```\nrefer to /pycocotools/cocoeval.py\ncoco results are stor in coco.eval dictionary\ndict_keys(['params', 'counts', 'date', 'precision', 'recall', 'scores'])\ncoco_eval.eval['precision'].shape\nOut[12]: (10, 101, 3, 4, 3) = [T, R, K, A, M],\n\n\n   #  imgIds     - [all] N img ids to use for evaluation\n    #  catIds     - [all] K cat ids to use for evaluation\n    #  iouThrs    - [.5:.05:.95] T=10 IoU thresholds for evaluation\n    #  recThrs    - [0:.01:1] R=101 recall thresholds for evaluation\n    #  areaRng    - [...] A=4 object area ranges for evaluation\n    #  maxDets    - [1 10 100] M=3 thresholds on max detections per image\n\n```\n\nhence T is 10, and for iou=0.6, it is index 2",
      "votes": 1,
      "replies": [
        {
          "id": 2349410,
          "postDate": "2023-07-18T11:48:16.043Z",
          "content": "<pre><code>def :\n    # without dataset:\n    #   test_pipeline = )\n    #   image = cv2.imread(image_file, cv2.IMREAD_COLOR)\n    #   r = test)\n\n    num_valid  = len(valid_dataset)\n    prediction = \n    start_timer = timer\n     t  range(num_valid):\n        r = valid_dataset\n        image_id  = r.img_id\n        print(, f'{t}/{num_valid} : id={image_id}', time - start_timer,'sec'), ='')\n\n        batch = dict(\n            inputs=.cuda],\n            data_samples=],\n        )\n\n        net.eval\n         torch.no:\n             torch.cuda.amp.autocast(enabled=True):\n                #results = net.test\n                data = net.data\n                d = net.\n\n        d = d.cpu#.numpy\n        d = d.\n        d= , 'counts': b'`&lt;Q2o=O2N2M3N2M2O2N2N2N3L5I8H=AmRh7'}\n            encode  m  d\n        ]\n        prediction.append(d)\n\n    #----\n    evaluator = \n    evaluator.dataset_meta = mmconfig.metainfo \n    eval_results = evaluator.offline\n</code></pre>",
          "rawMarkdown": "```\n\ndef do_valid():\n\t# without dataset:\n\t#   test_pipeline = Compose(get_test_pipeline_cfg(mmconfig))\n\t#   image = cv2.imread(image_file, cv2.IMREAD_COLOR)\n\t#   r = test_pipeline(dict(img=image, img_id=0))\n\n\tnum_valid  = len(valid_dataset)\n\tprediction = []\n\tstart_timer = timer()\n\tfor t in range(num_valid):\n\t\tr = valid_dataset[t]\n\t\timage_id  = r['data_samples'].img_id\n\t\tprint('\\r', f'{t}/{num_valid} : id={image_id}', time_to_str(timer() - start_timer,'sec'), end='')\n\n\t\tbatch = dict(\n\t\t\tinputs=[r['inputs'].cuda()],\n\t\t\tdata_samples=[r['data_samples']],\n\t\t)\n\n\t\tnet.eval()\n\t\twith torch.no_grad():\n\t\t\twith torch.cuda.amp.autocast(enabled=True):\n\t\t\t\t#results = net.test_step(batch)[0]\n\t\t\t\tdata = net.data_preprocessor(batch, training=False)\n\t\t\t\td = net._run_forward(data, mode='predict')[0]\n\n\t\td = d.cpu()#.numpy()\n\t\td = d.to_dict()\n\t\td['pred_instances']['masks']= [\n\t\t\t## e.g. {'size': [512, 512], 'counts': b'`<Q2o=00O2N2M3N2M2O2N2N2N3L5I8H=AmRh7'}\n\t\t\tencode_mask_for_pickle(m) for m in d['pred_instances']['masks']\n\t\t]\n\t\tprediction.append(d)\n\n\t#----\n\tevaluator = Evaluator(mmconfig.val_evaluator)\n\tevaluator.dataset_meta = mmconfig.metainfo \n\teval_results = evaluator.offline_evaluate(prediction)\n\n\n```",
          "replies": [
            {
              "id": 2349411,
              "postDate": "2023-07-18T11:49:27.790Z",
              "content": "<pre><code> pycocotools.mask  mask_util\ndef encode_mask_for_pickle(mask):\n    m = mask_util.encode(\n        np.( mask[:, :, np.newaxis], =, dtype=)\n    )[]\n     m\n</code></pre>",
              "rawMarkdown": "```\nimport pycocotools.mask as mask_util\ndef encode_mask_for_pickle(mask):\n\tm = mask_util.encode(\n\t\tnp.array( mask[:, :, np.newaxis], order='F', dtype='uint8')\n\t)[0]\n\treturn m\n\n```"
            }
          ]
        }
      ]
    },
    {
      "id": 2348285,
      "postDate": "2023-07-17T14:12:13.623Z",
      "content": "<p>how to trace the code?</p>\n<p>it is difficult to set breakpoint in pycharm IDE becuase of the many of the object are build dynamically using template, registry …<br>\ninstead use \"import inspect\" in debug console</p>\n<p>e.g.</p>\n<pre><code>#libsite-packagesrunner/loops.py\n\n        outputs = self.runner.model.train_step(\n            data_batch, optim_wrapper=self.runner.optim_wrapper)\n</code></pre>\n<p>i want to setp into the function \"train_step\"</p>\n<pre><code>\nimport inspect\ninspect.getfile(.runner.model.train_step)\n\n  \n</code></pre>\n<p>now i know where the function is located</p>",
      "rawMarkdown": "how to trace the code?\n\nit is difficult to set breakpoint in pycharm IDE becuase of the many of the object are build dynamically using template, registry ...\ninstead use \"import inspect\" in debug console\n\ne.g.\n```\n#/anaconda3.10/lib/python3.10/site-packages/mmengine/runner/loops.py\n\n        outputs = self.runner.model.train_step(\n            data_batch, optim_wrapper=self.runner.optim_wrapper)\n\n```\n\ni want to setp into the function \"train\\_step\"\n\n```\n#at debug console at runtime\nimport inspect\ninspect.getfile(self.runner.model.train_step)\n\n>> '/opt/anaconda3.10/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py' \n\n```\n\nnow i know where the function is located",
      "votes": 2
    },
    {
      "id": 2349601,
      "postDate": "2023-07-18T14:39:33.193Z",
      "content": "<p>Are you planning to add TTA for mmdet3 (like MultiScaleFlipAug in mmdet2)?</p>",
      "rawMarkdown": "Are you planning to add TTA for mmdet3 (like MultiScaleFlipAug in mmdet2)?"
    },
    {
      "id": 2349569,
      "postDate": "2023-07-18T14:18:15.993Z",
      "content": "<p>a better way to create the model</p>\n<pre><code> mmdet.registry  MODELS\n mmengine.model.utils  revert_sync_batchnorm\n mmdet.utils  register_all_modules\nregister_all_modules()\n\n\ncfg_file = \nmmconfig = Config.fromfile(cfg_file)\n\n\nnet = revert_sync_batchnorm(MODELS.build(mmconfig.model))\nnet.init_weights()# initialize the model  pretrained: mmengine - INFO - load model : ...\nnet.cuda()\nnet.train()\n</code></pre>",
      "rawMarkdown": "a better way to create the model\n\n```\nfrom mmdet.registry import MODELS\nfrom mmengine.model.utils import revert_sync_batchnorm\nfrom mmdet.utils import register_all_modules\nregister_all_modules()\n\n\ncfg_file = 'my_config_cascade_mask_rcnn_x101_64x4d_fpn_aug1.py'\nmmconfig = Config.fromfile(cfg_file)\n\n\nnet = revert_sync_batchnorm(MODELS.build(mmconfig.model))\nnet.init_weights()# initialize the model with pretrained: mmengine - INFO - load model from: ...\nnet.cuda()\nnet.train()\n```"
    },
    {
      "id": 2351754,
      "postDate": "2023-07-20T11:09:15.420Z",
      "rawMarkdown": "",
      "isDeleted": true,
      "replies": [
        {
          "id": 2351759,
          "postDate": "2023-07-20T11:11:33.870Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2348702,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2023-07-17T21:44:57.103000",
      "content": "<p>i confirm the code is working by using the fanous balloon toy dataset and the mmdet instance segmentation tutorial:</p>\n<pre><code>cfg_file = \ncfg = Config.fromfile(cfg_file)\n\ntrain_dataset = DATASETS.build(cfg.train_dataloader.dataset)\n\nnet = init_detector(cfg)\nmy_load_and_fix_state_dict(net, cfg[])\nnet.cuda()\nnet.train()\n\nscaler = torch.cuda.amp.GradScaler(=)\noptimizer = torch.optim.SGD(filter(lambda p: p.requires_grad, net.parameters()), =0.0025)\n\n\n\n=len(train_dataset)*12\niter_log = len(train_dataset)\n\n iteration  range(max_num_iteration):\n    =4\n    index = np.random.choice(batch_size,batch_size)\n    r = [\n        train_dataset[i]  i  index\n    ]\n    batch = dict(\n        inputs=[r[b][].cuda()  b  range(batch_size)],\n        data_samples=[r[b][]  b  range(batch_size)],\n    )\n\n    with torch.cuda.amp.autocast(=):\n        data = net.data_preprocessor(batch, =)\n        loss = net._run_forward(data, =)\n\n        loss_rpn_cls  = sum(loss[])\n        loss_rpn_bbox = sum(loss[])\n        loss_cls = loss[]\n        loss_bbox = loss[]\n        loss_mask = loss[]\n\n    optimizer.zero_grad()\n    scaler.scale(\n        loss_rpn_cls + loss_rpn_bbox + loss_cls + loss_bbox + loss_mask\n    ).backward()\n    scaler.(optimizer)\n    scaler.update()\n\n    acc = loss[]\n\n    text = \n    text +=f\n    text +=f\n    text +=f\n    text +=f\n    text +=f\n    text +=f\n    (, text, end =, =)\n     (iteration%==0):\n        ()\n</code></pre>\n<pre><code>   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n   . loss_rpn_bbox . loss_cls . loss_bbox . loss_mask . : acc .\n</code></pre>",
      "votes": 3,
      "replies": [
        {
          "id": 2351744,
          "author_name": "",
          "author_url": "",
          "post_date": "2023-07-20T11:05:20.993000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2349405,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2023-07-18T11:44:23.807000",
      "content": "<p>hack the coco evalutaion to output iou=0.6</p>\n<pre><code>#use our  \n = (  #   \n    #='',  #             \n    ='',\n</code></pre>\n<pre><code> mmdet.registry  METRICS\n mmdet.evaluation.metrics.coco_metric  CocoMetric\n\n@METRICS.register_module()\n (CocoMetric):\n    def compute_metrics Dict[str, float]:\n\n               ...\n        \n         iou  [, ]:\n               ...  IoU  @,   index= \n</code></pre>\n<pre><code>refer  /pycocotools/cocoeval.py\ncoco results are stor  coco. dictionary\ndict_keys([, , , , , ])\ncoco_eval.[].shape\nOut[]: (, , , , ) = [T, R, K, A, M],\n\n\n   \n    \n    \n    \n    \n    \n</code></pre>\n<p>hence T is 10, and for iou=0.6, it is index 2</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2349410,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2023-07-18T11:48:16.043000",
          "content": "<pre><code>def :\n    # without dataset:\n    #   test_pipeline = )\n    #   image = cv2.imread(image_file, cv2.IMREAD_COLOR)\n    #   r = test)\n\n    num_valid  = len(valid_dataset)\n    prediction = \n    start_timer = timer\n     t  range(num_valid):\n        r = valid_dataset\n        image_id  = r.img_id\n        print(, f'{t}/{num_valid} : id={image_id}', time - start_timer,'sec'), ='')\n\n        batch = dict(\n            inputs=.cuda],\n            data_samples=],\n        )\n\n        net.eval\n         torch.no:\n             torch.cuda.amp.autocast(enabled=True):\n                #results = net.test\n                data = net.data\n                d = net.\n\n        d = d.cpu#.numpy\n        d = d.\n        d= , 'counts': b'`&lt;Q2o=O2N2M3N2M2O2N2N2N3L5I8H=AmRh7'}\n            encode  m  d\n        ]\n        prediction.append(d)\n\n    #----\n    evaluator = \n    evaluator.dataset_meta = mmconfig.metainfo \n    eval_results = evaluator.offline\n</code></pre>",
          "votes": 0,
          "replies": [
            {
              "id": 2349411,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2023-07-18T11:49:27.790000",
              "content": "<pre><code> pycocotools.mask  mask_util\ndef encode_mask_for_pickle(mask):\n    m = mask_util.encode(\n        np.( mask[:, :, np.newaxis], =, dtype=)\n    )[]\n     m\n</code></pre>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2348285,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2023-07-17T14:12:13.623000",
      "content": "<p>how to trace the code?</p>\n<p>it is difficult to set breakpoint in pycharm IDE becuase of the many of the object are build dynamically using template, registry …<br>\ninstead use \"import inspect\" in debug console</p>\n<p>e.g.</p>\n<pre><code>#libsite-packagesrunner/loops.py\n\n        outputs = self.runner.model.train_step(\n            data_batch, optim_wrapper=self.runner.optim_wrapper)\n</code></pre>\n<p>i want to setp into the function \"train_step\"</p>\n<pre><code>\nimport inspect\ninspect.getfile(.runner.model.train_step)\n\n  \n</code></pre>\n<p>now i know where the function is located</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2349601,
      "author_name": "Araik Tamazian",
      "author_url": "",
      "post_date": "2023-07-18T14:39:33.193000",
      "content": "<p>Are you planning to add TTA for mmdet3 (like MultiScaleFlipAug in mmdet2)?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2349569,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2023-07-18T14:18:15.993000",
      "content": "<p>a better way to create the model</p>\n<pre><code> mmdet.registry  MODELS\n mmengine.model.utils  revert_sync_batchnorm\n mmdet.utils  register_all_modules\nregister_all_modules()\n\n\ncfg_file = \nmmconfig = Config.fromfile(cfg_file)\n\n\nnet = revert_sync_batchnorm(MODELS.build(mmconfig.model))\nnet.init_weights()# initialize the model  pretrained: mmengine - INFO - load model : ...\nnet.cuda()\nnet.train()\n</code></pre>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2351754,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-07-20T11:09:15.420000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 2351759,
          "author_name": "",
          "author_url": "",
          "post_date": "2023-07-20T11:11:33.870000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2348283": "I just want to use mmdet3.0 model and dataset.\n\nI want to write my own training loop, logging etc.\n\nHere is how it works:\n\nstep.1 create model from cfg\n\n```\n\ncfg_file='my_config.py'\ncfg = Config.fromfile(cfg_file)\nmodel = init_detector(cfg)\n\n```\n\nstep.2 create dataset from cfg\n```\n\nfrom mmdet.registry import DATASETS\ndataset = DATASETS.build(cfg.train_dataloader.dataset)\n\n#if you want to get one dataset record\nr0= dataset[0]  #format is dict(inputs=..., data_samples=...)\nr1= dataset[1]\n\n\n#if you want to make one bach for training\nbatch=dict(\n\tinputs=[r0['inputs'], r1['inputs'], ... ],\n\tdata_samples=[r0['data_samples'], r1['data_samples'], ...],\n)\n\n```\nstep.3 compute loss\n\n```\n# this is the function, but it is broken into 2 step:\n#model.forward(data, mode='loss') \n\n\ndata = model.data_preprocessor(batch, training=True)\nloss = model._run_forward(data, mode='loss')\n\n```\n\nif you print loss, it will be\n\n```\n{'loss_rpn_cls': [tensor(0.5263, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.1252, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0357, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0116, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0025, device='cuda:0', grad_fn=<MulBackward0>)],\n 'loss_rpn_bbox': [tensor(0., device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0., device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0., device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0.0037, device='cuda:0', grad_fn=<MulBackward0>),\n  tensor(0., device='cuda:0', grad_fn=<MulBackward0>)],\n 'loss_cls': tensor(0.7018, device='cuda:0', grad_fn=<MulBackward0>),\n 'acc': tensor([32.2266], device='cuda:0'),\n 'loss_bbox': tensor(0.0002, device='cuda:0', grad_fn=<MulBackward0>),\n 'loss_mask': tensor([0.7019], device='cuda:0', grad_fn=<MulBackward0>)}\n\n```\n",
    "2348702": "i confirm the code is working by using the fanous balloon toy dataset and the mmdet instance segmentation tutorial:\n\n```\ncfg_file = 'my_config.py'\ncfg = Config.fromfile(cfg_file)\n\ntrain_dataset = DATASETS.build(cfg.train_dataloader.dataset)\n\nnet = init_detector(cfg)\nmy_load_and_fix_state_dict(net, cfg['load_from'])\nnet.cuda()\nnet.train()\n\nscaler = torch.cuda.amp.GradScaler(enabled=True)\noptimizer = torch.optim.SGD(filter(lambda p: p.requires_grad, net.parameters()), lr=0.0025)\n\n#---\n\nmax_num_iteration=len(train_dataset)*12\niter_log = len(train_dataset)\n\nfor iteration in range(max_num_iteration):\n\tbatch_size=4\n\tindex = np.random.choice(batch_size,batch_size)\n\tr = [\n\t\ttrain_dataset[i] for i in index\n\t]\n\tbatch = dict(\n\t\tinputs=[r[b]['inputs'].cuda() for b in range(batch_size)],\n\t\tdata_samples=[r[b]['data_samples'] for b in range(batch_size)],\n\t)\n\n\twith torch.cuda.amp.autocast(enabled=True):\n\t\tdata = net.data_preprocessor(batch, training=True)\n\t\tloss = net._run_forward(data, mode='loss')\n\n\t\tloss_rpn_cls  = sum(loss['loss_rpn_cls'])\n\t\tloss_rpn_bbox = sum(loss['loss_rpn_bbox'])\n\t\tloss_cls = loss['loss_cls']\n\t\tloss_bbox = loss['loss_bbox']\n\t\tloss_mask = loss['loss_mask']\n\n\toptimizer.zero_grad()\n\tscaler.scale(\n\t\tloss_rpn_cls + loss_rpn_bbox + loss_cls + loss_bbox + loss_mask\n\t).backward()\n\tscaler.step(optimizer)\n\tscaler.update()\n\n\tacc = loss['acc']\n\n\ttext = ''\n\ttext +=f' loss_rpn_cls {loss_rpn_cls.item():0.3f}'\n\ttext +=f' loss_rpn_bbox {loss_rpn_bbox.item():0.3f}'\n\ttext +=f' loss_cls {loss_cls.item():0.3f}'\n\ttext +=f' loss_bbox {loss_bbox.item():0.3f}'\n\ttext +=f' loss_mask {loss_mask.item():0.3f}'\n\ttext +=f' : acc {acc.item():0.1f}'\n\tprint('\\r', text, end ='', flush=True)\n\tif (iteration%iter_log==0):\n\t\tprint('')\n\n```\n\n```\n  loss_rpn_cls 0.098 loss_rpn_bbox 0.018 loss_cls 0.709 loss_bbox 0.350 loss_mask 0.642 : acc 53.8\n  loss_rpn_cls 0.004 loss_rpn_bbox 0.001 loss_cls 0.044 loss_bbox 0.084 loss_mask 0.068 : acc 98.6\n  loss_rpn_cls 0.003 loss_rpn_bbox 0.006 loss_cls 0.061 loss_bbox 0.109 loss_mask 0.096 : acc 97.3\n  loss_rpn_cls 0.002 loss_rpn_bbox 0.005 loss_cls 0.044 loss_bbox 0.060 loss_mask 0.078 : acc 98.3\n  loss_rpn_cls 0.000 loss_rpn_bbox 0.008 loss_cls 0.062 loss_bbox 0.100 loss_mask 0.086 : acc 97.4\n  loss_rpn_cls 0.000 loss_rpn_bbox 0.001 loss_cls 0.012 loss_bbox 0.017 loss_mask 0.033 : acc 99.6\n\n```",
    "2349405": "hack the coco evalutaion to output iou=0.6\n\n```\n#use our override class\nval_evaluator = dict(  # Validation evaluator config\n    #type='CocoMetric',  #  coco metric  to evaluate AR, AP, and mAP for detection and instance segmentation\n\ttype='MyCocoMetric',\n\n\n```\n\n```\n\nfrom mmdet.registry import METRICS\nfrom mmdet.evaluation.metrics.coco_metric import CocoMetric\n\n@METRICS.register_module()\nclass MyCocoMetric(CocoMetric):\n\tdef compute_metrics(self, results: list) -> Dict[str, float]:\n\n               ...\n\t\t# indexes of IoU  @50 and @75 \n\t\tfor iou in [0, 5]:\n               ... for IoU  @60, it is index=2 ###<----modify here\n\n```\n\n```\nrefer to /pycocotools/cocoeval.py\ncoco results are stor in coco.eval dictionary\ndict_keys(['params', 'counts', 'date', 'precision', 'recall', 'scores'])\ncoco_eval.eval['precision'].shape\nOut[12]: (10, 101, 3, 4, 3) = [T, R, K, A, M],\n\n\n   #  imgIds     - [all] N img ids to use for evaluation\n    #  catIds     - [all] K cat ids to use for evaluation\n    #  iouThrs    - [.5:.05:.95] T=10 IoU thresholds for evaluation\n    #  recThrs    - [0:.01:1] R=101 recall thresholds for evaluation\n    #  areaRng    - [...] A=4 object area ranges for evaluation\n    #  maxDets    - [1 10 100] M=3 thresholds on max detections per image\n\n```\n\nhence T is 10, and for iou=0.6, it is index 2",
    "2348285": "how to trace the code?\n\nit is difficult to set breakpoint in pycharm IDE becuase of the many of the object are build dynamically using template, registry ...\ninstead use \"import inspect\" in debug console\n\ne.g.\n```\n#/anaconda3.10/lib/python3.10/site-packages/mmengine/runner/loops.py\n\n        outputs = self.runner.model.train_step(\n            data_batch, optim_wrapper=self.runner.optim_wrapper)\n\n```\n\ni want to setp into the function \"train\\_step\"\n\n```\n#at debug console at runtime\nimport inspect\ninspect.getfile(self.runner.model.train_step)\n\n>> '/opt/anaconda3.10/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py' \n\n```\n\nnow i know where the function is located",
    "2349601": "Are you planning to add TTA for mmdet3 (like MultiScaleFlipAug in mmdet2)?",
    "2349569": "a better way to create the model\n\n```\nfrom mmdet.registry import MODELS\nfrom mmengine.model.utils import revert_sync_batchnorm\nfrom mmdet.utils import register_all_modules\nregister_all_modules()\n\n\ncfg_file = 'my_config_cascade_mask_rcnn_x101_64x4d_fpn_aug1.py'\nmmconfig = Config.fromfile(cfg_file)\n\n\nnet = revert_sync_batchnorm(MODELS.build(mmconfig.model))\nnet.init_weights()# initialize the model with pretrained: mmengine - INFO - load model from: ...\nnet.cuda()\nnet.train()\n```",
    "2351754": ""
  }
}