Has anyone tried SSM architecture models like Mamba? How's the performance? I was able to train one mamba model locally with a decent CV but couldn't get it running on a Kaggle notebook yet.
", "messages": [ { "id": "2710771", "postDate": "03/22/2024 14:33:45", "content": "Has anyone tried SSM architecture models like Mamba? How's the performance? I was able to train one mamba model locally with a decent CV but couldn't get it running on a Kaggle notebook yet.
", "rawMarkdown": "Has anyone tried SSM architecture models like Mamba? How's the performance? I was able to train one mamba model locally with a decent CV but couldn't get it running on a Kaggle notebook yet.", "votes": null }, { "id": "2711420", "postDate": "03/22/2024 21:01:54", "content": "I have yet to see a Kaggle competition in which Mamba beats transformer. When I do, I will start to believe in Mamba—good luck, haha. Well, as I understand it, Mamba's main selling point is fast inference, so unless inference time is a problem (in this competition, it's not), there is apparently no reason to use Mamba over pure transformer(?).
", "rawMarkdown": "I have yet to see a Kaggle competition in which Mamba beats transformer. When I do, I will start to believe in Mamba—good luck, haha. Well, as I understand it, Mamba's main selling point is fast inference, so unless inference time is a problem (in this competition, it's not), there is apparently no reason to use Mamba over pure transformer(?).", "votes": null }, { "id": "2711753", "postDate": "03/23/2024 03:45:15", "content": "Haven't tried it yet,
\nif you share what are obstacles to run on kaggle
I will try mamba :v
", "rawMarkdown": "I will try mamba :v", "votes": null }, { "id": "2712406", "postDate": "03/23/2024 14:42:36", "content": "Yeah that's a good point! Although I think there's a potential for Mamba models considering it's still relatively new compared to the transformer models. :)
", "rawMarkdown": "Yeah that's a good point! Although I think there's a potential for Mamba models considering it's still relatively new compared to the transformer models. :)", "votes": null }, { "id": "2712408", "postDate": "03/23/2024 14:43:00", "content": "Awesome, can you please share the performance when you tested it? Thanks!
", "rawMarkdown": "Awesome, can you please share the performance when you tested it? Thanks!", "votes": null }, { "id": "2712435", "postDate": "03/23/2024 14:55:37", "content": "This is what I got, it seems it has to do something with the selective scan function from the mamba package
\nFile /kaggle/input/mamba-v1/MedMamba.py:, in SS2D.forward_corev0(, x)\n As = -torch.exp(.A_logs.()).view(, .d_state) # (k * d, d_state)\n dt_projs_bias = .dt_projs_bias.().view() # (k * d)\n--> out_y = .selective_scan(\n xs, dts, \n As, Bs, Cs, Ds, z=None,\n delta_bias=dt_projs_bias,\n delta_softplus=True,\n return_last_state=False,\n ).view(B, K, , L)\n assert out_y.dtype == torch.\n inv_y = torch.flip(out_y[:, :], dims=[]).view(B, , , L)\n\nFile /opt/conda/lib/python3/site-packages/mamba_ssm/ops/selective_scan_interface.py:, in selective_scan_fn(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n def selective_scan_fn(u, delta, A, B, C, D=None, z=None, delta_bias=None, delta_softplus=False,\n return_last_state=False):\n \n---> SelectiveScanFn.apply(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n\nFile /opt/conda/lib/python3/site-packages/torch/autograd/.py:, in Function.apply(cls, *args, **kwargs)\n not torch._C._are_functorch_transforms_active():\n # See NOTE: [functorch vjp and autograd interaction]\n args = _functorch.utils.unwrap_dead_wrappers(args)\n--> super().apply(*args, **kwargs) # type: ignore[misc]\n cls.setup_context == _SingleLevelFunction.setup_context:\n raise RuntimeError(\n \n \n \n \n )\n\nFile /opt/conda/lib/python3/site-packages/mamba_ssm/ops/selective_scan_interface.py:, in SelectiveScanFn.forward(ctx, u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n C = rearrange(C, )\n ctx.squeeze_C = True\n---> out, x, *rest = selective_scan_cuda.fwd(u, delta, A, B, C, D, z, delta_bias, delta_softplus)\n ctx.delta_softplus = delta_softplus\n ctx.has_z = z is not None\n\nRuntimeError: CUDA error: no kernel image is available execution on the device\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n\n raised from c10_cuda_check_implementation at /usr/local/src/pytorch/c10/cuda/CUDAException.cpp: (most recent call first):\nframe #: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<, std::char_traits<>, std::allocator<> >) + ( in /opt/conda/lib/python3/site-packages/torch/lib/libc10.so)\nframe #: c10::detail::torchCheckFail( *, *, , std::__cxx11::basic_string<, std::char_traits<>, std::allocator<> > &) + ( in /opt/conda/lib/python3/site-packages/torch/lib/libc10.so)\nframe #: c10::cuda::c10_cuda_check_implementation(, *, *, , ) + ( in /opt/conda/lib/python3/site-packages/torch/lib/libc10_cuda.so)\nframe #: selective_scan_fwd_launch<, , , >(SSMParamsBase&, CUstream_st*)::{lambda()#}::operator()() ::{lambda()#}::operator()() ::{lambda()#}::operator()() ::{lambda()#}::operator()() ::{lambda()#}::operator()() + ( in /opt/conda/lib/python3/site-packages/selective_scan_cuda.cpython-x86_64-linux-gnu.so)\nframe #: void selective_scan_fwd_cuda<, >(SSMParamsBase&, CUstream_st*) + ( in /opt/conda/lib/python3/site-packages/selective_scan_cuda.cpython-x86_64-linux-gnu.so)\nframe #: selective_scan_fwd(at::Tensor &, at::Tensor &, at::Tensor &, at::Tensor &, at::Tensor &, c10::optional<at::Tensor> &, c10::optional<at::Tensor> &, c10::optional<at::Tensor> &, ) + ( in /opt/conda/lib/python3/site-packages/selective_scan_cuda.cpython-x86_64-linux-gnu.so)\nframe #: <unknown Hi, I am facing a similar issue and was wondering if you had any luck fixing it (or finding a workaround).
", "rawMarkdown": "Hi, I am facing a similar issue and was wondering if you had any luck fixing it (or finding a workaround).", "votes": null } ], "comments": [ { "id": 2711420, "author_name": "shlomoron", "author_url": "", "post_date": "03/22/2024 21:01:54", "content": "I have yet to see a Kaggle competition in which Mamba beats transformer. When I do, I will start to believe in Mamba—good luck, haha. Well, as I understand it, Mamba's main selling point is fast inference, so unless inference time is a problem (in this competition, it's not), there is apparently no reason to use Mamba over pure transformer(?).
", "votes": null, "replies": [ { "id": 2712406, "author_name": "yyyu54", "author_url": "", "post_date": "03/23/2024 14:42:36", "content": "Yeah that's a good point! Although I think there's a potential for Mamba models considering it's still relatively new compared to the transformer models. :)
", "votes": null, "replies": [] } ] }, { "id": 2711753, "author_name": "letemoin", "author_url": "", "post_date": "03/23/2024 03:45:15", "content": "Haven't tried it yet,
\nif you share what are obstacles to run on kaggle
This is what I got, it seems it has to do something with the selective scan function from the mamba package
\nFile /kaggle/input/mamba-v1/MedMamba.py:, in SS2D.forward_corev0(, x)\n As = -torch.exp(.A_logs.()).view(, .d_state) # (k * d, d_state)\n dt_projs_bias = .dt_projs_bias.().view() # (k * d)\n--> out_y = .selective_scan(\n xs, dts, \n As, Bs, Cs, Ds, z=None,\n delta_bias=dt_projs_bias,\n delta_softplus=True,\n return_last_state=False,\n ).view(B, K, , L)\n assert out_y.dtype == torch.\n inv_y = torch.flip(out_y[:, :], dims=[]).view(B, , , L)\n\nFile /opt/conda/lib/python3/site-packages/mamba_ssm/ops/selective_scan_interface.py:, in selective_scan_fn(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n def selective_scan_fn(u, delta, A, B, C, D=None, z=None, delta_bias=None, delta_softplus=False,\n return_last_state=False):\n \n---> SelectiveScanFn.apply(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n\nFile /opt/conda/lib/python3/site-packages/torch/autograd/.py:, in Function.apply(cls, *args, **kwargs)\n not torch._C._are_functorch_transforms_active():\n # See NOTE: [functorch vjp and autograd interaction]\n args = _functorch.utils.unwrap_dead_wrappers(args)\n--> super().apply(*args, **kwargs) # type: ignore[misc]\n cls.setup_context == _SingleLevelFunction.setup_context:\n raise RuntimeError(\n \n \n \n \n )\n\nFile /opt/conda/lib/python3/site-packages/mamba_ssm/ops/selective_scan_interface.py:, in SelectiveScanFn.forward(ctx, u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n C = rearrange(C, )\n ctx.squeeze_C = True\n---> out, x, *rest = selective_scan_cuda.fwd(u, delta, A, B, C, D, z, delta_bias, delta_softplus)\n ctx.delta_softplus = delta_softplus\n ctx.has_z = z is not None\n\nRuntimeError: CUDA error: no kernel image is available execution on the device\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n\n raised from c10_cuda_check_implementation at /usr/local/src/pytorch/c10/cuda/CUDAException.cpp: (most recent call first):\nframe #: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<, std::char_traits<>, std::allocator<> >) + ( in /opt/conda/lib/python3/site-packages/torch/lib/libc10.so)\nframe #: c10::detail::torchCheckFail( *, *, , std::__cxx11::basic_string<, std::char_traits<>, std::allocator<> > &) + ( in /opt/conda/lib/python3/site-packages/torch/lib/libc10.so)\nframe #: c10::cuda::c10_cuda_check_implementation(, *, *, , ) + ( in /opt/conda/lib/python3/site-packages/torch/lib/libc10_cuda.so)\nframe #: selective_scan_fwd_launch<, , , >(SSMParamsBase&, CUstream_st*)::{lambda()#}::operator()() ::{lambda()#}::operator()() ::{lambda()#}::operator()() ::{lambda()#}::operator()() ::{lambda()#}::operator()() + ( in /opt/conda/lib/python3/site-packages/selective_scan_cuda.cpython-x86_64-linux-gnu.so)\nframe #: void selective_scan_fwd_cuda<, >(SSMParamsBase&, CUstream_st*) + ( in /opt/conda/lib/python3/site-packages/selective_scan_cuda.cpython-x86_64-linux-gnu.so)\nframe #: selective_scan_fwd(at::Tensor &, at::Tensor &, at::Tensor &, at::Tensor &, at::Tensor &, c10::optional<at::Tensor> &, c10::optional<at::Tensor> &, c10::optional<at::Tensor> &, ) + ( in /opt/conda/lib/python3/site-packages/selective_scan_cuda.cpython-x86_64-linux-gnu.so)\nframe #: <unknown Hi, I am facing a similar issue and was wondering if you had any luck fixing it (or finding a workaround).
", "votes": null, "replies": [] } ] } ] }, { "id": 2711772, "author_name": "quan0095", "author_url": "", "post_date": "03/23/2024 04:10:56", "content": "I will try mamba :v
", "votes": null, "replies": [ { "id": 2712408, "author_name": "yyyu54", "author_url": "", "post_date": "03/23/2024 14:43:00", "content": "Awesome, can you please share the performance when you tested it? Thanks!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2710771": "Has anyone tried SSM architecture models like Mamba? How's the performance? I was able to train one mamba model locally with a decent CV but couldn't get it running on a Kaggle notebook yet.", "2711420": "I have yet to see a Kaggle competition in which Mamba beats transformer. When I do, I will start to believe in Mamba—good luck, haha. Well, as I understand it, Mamba's main selling point is fast inference, so unless inference time is a problem (in this competition, it's not), there is apparently no reason to use Mamba over pure transformer(?).", "2711753": "Haven't tried it yet, \nif you share what are obstacles to run on kaggle", "2711772": "I will try mamba :v", "2712406": "Yeah that's a good point! Although I think there's a potential for Mamba models considering it's still relatively new compared to the transformer models. :)", "2712408": "Awesome, can you please share the performance when you tested it? Thanks!", "2712435": "This is what I got, it seems it has to do something with the selective scan function from the mamba package\n\n```\nFile /kaggle/input/mamba-v1-2/MedMamba.py:402, in SS2D.forward_corev0(self, x)\n 399 As = -torch.exp(self.A_logs.float()).view(-1, self.d_state) # (k * d, d_state)\n 400 dt_projs_bias = self.dt_projs_bias.float().view(-1) # (k * d)\n--> 402 out_y = self.selective_scan(\n 403 xs, dts, \n 404 As, Bs, Cs, Ds, z=None,\n 405 delta_bias=dt_projs_bias,\n 406 delta_softplus=True,\n 407 return_last_state=False,\n 408 ).view(B, K, -1, L)\n 409 assert out_y.dtype == torch.float\n 411 inv_y = torch.flip(out_y[:, 2:4], dims=[-1]).view(B, 2, -1, L)\n\nFile /opt/conda/lib/python3.10/site-packages/mamba_ssm/ops/selective_scan_interface.py:88, in selective_scan_fn(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n 82 def selective_scan_fn(u, delta, A, B, C, D=None, z=None, delta_bias=None, delta_softplus=False,\n 83 return_last_state=False):\n 84 \"\"\"if return_last_state is True, returns (out, last_state)\n 85 last_state has shape (batch, dim, dstate). Note that the gradient of the last state is\n 86 not considered in the backward pass.\n 87 \"\"\"\n---> 88 return SelectiveScanFn.apply(u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n\nFile /opt/conda/lib/python3.10/site-packages/torch/autograd/function.py:539, in Function.apply(cls, *args, **kwargs)\n 536 if not torch._C._are_functorch_transforms_active():\n 537 # See NOTE: [functorch vjp and autograd interaction]\n 538 args = _functorch.utils.unwrap_dead_wrappers(args)\n--> 539 return super().apply(*args, **kwargs) # type: ignore[misc]\n 541 if cls.setup_context == _SingleLevelFunction.setup_context:\n 542 raise RuntimeError(\n 543 \"In order to use an autograd.Function with functorch transforms \"\n 544 \"(vmap, grad, jvp, jacrev, ...), it must override the setup_context \"\n 545 \"staticmethod. For more details, please see \"\n 546 \"https://pytorch.org/docs/master/notes/extending.func.html\"\n 547 )\n\nFile /opt/conda/lib/python3.10/site-packages/mamba_ssm/ops/selective_scan_interface.py:42, in SelectiveScanFn.forward(ctx, u, delta, A, B, C, D, z, delta_bias, delta_softplus, return_last_state)\n 40 C = rearrange(C, \"b dstate l -> b 1 dstate l\")\n 41 ctx.squeeze_C = True\n---> 42 out, x, *rest = selective_scan_cuda.fwd(u, delta, A, B, C, D, z, delta_bias, delta_softplus)\n 43 ctx.delta_softplus = delta_softplus\n 44 ctx.has_z = z is not None\n\nRuntimeError: CUDA error: no kernel image is available for execution on the device\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1.\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n\nException raised from c10_cuda_check_implementation at /usr/local/src/pytorch/c10/cuda/CUDAException.cpp:44 (most recent call first):\nframe #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string