{
  "id": 420039,
  "title": "A small Polars hack that allowed me to implement a static computational graph and speed-up feature extraction 6x",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/420039",
  "author_name": "",
  "post_date": "2023-06-29T02:32:20.148615800Z",
  "votes": 10,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hey everyone!</p>\n<p>Congrats to the winners, and everybody who participated! This was a fun competition and I enjoyed it quite a lot.</p>\n<p>While we are all eagerly waiting for the solution write-ups of the top teams, I'd like to entertain you with a small Polars trick that I discovered during this competition. I found it in the time of great need, when I realized my submissions would fail the time limit due to slow feature extraction code.</p>\n<h2>A bit of context</h2>\n<p>I've created many different feature extractors each of which was returning a LazyFrame</p>\n<pre><code> () -&gt; LazyFrame:\n  ...\n\n () -&gt; LazyFrame:\n  ...\n\n...\n</code></pre>\n<p>each of the LazyFrames were then joined on <code>session_id</code> and only columns containing \"valuable\" features were selected (tens of thousands of features -&gt; thousands).</p>\n<pre><code>\nfeatures_lazy_frame = reduce(\n  partial(LazyFrame.join, on=),\n  [\n    features_a(events),\n    features_b(events),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n\n\nfeatures = features_lazy_frame.collect()\n</code></pre>\n<p>This worked perfectly fine in batch setting locally so I didn't expect any problems (and I wasn't submitting frequently).</p>\n<h2>Problem</h2>\n<p>~3 days before the end of the competition I realized that this was painfully slow in streaming setting and any of my submissions would go above 9 hour mark just because of feature extraction. What was the issue? Why was it fine when doing batch processing and suddenly didn't work when streaming?</p>\n<p>The problem is hidden in the fact that creating a <code>features_lazy_frame</code> also takes time.<br>\nHere are execution times for 1 game session:</p>\n<pre><code>%%timeit\nfeatures_lazy_frame = ...\n\n s ±  ms per  (mean ± . dev. of  runs,   )\n</code></pre>\n<pre><code>%%timeit\nfeatures_lazy_frame.collect()\n\n  .   loop (mean ± std. dev. of  runs,  loop each)\n</code></pre>\n<p>Constructing LazyFrame took 5/6th of the features extraction time! These 2.5 seconds didn't matter much in batch setting (we do it once for all the items), but in streaming we payed this cost for every single item coming our way.</p>\n<p>If only we could apply same computational graph to all new items, without rebuilding it from scratch every time…</p>\n<h2>Solution</h2>\n<p>I may be blind, but I couldn't find any documented solution to this problem - Polars doesn't seem to have an official support for static computational graphs. So I went digging through the source code.</p>\n<p>My initial understanding was, that LazyFrame is a computational graph that is tied to its original data and there is no way around it. Basically each LazyFrame contains and DataFrame inside.</p>\n<p>But then I came across this undocumented method <a href=\"https://github.com/pola-rs/polars/blob/822ed96f593ef71a7c05b7a114aca08ae926c854/py-polars/polars/lazyframe/frame.py#L505\" target=\"_blank\">https://github.com/pola-rs/polars/blob/822ed96f593ef71a7c05b7a114aca08ae926c854/py-polars/polars/lazyframe/frame.py#L505</a></p>\n<pre><code> :\n  ...\n\n   () -&gt; Self:\n</code></pre>\n<p>Using this method we can create a LazyFrame without any underlying data, that would be getting the data in compute time by calling <code>scan_fn</code>. I spent another hour to figure out that <code>scan_fn</code> should be actually pickle bytes of the function I want to run.</p>\n<p>Finally, I came up with the following unholy abomination:</p>\n<pre><code> ():\n   SESSION\n\nfunction_lazy_frame = pl.LazyFrame._scan_python_function(\n  schema,  \n  pickle.dumps(function_that_returns_global_session_variable),\n)\n</code></pre>\n<p>if we now call <code>features_lazy_frame.collect()</code> it will simply return the DataFrame from the global variable called <code>SESSION</code>.</p>\n<p>Now we can build a computation graph starting from this node that would be computing our features:</p>\n<pre><code>features_lazy_frame = reduce(\n  partial(LazyFrame.join, on=),\n  [\n    features_a(function_lazy_frame),\n    features_b(function_lazy_frame),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n</code></pre>\n<p>Computation graph is built and now we can start computing features</p>\n<pre><code>SESSION = session_1\nfeatures_lazy_frame.collect()  \n\nSESSION = session_2\nfeatures_lazy_frame.collect()  \n</code></pre>\n<p>Each call of <code>features_lazy_frame.collect()</code> takes input value from <code>SESSION</code> variable.</p>\n<p>This hack sped up my feature extraction 6x and allowed me to continue with my submissions. In my case it saved my submissions, but for someone else it could potentially win efficiency price in the future :)</p>\n<p>This is definitely useless for any production use (I hope polars comes up with official API for static graphs and will decouple them from data frames), but could come handy in similar competitions with efficiency price (if done carefully).</p>",
  "messages": [
    {
      "id": "2321950",
      "postDate": "06/29/2023 02:32:20",
      "content": "<p>Hey everyone!</p>\n<p>Congrats to the winners, and everybody who participated! This was a fun competition and I enjoyed it quite a lot.</p>\n<p>While we are all eagerly waiting for the solution write-ups of the top teams, I'd like to entertain you with a small Polars trick that I discovered during this competition. I found it in the time of great need, when I realized my submissions would fail the time limit due to slow feature extraction code.</p>\n<h2>A bit of context</h2>\n<p>I've created many different feature extractors each of which was returning a LazyFrame</p>\n<pre><code> () -&gt; LazyFrame:\n  ...\n\n () -&gt; LazyFrame:\n  ...\n\n...\n</code></pre>\n<p>each of the LazyFrames were then joined on <code>session_id</code> and only columns containing \"valuable\" features were selected (tens of thousands of features -&gt; thousands).</p>\n<pre><code>\nfeatures_lazy_frame = reduce(\n  partial(LazyFrame.join, on=),\n  [\n    features_a(events),\n    features_b(events),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n\n\nfeatures = features_lazy_frame.collect()\n</code></pre>\n<p>This worked perfectly fine in batch setting locally so I didn't expect any problems (and I wasn't submitting frequently).</p>\n<h2>Problem</h2>\n<p>~3 days before the end of the competition I realized that this was painfully slow in streaming setting and any of my submissions would go above 9 hour mark just because of feature extraction. What was the issue? Why was it fine when doing batch processing and suddenly didn't work when streaming?</p>\n<p>The problem is hidden in the fact that creating a <code>features_lazy_frame</code> also takes time.<br>\nHere are execution times for 1 game session:</p>\n<pre><code>%%timeit\nfeatures_lazy_frame = ...\n\n s ±  ms per  (mean ± . dev. of  runs,   )\n</code></pre>\n<pre><code>%%timeit\nfeatures_lazy_frame.collect()\n\n  .   loop (mean ± std. dev. of  runs,  loop each)\n</code></pre>\n<p>Constructing LazyFrame took 5/6th of the features extraction time! These 2.5 seconds didn't matter much in batch setting (we do it once for all the items), but in streaming we payed this cost for every single item coming our way.</p>\n<p>If only we could apply same computational graph to all new items, without rebuilding it from scratch every time…</p>\n<h2>Solution</h2>\n<p>I may be blind, but I couldn't find any documented solution to this problem - Polars doesn't seem to have an official support for static computational graphs. So I went digging through the source code.</p>\n<p>My initial understanding was, that LazyFrame is a computational graph that is tied to its original data and there is no way around it. Basically each LazyFrame contains and DataFrame inside.</p>\n<p>But then I came across this undocumented method <a href=\"https://github.com/pola-rs/polars/blob/822ed96f593ef71a7c05b7a114aca08ae926c854/py-polars/polars/lazyframe/frame.py#L505\" target=\"_blank\">https://github.com/pola-rs/polars/blob/822ed96f593ef71a7c05b7a114aca08ae926c854/py-polars/polars/lazyframe/frame.py#L505</a></p>\n<pre><code> :\n  ...\n\n   () -&gt; Self:\n</code></pre>\n<p>Using this method we can create a LazyFrame without any underlying data, that would be getting the data in compute time by calling <code>scan_fn</code>. I spent another hour to figure out that <code>scan_fn</code> should be actually pickle bytes of the function I want to run.</p>\n<p>Finally, I came up with the following unholy abomination:</p>\n<pre><code> ():\n   SESSION\n\nfunction_lazy_frame = pl.LazyFrame._scan_python_function(\n  schema,  \n  pickle.dumps(function_that_returns_global_session_variable),\n)\n</code></pre>\n<p>if we now call <code>features_lazy_frame.collect()</code> it will simply return the DataFrame from the global variable called <code>SESSION</code>.</p>\n<p>Now we can build a computation graph starting from this node that would be computing our features:</p>\n<pre><code>features_lazy_frame = reduce(\n  partial(LazyFrame.join, on=),\n  [\n    features_a(function_lazy_frame),\n    features_b(function_lazy_frame),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n</code></pre>\n<p>Computation graph is built and now we can start computing features</p>\n<pre><code>SESSION = session_1\nfeatures_lazy_frame.collect()  \n\nSESSION = session_2\nfeatures_lazy_frame.collect()  \n</code></pre>\n<p>Each call of <code>features_lazy_frame.collect()</code> takes input value from <code>SESSION</code> variable.</p>\n<p>This hack sped up my feature extraction 6x and allowed me to continue with my submissions. In my case it saved my submissions, but for someone else it could potentially win efficiency price in the future :)</p>\n<p>This is definitely useless for any production use (I hope polars comes up with official API for static graphs and will decouple them from data frames), but could come handy in similar competitions with efficiency price (if done carefully).</p>",
      "rawMarkdown": "Hey everyone!\n\nCongrats to the winners, and everybody who participated! This was a fun competition and I enjoyed it quite a lot.\n\nWhile we are all eagerly waiting for the solution write-ups of the top teams, I'd like to entertain you with a small Polars trick that I discovered during this competition. I found it in the time of great need, when I realized my submissions would fail the time limit due to slow feature extraction code.\n\n## A bit of context\n\nI've created many different feature extractors each of which was returning a LazyFrame\n\n```python\ndef features_a(events: DataFrame) -> LazyFrame:\n  ...\n  \ndef features_b(events: DataFrame) -> LazyFrame:\n  ...\n\n...\n```\n\neach of the LazyFrames were then joined on `session_id` and only columns containing \"valuable\" features were selected (tens of thousands of features -> thousands).\n\n```python\n# Create a computational graph (LazyFrame)\nfeatures_lazy_frame = reduce(\n  partial(LazyFrame.join, on='session_id'),\n  [\n    features_a(events),\n    features_b(events),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n\n# Calculate features\nfeatures = features_lazy_frame.collect()\n```\n\nThis worked perfectly fine in batch setting locally so I didn't expect any problems (and I wasn't submitting frequently).\n\n## Problem\n\n~3 days before the end of the competition I realized that this was painfully slow in streaming setting and any of my submissions would go above 9 hour mark just because of feature extraction. What was the issue? Why was it fine when doing batch processing and suddenly didn't work when streaming?\n\nThe problem is hidden in the fact that creating a `features_lazy_frame` also takes time.\nHere are execution times for 1 game session:\n\n```\n%%timeit\nfeatures_lazy_frame = ...\n\n2.49 s ± 13 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n```\n\n```\n%%timeit\nfeatures_lazy_frame.collect()\n\n486 ms ± 3.93 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n```\n\nConstructing LazyFrame took 5/6th of the features extraction time! These 2.5 seconds didn't matter much in batch setting (we do it once for all the items), but in streaming we payed this cost for every single item coming our way.\n\nIf only we could apply same computational graph to all new items, without rebuilding it from scratch every time...\n\n## Solution\n\nI may be blind, but I couldn't find any documented solution to this problem - Polars doesn't seem to have an official support for static computational graphs. So I went digging through the source code.\n\nMy initial understanding was, that LazyFrame is a computational graph that is tied to its original data and there is no way around it. Basically each LazyFrame contains and DataFrame inside.\n\nBut then I came across this undocumented method https://github.com/pola-rs/polars/blob/822ed96f593ef71a7c05b7a114aca08ae926c854/py-polars/polars/lazyframe/frame.py#L505\n\n\n```python\nclass LazyFrame:\n  ...\n  @classmethod\n  def _scan_python_function(\n      cls,\n      schema: pa.schema | dict[str, PolarsDataType],\n      scan_fn: Any,\n      pyarrow: bool = False,\n  ) -> Self:\n```\n\nUsing this method we can create a LazyFrame without any underlying data, that would be getting the data in compute time by calling `scan_fn`. I spent another hour to figure out that `scan_fn` should be actually pickle bytes of the function I want to run.\n\nFinally, I came up with the following unholy abomination:\n\n```python\ndef function_that_returns_global_session_variable(*args, **kwargs):\n  return SESSION\n\nfunction_lazy_frame = pl.LazyFrame._scan_python_function(\n  schema,  # Schema of the dataframe\n  pickle.dumps(function_that_returns_global_session_variable),\n)\n```\n\nif we now call `features_lazy_frame.collect()` it will simply return the DataFrame from the global variable called `SESSION`.\n\nNow we can build a computation graph starting from this node that would be computing our features:\n\n```python\nfeatures_lazy_frame = reduce(\n  partial(LazyFrame.join, on='session_id'),\n  [\n    features_a(function_lazy_frame),\n    features_b(function_lazy_frame),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n```\n\nComputation graph is built and now we can start computing features\n\n```python\nSESSION = session_1\nfeatures_lazy_frame.collect()  # Will build features for session_1!\n\nSESSION = session_2\nfeatures_lazy_frame.collect()  # Will build features for session_2!\n```\n\nEach call of `features_lazy_frame.collect()` takes input value from `SESSION` variable.\n\nThis hack sped up my feature extraction 6x and allowed me to continue with my submissions. In my case it saved my submissions, but for someone else it could potentially win efficiency price in the future :)\n\nThis is definitely useless for any production use (I hope polars comes up with official API for static graphs and will decouple them from data frames), but could come handy in similar competitions with efficiency price (if done carefully).",
      "votes": null
    },
    {
      "id": "2323392",
      "postDate": "06/29/2023 23:16:19",
      "content": "<p>Interesting! Did you compare the times if you did without lazy evaluation at all?</p>",
      "rawMarkdown": "Interesting! Did you compare the times if you did without lazy evaluation at all?",
      "votes": null
    },
    {
      "id": "2323713",
      "postDate": "06/30/2023 06:23:46",
      "content": "<p>For me in this competition I switched several times between different tools: Pandas -&gt; Polars -&gt; Lazy Polars -&gt; Lazy Polars with this hack, and every time I got a speed boost. It is very likely though that I've created overly complicated OOP code (tried to mimic scikit-learn transformers) that led to big performance degradation. Top contenders seem to have execution times &lt; 5 minutes, which is much better than anything I had.</p>\n<p>I'd say logically this should work same or faster on complicated feature engineering because it does most of the computations in Rust without returning control to Python.</p>\n<p>Made a small demo notebook: <a href=\"https://www.kaggle.com/code/informhunter/polars-static-graph-tests/notebook?scriptVersionId=135312235\" target=\"_blank\">https://www.kaggle.com/code/informhunter/polars-static-graph-tests/notebook?scriptVersionId=135312235</a></p>\n<p>In this simplistic example, when streaming, this hack improves execution time from 1:04 to 1:00, which is not a lot, but the difference gets much higher the more complex the computational graph is.</p>",
      "rawMarkdown": "For me in this competition I switched several times between different tools: Pandas -> Polars -> Lazy Polars -> Lazy Polars with this hack, and every time I got a speed boost. It is very likely though that I've created overly complicated OOP code (tried to mimic scikit-learn transformers) that led to big performance degradation. Top contenders seem to have execution times < 5 minutes, which is much better than anything I had.\n\nI'd say logically this should work same or faster on complicated feature engineering because it does most of the computations in Rust without returning control to Python.\n\nMade a small demo notebook: https://www.kaggle.com/code/informhunter/polars-static-graph-tests/notebook?scriptVersionId=135312235\n\nIn this simplistic example, when streaming, this hack improves execution time from 1:04 to 1:00, which is not a lot, but the difference gets much higher the more complex the computational graph is.",
      "votes": null
    },
    {
      "id": "2324445",
      "postDate": "06/30/2023 15:53:42",
      "content": "<p>That makes sense: when I compared lazy vs non-lazy my feature engineering just wasn't that complicated so it appeared that non-lazy was faster. </p>",
      "rawMarkdown": "That makes sense: when I compared lazy vs non-lazy my feature engineering just wasn't that complicated so it appeared that non-lazy was faster.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2323392,
      "author_name": "roberthatch",
      "author_url": "",
      "post_date": "06/29/2023 23:16:19",
      "content": "<p>Interesting! Did you compare the times if you did without lazy evaluation at all?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2323713,
          "author_name": "informhunter",
          "author_url": "",
          "post_date": "06/30/2023 06:23:46",
          "content": "<p>For me in this competition I switched several times between different tools: Pandas -&gt; Polars -&gt; Lazy Polars -&gt; Lazy Polars with this hack, and every time I got a speed boost. It is very likely though that I've created overly complicated OOP code (tried to mimic scikit-learn transformers) that led to big performance degradation. Top contenders seem to have execution times &lt; 5 minutes, which is much better than anything I had.</p>\n<p>I'd say logically this should work same or faster on complicated feature engineering because it does most of the computations in Rust without returning control to Python.</p>\n<p>Made a small demo notebook: <a href=\"https://www.kaggle.com/code/informhunter/polars-static-graph-tests/notebook?scriptVersionId=135312235\" target=\"_blank\">https://www.kaggle.com/code/informhunter/polars-static-graph-tests/notebook?scriptVersionId=135312235</a></p>\n<p>In this simplistic example, when streaming, this hack improves execution time from 1:04 to 1:00, which is not a lot, but the difference gets much higher the more complex the computational graph is.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2324445,
              "author_name": "roberthatch",
              "author_url": "",
              "post_date": "06/30/2023 15:53:42",
              "content": "<p>That makes sense: when I compared lazy vs non-lazy my feature engineering just wasn't that complicated so it appeared that non-lazy was faster. </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2321950": "Hey everyone!\n\nCongrats to the winners, and everybody who participated! This was a fun competition and I enjoyed it quite a lot.\n\nWhile we are all eagerly waiting for the solution write-ups of the top teams, I'd like to entertain you with a small Polars trick that I discovered during this competition. I found it in the time of great need, when I realized my submissions would fail the time limit due to slow feature extraction code.\n\n## A bit of context\n\nI've created many different feature extractors each of which was returning a LazyFrame\n\n```python\ndef features_a(events: DataFrame) -> LazyFrame:\n  ...\n  \ndef features_b(events: DataFrame) -> LazyFrame:\n  ...\n\n...\n```\n\neach of the LazyFrames were then joined on `session_id` and only columns containing \"valuable\" features were selected (tens of thousands of features -> thousands).\n\n```python\n# Create a computational graph (LazyFrame)\nfeatures_lazy_frame = reduce(\n  partial(LazyFrame.join, on='session_id'),\n  [\n    features_a(events),\n    features_b(events),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n\n# Calculate features\nfeatures = features_lazy_frame.collect()\n```\n\nThis worked perfectly fine in batch setting locally so I didn't expect any problems (and I wasn't submitting frequently).\n\n## Problem\n\n~3 days before the end of the competition I realized that this was painfully slow in streaming setting and any of my submissions would go above 9 hour mark just because of feature extraction. What was the issue? Why was it fine when doing batch processing and suddenly didn't work when streaming?\n\nThe problem is hidden in the fact that creating a `features_lazy_frame` also takes time.\nHere are execution times for 1 game session:\n\n```\n%%timeit\nfeatures_lazy_frame = ...\n\n2.49 s ± 13 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n```\n\n```\n%%timeit\nfeatures_lazy_frame.collect()\n\n486 ms ± 3.93 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n```\n\nConstructing LazyFrame took 5/6th of the features extraction time! These 2.5 seconds didn't matter much in batch setting (we do it once for all the items), but in streaming we payed this cost for every single item coming our way.\n\nIf only we could apply same computational graph to all new items, without rebuilding it from scratch every time...\n\n## Solution\n\nI may be blind, but I couldn't find any documented solution to this problem - Polars doesn't seem to have an official support for static computational graphs. So I went digging through the source code.\n\nMy initial understanding was, that LazyFrame is a computational graph that is tied to its original data and there is no way around it. Basically each LazyFrame contains and DataFrame inside.\n\nBut then I came across this undocumented method https://github.com/pola-rs/polars/blob/822ed96f593ef71a7c05b7a114aca08ae926c854/py-polars/polars/lazyframe/frame.py#L505\n\n\n```python\nclass LazyFrame:\n  ...\n  @classmethod\n  def _scan_python_function(\n      cls,\n      schema: pa.schema | dict[str, PolarsDataType],\n      scan_fn: Any,\n      pyarrow: bool = False,\n  ) -> Self:\n```\n\nUsing this method we can create a LazyFrame without any underlying data, that would be getting the data in compute time by calling `scan_fn`. I spent another hour to figure out that `scan_fn` should be actually pickle bytes of the function I want to run.\n\nFinally, I came up with the following unholy abomination:\n\n```python\ndef function_that_returns_global_session_variable(*args, **kwargs):\n  return SESSION\n\nfunction_lazy_frame = pl.LazyFrame._scan_python_function(\n  schema,  # Schema of the dataframe\n  pickle.dumps(function_that_returns_global_session_variable),\n)\n```\n\nif we now call `features_lazy_frame.collect()` it will simply return the DataFrame from the global variable called `SESSION`.\n\nNow we can build a computation graph starting from this node that would be computing our features:\n\n```python\nfeatures_lazy_frame = reduce(\n  partial(LazyFrame.join, on='session_id'),\n  [\n    features_a(function_lazy_frame),\n    features_b(function_lazy_frame),\n    ...\n  ]),\n).select(SELECTED_FEATURES)\n```\n\nComputation graph is built and now we can start computing features\n\n```python\nSESSION = session_1\nfeatures_lazy_frame.collect()  # Will build features for session_1!\n\nSESSION = session_2\nfeatures_lazy_frame.collect()  # Will build features for session_2!\n```\n\nEach call of `features_lazy_frame.collect()` takes input value from `SESSION` variable.\n\nThis hack sped up my feature extraction 6x and allowed me to continue with my submissions. In my case it saved my submissions, but for someone else it could potentially win efficiency price in the future :)\n\nThis is definitely useless for any production use (I hope polars comes up with official API for static graphs and will decouple them from data frames), but could come handy in similar competitions with efficiency price (if done carefully).",
    "2323392": "Interesting! Did you compare the times if you did without lazy evaluation at all?",
    "2323713": "For me in this competition I switched several times between different tools: Pandas -> Polars -> Lazy Polars -> Lazy Polars with this hack, and every time I got a speed boost. It is very likely though that I've created overly complicated OOP code (tried to mimic scikit-learn transformers) that led to big performance degradation. Top contenders seem to have execution times < 5 minutes, which is much better than anything I had.\n\nI'd say logically this should work same or faster on complicated feature engineering because it does most of the computations in Rust without returning control to Python.\n\nMade a small demo notebook: https://www.kaggle.com/code/informhunter/polars-static-graph-tests/notebook?scriptVersionId=135312235\n\nIn this simplistic example, when streaming, this hack improves execution time from 1:04 to 1:00, which is not a lot, but the difference gets much higher the more complex the computational graph is.",
    "2324445": "That makes sense: when I compared lazy vs non-lazy my feature engineering just wasn't that complicated so it appeared that non-lazy was faster."
  },
  "source": "meta"
}