{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":87793,"databundleVersionId":12276181,"sourceType":"competition"},{"sourceId":10855324,"sourceType":"datasetVersion","datasetId":6742586},{"sourceId":11855248,"sourceType":"datasetVersion","datasetId":7449300},{"sourceId":11855563,"sourceType":"datasetVersion","datasetId":7449485},{"sourceId":240377984,"sourceType":"kernelVersion"}],"dockerImageVersionId":31041,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![RNA Types](https://raw.githubusercontent.com/ammomahdavikia/asset-holding/main/NuFold_logo.png)","metadata":{}},{"cell_type":"markdown","source":"In this notebook, we are implementing [NuFold](https://github.com/kiharalab/NuFold) RNA folding framework for inference. We will be using CASP15 RNA targets as validation set and submit the notebook to Staford RNA 3D Folding challenge to evaluate its real-world performance on recent RNA structures.\n\nNuFold shares the basic framework of deep learning architecture as AF2, which consists of three main modules:\n1. **Preprocessing**: NuFold incorporates predicted secondary structure information along with an MSA for a target RNA.\n    + **Secondary Structure**: predicted using IPknot, as it can predict pseudo-knots.\n    + **Multiple Sequence Alignment**: generated by rMSA pipeline.\n2. **EvoFormer**: The EvoFormer block is essentially the same as AF2 except that it considers the seven letters to represent RNA sequences, which are four standard nucleotides, unknown, gaps, and masked\nposition.\n3. **Structure**: The Structure module defines the base frame with the following four atoms: O4’, C1’, C2’, and the first nitrogen of the base (N1 for `C` and `U`, N9 for `G` and `A`). All other atoms are partitioned into ten frames, which are then iteratively bonded using predicted torsion angles on the bonds between frames as a guiding principle.\n\n![NuFold Architecture](https://raw.githubusercontent.com/ammomahdavikia/asset-holding/main/NuFold_architecture.png)","metadata":{}},{"cell_type":"markdown","source":"### ↓ Libraries","metadata":{}},{"cell_type":"code","source":"deps_path = '/kaggle/input/nufold-dependencies/NuFold/dependencies'\n\npackages = [\n    \"torch\",\n    \"torchvision\",\n    \"torchaudio\"\n    \"protobuf\",\n    \"biopython\",\n    \"ml_collections\",\n    \"dm_tree\",\n    \"deepspeed\"\n]\n\nfor pkg in packages:\n    print(f\"Installing {pkg}...\")\n    !pip install --no-index --find-links={deps_path} {pkg}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:38:46.910481Z","iopub.execute_input":"2025-05-18T12:38:46.910718Z","iopub.status.idle":"2025-05-18T12:42:33.483462Z","shell.execute_reply.started":"2025-05-18T12:38:46.910700Z","shell.execute_reply":"2025-05-18T12:42:33.482741Z"},"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### ↓ Imports","metadata":{}},{"cell_type":"code","source":"import os\nimport re\nimport requests\nimport zipfile\nfrom glob import glob\nfrom tqdm import tqdm\nimport subprocess\nfrom datetime import datetime\nimport warnings\n\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nimport matplotlib.font_manager as fm\nimport seaborn as sns\nfrom Bio import SeqIO\nfrom Bio.PDB import PDBParser","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:33.485087Z","iopub.execute_input":"2025-05-18T12:42:33.485619Z","iopub.status.idle":"2025-05-18T12:42:35.206840Z","shell.execute_reply.started":"2025-05-18T12:42:33.485585Z","shell.execute_reply":"2025-05-18T12:42:35.206176Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"warnings.simplefilter(action='ignore', category=FutureWarning)\n\ncolors = ['#c1121f', '#ffb703', '#003049']\nprint('\\n----- Color -----\\n')\nsns.palplot(sns.color_palette(colors))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:35.207671Z","iopub.execute_input":"2025-05-18T12:42:35.208081Z","iopub.status.idle":"2025-05-18T12:42:35.414251Z","shell.execute_reply.started":"2025-05-18T12:42:35.208059Z","shell.execute_reply":"2025-05-18T12:42:35.413681Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"PATH = '/kaggle/input/stanford-rna-3d-folding'\nSS = '/kaggle/working/secondary_structure'\nMODEL = '/kaggle/input/nufold-checkpoint/global_step145245.pt'\nOUTPUT = '/kaggle/working/nufold_pdb'\n\nos.makedirs(OUTPUT, exist_ok=True)\nos.makedirs(SS, exist_ok=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:35.415611Z","iopub.execute_input":"2025-05-18T12:42:35.415794Z","iopub.status.idle":"2025-05-18T12:42:35.419794Z","shell.execute_reply.started":"2025-05-18T12:42:35.415779Z","shell.execute_reply":"2025-05-18T12:42:35.419148Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Load the data","metadata":{}},{"cell_type":"code","source":"test_seq = pd.read_csv(os.path.join(PATH, 'test_sequences.csv'))\ntest_seq.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:35.420481Z","iopub.execute_input":"2025-05-18T12:42:35.420919Z","iopub.status.idle":"2025-05-18T12:42:35.469272Z","shell.execute_reply.started":"2025-05-18T12:42:35.420896Z","shell.execute_reply":"2025-05-18T12:42:35.468689Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Preprocessing\n\nFor the secondary structure, we will use IPknot which can predict pseudoknots. For the MSA, we will use kaggle provided MSAs which are already extracted using rMSA pipeline. Provided MSA structures are in `FASTA` format and we will change it to `A3M`.","metadata":{}},{"cell_type":"code","source":"!cp /kaggle/input/ipknot-1-1-0-x86-64-linux/ipknot /kaggle/working/\n!chmod +x /kaggle/working/ipknot","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:35.469931Z","iopub.execute_input":"2025-05-18T12:42:35.470118Z","iopub.status.idle":"2025-05-18T12:42:35.778648Z","shell.execute_reply.started":"2025-05-18T12:42:35.470103Z","shell.execute_reply":"2025-05-18T12:42:35.777826Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def fasta_to_a3m_msa(input_file, output_file):\n    records = list(SeqIO.parse(input_file, \"fasta\"))\n    if not records:\n        raise ValueError(\"Empty MSA\")\n\n    query = str(records[0].seq)\n    output_lines = [f\">{records[0].id}\\n{query}\"]  # Keep query as-is\n\n    for record in records[1:]:\n        aligned = str(record.seq)\n        # Remove characters aligned to gaps in the query\n        compressed = ''.join([\n            res for res, q in zip(aligned, query) if q != '-'\n        ])\n        output_lines.append(f\">{record.id}\\n{compressed}\")\n\n    with open(output_file, \"w\") as f:\n        f.write(\"\\n\".join(output_lines) + \"\\n\")\n\ndef write_fasta(sequence, target_id, output_file):\n    with open(output_file, \"w\") as f:\n        f.write(f\">{target_id}\\n{sequence}\\n\")\n\nfor idx, rna in tqdm(test_seq.iterrows(), total=len(test_seq)):\n    target_id = rna.target_id\n    sequence = rna.sequence\n\n    os.makedirs(os.path.join(SS, target_id), exist_ok=True)\n    \n    fasta_path = os.path.join(SS, target_id, f\"{target_id}.fasta\")\n    msa_path = os.path.join(PATH, 'MSA', f'{target_id}.MSA.fasta')\n    a3m_path = os.path.join(SS, target_id, f'{target_id}.a3m')\n    ipknot_path = os.path.join(SS, target_id, f\"{target_id}.ipknot.ss\")\n    \n    write_fasta(sequence, target_id, fasta_path)\n    fasta_to_a3m_msa(msa_path, a3m_path)\n    os.system(f'/kaggle/working/ipknot {fasta_path} > {ipknot_path}')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:35.779685Z","iopub.execute_input":"2025-05-18T12:42:35.779939Z","iopub.status.idle":"2025-05-18T12:42:41.237092Z","shell.execute_reply.started":"2025-05-18T12:42:35.779903Z","shell.execute_reply":"2025-05-18T12:42:41.236475Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!rm /kaggle/working/ipknot","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:41.237865Z","iopub.execute_input":"2025-05-18T12:42:41.238177Z","iopub.status.idle":"2025-05-18T12:42:41.363343Z","shell.execute_reply.started":"2025-05-18T12:42:41.238149Z","shell.execute_reply":"2025-05-18T12:42:41.362447Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Folding","metadata":{}},{"cell_type":"code","source":"!cp -r /kaggle/input/nufold-dependencies/NuFold /kaggle/working/\n%cd NuFold","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:41.364405Z","iopub.execute_input":"2025-05-18T12:42:41.364762Z","iopub.status.idle":"2025-05-18T12:42:47.464340Z","shell.execute_reply.started":"2025-05-18T12:42:41.364737Z","shell.execute_reply":"2025-05-18T12:42:47.463285Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for idx, rna in test_seq.iterrows():\n    target_id = rna.target_id\n    length = len(rna.sequence)\n    warnings.filterwarnings('ignore', category=FutureWarning)\n    fasta_path = os.path.join(SS, target_id, f'{target_id}.fasta')\n    \n    command = [\n        \"python3\", \"run_nufold.py\",\n        \"--ckpt_path\", MODEL,\n        \"--input_fasta\", fasta_path,\n        \"--input_dir\", SS,\n        \"--output_dir\", OUTPUT,\n        \"--recycle\", \"4\",\n        \"--config_preset\", \"initial_training\"\n    ]\n    \n    print(f\"\\n----- Running NuFold for {target_id} (#{length}) -----\\n\")\n    \n    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n    for line in process.stdout:\n        print(line, end='')\n    process.wait()\n    \n    if process.returncode != 0:\n        print(f\"[{datetime.now()}] [FAIL] NuFolding failed for {target_id} with return code {process.returncode}!\")\n    else:\n        print(f\"[{datetime.now()}] [SUCCESS] NuFolding completed for {target_id}!\")\n    \n    os.system(f'rm {os.path.join(OUTPUT, target_id, f\"{target_id}_in_0.pkl\")}')\n    os.system(f'rm {os.path.join(OUTPUT, target_id, f\"{target_id}_out_0.pkl\")}')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T12:42:47.467507Z","iopub.execute_input":"2025-05-18T12:42:47.468092Z","iopub.status.idle":"2025-05-18T13:04:29.148478Z","shell.execute_reply.started":"2025-05-18T12:42:47.468067Z","shell.execute_reply":"2025-05-18T13:04:29.147699Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!rm -r /kaggle/working/NuFold\n%cd /kaggle/working/","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:29.149474Z","iopub.execute_input":"2025-05-18T13:04:29.149772Z","iopub.status.idle":"2025-05-18T13:04:29.572617Z","shell.execute_reply.started":"2025-05-18T13:04:29.149747Z","shell.execute_reply":"2025-05-18T13:04:29.571690Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Submission","metadata":{}},{"cell_type":"code","source":"submission = pd.read_csv(os.path.join(PATH, 'sample_submission.csv'))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:29.573825Z","iopub.execute_input":"2025-05-18T13:04:29.574151Z","iopub.status.idle":"2025-05-18T13:04:29.592708Z","shell.execute_reply.started":"2025-05-18T13:04:29.574094Z","shell.execute_reply":"2025-05-18T13:04:29.592179Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for idx, rna in test_seq.iterrows():\n    target_id = rna.target_id\n    prediction_path = os.path.join(OUTPUT, target_id)\n    predictions = glob(os.path.join(prediction_path, '*.pdb'))\n\n    for i, sample in enumerate(predictions):\n        parser = PDBParser(QUIET=True)\n        structure = parser.get_structure(\"rna\", sample)\n        for model in structure:\n            for chain in model:\n                for residue in chain:\n                    res_number = residue.get_id()[1]\n                    res_id = f'{target_id}_{res_number}'\n                    for atom in residue:\n                        if atom.get_name() == \"C1'\":\n                            x, y, z = atom.coord\n                            submission.loc[submission.ID == res_id, f'x_{i+1}'] = float(x)\n                            submission.loc[submission.ID == res_id, f'y_{i+1}'] = float(y)\n                            submission.loc[submission.ID == res_id, f'z_{i+1}'] = float(z)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:29.593500Z","iopub.execute_input":"2025-05-18T13:04:29.593733Z","iopub.status.idle":"2025-05-18T13:04:50.407072Z","shell.execute_reply.started":"2025-05-18T13:04:29.593715Z","shell.execute_reply":"2025-05-18T13:04:50.406307Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:50.407905Z","iopub.execute_input":"2025-05-18T13:04:50.408667Z","iopub.status.idle":"2025-05-18T13:04:50.425798Z","shell.execute_reply.started":"2025-05-18T13:04:50.408640Z","shell.execute_reply":"2025-05-18T13:04:50.425191Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print('Submission.csv shape:', submission.shape)\nsubmission.to_csv('/kaggle/working/submission.csv', index=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:50.426516Z","iopub.execute_input":"2025-05-18T13:04:50.426731Z","iopub.status.idle":"2025-05-18T13:04:50.507926Z","shell.execute_reply.started":"2025-05-18T13:04:50.426716Z","shell.execute_reply":"2025-05-18T13:04:50.507173Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!rm -r {SS}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:07:25.718362Z","iopub.execute_input":"2025-05-18T13:07:25.718668Z","iopub.status.idle":"2025-05-18T13:07:25.852922Z","shell.execute_reply.started":"2025-05-18T13:07:25.718642Z","shell.execute_reply":"2025-05-18T13:07:25.852023Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Evaluation","metadata":{}},{"cell_type":"code","source":"max_images = 12\ntest_seq_subset = test_seq.iloc[:max_images]\n\nncols = 2\nnrows = max_images // ncols + (max_images % ncols > 0)\n\nfig, axs = plt.subplots(nrows=nrows, ncols=ncols, figsize=(15, 5 * nrows), dpi=150)\naxs = axs.flatten()\n\nfor idx, (i, rna) in enumerate(test_seq_subset.iterrows()):\n    target_id = rna.target_id\n    plddt_plot_path = os.path.join(OUTPUT, target_id, f'{target_id}_plddt_plot.png')\n\n    if os.path.exists(plddt_plot_path):\n        img = mpimg.imread(plddt_plot_path)\n        axs[idx].imshow(img, interpolation='none')\n        axs[idx].axis('off')\n        axs[idx].set_title(target_id, fontsize=10)\n    else:\n        axs[idx].set_visible(False)\n\nfor j in range(len(test_seq_subset), len(axs)):\n    axs[j].set_visible(False)\n\nplt.tight_layout(rect=[0, 0, 1, 0.95])\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:50.636615Z","iopub.execute_input":"2025-05-18T13:04:50.636885Z","iopub.status.idle":"2025-05-18T13:04:59.471698Z","shell.execute_reply.started":"2025-05-18T13:04:50.636861Z","shell.execute_reply":"2025-05-18T13:04:59.470896Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!rm -r {OUTPUT}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:07:20.323732Z","iopub.execute_input":"2025-05-18T13:07:20.324002Z","iopub.status.idle":"2025-05-18T13:07:20.463435Z","shell.execute_reply.started":"2025-05-18T13:07:20.323986Z","shell.execute_reply":"2025-05-18T13:07:20.462322Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"solution = pd.read_csv(os.path.join(PATH, 'validation_labels.csv'))\nsolution.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:59.606689Z","iopub.execute_input":"2025-05-18T13:04:59.606975Z","iopub.status.idle":"2025-05-18T13:04:59.693575Z","shell.execute_reply.started":"2025-05-18T13:04:59.606947Z","shell.execute_reply":"2025-05-18T13:04:59.692973Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def parse_tmscore_output(output):\n    # Extract TM-score based on length of reference structure (second)\n    tm_score_match = re.findall(r'TM-score=\\s+([\\d.]+)', output)[1]\n    if not tm_score_match:\n        raise ValueError('No TM score found')\n    return float(tm_score_match)\n\n\ndef write_target_line(\n    atom_name, atom_serial, residue_name, chain_id, residue_num, x_coord, y_coord, z_coord, occupancy=1.0, b_factor=0.0, atom_type='P'\n) -> str:\n    \"\"\"\n    Writes a single line of PDB format based on provided atom information.\n\n    Args:\n        atom_name (str): Name of the atom (e.g., \"N\", \"CA\").\n        atom_serial (int): Atom serial number.\n        residue_name (str): Residue name (e.g., \"ALA\").\n        chain_id (str): Chain identifier.\n        residue_num (int): Residue number.\n        x_coord (float): X coordinate.\n        y_coord (float): Y coordinate.\n        z_coord (float): Z coordinate.\n        occupancy (float, optional): Occupancy value (default: 1.0).\n        b_factor (float, optional): B-factor value (default: 0.0).\n\n    Returns:\n        str: A single line of PDB string.\n    \"\"\"\n    return f'ATOM  {atom_serial:>5d}  {atom_name:<5s} {residue_name:<3s} {residue_num:>3d}    {x_coord:>8.3f}{y_coord:>8.3f}{z_coord:>8.3f}{occupancy:>6.2f}{b_factor:>6.2f}           {atom_type}\\n'\n\n\ndef write2pdb(df: pd.DataFrame, xyz_id: str, target_path: str) -> int:\n    resolved_cnt = 0\n    with open(target_path, 'w') as target_file:\n        for _, row in df.iterrows():\n            x_coord = row[f'x_{xyz_id}']\n            y_coord = row[f'y_{xyz_id}']\n            z_coord = row[f'z_{xyz_id}']\n\n            if x_coord > -1e17 and y_coord > -1e17 and z_coord > -1e17:\n                # if True:\n                resolved_cnt += 1\n                target_line = write_target_line(\n                    atom_name=\"C1'\",\n                    atom_serial=int(row['resid']),\n                    residue_name=row['resname'],\n                    chain_id='0',\n                    residue_num=int(row['resid']),\n                    x_coord=x_coord,\n                    y_coord=y_coord,\n                    z_coord=z_coord,\n                    atom_type='C',\n                )\n                target_file.write(target_line)\n    return resolved_cnt\n\n\ndef score(solution: pd.DataFrame, submission: pd.DataFrame, row_id_column_name: str) -> float:\n    \"\"\"\n    Computes the TM-score between predicted and native RNA structures using USalign.\n\n    This function evaluates the structural similarity of RNA predictions to native structures\n    by computing the TM-score. It uses USalign, a structural alignment tool, to compare\n    the predicted structures with the native structures.\n\n    Workflow:\n    1. Copies the USalign binary to the working directory and grants execution permissions.\n    2. Extracts the `target_id` from the `ID` column of both the solution and submission DataFrames.\n    3. Iterates over each unique `target_id`, grouping the native and predicted structures.\n    4. Writes PDB files for native and predicted structures.\n    5. Runs USalign on each predicted-native pair and extracts the TM-score.\n    6. Computes the highest TM-score per target and returns aggregated results.\n\n    Args:\n        solution (pd.DataFrame): A DataFrame containing the native RNA structures.\n        submission (pd.DataFrame): A DataFrame containing the predicted RNA structures.\n        row_id_column_name (str): The name of the column containing unique row identifiers.\n\n    Returns:\n        float: the average highest TM-scores.\n    \"\"\"\n\n    os.system('cp //kaggle/input/usalign/USalign /kaggle/working/')\n    os.system('sudo chmod u+x /kaggle/working//USalign')\n\n    # Extract target_id from ID (target_resid)\n    solution['target_id'] = solution['ID'].apply(lambda x: x.split('_')[0])\n    submission['target_id'] = submission['ID'].apply(lambda x: x.split('_')[0])\n\n    results = {}\n    # Iterate through each target_id and generate PDB files for both clean and corrupted data\n    for target_id, group_native in tqdm(solution.groupby('target_id'), desc='TM-scoring'):\n        group_predicted = submission[submission['target_id'] == target_id]\n        native_pdb = 'native.pdb'\n        predicted_pdb = 'predicted.pdb'\n\n        target_id_scores = []\n        for pred_cnt in range(1, 6):\n            prediction_scores = []\n            for native_cnt in range(1, 41):\n                # Write solution PDB\n                resolved_cnt = write2pdb(group_native, native_cnt, native_pdb)\n\n                # Write predicted PDB\n                _ = write2pdb(group_predicted, pred_cnt, predicted_pdb)\n\n                if resolved_cnt > 0:\n                    command = f'/kaggle/working/USalign {predicted_pdb} {native_pdb} -atom \" C1\\'\"'\n                    usalign_output = os.popen(command).read()\n                    prediction_scores.append(parse_tmscore_output(usalign_output))\n\n            target_id_scores.append(max(prediction_scores))\n        results[target_id] = max(target_id_scores)\n\n    return results","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:59.694245Z","iopub.execute_input":"2025-05-18T13:04:59.694420Z","iopub.status.idle":"2025-05-18T13:04:59.705619Z","shell.execute_reply.started":"2025-05-18T13:04:59.694407Z","shell.execute_reply":"2025-05-18T13:04:59.704967Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"scores = score(solution, submission, 'ID')\nresult = {'target_id': [],\n          'length': [],\n          'tm_score': []}\n\nfor idx, rna in test_seq.iterrows():\n    \n    target_id = rna.target_id\n    length = len(rna.sequence)\n    tm_score = scores[target_id]\n    \n    result['target_id'].append(target_id)\n    result['length'].append(length)\n    result['tm_score'].append(tm_score)\n\nresult_df = pd.DataFrame(result)\nresult_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:04:59.706439Z","iopub.execute_input":"2025-05-18T13:04:59.706663Z","iopub.status.idle":"2025-05-18T13:06:17.631968Z","shell.execute_reply.started":"2025-05-18T13:04:59.706645Z","shell.execute_reply":"2025-05-18T13:06:17.631397Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def plot_results(df, desc, axs=None):\n    if axs is None:\n        fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(15, 5), gridspec_kw={'width_ratios': [1, 2]})\n    else:\n        fig = None\n\n    df['color'] = df['length'].apply(lambda x: colors[0] if x > 400 else colors[-1])\n\n    sns.scatterplot(data=df, x='length', y='tm_score', hue='color', palette={colors[0]: colors[0], colors[-1]: colors[-1]}, ax=axs[0], legend=False)\n    axs[0].set_title('TM-score per length')\n    axs[0].set_xlabel('RNA length')\n    axs[0].set_ylabel('TM-score')\n    axs[0].set_ylim(0., 1.)\n\n    mean_tm = df['tm_score'].mean()\n    axs[0].axhline(mean_tm, color=colors[1], linestyle='--', label=f'Mean = {mean_tm:.2f}')\n    axs[0].legend()\n\n    sns.barplot(data=df, x='target_id', y='tm_score', palette=df['color'].to_list(), ax=axs[1])\n    axs[1].set_title('TM-score per target')\n    axs[1].set_xlabel('Target id')\n    axs[1].set_ylabel('TM-score')\n    axs[1].set_ylim(0., 1.)\n    axs[1].axhline(mean_tm, color=colors[1], linestyle='--', label=f'Mean = {mean_tm:.2f}')\n    axs[1].legend()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:06:17.639822Z","iopub.execute_input":"2025-05-18T13:06:17.640084Z","iopub.status.idle":"2025-05-18T13:06:17.661894Z","shell.execute_reply.started":"2025-05-18T13:06:17.640063Z","shell.execute_reply":"2025-05-18T13:06:17.661080Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plot_results(result_df, desc='Results')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:06:17.662628Z","iopub.execute_input":"2025-05-18T13:06:17.662869Z","iopub.status.idle":"2025-05-18T13:06:18.073982Z","shell.execute_reply.started":"2025-05-18T13:06:17.662844Z","shell.execute_reply":"2025-05-18T13:06:18.073287Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def log_resluts(result_df):\n    long_score = result_df.loc[result_df.length > 400, 'tm_score'].mean()\n    short_score = result_df.loc[result_df.length <= 400, 'tm_score'].mean()\n    all_score = result_df.tm_score.mean()\n\n    print(f'[TM-score] > 400: {long_score:.4f}')\n    print(f'[TM-score] <= 400: {short_score:.4f}')\n    print(f'[TM-score] all: {all_score:.4f}')\n\nlog_resluts(result_df)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-05-18T13:06:18.074687Z","iopub.execute_input":"2025-05-18T13:06:18.074906Z","iopub.status.idle":"2025-05-18T13:06:18.081081Z","shell.execute_reply.started":"2025-05-18T13:06:18.074890Z","shell.execute_reply":"2025-05-18T13:06:18.080321Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Reference\n\n[Kagaya et al., 2025]  \nYuki Kagaya, Zicong Zhang, Nabil Ibtehaz, Xiao Wang, Tsukasa Nakamura, Pranav Deep Punuru, and Daisuke Kihara.  \n**NuFold: end-to-end approach for RNA tertiary structure prediction with flexible nucleobase center representation.**  \n*Nature Communications*, 2025. [https://www.nature.com/articles/s41467-025-56261-7](https://www.nature.com/articles/s41467-025-56261-7)\n\n[![GitHub](https://img.shields.io/badge/GitHub-Repo-black?logo=github)](https://github.com/kiharalab/NuFold)","metadata":{}}]}