{
  "id": 579625,
  "title": "Cluster score vulnerability",
  "url": "/competitions/image-matching-challenge-2025/discussion/579625",
  "author_name": "",
  "post_date": "2025-05-19T05:40:29.580256300Z",
  "votes": 20,
  "comment_count": 10,
  "views": 0,
  "content": "<p>The cluster score’s been odd to me all along, and here’s what I got on my CV set today:</p>\n<pre><code>final_score, dataset_scores = metric.score(\n    gt_csv=,\n    user_csv=submission_file,\n    thresholds_csv=f,\n    mask_csv=   RERUN  os.path.(data_dir, ),\n    inl_cf=,\n    strict_cf=,\n    =,\n)\n</code></pre>\n<p>./filtered_train_labels.csv</p>\n<p>dataset    scene   image   rotation_matrix translation_vector<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1310.png -0.45096986446600007;0.27640276616340004;-0.8486619380236001;-0.22647640738860003;0.88428353942478;0.40835157692580004;0.8633272908284001;0.3763563262902;-0.3361864742252201   530.376;-32.5758;8.66025<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1200.png -0.49438604111399975;0.2992504042362;-0.8161052283251999;-0.2008084914918;0.8741562895071799;0.4421836760036;0.8457272006268;0.38249137133239997;-0.37207832170281985   540.488;-56.942;30.3337<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1255.png -0.480305983746;0.25415113510376;-0.8394725706204399;-0.25052886194824;0.8774722137962327;0.40899617651612;0.8405606971715599;0.40675518877988;-0.35778302927376715 536.308;-28.765;27.6173<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1220.png -0.4723317373060001;0.31990109336960004;-0.8213197004336;-0.21688922167040003;0.8609798276824799;0.460079128708;0.8543195101264;0.395445356036;-0.33728500142751994 537.855;-63.0646;13.3448<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1160.png -0.5179049924259997;0.26548250388814;-0.8132001242871599;-0.23628750129585996;0.8692429742569598;0.43426369078175997;0.82215790377684;0.4170560831942399;-0.38745516311304007   544.394;-46.0541;47.5699<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1155.png -0.511186782162;0.2588402620089;-0.8195656146679201;-0.23862885722309998;0.8733372238454078;0.42466244791394;0.8256769019920801;0.41265438920605996;-0.38467152209259226    545.661;-40.4273;44.9054<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1190.png -0.5485391152799999;0.21904702067452;-0.8069216936175599;-0.22989207292548;0.8883665315084622;0.39743489486199995;0.80389915518244;0.40351364733;-0.4369465727475377    539.761;-27.2237;73.5853<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1235.png -0.51286548354;0.272770387957;-0.813981831169;-0.229720591739;0.8699931798735;0.436280199082;0.8271629559829999;0.41074113238599996;-0.38352859310249987    536.302;-47.537;43.3903<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1230.png -0.5217347486800001;0.28092921213600003;-0.8055244708920001;-0.22363741749600002;0.8661653987119999;0.446926879356;0.823272252012;0.41332324851600005;-0.38908213179200013  535.852;-54.4949;47.289<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1215.png -0.4846348040199999;0.29270774428199997;-0.824289704106;-0.203701108158;0.87867415198;0.431784408786;0.850668677214;0.377165870106;-0.36621136387599984 539.547;-50.5375;25.704<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1475.png -0.43655440942599966;0.2441491108092;-0.8659165170531999;-0.1735002444348;0.92156072964912;0.3473089176852;0.8827898209628;0.30185597693879995;-0.3599514672608797  512.368;-10.5897;6.65932<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0090.png 0.60940070194928;-0.1179033225381;0.78404708259042;0.1108402345611;0.991839419534035;0.0630001048134;-0.78507672464958;0.0485116206546;0.617496065484755    -488.723;62.0719;-94.8123<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0190.png 0.53043187119032;-0.11174926732833998;0.8403297313677439;0.11627415416605999;0.9914953705220798;0.05845724906793999;-0.8397155930682559;0.06670105057045998;0.5389142975357599  -505.153;63.6353;-47.3736<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0265.png 0.5024316490966199;-0.1440664075504;0.8525304111638;0.11003796779840001;0.98865702691062;0.10222009038440001;-0.8575866588202;0.0424519899524;0.51258533799 -515.543;45.4976;-34.9178<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0290.png 0.1827152068924799;-0.1825841649328;0.96606405046064;0.0934499591504;0.981381045126;0.16780460438720002;-0.97871539888336;0.05961797408960001;0.19637569237751995   -502.175;18.5164;138.079<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1390.png -0.61088428465;0.24389941925167996;-0.7532147358502399;-0.21021443341232;0.8672501987647968;0.45131685122223997;0.76330155050176;0.43403926262576;-0.4785182546492033   508.898;-57.0345;102.976<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1555.png -0.3896142358800001;0.306820506496;-0.8683673801680001;-0.262779872416;0.866645016368;0.42411455129600006;0.882693303928;0.3934305412160001;-0.25703098064800023    502.466;-31.5815;-28.0105<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0075.png 0.60266391944928;-0.1765441431392;0.77822118014776;0.128370227696;0.9839678709467;0.12380754060399998;-0.78760213582024;0.025286122525599995;0.61566495143342   -490.228;30.7243;-95.8173<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0140.png 0.61631690468582;-0.1323900181294;0.77629008172948;0.0989362256134;0.99097406046054;0.0904545920566;-0.78125861989052;0.02105452593739999;0.62385223567472  -492.994;52.8367;-102.547<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0280.png 0.3507627593310202;-0.15963154405563998;0.9227583533427759;0.14714297501276;0.9825194375641688;0.11403724051375999;-0.924831959561224;0.09577743712383999;0.36811989677714896   -516.737;31.1435;52.1067<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0275.png 0.54473792542488;-0.08624649615808;0.834159695580608;0.07852330167232;0.9955730324064128;0.05165685626512;-0.8349221202753919;0.037361498968480004;0.5490987453775328   -510.948;76.3842;-57.1832<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0120.png 0.58835908389912;-0.1421097419064;0.7960142112715201;0.11529874805519999;0.98912028211904;0.0913634055704;-0.80033743008048;0.038024918616800005;0.59834297293592   -494.821;48.5412;-84.8384<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1535.png -0.47973478756199994;0.2492251514408;-0.8412746576420002;-0.2867349429592;0.86163642122928;0.41876712898479995;0.829240193302;0.4421195084152;-0.34189556383272013  504.914;-24.8421;26.4884<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0205.png 0.6106958808574999;-0.06976742683359999;0.7887859260567;0.0884298177724;0.99588914017628;0.019621157275200002;-0.7869122557953;0.0577696169028;0.61435491464278 -496.436;87.8103;-94.0397<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1410.png -0.5233981321799999;0.22283728763796;-0.8224340018261199;-0.24712970381404004;0.8840135385507288;0.39679585939524;0.81546370465788;0.41093014951275997;-0.4076213029052711  516.426;-21.8037;56.0843<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0305.png 0.4627330191951199;-0.1111267027116;0.8795049273668402;0.0602394287796;0.9937604130175;0.0938693329872;-0.8844485697531602;0.009544443388800006;0.4665399662223799  -518.488;61.2659;-11.9061<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1520.png -0.4426463281319999;0.256743921225;-0.859154084569;-0.100230111575;0.9379624663575;0.33193416735;0.891076375431;0.23304275495;-0.3894521280104999   507.62;-17.5998;10.1222<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1465.png -0.4968166092199997;0.27276438345582;-0.8238766921380599;-0.24798936723618;0.8651227477203262;0.43596318783042;0.83166969793794;0.42090642490158003;-0.36216467925567364    510.913;-41.1816;32.9071<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0110.png 0.64203171652358;-0.07928514436382;0.762567666990514;0.06655954761538;0.9966471779814662;0.04758394800508;-0.763783612993486;0.02020572616172;0.6451562790218862    -483.736;79.1617;-115.213<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0200.png 0.5985017101599199;-0.08324152791520001;0.7967851671261401;0.09944926157440001;0.994613859171675;0.029208153107800002;-0.7949249014178601;0.061758561074599996;0.603556403475755    -498.297;80.877;-86.8741<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1440.png -0.517530307396;0.3034291605098;-0.8000601703978;-0.24403450075420002;0.8438411834922199;0.4778907236068;0.8201296812982001;0.44256426179720004;-0.36266652626777995    510.974;-63.7647;38.151<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0230.png 0.56873842000062;-0.10835957185299999;0.81534897651194;0.07321356390020001;0.9940186344654;0.0810352792214;-0.8192530282960601;0.013606822013000001;0.57326998852878    -506.037;63.6934;-73.9118<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1400.png -0.5600271884200001;0.19081582154300003;-0.806200089329;-0.237897472313;0.8950932793455;0.37711111714600004;0.793583048239;0.40298551370600005;-0.45588195179049995 514.387;-13.5811;82.1421<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1485.png -0.5026803372659998;0.25712356923519997;-0.8253468535042;-0.2184205547408;0.88598782423302;0.40904504076279996;0.8364223931197999;0.3858922832212;-0.38920739491697987  508.936;-32.7062;39.5221<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1470.png -0.45382847489800016;0.251393940534;-0.8548931472132;-0.20805190645799998;0.90297008024312;0.3759781083036;0.8664615460067999;0.3484913490564;-0.35749064495887994  512.296;-18.2419;14.6334<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1565.png -0.4680401747999998;0.29573148083428;-0.83275597003404;-0.26410201542972;0.8524664788353342;0.45116643323185995;0.84332066251796;0.43109617984014;-0.3208854020606655   500.953;-44.7338;12.9665<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1055.png -0.49401401832400005;0.2221184836632;-0.8406026605248;-0.2824626932448;0.87337372166128;0.39677841453839996;0.8222920895832;0.4334531837856;-0.3687186179387201 557.841;-17.2461;40.733<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0935.png -0.56070208269;0.2776073522472;-0.7800951234864001;-0.21657491425680003;0.86015662933688;0.46176398227840004;0.7991930619216;0.42786075631360004;-0.42216892035711995   563.852;-68.1348;72.5404<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0960.png -0.576299799906;0.30025187450199997;-0.760084830064;-0.20935704493399998;0.84478740867;0.49244697083199995;0.789968204912;0.442925731024;-0.42399112512000015   557.94;-86.884;77.5747<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1095.png -0.6406246538179998;0.1787753640684;-0.746751722238;-0.2711690117316;0.85718152040408;0.437843689094;0.7183774304220001;0.482989749586;-0.5006531650139199  536.785;-41.7236;135.597<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0945.png -0.49943696121000025;0.2180181433032;-0.8384691656664;-0.2978660743128;0.86560416794048;0.40249869824560003;0.8135344211256;0.4507743221464;-0.36737448285352015    569.414;-17.9502;44.7769<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0905.png -0.6298631872799998;0.248062909785;-0.736028307195;-0.22765754905500002;0.847044929223875;0.48029884049349997;0.742593372285;0.47008481001849994;-0.47704907276012487   557.911;-76.8757;120.016<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1065.png -0.5106840009040001;0.22467611040816;-0.82989415920064;-0.24511506834384;0.8871469872537312;0.39101018164608004;0.82408875331136;0.40310172945792;-0.39798040846626903  555.788;-21.4594;50.7536<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1110.png -0.3785696427280001;0.282491242764;-0.8814082541264001;-0.22357147045200002;0.89617798150968;0.3832500242496;0.8981634652336001;0.34214540951039996;-0.27610855536232015    552.442;-21.6061;-33.6109<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1100.png -0.5769026812160001;0.19173622887359998;-0.7939908723744;-0.24697893867839998;0.88561096761848;0.3933124581452;0.7785792770976001;0.4230017782708;-0.4635566029415201   545.921;-22.137;95.8452<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0940.png -0.4529469422499999;0.30956856345300005;-0.836064870586;-0.260909883747;0.85069551742782;0.456336469632;0.8525040738140001;0.424834374168;-0.3045502404221798   569.194;-55.8551;0.284154<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1010.png -0.491117283016;0.24408210573999997;-0.836197569146;-0.272647621804;0.8686366413915;0.41368289700499994;0.8273244365499999;0.43115448577899995;-0.3600539414485 562.834;-28.8324;35.0702<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1105.png -0.40761659251200033;0.23662429191016002;-0.8819627063426241;-0.23530146904984003;0.9060090616669695;0.35182500772279;0.8823165472573762;0.35093669211721;-0.31362632022103076  554.225;-2.17632;-10.6082<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0985.png -0.49911414688800027;0.2679426943970001;-0.8240711404720001;-0.26430339558700006;0.858604704175875;0.43925142583700005;0.8252455677680001;0.4370409735230001;-0.3577237285361252    564.567;-45.0501;35.2734</p>\n<p>submission.csv</p>\n<p>dataset    scene   image   rotation_matrix translation_vector<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1310.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1200.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1255.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1220.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1160.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1155.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1190.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1235.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1230.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1215.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1475.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0090.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0190.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0265.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0290.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1390.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1555.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0075.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0140.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0280.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0275.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0120.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1535.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0205.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1410.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0305.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1520.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1465.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0110.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0200.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1440.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0230.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1400.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1485.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1470.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1565.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_1055.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0935.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0960.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1095.png 0.996849134;0.023781899;0.075671823;-0.023495817;0.999713034;-0.004668714;-0.075761139;0.002876032;0.997121847  5.189118625;-0.209417331;0.875569692<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0945.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0905.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_1065.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1110.png 0.935267251;-0.012903738;0.353707030;-0.013198564;0.997368669;0.071284895;-0.353696152;-0.071338853;0.932635942 -9.860201617;-1.951112114;0.506709770<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1100.png 0.989890721;-0.001467119;0.141824565;-0.005763828;0.998704347;0.050560904;-0.141714988;-0.050867222;0.988599711 -0.622975224;0.187419191;0.607450304<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0940.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_1010.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1105.png 0.946100098;-0.047238793;0.320410831;0.019006687;0.995699106;0.090675446;-0.323316177;-0.079698100;0.942928875  -4.739468462;-0.319373064;-0.255384166<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0985.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan</p>\n<h2>-----------------------------------------------------------------------------</h2>\n<p><code>fbk_vineyard: score=40.00% (mAA=25.00%, clusterness=100.00%)</code></p>\n<p>In the submission.csv, there are only 1 cluster and 1 outliers cluster, which means the clusterness should be under 33.33%, but it’s showing 100.00%. Enlighten me if I'm mistaken; otherwise this isn't just a fluke.</p>\n<p>[Follow-up]<br>\nNow, I've got the reason. I think this is a critical vulnerability, so that the scoring module (including mAA. which doesn't seem to be  \"equivalent to recall\" in Overview) should be revisited by the host and rescoring may benefit fairness unless the hidden set is protected.</p>",
  "messages": [
    {
      "id": "3204965",
      "postDate": "05/19/2025 05:40:29",
      "content": "<p>The cluster score’s been odd to me all along, and here’s what I got on my CV set today:</p>\n<pre><code>final_score, dataset_scores = metric.score(\n    gt_csv=,\n    user_csv=submission_file,\n    thresholds_csv=f,\n    mask_csv=   RERUN  os.path.(data_dir, ),\n    inl_cf=,\n    strict_cf=,\n    =,\n)\n</code></pre>\n<p>./filtered_train_labels.csv</p>\n<p>dataset    scene   image   rotation_matrix translation_vector<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1310.png -0.45096986446600007;0.27640276616340004;-0.8486619380236001;-0.22647640738860003;0.88428353942478;0.40835157692580004;0.8633272908284001;0.3763563262902;-0.3361864742252201   530.376;-32.5758;8.66025<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1200.png -0.49438604111399975;0.2992504042362;-0.8161052283251999;-0.2008084914918;0.8741562895071799;0.4421836760036;0.8457272006268;0.38249137133239997;-0.37207832170281985   540.488;-56.942;30.3337<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1255.png -0.480305983746;0.25415113510376;-0.8394725706204399;-0.25052886194824;0.8774722137962327;0.40899617651612;0.8405606971715599;0.40675518877988;-0.35778302927376715 536.308;-28.765;27.6173<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1220.png -0.4723317373060001;0.31990109336960004;-0.8213197004336;-0.21688922167040003;0.8609798276824799;0.460079128708;0.8543195101264;0.395445356036;-0.33728500142751994 537.855;-63.0646;13.3448<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1160.png -0.5179049924259997;0.26548250388814;-0.8132001242871599;-0.23628750129585996;0.8692429742569598;0.43426369078175997;0.82215790377684;0.4170560831942399;-0.38745516311304007   544.394;-46.0541;47.5699<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1155.png -0.511186782162;0.2588402620089;-0.8195656146679201;-0.23862885722309998;0.8733372238454078;0.42466244791394;0.8256769019920801;0.41265438920605996;-0.38467152209259226    545.661;-40.4273;44.9054<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1190.png -0.5485391152799999;0.21904702067452;-0.8069216936175599;-0.22989207292548;0.8883665315084622;0.39743489486199995;0.80389915518244;0.40351364733;-0.4369465727475377    539.761;-27.2237;73.5853<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1235.png -0.51286548354;0.272770387957;-0.813981831169;-0.229720591739;0.8699931798735;0.436280199082;0.8271629559829999;0.41074113238599996;-0.38352859310249987    536.302;-47.537;43.3903<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1230.png -0.5217347486800001;0.28092921213600003;-0.8055244708920001;-0.22363741749600002;0.8661653987119999;0.446926879356;0.823272252012;0.41332324851600005;-0.38908213179200013  535.852;-54.4949;47.289<br>\nfbk_vineyard    vineyard_split_2    vineyard_split_2_frame_1215.png -0.4846348040199999;0.29270774428199997;-0.824289704106;-0.203701108158;0.87867415198;0.431784408786;0.850668677214;0.377165870106;-0.36621136387599984 539.547;-50.5375;25.704<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1475.png -0.43655440942599966;0.2441491108092;-0.8659165170531999;-0.1735002444348;0.92156072964912;0.3473089176852;0.8827898209628;0.30185597693879995;-0.3599514672608797  512.368;-10.5897;6.65932<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0090.png 0.60940070194928;-0.1179033225381;0.78404708259042;0.1108402345611;0.991839419534035;0.0630001048134;-0.78507672464958;0.0485116206546;0.617496065484755    -488.723;62.0719;-94.8123<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0190.png 0.53043187119032;-0.11174926732833998;0.8403297313677439;0.11627415416605999;0.9914953705220798;0.05845724906793999;-0.8397155930682559;0.06670105057045998;0.5389142975357599  -505.153;63.6353;-47.3736<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0265.png 0.5024316490966199;-0.1440664075504;0.8525304111638;0.11003796779840001;0.98865702691062;0.10222009038440001;-0.8575866588202;0.0424519899524;0.51258533799 -515.543;45.4976;-34.9178<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0290.png 0.1827152068924799;-0.1825841649328;0.96606405046064;0.0934499591504;0.981381045126;0.16780460438720002;-0.97871539888336;0.05961797408960001;0.19637569237751995   -502.175;18.5164;138.079<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1390.png -0.61088428465;0.24389941925167996;-0.7532147358502399;-0.21021443341232;0.8672501987647968;0.45131685122223997;0.76330155050176;0.43403926262576;-0.4785182546492033   508.898;-57.0345;102.976<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1555.png -0.3896142358800001;0.306820506496;-0.8683673801680001;-0.262779872416;0.866645016368;0.42411455129600006;0.882693303928;0.3934305412160001;-0.25703098064800023    502.466;-31.5815;-28.0105<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0075.png 0.60266391944928;-0.1765441431392;0.77822118014776;0.128370227696;0.9839678709467;0.12380754060399998;-0.78760213582024;0.025286122525599995;0.61566495143342   -490.228;30.7243;-95.8173<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0140.png 0.61631690468582;-0.1323900181294;0.77629008172948;0.0989362256134;0.99097406046054;0.0904545920566;-0.78125861989052;0.02105452593739999;0.62385223567472  -492.994;52.8367;-102.547<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0280.png 0.3507627593310202;-0.15963154405563998;0.9227583533427759;0.14714297501276;0.9825194375641688;0.11403724051375999;-0.924831959561224;0.09577743712383999;0.36811989677714896   -516.737;31.1435;52.1067<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0275.png 0.54473792542488;-0.08624649615808;0.834159695580608;0.07852330167232;0.9955730324064128;0.05165685626512;-0.8349221202753919;0.037361498968480004;0.5490987453775328   -510.948;76.3842;-57.1832<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0120.png 0.58835908389912;-0.1421097419064;0.7960142112715201;0.11529874805519999;0.98912028211904;0.0913634055704;-0.80033743008048;0.038024918616800005;0.59834297293592   -494.821;48.5412;-84.8384<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1535.png -0.47973478756199994;0.2492251514408;-0.8412746576420002;-0.2867349429592;0.86163642122928;0.41876712898479995;0.829240193302;0.4421195084152;-0.34189556383272013  504.914;-24.8421;26.4884<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0205.png 0.6106958808574999;-0.06976742683359999;0.7887859260567;0.0884298177724;0.99588914017628;0.019621157275200002;-0.7869122557953;0.0577696169028;0.61435491464278 -496.436;87.8103;-94.0397<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1410.png -0.5233981321799999;0.22283728763796;-0.8224340018261199;-0.24712970381404004;0.8840135385507288;0.39679585939524;0.81546370465788;0.41093014951275997;-0.4076213029052711  516.426;-21.8037;56.0843<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0305.png 0.4627330191951199;-0.1111267027116;0.8795049273668402;0.0602394287796;0.9937604130175;0.0938693329872;-0.8844485697531602;0.009544443388800006;0.4665399662223799  -518.488;61.2659;-11.9061<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1520.png -0.4426463281319999;0.256743921225;-0.859154084569;-0.100230111575;0.9379624663575;0.33193416735;0.891076375431;0.23304275495;-0.3894521280104999   507.62;-17.5998;10.1222<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1465.png -0.4968166092199997;0.27276438345582;-0.8238766921380599;-0.24798936723618;0.8651227477203262;0.43596318783042;0.83166969793794;0.42090642490158003;-0.36216467925567364    510.913;-41.1816;32.9071<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0110.png 0.64203171652358;-0.07928514436382;0.762567666990514;0.06655954761538;0.9966471779814662;0.04758394800508;-0.763783612993486;0.02020572616172;0.6451562790218862    -483.736;79.1617;-115.213<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0200.png 0.5985017101599199;-0.08324152791520001;0.7967851671261401;0.09944926157440001;0.994613859171675;0.029208153107800002;-0.7949249014178601;0.061758561074599996;0.603556403475755    -498.297;80.877;-86.8741<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1440.png -0.517530307396;0.3034291605098;-0.8000601703978;-0.24403450075420002;0.8438411834922199;0.4778907236068;0.8201296812982001;0.44256426179720004;-0.36266652626777995    510.974;-63.7647;38.151<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_0230.png 0.56873842000062;-0.10835957185299999;0.81534897651194;0.07321356390020001;0.9940186344654;0.0810352792214;-0.8192530282960601;0.013606822013000001;0.57326998852878    -506.037;63.6934;-73.9118<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1400.png -0.5600271884200001;0.19081582154300003;-0.806200089329;-0.237897472313;0.8950932793455;0.37711111714600004;0.793583048239;0.40298551370600005;-0.45588195179049995 514.387;-13.5811;82.1421<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1485.png -0.5026803372659998;0.25712356923519997;-0.8253468535042;-0.2184205547408;0.88598782423302;0.40904504076279996;0.8364223931197999;0.3858922832212;-0.38920739491697987  508.936;-32.7062;39.5221<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1470.png -0.45382847489800016;0.251393940534;-0.8548931472132;-0.20805190645799998;0.90297008024312;0.3759781083036;0.8664615460067999;0.3484913490564;-0.35749064495887994  512.296;-18.2419;14.6334<br>\nfbk_vineyard    vineyard_split_3    vineyard_split_3_frame_1565.png -0.4680401747999998;0.29573148083428;-0.83275597003404;-0.26410201542972;0.8524664788353342;0.45116643323185995;0.84332066251796;0.43109617984014;-0.3208854020606655   500.953;-44.7338;12.9665<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1055.png -0.49401401832400005;0.2221184836632;-0.8406026605248;-0.2824626932448;0.87337372166128;0.39677841453839996;0.8222920895832;0.4334531837856;-0.3687186179387201 557.841;-17.2461;40.733<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0935.png -0.56070208269;0.2776073522472;-0.7800951234864001;-0.21657491425680003;0.86015662933688;0.46176398227840004;0.7991930619216;0.42786075631360004;-0.42216892035711995   563.852;-68.1348;72.5404<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0960.png -0.576299799906;0.30025187450199997;-0.760084830064;-0.20935704493399998;0.84478740867;0.49244697083199995;0.789968204912;0.442925731024;-0.42399112512000015   557.94;-86.884;77.5747<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1095.png -0.6406246538179998;0.1787753640684;-0.746751722238;-0.2711690117316;0.85718152040408;0.437843689094;0.7183774304220001;0.482989749586;-0.5006531650139199  536.785;-41.7236;135.597<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0945.png -0.49943696121000025;0.2180181433032;-0.8384691656664;-0.2978660743128;0.86560416794048;0.40249869824560003;0.8135344211256;0.4507743221464;-0.36737448285352015    569.414;-17.9502;44.7769<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0905.png -0.6298631872799998;0.248062909785;-0.736028307195;-0.22765754905500002;0.847044929223875;0.48029884049349997;0.742593372285;0.47008481001849994;-0.47704907276012487   557.911;-76.8757;120.016<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1065.png -0.5106840009040001;0.22467611040816;-0.82989415920064;-0.24511506834384;0.8871469872537312;0.39101018164608004;0.82408875331136;0.40310172945792;-0.39798040846626903  555.788;-21.4594;50.7536<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1110.png -0.3785696427280001;0.282491242764;-0.8814082541264001;-0.22357147045200002;0.89617798150968;0.3832500242496;0.8981634652336001;0.34214540951039996;-0.27610855536232015    552.442;-21.6061;-33.6109<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1100.png -0.5769026812160001;0.19173622887359998;-0.7939908723744;-0.24697893867839998;0.88561096761848;0.3933124581452;0.7785792770976001;0.4230017782708;-0.4635566029415201   545.921;-22.137;95.8452<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0940.png -0.4529469422499999;0.30956856345300005;-0.836064870586;-0.260909883747;0.85069551742782;0.456336469632;0.8525040738140001;0.424834374168;-0.3045502404221798   569.194;-55.8551;0.284154<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1010.png -0.491117283016;0.24408210573999997;-0.836197569146;-0.272647621804;0.8686366413915;0.41368289700499994;0.8273244365499999;0.43115448577899995;-0.3600539414485 562.834;-28.8324;35.0702<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_1105.png -0.40761659251200033;0.23662429191016002;-0.8819627063426241;-0.23530146904984003;0.9060090616669695;0.35182500772279;0.8823165472573762;0.35093669211721;-0.31362632022103076  554.225;-2.17632;-10.6082<br>\nfbk_vineyard    vineyard_split_1    vineyard_split_1_frame_0985.png -0.49911414688800027;0.2679426943970001;-0.8240711404720001;-0.26430339558700006;0.858604704175875;0.43925142583700005;0.8252455677680001;0.4370409735230001;-0.3577237285361252    564.567;-45.0501;35.2734</p>\n<p>submission.csv</p>\n<p>dataset    scene   image   rotation_matrix translation_vector<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1310.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1200.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1255.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1220.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1160.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1155.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1190.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1235.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1230.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_2_frame_1215.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1475.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0090.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0190.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0265.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0290.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1390.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1555.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0075.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0140.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0280.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0275.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0120.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1535.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0205.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1410.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0305.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1520.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1465.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0110.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0200.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1440.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_0230.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1400.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1485.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1470.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_3_frame_1565.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_1055.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0935.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0960.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1095.png 0.996849134;0.023781899;0.075671823;-0.023495817;0.999713034;-0.004668714;-0.075761139;0.002876032;0.997121847  5.189118625;-0.209417331;0.875569692<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0945.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0905.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_1065.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1110.png 0.935267251;-0.012903738;0.353707030;-0.013198564;0.997368669;0.071284895;-0.353696152;-0.071338853;0.932635942 -9.860201617;-1.951112114;0.506709770<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1100.png 0.989890721;-0.001467119;0.141824565;-0.005763828;0.998704347;0.050560904;-0.141714988;-0.050867222;0.988599711 -0.622975224;0.187419191;0.607450304<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0940.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_1010.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan<br>\nfbk_vineyard    cluster0    vineyard_split_1_frame_1105.png 0.946100098;-0.047238793;0.320410831;0.019006687;0.995699106;0.090675446;-0.323316177;-0.079698100;0.942928875  -4.739468462;-0.319373064;-0.255384166<br>\nfbk_vineyard    outliers    vineyard_split_1_frame_0985.png nan;nan;nan;nan;nan;nan;nan;nan;nan nan;nan;nan</p>\n<h2>-----------------------------------------------------------------------------</h2>\n<p><code>fbk_vineyard: score=40.00% (mAA=25.00%, clusterness=100.00%)</code></p>\n<p>In the submission.csv, there are only 1 cluster and 1 outliers cluster, which means the clusterness should be under 33.33%, but it’s showing 100.00%. Enlighten me if I'm mistaken; otherwise this isn't just a fluke.</p>\n<p>[Follow-up]<br>\nNow, I've got the reason. I think this is a critical vulnerability, so that the scoring module (including mAA. which doesn't seem to be  \"equivalent to recall\" in Overview) should be revisited by the host and rescoring may benefit fairness unless the hidden set is protected.</p>",
      "rawMarkdown": "The cluster score’s been odd to me all along, and here’s what I got on my CV set today:\n\n    final_score, dataset_scores = metric.score(\n        gt_csv='./filtered_train_labels.csv',\n        user_csv=submission_file,\n        thresholds_csv=f'{data_dir}/train_thresholds.csv',\n        mask_csv=None if not RERUN else os.path.join(data_dir, 'mask.csv'),\n        inl_cf=0,\n        strict_cf=-1,\n        verbose=True,\n    )\n\n./filtered_train_labels.csv\n\ndataset\tscene\timage\trotation_matrix\ttranslation_vector\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1310.png\t-0.45096986446600007;0.27640276616340004;-0.8486619380236001;-0.22647640738860003;0.88428353942478;0.40835157692580004;0.8633272908284001;0.3763563262902;-0.3361864742252201\t530.376;-32.5758;8.66025\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1200.png\t-0.49438604111399975;0.2992504042362;-0.8161052283251999;-0.2008084914918;0.8741562895071799;0.4421836760036;0.8457272006268;0.38249137133239997;-0.37207832170281985\t540.488;-56.942;30.3337\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1255.png\t-0.480305983746;0.25415113510376;-0.8394725706204399;-0.25052886194824;0.8774722137962327;0.40899617651612;0.8405606971715599;0.40675518877988;-0.35778302927376715\t536.308;-28.765;27.6173\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1220.png\t-0.4723317373060001;0.31990109336960004;-0.8213197004336;-0.21688922167040003;0.8609798276824799;0.460079128708;0.8543195101264;0.395445356036;-0.33728500142751994\t537.855;-63.0646;13.3448\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1160.png\t-0.5179049924259997;0.26548250388814;-0.8132001242871599;-0.23628750129585996;0.8692429742569598;0.43426369078175997;0.82215790377684;0.4170560831942399;-0.38745516311304007\t544.394;-46.0541;47.5699\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1155.png\t-0.511186782162;0.2588402620089;-0.8195656146679201;-0.23862885722309998;0.8733372238454078;0.42466244791394;0.8256769019920801;0.41265438920605996;-0.38467152209259226\t545.661;-40.4273;44.9054\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1190.png\t-0.5485391152799999;0.21904702067452;-0.8069216936175599;-0.22989207292548;0.8883665315084622;0.39743489486199995;0.80389915518244;0.40351364733;-0.4369465727475377\t539.761;-27.2237;73.5853\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1235.png\t-0.51286548354;0.272770387957;-0.813981831169;-0.229720591739;0.8699931798735;0.436280199082;0.8271629559829999;0.41074113238599996;-0.38352859310249987\t536.302;-47.537;43.3903\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1230.png\t-0.5217347486800001;0.28092921213600003;-0.8055244708920001;-0.22363741749600002;0.8661653987119999;0.446926879356;0.823272252012;0.41332324851600005;-0.38908213179200013\t535.852;-54.4949;47.289\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1215.png\t-0.4846348040199999;0.29270774428199997;-0.824289704106;-0.203701108158;0.87867415198;0.431784408786;0.850668677214;0.377165870106;-0.36621136387599984\t539.547;-50.5375;25.704\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1475.png\t-0.43655440942599966;0.2441491108092;-0.8659165170531999;-0.1735002444348;0.92156072964912;0.3473089176852;0.8827898209628;0.30185597693879995;-0.3599514672608797\t512.368;-10.5897;6.65932\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0090.png\t0.60940070194928;-0.1179033225381;0.78404708259042;0.1108402345611;0.991839419534035;0.0630001048134;-0.78507672464958;0.0485116206546;0.617496065484755\t-488.723;62.0719;-94.8123\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0190.png\t0.53043187119032;-0.11174926732833998;0.8403297313677439;0.11627415416605999;0.9914953705220798;0.05845724906793999;-0.8397155930682559;0.06670105057045998;0.5389142975357599\t-505.153;63.6353;-47.3736\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0265.png\t0.5024316490966199;-0.1440664075504;0.8525304111638;0.11003796779840001;0.98865702691062;0.10222009038440001;-0.8575866588202;0.0424519899524;0.51258533799\t-515.543;45.4976;-34.9178\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0290.png\t0.1827152068924799;-0.1825841649328;0.96606405046064;0.0934499591504;0.981381045126;0.16780460438720002;-0.97871539888336;0.05961797408960001;0.19637569237751995\t-502.175;18.5164;138.079\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1390.png\t-0.61088428465;0.24389941925167996;-0.7532147358502399;-0.21021443341232;0.8672501987647968;0.45131685122223997;0.76330155050176;0.43403926262576;-0.4785182546492033\t508.898;-57.0345;102.976\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1555.png\t-0.3896142358800001;0.306820506496;-0.8683673801680001;-0.262779872416;0.866645016368;0.42411455129600006;0.882693303928;0.3934305412160001;-0.25703098064800023\t502.466;-31.5815;-28.0105\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0075.png\t0.60266391944928;-0.1765441431392;0.77822118014776;0.128370227696;0.9839678709467;0.12380754060399998;-0.78760213582024;0.025286122525599995;0.61566495143342\t-490.228;30.7243;-95.8173\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0140.png\t0.61631690468582;-0.1323900181294;0.77629008172948;0.0989362256134;0.99097406046054;0.0904545920566;-0.78125861989052;0.02105452593739999;0.62385223567472\t-492.994;52.8367;-102.547\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0280.png\t0.3507627593310202;-0.15963154405563998;0.9227583533427759;0.14714297501276;0.9825194375641688;0.11403724051375999;-0.924831959561224;0.09577743712383999;0.36811989677714896\t-516.737;31.1435;52.1067\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0275.png\t0.54473792542488;-0.08624649615808;0.834159695580608;0.07852330167232;0.9955730324064128;0.05165685626512;-0.8349221202753919;0.037361498968480004;0.5490987453775328\t-510.948;76.3842;-57.1832\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0120.png\t0.58835908389912;-0.1421097419064;0.7960142112715201;0.11529874805519999;0.98912028211904;0.0913634055704;-0.80033743008048;0.038024918616800005;0.59834297293592\t-494.821;48.5412;-84.8384\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1535.png\t-0.47973478756199994;0.2492251514408;-0.8412746576420002;-0.2867349429592;0.86163642122928;0.41876712898479995;0.829240193302;0.4421195084152;-0.34189556383272013\t504.914;-24.8421;26.4884\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0205.png\t0.6106958808574999;-0.06976742683359999;0.7887859260567;0.0884298177724;0.99588914017628;0.019621157275200002;-0.7869122557953;0.0577696169028;0.61435491464278\t-496.436;87.8103;-94.0397\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1410.png\t-0.5233981321799999;0.22283728763796;-0.8224340018261199;-0.24712970381404004;0.8840135385507288;0.39679585939524;0.81546370465788;0.41093014951275997;-0.4076213029052711\t516.426;-21.8037;56.0843\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0305.png\t0.4627330191951199;-0.1111267027116;0.8795049273668402;0.0602394287796;0.9937604130175;0.0938693329872;-0.8844485697531602;0.009544443388800006;0.4665399662223799\t-518.488;61.2659;-11.9061\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1520.png\t-0.4426463281319999;0.256743921225;-0.859154084569;-0.100230111575;0.9379624663575;0.33193416735;0.891076375431;0.23304275495;-0.3894521280104999\t507.62;-17.5998;10.1222\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1465.png\t-0.4968166092199997;0.27276438345582;-0.8238766921380599;-0.24798936723618;0.8651227477203262;0.43596318783042;0.83166969793794;0.42090642490158003;-0.36216467925567364\t510.913;-41.1816;32.9071\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0110.png\t0.64203171652358;-0.07928514436382;0.762567666990514;0.06655954761538;0.9966471779814662;0.04758394800508;-0.763783612993486;0.02020572616172;0.6451562790218862\t-483.736;79.1617;-115.213\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0200.png\t0.5985017101599199;-0.08324152791520001;0.7967851671261401;0.09944926157440001;0.994613859171675;0.029208153107800002;-0.7949249014178601;0.061758561074599996;0.603556403475755\t-498.297;80.877;-86.8741\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1440.png\t-0.517530307396;0.3034291605098;-0.8000601703978;-0.24403450075420002;0.8438411834922199;0.4778907236068;0.8201296812982001;0.44256426179720004;-0.36266652626777995\t510.974;-63.7647;38.151\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0230.png\t0.56873842000062;-0.10835957185299999;0.81534897651194;0.07321356390020001;0.9940186344654;0.0810352792214;-0.8192530282960601;0.013606822013000001;0.57326998852878\t-506.037;63.6934;-73.9118\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1400.png\t-0.5600271884200001;0.19081582154300003;-0.806200089329;-0.237897472313;0.8950932793455;0.37711111714600004;0.793583048239;0.40298551370600005;-0.45588195179049995\t514.387;-13.5811;82.1421\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1485.png\t-0.5026803372659998;0.25712356923519997;-0.8253468535042;-0.2184205547408;0.88598782423302;0.40904504076279996;0.8364223931197999;0.3858922832212;-0.38920739491697987\t508.936;-32.7062;39.5221\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1470.png\t-0.45382847489800016;0.251393940534;-0.8548931472132;-0.20805190645799998;0.90297008024312;0.3759781083036;0.8664615460067999;0.3484913490564;-0.35749064495887994\t512.296;-18.2419;14.6334\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1565.png\t-0.4680401747999998;0.29573148083428;-0.83275597003404;-0.26410201542972;0.8524664788353342;0.45116643323185995;0.84332066251796;0.43109617984014;-0.3208854020606655\t500.953;-44.7338;12.9665\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1055.png\t-0.49401401832400005;0.2221184836632;-0.8406026605248;-0.2824626932448;0.87337372166128;0.39677841453839996;0.8222920895832;0.4334531837856;-0.3687186179387201\t557.841;-17.2461;40.733\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0935.png\t-0.56070208269;0.2776073522472;-0.7800951234864001;-0.21657491425680003;0.86015662933688;0.46176398227840004;0.7991930619216;0.42786075631360004;-0.42216892035711995\t563.852;-68.1348;72.5404\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0960.png\t-0.576299799906;0.30025187450199997;-0.760084830064;-0.20935704493399998;0.84478740867;0.49244697083199995;0.789968204912;0.442925731024;-0.42399112512000015\t557.94;-86.884;77.5747\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1095.png\t-0.6406246538179998;0.1787753640684;-0.746751722238;-0.2711690117316;0.85718152040408;0.437843689094;0.7183774304220001;0.482989749586;-0.5006531650139199\t536.785;-41.7236;135.597\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0945.png\t-0.49943696121000025;0.2180181433032;-0.8384691656664;-0.2978660743128;0.86560416794048;0.40249869824560003;0.8135344211256;0.4507743221464;-0.36737448285352015\t569.414;-17.9502;44.7769\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0905.png\t-0.6298631872799998;0.248062909785;-0.736028307195;-0.22765754905500002;0.847044929223875;0.48029884049349997;0.742593372285;0.47008481001849994;-0.47704907276012487\t557.911;-76.8757;120.016\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1065.png\t-0.5106840009040001;0.22467611040816;-0.82989415920064;-0.24511506834384;0.8871469872537312;0.39101018164608004;0.82408875331136;0.40310172945792;-0.39798040846626903\t555.788;-21.4594;50.7536\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1110.png\t-0.3785696427280001;0.282491242764;-0.8814082541264001;-0.22357147045200002;0.89617798150968;0.3832500242496;0.8981634652336001;0.34214540951039996;-0.27610855536232015\t552.442;-21.6061;-33.6109\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1100.png\t-0.5769026812160001;0.19173622887359998;-0.7939908723744;-0.24697893867839998;0.88561096761848;0.3933124581452;0.7785792770976001;0.4230017782708;-0.4635566029415201\t545.921;-22.137;95.8452\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0940.png\t-0.4529469422499999;0.30956856345300005;-0.836064870586;-0.260909883747;0.85069551742782;0.456336469632;0.8525040738140001;0.424834374168;-0.3045502404221798\t569.194;-55.8551;0.284154\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1010.png\t-0.491117283016;0.24408210573999997;-0.836197569146;-0.272647621804;0.8686366413915;0.41368289700499994;0.8273244365499999;0.43115448577899995;-0.3600539414485\t562.834;-28.8324;35.0702\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1105.png\t-0.40761659251200033;0.23662429191016002;-0.8819627063426241;-0.23530146904984003;0.9060090616669695;0.35182500772279;0.8823165472573762;0.35093669211721;-0.31362632022103076\t554.225;-2.17632;-10.6082\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0985.png\t-0.49911414688800027;0.2679426943970001;-0.8240711404720001;-0.26430339558700006;0.858604704175875;0.43925142583700005;0.8252455677680001;0.4370409735230001;-0.3577237285361252\t564.567;-45.0501;35.2734\n\n\n\nsubmission.csv\n\ndataset\tscene\timage\trotation_matrix\ttranslation_vector\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1310.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1200.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1255.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1220.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1160.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1155.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1190.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1235.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1230.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1215.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1475.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0090.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0190.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0265.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0290.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1390.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1555.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0075.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0140.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0280.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0275.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0120.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1535.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0205.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1410.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0305.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1520.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1465.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0110.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0200.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1440.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0230.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1400.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1485.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1470.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1565.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_1055.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0935.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0960.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1095.png\t0.996849134;0.023781899;0.075671823;-0.023495817;0.999713034;-0.004668714;-0.075761139;0.002876032;0.997121847\t5.189118625;-0.209417331;0.875569692\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0945.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0905.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_1065.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1110.png\t0.935267251;-0.012903738;0.353707030;-0.013198564;0.997368669;0.071284895;-0.353696152;-0.071338853;0.932635942\t-9.860201617;-1.951112114;0.506709770\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1100.png\t0.989890721;-0.001467119;0.141824565;-0.005763828;0.998704347;0.050560904;-0.141714988;-0.050867222;0.988599711\t-0.622975224;0.187419191;0.607450304\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0940.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_1010.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1105.png\t0.946100098;-0.047238793;0.320410831;0.019006687;0.995699106;0.090675446;-0.323316177;-0.079698100;0.942928875\t-4.739468462;-0.319373064;-0.255384166\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0985.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\n\n\n-----------------------------------------------------------------------------\n-----------------------------------------------------------------------------\n`fbk_vineyard: score=40.00% (mAA=25.00%, clusterness=100.00%)`\n\nIn the submission.csv, there are only 1 cluster and 1 outliers cluster, which means the clusterness should be under 33.33%, but it’s showing 100.00%. Enlighten me if I'm mistaken; otherwise this isn't just a fluke.\n\n[Follow-up]\nNow, I've got the reason. I think this is a critical vulnerability, so that the scoring module (including mAA. which doesn't seem to be  \"equivalent to recall\" in Overview) should be revisited by the host and rescoring may benefit fairness unless the hidden set is protected.",
      "votes": null
    },
    {
      "id": "3206519",
      "postDate": "05/21/2025 12:35:05",
      "content": "<p>It seems there is a bug in metric computation <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> <a href=\"https://www.kaggle.com/addisonhoward\" target=\"_blank\">@addisonhoward</a> Please take a look. </p>",
      "rawMarkdown": "It seems there is a bug in metric computation @sohier @addisonhoward Please take a look.",
      "votes": null
    },
    {
      "id": "3206725",
      "postDate": "05/21/2025 17:55:57",
      "content": "<p>As you suggested, I've now removed my post about why I think this happens.<br>\nIf an organiser would like my notes on this, just let me know.</p>",
      "rawMarkdown": "As you suggested, I've now removed my post about why I think this happens.\nIf an organiser would like my notes on this, just let me know.",
      "votes": null
    },
    {
      "id": "3206853",
      "postDate": "05/21/2025 20:46:09",
      "content": "<p><a href=\"https://www.kaggle.com/gowillgo\" target=\"_blank\">@gowillgo</a> Eduard and I will take a look. It may take us some time to coordinate.</p>",
      "rawMarkdown": "gowillgo Eduard and I will take a look. It may take us some time to coordinate.",
      "votes": null
    },
    {
      "id": "3207296",
      "postDate": "05/22/2025 14:45:17",
      "content": "<p>I also noticed case that the mAA exceeded 100%.<br>\n<code>ETs: score=104.00% (mAA=108.33%, clusterness=100.00%)</code></p>\n<p><a href=\"https://www.kaggle.com/code/motono0223/imc2025-maa108-ets-example\" target=\"_blank\">https://www.kaggle.com/code/motono0223/imc2025-maa108-ets-example</a></p>",
      "rawMarkdown": "I also noticed case that the mAA exceeded 100%.\n`ETs: score=104.00% (mAA=108.33%, clusterness=100.00%)`\n\nhttps://www.kaggle.com/code/motono0223/imc2025-maa108-ets-example",
      "votes": null
    },
    {
      "id": "3207610",
      "postDate": "05/23/2025 03:47:32",
      "content": "<p>i also see this problem, seems bug in metric function?</p>",
      "rawMarkdown": "i also see this problem, seems bug in metric function?",
      "votes": null
    },
    {
      "id": "3207612",
      "postDate": "05/23/2025 03:51:55",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> ,Does this mean the duration of the competition will be extended?</p>",
      "rawMarkdown": "Hi, @sohier ,Does this mean the duration of the competition will be extended?",
      "votes": null
    },
    {
      "id": "3208082",
      "postDate": "05/23/2025 17:20:24",
      "content": "<p><a href=\"https://www.kaggle.com/petermoorhouse\" target=\"_blank\">@petermoorhouse</a> Thanks again for your nice, insightful post. The core problem in the above submission was actually in mAA calculation. In my local test for the metric after the initial draft of this post, I was even able to produce <code>fbk_vineyard: score=127.27% (mAA=175.00%, clusterness=100.00%)</code>, which is definitely vulnerable in many senses. We'll see how the fix works. Thanks again!</p>",
      "rawMarkdown": "petermoorhouse Thanks again for your nice, insightful post. The core problem in the above submission was actually in mAA calculation. In my local test for the metric after the initial draft of this post, I was even able to produce `fbk_vineyard: score=127.27% (mAA=175.00%, clusterness=100.00%)`, which is definitely vulnerable in many senses. We'll see how the fix works. Thanks again!",
      "votes": null
    },
    {
      "id": "3208300",
      "postDate": "05/23/2025 20:40:13",
      "content": "<p><a href=\"https://www.kaggle.com/gowillgo\" target=\"_blank\">@gowillgo</a> <a href=\"https://www.kaggle.com/petermoorhouse\" target=\"_blank\">@petermoorhouse</a> Thank you the reporting, we really appreciate it!</p>\n<p>The fixed version of the metric is available here now<br>\n<a href=\"https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py\" target=\"_blank\">https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py</a></p>",
      "rawMarkdown": "gowillgo @petermoorhouse Thank you the reporting, we really appreciate it!\n\nThe fixed version of the metric is available here now\nhttps://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py",
      "votes": null
    },
    {
      "id": "3208448",
      "postDate": "05/24/2025 06:59:35",
      "content": "<p>Now recall and precision complement each other well. Thank you for all the effort. I had a moment of uncertainty, thinking things might just carry on given the high volume of rework, possibly ending up with some kind of magic <a href=\"https://www.kaggle.com/competitions/uspto-explainable-ai\" target=\"_blank\">end</a> which I’m not a fan of. But my belief that academia and scholarship would do the right thing has far exceeded, and I believe many participants will appreciate the action taken.</p>\n<p>Thanks again!</p>",
      "rawMarkdown": "Now recall and precision complement each other well. Thank you for all the effort. I had a moment of uncertainty, thinking things might just carry on given the high volume of rework, possibly ending up with some kind of magic [end](https://www.kaggle.com/competitions/uspto-explainable-ai) which I’m not a fan of. But my belief that academia and scholarship would do the right thing has far exceeded, and I believe many participants will appreciate the action taken.\n\nThanks again!",
      "votes": null
    },
    {
      "id": "3208822",
      "postDate": "05/24/2025 18:54:49",
      "content": "<p>Thank you for reporting, that's the main thing, that matters. We just did what we had to do, with full support from Kaggle orgs.</p>",
      "rawMarkdown": "Thank you for reporting, that's the main thing, that matters. We just did what we had to do, with full support from Kaggle orgs.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3206519,
      "author_name": "rohitsingh9990",
      "author_url": "",
      "post_date": "05/21/2025 12:35:05",
      "content": "<p>It seems there is a bug in metric computation <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> <a href=\"https://www.kaggle.com/addisonhoward\" target=\"_blank\">@addisonhoward</a> Please take a look. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3206725,
      "author_name": "petermoorhouse",
      "author_url": "",
      "post_date": "05/21/2025 17:55:57",
      "content": "<p>As you suggested, I've now removed my post about why I think this happens.<br>\nIf an organiser would like my notes on this, just let me know.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3208082,
          "author_name": "gowillgo",
          "author_url": "",
          "post_date": "05/23/2025 17:20:24",
          "content": "<p><a href=\"https://www.kaggle.com/petermoorhouse\" target=\"_blank\">@petermoorhouse</a> Thanks again for your nice, insightful post. The core problem in the above submission was actually in mAA calculation. In my local test for the metric after the initial draft of this post, I was even able to produce <code>fbk_vineyard: score=127.27% (mAA=175.00%, clusterness=100.00%)</code>, which is definitely vulnerable in many senses. We'll see how the fix works. Thanks again!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3206853,
      "author_name": "sohier",
      "author_url": "",
      "post_date": "05/21/2025 20:46:09",
      "content": "<p><a href=\"https://www.kaggle.com/gowillgo\" target=\"_blank\">@gowillgo</a> Eduard and I will take a look. It may take us some time to coordinate.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3207612,
          "author_name": "cswwp347724",
          "author_url": "",
          "post_date": "05/23/2025 03:51:55",
          "content": "<p>Hi, <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> ,Does this mean the duration of the competition will be extended?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3207296,
      "author_name": "motono0223",
      "author_url": "",
      "post_date": "05/22/2025 14:45:17",
      "content": "<p>I also noticed case that the mAA exceeded 100%.<br>\n<code>ETs: score=104.00% (mAA=108.33%, clusterness=100.00%)</code></p>\n<p><a href=\"https://www.kaggle.com/code/motono0223/imc2025-maa108-ets-example\" target=\"_blank\">https://www.kaggle.com/code/motono0223/imc2025-maa108-ets-example</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 3207610,
          "author_name": "cswwp347724",
          "author_url": "",
          "post_date": "05/23/2025 03:47:32",
          "content": "<p>i also see this problem, seems bug in metric function?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3208300,
      "author_name": "oldufo",
      "author_url": "",
      "post_date": "05/23/2025 20:40:13",
      "content": "<p><a href=\"https://www.kaggle.com/gowillgo\" target=\"_blank\">@gowillgo</a> <a href=\"https://www.kaggle.com/petermoorhouse\" target=\"_blank\">@petermoorhouse</a> Thank you the reporting, we really appreciate it!</p>\n<p>The fixed version of the metric is available here now<br>\n<a href=\"https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py\" target=\"_blank\">https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 3208448,
          "author_name": "gowillgo",
          "author_url": "",
          "post_date": "05/24/2025 06:59:35",
          "content": "<p>Now recall and precision complement each other well. Thank you for all the effort. I had a moment of uncertainty, thinking things might just carry on given the high volume of rework, possibly ending up with some kind of magic <a href=\"https://www.kaggle.com/competitions/uspto-explainable-ai\" target=\"_blank\">end</a> which I’m not a fan of. But my belief that academia and scholarship would do the right thing has far exceeded, and I believe many participants will appreciate the action taken.</p>\n<p>Thanks again!</p>",
          "votes": null,
          "replies": [
            {
              "id": 3208822,
              "author_name": "oldufo",
              "author_url": "",
              "post_date": "05/24/2025 18:54:49",
              "content": "<p>Thank you for reporting, that's the main thing, that matters. We just did what we had to do, with full support from Kaggle orgs.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3204965": "The cluster score’s been odd to me all along, and here’s what I got on my CV set today:\n\n    final_score, dataset_scores = metric.score(\n        gt_csv='./filtered_train_labels.csv',\n        user_csv=submission_file,\n        thresholds_csv=f'{data_dir}/train_thresholds.csv',\n        mask_csv=None if not RERUN else os.path.join(data_dir, 'mask.csv'),\n        inl_cf=0,\n        strict_cf=-1,\n        verbose=True,\n    )\n\n./filtered_train_labels.csv\n\ndataset\tscene\timage\trotation_matrix\ttranslation_vector\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1310.png\t-0.45096986446600007;0.27640276616340004;-0.8486619380236001;-0.22647640738860003;0.88428353942478;0.40835157692580004;0.8633272908284001;0.3763563262902;-0.3361864742252201\t530.376;-32.5758;8.66025\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1200.png\t-0.49438604111399975;0.2992504042362;-0.8161052283251999;-0.2008084914918;0.8741562895071799;0.4421836760036;0.8457272006268;0.38249137133239997;-0.37207832170281985\t540.488;-56.942;30.3337\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1255.png\t-0.480305983746;0.25415113510376;-0.8394725706204399;-0.25052886194824;0.8774722137962327;0.40899617651612;0.8405606971715599;0.40675518877988;-0.35778302927376715\t536.308;-28.765;27.6173\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1220.png\t-0.4723317373060001;0.31990109336960004;-0.8213197004336;-0.21688922167040003;0.8609798276824799;0.460079128708;0.8543195101264;0.395445356036;-0.33728500142751994\t537.855;-63.0646;13.3448\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1160.png\t-0.5179049924259997;0.26548250388814;-0.8132001242871599;-0.23628750129585996;0.8692429742569598;0.43426369078175997;0.82215790377684;0.4170560831942399;-0.38745516311304007\t544.394;-46.0541;47.5699\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1155.png\t-0.511186782162;0.2588402620089;-0.8195656146679201;-0.23862885722309998;0.8733372238454078;0.42466244791394;0.8256769019920801;0.41265438920605996;-0.38467152209259226\t545.661;-40.4273;44.9054\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1190.png\t-0.5485391152799999;0.21904702067452;-0.8069216936175599;-0.22989207292548;0.8883665315084622;0.39743489486199995;0.80389915518244;0.40351364733;-0.4369465727475377\t539.761;-27.2237;73.5853\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1235.png\t-0.51286548354;0.272770387957;-0.813981831169;-0.229720591739;0.8699931798735;0.436280199082;0.8271629559829999;0.41074113238599996;-0.38352859310249987\t536.302;-47.537;43.3903\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1230.png\t-0.5217347486800001;0.28092921213600003;-0.8055244708920001;-0.22363741749600002;0.8661653987119999;0.446926879356;0.823272252012;0.41332324851600005;-0.38908213179200013\t535.852;-54.4949;47.289\nfbk_vineyard\tvineyard_split_2\tvineyard_split_2_frame_1215.png\t-0.4846348040199999;0.29270774428199997;-0.824289704106;-0.203701108158;0.87867415198;0.431784408786;0.850668677214;0.377165870106;-0.36621136387599984\t539.547;-50.5375;25.704\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1475.png\t-0.43655440942599966;0.2441491108092;-0.8659165170531999;-0.1735002444348;0.92156072964912;0.3473089176852;0.8827898209628;0.30185597693879995;-0.3599514672608797\t512.368;-10.5897;6.65932\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0090.png\t0.60940070194928;-0.1179033225381;0.78404708259042;0.1108402345611;0.991839419534035;0.0630001048134;-0.78507672464958;0.0485116206546;0.617496065484755\t-488.723;62.0719;-94.8123\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0190.png\t0.53043187119032;-0.11174926732833998;0.8403297313677439;0.11627415416605999;0.9914953705220798;0.05845724906793999;-0.8397155930682559;0.06670105057045998;0.5389142975357599\t-505.153;63.6353;-47.3736\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0265.png\t0.5024316490966199;-0.1440664075504;0.8525304111638;0.11003796779840001;0.98865702691062;0.10222009038440001;-0.8575866588202;0.0424519899524;0.51258533799\t-515.543;45.4976;-34.9178\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0290.png\t0.1827152068924799;-0.1825841649328;0.96606405046064;0.0934499591504;0.981381045126;0.16780460438720002;-0.97871539888336;0.05961797408960001;0.19637569237751995\t-502.175;18.5164;138.079\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1390.png\t-0.61088428465;0.24389941925167996;-0.7532147358502399;-0.21021443341232;0.8672501987647968;0.45131685122223997;0.76330155050176;0.43403926262576;-0.4785182546492033\t508.898;-57.0345;102.976\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1555.png\t-0.3896142358800001;0.306820506496;-0.8683673801680001;-0.262779872416;0.866645016368;0.42411455129600006;0.882693303928;0.3934305412160001;-0.25703098064800023\t502.466;-31.5815;-28.0105\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0075.png\t0.60266391944928;-0.1765441431392;0.77822118014776;0.128370227696;0.9839678709467;0.12380754060399998;-0.78760213582024;0.025286122525599995;0.61566495143342\t-490.228;30.7243;-95.8173\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0140.png\t0.61631690468582;-0.1323900181294;0.77629008172948;0.0989362256134;0.99097406046054;0.0904545920566;-0.78125861989052;0.02105452593739999;0.62385223567472\t-492.994;52.8367;-102.547\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0280.png\t0.3507627593310202;-0.15963154405563998;0.9227583533427759;0.14714297501276;0.9825194375641688;0.11403724051375999;-0.924831959561224;0.09577743712383999;0.36811989677714896\t-516.737;31.1435;52.1067\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0275.png\t0.54473792542488;-0.08624649615808;0.834159695580608;0.07852330167232;0.9955730324064128;0.05165685626512;-0.8349221202753919;0.037361498968480004;0.5490987453775328\t-510.948;76.3842;-57.1832\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0120.png\t0.58835908389912;-0.1421097419064;0.7960142112715201;0.11529874805519999;0.98912028211904;0.0913634055704;-0.80033743008048;0.038024918616800005;0.59834297293592\t-494.821;48.5412;-84.8384\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1535.png\t-0.47973478756199994;0.2492251514408;-0.8412746576420002;-0.2867349429592;0.86163642122928;0.41876712898479995;0.829240193302;0.4421195084152;-0.34189556383272013\t504.914;-24.8421;26.4884\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0205.png\t0.6106958808574999;-0.06976742683359999;0.7887859260567;0.0884298177724;0.99588914017628;0.019621157275200002;-0.7869122557953;0.0577696169028;0.61435491464278\t-496.436;87.8103;-94.0397\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1410.png\t-0.5233981321799999;0.22283728763796;-0.8224340018261199;-0.24712970381404004;0.8840135385507288;0.39679585939524;0.81546370465788;0.41093014951275997;-0.4076213029052711\t516.426;-21.8037;56.0843\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0305.png\t0.4627330191951199;-0.1111267027116;0.8795049273668402;0.0602394287796;0.9937604130175;0.0938693329872;-0.8844485697531602;0.009544443388800006;0.4665399662223799\t-518.488;61.2659;-11.9061\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1520.png\t-0.4426463281319999;0.256743921225;-0.859154084569;-0.100230111575;0.9379624663575;0.33193416735;0.891076375431;0.23304275495;-0.3894521280104999\t507.62;-17.5998;10.1222\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1465.png\t-0.4968166092199997;0.27276438345582;-0.8238766921380599;-0.24798936723618;0.8651227477203262;0.43596318783042;0.83166969793794;0.42090642490158003;-0.36216467925567364\t510.913;-41.1816;32.9071\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0110.png\t0.64203171652358;-0.07928514436382;0.762567666990514;0.06655954761538;0.9966471779814662;0.04758394800508;-0.763783612993486;0.02020572616172;0.6451562790218862\t-483.736;79.1617;-115.213\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0200.png\t0.5985017101599199;-0.08324152791520001;0.7967851671261401;0.09944926157440001;0.994613859171675;0.029208153107800002;-0.7949249014178601;0.061758561074599996;0.603556403475755\t-498.297;80.877;-86.8741\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1440.png\t-0.517530307396;0.3034291605098;-0.8000601703978;-0.24403450075420002;0.8438411834922199;0.4778907236068;0.8201296812982001;0.44256426179720004;-0.36266652626777995\t510.974;-63.7647;38.151\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_0230.png\t0.56873842000062;-0.10835957185299999;0.81534897651194;0.07321356390020001;0.9940186344654;0.0810352792214;-0.8192530282960601;0.013606822013000001;0.57326998852878\t-506.037;63.6934;-73.9118\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1400.png\t-0.5600271884200001;0.19081582154300003;-0.806200089329;-0.237897472313;0.8950932793455;0.37711111714600004;0.793583048239;0.40298551370600005;-0.45588195179049995\t514.387;-13.5811;82.1421\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1485.png\t-0.5026803372659998;0.25712356923519997;-0.8253468535042;-0.2184205547408;0.88598782423302;0.40904504076279996;0.8364223931197999;0.3858922832212;-0.38920739491697987\t508.936;-32.7062;39.5221\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1470.png\t-0.45382847489800016;0.251393940534;-0.8548931472132;-0.20805190645799998;0.90297008024312;0.3759781083036;0.8664615460067999;0.3484913490564;-0.35749064495887994\t512.296;-18.2419;14.6334\nfbk_vineyard\tvineyard_split_3\tvineyard_split_3_frame_1565.png\t-0.4680401747999998;0.29573148083428;-0.83275597003404;-0.26410201542972;0.8524664788353342;0.45116643323185995;0.84332066251796;0.43109617984014;-0.3208854020606655\t500.953;-44.7338;12.9665\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1055.png\t-0.49401401832400005;0.2221184836632;-0.8406026605248;-0.2824626932448;0.87337372166128;0.39677841453839996;0.8222920895832;0.4334531837856;-0.3687186179387201\t557.841;-17.2461;40.733\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0935.png\t-0.56070208269;0.2776073522472;-0.7800951234864001;-0.21657491425680003;0.86015662933688;0.46176398227840004;0.7991930619216;0.42786075631360004;-0.42216892035711995\t563.852;-68.1348;72.5404\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0960.png\t-0.576299799906;0.30025187450199997;-0.760084830064;-0.20935704493399998;0.84478740867;0.49244697083199995;0.789968204912;0.442925731024;-0.42399112512000015\t557.94;-86.884;77.5747\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1095.png\t-0.6406246538179998;0.1787753640684;-0.746751722238;-0.2711690117316;0.85718152040408;0.437843689094;0.7183774304220001;0.482989749586;-0.5006531650139199\t536.785;-41.7236;135.597\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0945.png\t-0.49943696121000025;0.2180181433032;-0.8384691656664;-0.2978660743128;0.86560416794048;0.40249869824560003;0.8135344211256;0.4507743221464;-0.36737448285352015\t569.414;-17.9502;44.7769\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0905.png\t-0.6298631872799998;0.248062909785;-0.736028307195;-0.22765754905500002;0.847044929223875;0.48029884049349997;0.742593372285;0.47008481001849994;-0.47704907276012487\t557.911;-76.8757;120.016\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1065.png\t-0.5106840009040001;0.22467611040816;-0.82989415920064;-0.24511506834384;0.8871469872537312;0.39101018164608004;0.82408875331136;0.40310172945792;-0.39798040846626903\t555.788;-21.4594;50.7536\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1110.png\t-0.3785696427280001;0.282491242764;-0.8814082541264001;-0.22357147045200002;0.89617798150968;0.3832500242496;0.8981634652336001;0.34214540951039996;-0.27610855536232015\t552.442;-21.6061;-33.6109\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1100.png\t-0.5769026812160001;0.19173622887359998;-0.7939908723744;-0.24697893867839998;0.88561096761848;0.3933124581452;0.7785792770976001;0.4230017782708;-0.4635566029415201\t545.921;-22.137;95.8452\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0940.png\t-0.4529469422499999;0.30956856345300005;-0.836064870586;-0.260909883747;0.85069551742782;0.456336469632;0.8525040738140001;0.424834374168;-0.3045502404221798\t569.194;-55.8551;0.284154\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1010.png\t-0.491117283016;0.24408210573999997;-0.836197569146;-0.272647621804;0.8686366413915;0.41368289700499994;0.8273244365499999;0.43115448577899995;-0.3600539414485\t562.834;-28.8324;35.0702\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_1105.png\t-0.40761659251200033;0.23662429191016002;-0.8819627063426241;-0.23530146904984003;0.9060090616669695;0.35182500772279;0.8823165472573762;0.35093669211721;-0.31362632022103076\t554.225;-2.17632;-10.6082\nfbk_vineyard\tvineyard_split_1\tvineyard_split_1_frame_0985.png\t-0.49911414688800027;0.2679426943970001;-0.8240711404720001;-0.26430339558700006;0.858604704175875;0.43925142583700005;0.8252455677680001;0.4370409735230001;-0.3577237285361252\t564.567;-45.0501;35.2734\n\n\n\nsubmission.csv\n\ndataset\tscene\timage\trotation_matrix\ttranslation_vector\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1310.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1200.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1255.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1220.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1160.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1155.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1190.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1235.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1230.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_2_frame_1215.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1475.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0090.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0190.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0265.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0290.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1390.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1555.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0075.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0140.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0280.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0275.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0120.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1535.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0205.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1410.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0305.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1520.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1465.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0110.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0200.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1440.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_0230.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1400.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1485.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1470.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_3_frame_1565.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_1055.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0935.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0960.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1095.png\t0.996849134;0.023781899;0.075671823;-0.023495817;0.999713034;-0.004668714;-0.075761139;0.002876032;0.997121847\t5.189118625;-0.209417331;0.875569692\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0945.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0905.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_1065.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1110.png\t0.935267251;-0.012903738;0.353707030;-0.013198564;0.997368669;0.071284895;-0.353696152;-0.071338853;0.932635942\t-9.860201617;-1.951112114;0.506709770\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1100.png\t0.989890721;-0.001467119;0.141824565;-0.005763828;0.998704347;0.050560904;-0.141714988;-0.050867222;0.988599711\t-0.622975224;0.187419191;0.607450304\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0940.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\toutliers\tvineyard_split_1_frame_1010.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\nfbk_vineyard\tcluster0\tvineyard_split_1_frame_1105.png\t0.946100098;-0.047238793;0.320410831;0.019006687;0.995699106;0.090675446;-0.323316177;-0.079698100;0.942928875\t-4.739468462;-0.319373064;-0.255384166\nfbk_vineyard\toutliers\tvineyard_split_1_frame_0985.png\tnan;nan;nan;nan;nan;nan;nan;nan;nan\tnan;nan;nan\n\n\n-----------------------------------------------------------------------------\n-----------------------------------------------------------------------------\n`fbk_vineyard: score=40.00% (mAA=25.00%, clusterness=100.00%)`\n\nIn the submission.csv, there are only 1 cluster and 1 outliers cluster, which means the clusterness should be under 33.33%, but it’s showing 100.00%. Enlighten me if I'm mistaken; otherwise this isn't just a fluke.\n\n[Follow-up]\nNow, I've got the reason. I think this is a critical vulnerability, so that the scoring module (including mAA. which doesn't seem to be  \"equivalent to recall\" in Overview) should be revisited by the host and rescoring may benefit fairness unless the hidden set is protected.",
    "3206519": "It seems there is a bug in metric computation @sohier @addisonhoward Please take a look.",
    "3206725": "As you suggested, I've now removed my post about why I think this happens.\nIf an organiser would like my notes on this, just let me know.",
    "3206853": "gowillgo Eduard and I will take a look. It may take us some time to coordinate.",
    "3207296": "I also noticed case that the mAA exceeded 100%.\n`ETs: score=104.00% (mAA=108.33%, clusterness=100.00%)`\n\nhttps://www.kaggle.com/code/motono0223/imc2025-maa108-ets-example",
    "3207610": "i also see this problem, seems bug in metric function?",
    "3207612": "Hi, @sohier ,Does this mean the duration of the competition will be extended?",
    "3208082": "petermoorhouse Thanks again for your nice, insightful post. The core problem in the above submission was actually in mAA calculation. In my local test for the metric after the initial draft of this post, I was even able to produce `fbk_vineyard: score=127.27% (mAA=175.00%, clusterness=100.00%)`, which is definitely vulnerable in many senses. We'll see how the fix works. Thanks again!",
    "3208300": "gowillgo @petermoorhouse Thank you the reporting, we really appreciate it!\n\nThe fixed version of the metric is available here now\nhttps://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py",
    "3208448": "Now recall and precision complement each other well. Thank you for all the effort. I had a moment of uncertainty, thinking things might just carry on given the high volume of rework, possibly ending up with some kind of magic [end](https://www.kaggle.com/competitions/uspto-explainable-ai) which I’m not a fan of. But my belief that academia and scholarship would do the right thing has far exceeded, and I believe many participants will appreciate the action taken.\n\nThanks again!",
    "3208822": "Thank you for reporting, that's the main thing, that matters. We just did what we had to do, with full support from Kaggle orgs."
  },
  "source": "meta"
}