This seems a bit quirky - most DoubleHeaders have 2 games on the same day with one GameNumber 1 the other 2 and both with Y. But for the Twins on the 14th of April, one is Y and one is N and both have GameNumber 1. It is possible one of the games is for the postponed on the 12th. Seems odd.
\nIt may be some of these aberrations in the data are causing problems when putting it altogether for a submission. Just thought I'd point it out.
", "messages": [ { "id": "1380803", "postDate": "07/08/2021 10:19:21", "content": "This seems a bit quirky - most DoubleHeaders have 2 games on the same day with one GameNumber 1 the other 2 and both with Y. But for the Twins on the 14th of April, one is Y and one is N and both have GameNumber 1. It is possible one of the games is for the postponed on the 12th. Seems odd.
\nIt may be some of these aberrations in the data are causing problems when putting it altogether for a submission. Just thought I'd point it out.
", "rawMarkdown": "This seems a bit quirky - most DoubleHeaders have 2 games on the same day with one GameNumber 1 the other 2 and both with Y. But for the Twins on the 14th of April, one is Y and one is N and both have GameNumber 1. It is possible one of the games is for the postponed on the 12th. Seems odd.\n\nIt may be some of these aberrations in the data are causing problems when putting it altogether for a submission. Just thought I'd point it out.", "votes": null }, { "id": "1381238", "postDate": "07/08/2021 18:41:09", "content": "There are a few quirky things in the data.
\nFrom my notes:
\ndailyDataDate==2020918 gamePk==631122 Start time of 2020-09-18T03:33:00Z is not accurate; that would imply the game started the day before (2020-09-17) at ~11:30PM local time
But both teams had games the day before against other opponents. This game was actually the 2nd game of a doubleheader on 2020-09-18 at 4PM local time. The first game of the doubleheader (gamePk==630907) is a makeup game from 2020-08-02 that started on 2020-08-18 at 1PM local time (according to pretty much every stats site I looked at). But in the playerBoxScores data it shows gameTimeUTC==2020-09-18T20:05:00Z which implies a local start time of 4PM. So the start times for these games are both incorrect. It's a very confusing situation!!
Then there's also this case:
\n\n\nOn July 27, 2019, the Royals traded Diekman to the Oakland Athletics for Ismael Aquino and Dairon Blanco.[28] On September 6, Diekman pitched in a suspended game between the Athletics and Detroit Tigers that originally started on May 19, allowing a double in a scoreless inning. This appearance would be recorded to have occurred on May 19, when Diekman threw an inning of relief for the Royals against the Los Angeles Angels. Diekman thus holds the rare feat of having pitched for two different teams on the same day.
\n
dailyDataDate engagementMetricsDate playerId target1 target2 \\\n1038525 20190519 2019-05-20 518617 0.00124 1.060508 \n1038526 20190519 2019-05-20 518617 0.00124 1.060508 \n\n target3 target4 home gamePk gameDate gameTimeUTC \\\n1038525 0.018201 1.460091 0.0 565534.0 2019-05-19 2019-05-19T17:10:00Z \n1038526 0.018201 1.460091 0.0 564756.0 2019-05-19 2019-05-19T20:07:00Z \n\n teamId teamName playerName jerseyNum positionCode \\\n1038525 133.0 Oakland Athletics Jake Diekman 35 1.0 \n1038526 118.0 Kansas City Royals Jake Diekman 40 1.0 \nSo the game continued on September 6, when Diekman was now on the A's, but the dailyDataDate is still 20190519… in real life this data wouldn't have been available on that day.
Interesting for Jake Diekman, maybe an alternative universe or some field of dreams games here.
\nI agree there are issues with dates and gameTimeUTC and had posted here about transaction data and rosters. So players have been called up/traded per a transaction and play before even on a roster. From other examples rosters are probably out by one day so again they play on one date and appear on a roster for the team the next. My thought was that it could be East Coast time zones are off maybe? And maybe some datetimes were in UTC to begin with and others not? Some players are even on rosters for up to 3 different teams in a month. Since rosters seem to be an issue in the submit test and probably will be in future, was trying to get a baseline roster and just use transactions to update but even that is problematic.
\nBeginning to think it might be better/easier to scrape your own data. DIY competition.
", "rawMarkdown": "Interesting for Jake Diekman, maybe an alternative universe or some field of dreams games here.\n \nI agree there are issues with dates and gameTimeUTC and had posted [here](https://www.kaggle.com/c/mlb-player-digital-engagement-forecasting/discussion/248359) about transaction data and rosters. So players have been called up/traded per a transaction and play before even on a roster. From other examples rosters are probably out by one day so again they play on one date and appear on a roster for the team the next. My thought was that it could be East Coast time zones are off maybe? And maybe some datetimes were in UTC to begin with and others not? Some players are even on rosters for up to 3 different teams in a month. Since rosters seem to be an issue in the submit test and probably will be in future, was trying to get a baseline roster and just use transactions to update but even that is problematic.\n \nBeginning to think it might be better/easier to scrape your own data. DIY competition.", "votes": null } ], "comments": [ { "id": 1381238, "author_name": "brandenkmurray", "author_url": "", "post_date": "07/08/2021 18:41:09", "content": "There are a few quirky things in the data.
\nFrom my notes:
\ndailyDataDate==2020918 gamePk==631122 Start time of 2020-09-18T03:33:00Z is not accurate; that would imply the game started the day before (2020-09-17) at ~11:30PM local time
But both teams had games the day before against other opponents. This game was actually the 2nd game of a doubleheader on 2020-09-18 at 4PM local time. The first game of the doubleheader (gamePk==630907) is a makeup game from 2020-08-02 that started on 2020-08-18 at 1PM local time (according to pretty much every stats site I looked at). But in the playerBoxScores data it shows gameTimeUTC==2020-09-18T20:05:00Z which implies a local start time of 4PM. So the start times for these games are both incorrect. It's a very confusing situation!!
Then there's also this case:
\n\n\nOn July 27, 2019, the Royals traded Diekman to the Oakland Athletics for Ismael Aquino and Dairon Blanco.[28] On September 6, Diekman pitched in a suspended game between the Athletics and Detroit Tigers that originally started on May 19, allowing a double in a scoreless inning. This appearance would be recorded to have occurred on May 19, when Diekman threw an inning of relief for the Royals against the Los Angeles Angels. Diekman thus holds the rare feat of having pitched for two different teams on the same day.
\n
dailyDataDate engagementMetricsDate playerId target1 target2 \\\n1038525 20190519 2019-05-20 518617 0.00124 1.060508 \n1038526 20190519 2019-05-20 518617 0.00124 1.060508 \n\n target3 target4 home gamePk gameDate gameTimeUTC \\\n1038525 0.018201 1.460091 0.0 565534.0 2019-05-19 2019-05-19T17:10:00Z \n1038526 0.018201 1.460091 0.0 564756.0 2019-05-19 2019-05-19T20:07:00Z \n\n teamId teamName playerName jerseyNum positionCode \\\n1038525 133.0 Oakland Athletics Jake Diekman 35 1.0 \n1038526 118.0 Kansas City Royals Jake Diekman 40 1.0 \nSo the game continued on September 6, when Diekman was now on the A's, but the dailyDataDate is still 20190519… in real life this data wouldn't have been available on that day.
Interesting for Jake Diekman, maybe an alternative universe or some field of dreams games here.
\nI agree there are issues with dates and gameTimeUTC and had posted here about transaction data and rosters. So players have been called up/traded per a transaction and play before even on a roster. From other examples rosters are probably out by one day so again they play on one date and appear on a roster for the team the next. My thought was that it could be East Coast time zones are off maybe? And maybe some datetimes were in UTC to begin with and others not? Some players are even on rosters for up to 3 different teams in a month. Since rosters seem to be an issue in the submit test and probably will be in future, was trying to get a baseline roster and just use transactions to update but even that is problematic.
\nBeginning to think it might be better/easier to scrape your own data. DIY competition.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1380803": "This seems a bit quirky - most DoubleHeaders have 2 games on the same day with one GameNumber 1 the other 2 and both with Y. But for the Twins on the 14th of April, one is Y and one is N and both have GameNumber 1. It is possible one of the games is for the postponed on the 12th. Seems odd.\n\nIt may be some of these aberrations in the data are causing problems when putting it altogether for a submission. Just thought I'd point it out.", "1381238": "There are a few quirky things in the data.\n\nFrom my notes:\n`dailyDataDate==2020918 gamePk==631122 Start time of 2020-09-18T03:33:00Z is not accurate; that would imply the game started the day before (2020-09-17) at ~11:30PM local time`\n\nBut both teams had games the day before against other opponents. This game was actually the 2nd game of a doubleheader on 2020-09-18 at 4PM local time. The first game of the doubleheader (`gamePk==630907`) is a makeup game from 2020-08-02 that started on 2020-08-18 at 1PM local time (according to pretty much every stats site I looked at). But in the `playerBoxScores` data it shows `gameTimeUTC==2020-09-18T20:05:00Z` which implies a local start time of 4PM. So the start times for these games are both incorrect. It's a very confusing situation!!\n\nThen there's also this case:\n>On July 27, 2019, the Royals traded Diekman to the Oakland Athletics for Ismael Aquino and Dairon Blanco.[28] On September 6, Diekman pitched in a suspended game between the Athletics and Detroit Tigers that originally started on May 19, allowing a double in a scoreless inning. This appearance would be recorded to have occurred on May 19, when Diekman threw an inning of relief for the Royals against the Los Angeles Angels. Diekman thus holds the rare feat of having pitched for two different teams on the same day.\n\n```\n dailyDataDate engagementMetricsDate playerId target1 target2 \\\n1038525 20190519 2019-05-20 518617 0.00124 1.060508 \n1038526 20190519 2019-05-20 518617 0.00124 1.060508 \n\n target3 target4 home gamePk gameDate gameTimeUTC \\\n1038525 0.018201 1.460091 0.0 565534.0 2019-05-19 2019-05-19T17:10:00Z \n1038526 0.018201 1.460091 0.0 564756.0 2019-05-19 2019-05-19T20:07:00Z \n\n teamId teamName playerName jerseyNum positionCode \\\n1038525 133.0 Oakland Athletics Jake Diekman 35 1.0 \n1038526 118.0 Kansas City Royals Jake Diekman 40 1.0 \n```\n\nSo the game continued on September 6, when Diekman was now on the A's, but the `dailyDataDate` is still 20190519... in real life this data wouldn't have been available on that day.", "1381856": "Interesting for Jake Diekman, maybe an alternative universe or some field of dreams games here.\n \nI agree there are issues with dates and gameTimeUTC and had posted [here](https://www.kaggle.com/c/mlb-player-digital-engagement-forecasting/discussion/248359) about transaction data and rosters. So players have been called up/traded per a transaction and play before even on a roster. From other examples rosters are probably out by one day so again they play on one date and appear on a roster for the team the next. My thought was that it could be East Coast time zones are off maybe? And maybe some datetimes were in UTC to begin with and others not? Some players are even on rosters for up to 3 different teams in a month. Since rosters seem to be an issue in the submit test and probably will be in future, was trying to get a baseline roster and just use transactions to update but even that is problematic.\n \nBeginning to think it might be better/easier to scrape your own data. DIY competition." }, "source": "meta" }