Having Your Voice Heard

[Hermes_]

I can tell you that I've got a largish group (Both the PlayEDH and a personal server on Discord) who are excited to use a form-submission system to submit data. I'm just slowly fleshing out a proper form that covers as many basis as possible, and having people provide feedback in the meantime. I should be ready to show you awesome people it in the next week!

how many is largish? I ask because we have a facebook group for our local magic player and it has supposedly 98 members,when only 6 to 8 show up to play on a good day.

[Hawk]

Hi Sheldon (and thread!) Some initial background - my name's Jason and I've played MTG for twenty years now. I've played EDH since shortly before the first precons. I used to play a ton with a robust playgroup in grad school; now I play much more rarely and usually with my wife, sister and brother-in-law, and a few close friends. I very occasionally will show up to a cardshop on casual night, but honestly haven't done so for the last few years. I want ya'll to know what kind of filthy casual player I am to understand my argument better

I voted "1" since I think I have next to no influence on the banlist - but I also think that's a good thing. I think it's good I have next-to-no influence, and good that my playgroup also has no influence. Why? Because we're one playgroup. We have our own tastes, our own idiosyncrasies, our own gentleman's agreements and social contracts on what is fair play (no Mass Land destruction, no easy infinites, budget, basically everyone is running tribal beatdown). In our world, every single STAX effect no matter how innocuous would be banned and is already effectively shadow-banned, but we could probably let Coalition Victory go because we're all too poor to own original ABU duals (I own two after desperately trading for them but they're the only ones in our entire combined collections) and because games are beatdown-y enough and we all pack enough counterspells and cheap instant-speed removal to make it a cool once-in-a-while trick, and not a cohesive and consistent strategy. But I'd never dream of making those decisions for all players worldwide, since our little family carebear powwow isn't everyone's experience.

I think that's why you guys experience such blowback and there's such a call for data here - because it seems to us like you and the Rules Committee are also just one or two playgroups with your own tastes, contracts, and agreements and you're imposing "your way" of having fun and playing onto the rest of us that have come to love and cherish this format as dearly as its creator does. WotC has kind of put you guys in a weird position - you created a format as fans, and now are expected to behave like fair, impartial, logical game designers with a robust understanding of what is good for the hundreds of thousands of other players out there. I don't envy you at all.

And I don't know that I have a good solution. Data is alluring to me, too, because it's impartial and fair. But it's also somewhat biasing especially if you are asking for it. WotC has an entire department now dedicated to analyzing MODO and Arena data to see win percentages and game outcomes and matchup analyses - you guys don't (I presume) have access to that data, at least not easily, and don't have time to do comprehensive analysis of how often, say, turn 1 Sol Ring leads to a win or how often Deadeye Navigator or Flash effectively ends the game on the turn they were cast even if you DID have access. So, you'd have to crowdsource, but that will largely bias your data towards people who take this format very, very seriously - I get the impression posting here that my way of playing is not the norm for enfranchised players who are slinging cardboard in a non-family playgroup weekly. Even if WOTC can give you MODO data, that is also biased - most likely to competitive queues and highly-enfranchised players. If MaRo is to be believed (and he has no reason to lie about stuff like this), those folks represent the smallest percentage of actual Magic players. The world of Magic and Commander especially is made up of MY anti-LD, anti-Stax, anti-counterspell, pro-turning dudes sideways people, not them.

At the same time, part of me wonders if perhaps the more competitive, enfranchised folks SHOULD have the largest voice despite being the minority. I normally hate entitlement, but here's the thing - I think when you guys first made a banlist, it made sense to keep it as minimal as possible. The format was new, 100 card decks were hard, people were scraping decks together from boxes of old stuff that had collected dust because other formats didn't allow for it to shine. No one had any clue what they were doing. It was the wild west and 40 life meant playing all the dumb six drops and nostalgic cute things you wanted. Folks were just playing with friends and family, or showing up to cons to get in a few friendly matches. In this world, you wanted the ban list small because everyone was scrapping decks together and you didn't want lil' Timmy showing up and finding out that his very favorite card that he was going to use totally fairly is not allowed in the format unless that card was going to just blue screen the game like Biorhythm or Shahrazad tends to do.

But the format's evolved, now. Obviously I don't have data, but now that WOTC officially supports Commander I bet WOTC could tell you that the largest consumers of the game, the silent casual majority, are not scrapping together their decks from 20-year old collections. They're buying precons and peppering in commons from bulk lots and random booster junk - so they're unlikely to ever look at a banned list, and also unlikely to ever have a #feelsbad moment of their deck being gutted if they do show up at FNM. Meanwhile, the format has grown increasingly popular and prolific - tomes are written about it, an entire website exists to collate data on it and tell you the most common and synergistic builds, podcasts come out weekly for it, and enfranchised players post passionately about it. And for the enfranchised, I think we need a firmer hand and a more robust banlist - to help make games more fun, and more fair. Our inclination as nerds is to tweak, to optimize, to achieve bigger and better faster and more efficiently. And we have more data at our fingertips than ever before, making it easy to optimize. You yourself said a fear is that the game is "descending to the infinite" earlier and more often - and I think without data or crowd-driven decision making, you're never going to be able to combat that with your playgroup and the social contract alone. I'm not convinced that Johnny and Jane the casuals have a noticeably worse experience if you did allow r/EDH and MTGNexus to straight up vote on banned cards, and meanwhile all the MODO players and all the CEDH folks have a significantly improved experience. Food for thought, at least.

I think this thread is the best possible start - being out here, being present, and sincerely trying to get a pulse on the format. I don't know for sure that we should all vote on banning Flash or not - and you shouldn't give us a poll if you don't have any intention of utilizing it. But knowing that you and others will at least read and consider that everyone hates Flash-Hulk helps some.

[SkyeDragonQueen]

I can tell you that I've got a largish group (Both the PlayEDH and a personal server on Discord) who are excited to use a form-submission system to submit data. I'm just slowly fleshing out a proper form that covers as many basis as possible, and having people provide feedback in the meantime. I should be ready to show you awesome people it in the next week!

how many is largish? I ask because we have a facebook group for our local magic player and it has supposedly 98 members,when only 6 to 8 show up to play on a good day.

One of the servers is "PlayEDH", which boasts around 5,000 members (a little under I believe), and has near round the clock pods firing, day and night. While I wouldn't expect to have every pod be gracious enough to record and submit data, a good number of members have already agreed to once the form is finalized. A few smaller discords have also shown collective interest.

[Sheldon]

As a side note to the engaging data discussion, someone help me understand how "use data" doesn't become "make it a tournament format."

If we're not using tournament data (or win/loss rates of any kind), then how are we applying it? A card being in X% decks (in a strongly-biased reporting system, btw) is only part of the puzzle.

[SkyeDragonQueen]

As a side note to the engaging data discussion, someone help me understand how "use data" doesn't become "make it a tournament format."

If we're not using tournament data (or win/loss rates of any kind), then how are we applying it? A card being in X% decks (in a strongly-biased reporting system, btw) is only part of the puzzle.

While I definitely have bias in this, in my opinion it comes down to helping players better understand where problematic cards lay, rather than strictly a "This deck has a high win rate, you should play it!". If the data begins to show a correlation between certain strategies at any given power level and a higher than expected win rate, it's more possible to make educated decisions on what to do to continue the health of the format. While there will be players who see data as a means to power up their lists, I don't necessarily believe that will be the ultimate goal of this data collection. I'm interested in winrates not because it can show the strength of a deck (though of course, that is interesting to gleam), but instead to point to what is causing degeneracy, if at all.

As another user here said, it's important to get the social aspect as a major datapoint, and I've taken that advice to heart. As more and more players begin to use the form, and it becomes stronger and more accepted, I hope to find when friendly games of EDH go arry, and why. It's possibile that through enough data collection, we may even be able to see more subtle information, such as certain strategies or archetypes causing more headaches for players than others, or even certain commanders that seem underused being more popular. What we do with that information is of course, the major puzzle.

[Sheldon]

So maybe the other way to ask the question is "what data points that aren't win rate are important?"

[SkyeDragonQueen]

So maybe the other way to ask the question is "what data points that aren't win rate are important?"

While I don't have a complete answer to that (and I seriously would love anyone who reads this to chime in!) so far my guesses are: Complete Decklists or Sample lists of commonly played archetypes for each commander (samples would make it easier for each player who submits a report to easily choose from, even if the cards aren't 100% correct, if they don't have a decklist online it at least gets a bit more information than "none"); Feelings/Prejudice for or against certain commanders or strategies (such as how players feel about certain cards or strategies in regards to winning against them, with them, or losing with/against them); Win Conditions (while winrate itself isn't what is important here, it's how the game ultimately ended. This can provide more insight into degenerate combos, or cards with enough gamedata); and Venue. While venue may seem a bit strange, I think it's important to note which players are providing the majority of the data. If this were widely accepted and used across the format, it'd be interesting to see if there are players using more at FNM/Hobby stores, Kitchen Table, side events at larger venues like GPs, etc. to better understand if the voices we're hearing from reflect enough of the player base or not. It would also help with better communicating the data collection to the areas that are lacking, to ensure a more even and unbiased distribution of data.

Turn order and mulligan data are likely more importantly tied to win rate, so they're less important, albeit still currently included. But it may be interesting to at least note if going first has a much larger connection to winning the game than perhaps originally thought, or the inverse.

[HoffOccultist]

So maybe the other way to ask the question is "what data points that aren't win rate are important?"

While I don't have a complete answer to that (and I seriously would love anyone who reads this to chime in!) so far my guesses are: Complete Decklists or Sample lists of commonly played archetypes for each commander (samples would make it easier for each player who submits a report to easily choose from, even if the cards aren't 100% correct, if they don't have a decklist online it at least gets a bit more information than "none"); Feelings/Prejudice for or against certain commanders or strategies (such as how players feel about certain cards or strategies in regards to winning against them, with them, or losing with/against them); Win Conditions (while winrate itself isn't what is important here, it's how the game ultimately ended. This can provide more insight into degenerate combos, or cards with enough gamedata); and Venue. While venue may seem a bit strange, I think it's important to note which players are providing the majority of the data. If this were widely accepted and used across the format, it'd be interesting to see if there are players using more at FNM/Hobby stores, Kitchen Table, side events at larger venues like GPs, etc. to better understand if the voices we're hearing from reflect enough of the player base or not. It would also help with better communicating the data collection to the areas that are lacking, to ensure a more even and unbiased distribution of data.

Turn order and mulligan data are likely more importantly tied to win rate, so they're less important, albeit still currently included. But it may be interesting to at least note if going first has a much larger connection to winning the game than perhaps originally thought, or the inverse.

I think something that would be interesting to collect, though I don't know how well one could collect it because it would probably require a bit of a narrative answer, would be Memorable Experience from the game (positive or negative, with some indication of which it is). If someone goes off explosively with fast mana, but it ends up being enjoyable for the table to play archenemy for a pod, that should be recorded, just like someone going off explosively with fast mana that leads to feel bads (replace fast mana in these examples with literally anything else and it's still good information). This helps get to both the social information--what are people enjoying/not enjoying about games--as well as providing some data on card use--specifically, how things work together. I bring up that point about how things work together, because sometimes there's something innocuous in a decklist that doesn't make sense to an observer until it's pointed out or seen in action, and those interactions can end up having huge impacts on games even though they seem like edge cases. But, as I mentioned, I don't know how easy any of this would be to gather, because it's inherently biased information (what one player answers was fun another might have found miserable), and because it's more than a drop down or checklist style answer.

With regard to what Skye said above, I echo the sentiment that venue information is quite important, and in extension should include something about what level of familiarity the group has. For instance, when I play EDH (at any end of the spectrum), I am almost always playing it at an LGS, but I also am almost always playing with a very similar group of people (one group for lower powered games, one group for cEDH games). That familiarity probably colors all of our experiences as players more than the fact that we are playing in an LGS (doubly so since that lower power group had to shift venues when the LGS running a league we were all enjoying closed).

[SkyeDragonQueen]

[...]
I think something that would be interesting to collect, though I don't know how well one could collect it because it would probably require a bit of a narrative answer, would be Memorable Experience from the game (positive or negative, with some indication of which it is). If someone goes off explosively with fast mana, but it ends up being enjoyable for the table to play archenemy for a pod, that should be recorded, just like someone going off explosively with fast mana that leads to feel bads (replace fast mana in these examples with literally anything else and it's still good information). This helps get to both the social information--what are people enjoying/not enjoying about games--as well as providing some data on card use--specifically, how things work together. I bring up that point about how things work together, because sometimes there's something innocuous in a decklist that doesn't make sense to an observer until it's pointed out or seen in action, and those interactions can end up having huge impacts on games even though they seem like edge cases. But, as I mentioned, I don't know how easy any of this would be to gather, because it's inherently biased information (what one player answers was fun another might have found miserable), and because it's more than a drop down or checklist style answer.

With regard to what Skye said above, I echo the sentiment that venue information is quite important, and in extension should include something about what level of familiarity the group has. For instance, when I play EDH (at any end of the spectrum), I am almost always playing it at an LGS, but I also am almost always playing with a very similar group of people (one group for lower powered games, one group for cEDH games). That familiarity probably colors all of our experiences as players more than the fact that we are playing in an LGS (doubly so since that lower power group had to shift venues when the LGS running a league we were all enjoying closed).

Ah! Thank you SO much! How did I think of familiarity? That'll be added to the form on the next update (likely later tonight!). Brilliant idea!

As it stands now, I'm going to try and add in an optional submission at the end of the form for each player to be able to submit their thoughts on the game in an in-depth way. While many will likely opt not to do this, even just a few players giving that bigger feedback can help a lot.

[pokken]

More than how games go a more accurate edhrec would be amazing. If it was even say halfway representative of the general population it'd be the best resource.

[Styrofoam]

Not to be a negative Nancy, but doesn't the very act of gathering, disseminating, consuming, and applying data move commander to a more homogenized, more competitive place?

I would argue that the large amount of commander related content is responsible for the move away from the original intent of the format, and more towards the 75% optimized lists that are becoming the norm when people say they are "casual"

I've got a casual deck, I've mentioned it before. It's Soraya the Falconer mono white bird tribal deck. When people say "Casual" and pull out their 75% Muldrotha deck, I feel like that bar for casual is moving towards the more streamlined, optimized lists, where non-optimized lists like mine can't compete... Quantifying data is going to further push this line on the spectrum to further optimize (for those who care to interpret it and apply it) and for the truly casual players who have no desire to consume that type of content, they will continue to left further and further from where the format is going.

Now, I could be wrong, but [mention]Sheldon[/mention] has said several times that the format is moving more and more towards this optimized deckbuilding, and I think this will only exacerbate this.

(as a side note, I think there is a large problem with 75% decks claiming to be 'casual' and steamrolling over casuals. almost to the point, where there's not a place to play some of my favorite decks, like my very tuned Lord Windgrace)

[HoffOccultist]

Not to be a negative Nancy, but doesn't the very act of gathering, disseminating, consuming, and applying data move commander to a more homogenized, more competitive place?

I would argue that the large amount of commander related content is responsible for the move away from the original intent of the format, and more towards the 75% optimized lists that are becoming the norm when people say they are "casual"

I've got a casual deck, I've mentioned it before. It's Soraya the Falconer mono white bird tribal deck. When people say "Casual" and pull out their 75% Muldrotha deck, I feel like that bar for casual is moving towards the more streamlined, optimized lists, where non-optimized lists like mine can't compete... Quantifying data is going to further push this line on the spectrum to further optimize (for those who care to interpret it and apply it) and for the truly casual players who have no desire to consume that type of content, they will continue to left further and further from where the format is going.

Now, I could be wrong, but @Sheldon has said several times that the format is moving more and more towards this optimized deckbuilding, and I think this will only exacerbate this.

(as a side note, I think there is a large problem with 75% decks claiming to be 'casual' and steamrolling over casuals. almost to the point, where there's not a place to play some of my favorite decks, like my very tuned Lord Windgrace)

You've got a couple good points here.

First, I think one issue that I'm seeing throughout a couple of these different threads is the nomenclature between "casual" and "competitive". I could go off on a tangent about these, but ultimately I think they're largely meaningless without further conversation (such as Rule 0 promotes), and don't help make the conversations all that much clearer. Your point about your Soraya deck really illustrates that.

Secondly, your point about Sheldon not wanting the format to hastily homogenize is well taken, and I think that's why he has posted a couple times looking for very specific ideas about data collection. And it's a fair fear to have. Even with resources like EDHRec and the boatload of content out there, a lot of more streamlined ideas have moved to the forefront (I think there's a discussion to be had about whether that's inevitable and whether that's a bad thing, but I don't think this is the place, nor do I have fully formed thoughts on it). I agree to an extent that more data does potentially lead to potentially pushing people in specific directions, which is why I think it's really important to balance any quantitative data against qualitative data--effectively, numbers can point out a trend, but because EDH is a social format at it's core, the numbers should, ultimately, be less important than the overall enjoyment of people (though this too can't be a hard and fast rule--people may have really enjoyed Tolarian Academy back in the day, but it needed to go and rightfully did).

[SkyeDragonQueen]

I believe players are going to use any information, data or experiences they can in an effort to play the format the way they prefer. That's not necessarily a problem, though! The problem is when those players then "force" others around them to similarly power their deck up, or move towards a more linear/infinite gameplan. While there is effectively no way for the community to stop this, what we can do is collect and provide resources for as many players as possible to use in their own way. One of the biggest things I believe needs to be communicated to each player is that the most important part of EDH is to ENJOY it. If you want to play with cards that aren't optimal, cards that are there for flavour instead of function, commanders that don't seem particularly good, or combos that are as silly as our lord and savior SethPBKASO, that's EDH, no? There will always be players who want to push themselves, and by extention their playgroup(s) up to the most powerful level as possible, but rather than try to restrict, or undermine that, I believe it's better to encourage players to play where they enjoy it the most, and if possible, play at multiple levels if their group has that interest.

I play dragons. I have a collection of hundreds of dragon cards, be them silly Viashino, powerful Glorybringers, or really REALLY bad Sarkhans (Sorry buddy, but a lot of your cards are... well.. Shivan Dragon is definitely with you). The decks I build with that collection can barely stand up to Pre-Cons most days, but that's one of my favourite places to experience this format. When my old group started to push up, I sat down with them, and asked each player what they wanted out of the format. Of the 7 other players who regularly played in the group, two of them really wanted to push up and beyond what we were at, and experience a powerful EDH, some wanted to be near the middle of the pack, while one wanted nothing more than to play with their unchanged (minus foiling) pre-constructed Kaalia deck they'd bought years prior. Ultimately, there was no way to make every player happy every game, but collectively, we tried to make changes to help the group prosper, and not divide too much. The more spikey players built their decks up, while creating more casual decks to play with the group easily at a lower power. The middle group kept their decks, while trying to create something new that was a lot more powerful, and one even re-built a pre-con they'd previously bought, to play with the Kaalia player. Finally, that Kaalia player, who didn't have the means to build another deck, was happy to borrow a deck to play, as long as they'd still get games with Kaalia. At the end of the day, most of the players found that the alternate power levels interested them in different ways and I believe the group was stronger because of the experimenting.

That's not how real life works in most cases though. Some players don't enjoy certain levels of power, or strategies. Die-Hard Timmy players aren't easily going to give up their Craw Wurms for a Counterspell. But that's okay, because that's what makes the game great, in my opinion. There's something for everyone. EDH has a huge player base (I'd argue it might be the biggest in Magic, but of course there's no data to back that up!), and while I'd love if every player played at the same power level to allow pick up games with people you ultimately may not know to be fair, balanced and as fun as possible, that's impossible at the face of it. Instead, what I'd like this data to help provide, is a means for players to explore and experience new levels of play. Having one EDH deck for many players is all they can afford, and it's going to be difficult to have that deck operate against the multitude of others in a balanced fashion. But if this data can help even a single player to find interest in multiple levels of play, or help playgroups stay together and explore EDH as a whole, rather than feel pulled along by one or two players, then I'd consider it a win. However, I truly don't know how to do that myself. I've pushed the button to start this blender of ideas, but ultimately I do need help from all of you with ideas and strategies to make it work as well as possible.

Jeez that's a wall of text. Thanks everyone, though. Sheldon I'm sorry I'm derailing your thread a bit with all this data talk, but thanks for opening up the dialogue a lot, and asking the important questions of me. As for everyone else, seriously bite into me, and break this idea down. The better we can understand its flaws, the better we as a community can make it, right?

[Styrofoam]

I believe players are going to use any information, data or experiences they can in an effort to play the format the way they prefer. That's not necessarily a problem, though! The problem is when those players then "force" others around them to similarly power their deck up, or move towards a more linear/infinite gameplan. While there is effectively no way for the community to stop this, what we can do is collect and provide resources for as many players as possible to use in their own way. One of the biggest things I believe needs to be communicated to each player is that the most important part of EDH is to ENJOY it. If you want to play with cards that aren't optimal, cards that are there for flavour instead of function, commanders that don't seem particularly good, or combos that are as silly as our lord and savior SethPBKASO, that's EDH, no? There will always be players who want to push themselves, and by extention their playgroup(s) up to the most powerful level as possible, but rather than try to restrict, or undermine that, I believe it's better to encourage players to play where they enjoy it the most, and if possible, play at multiple levels if their group has that interest.

I play dragons. I have a collection of hundreds of dragon cards, be them silly Viashino, powerful Glorybringers, or really REALLY bad Sarkhans (Sorry buddy, but a lot of your cards are... well.. Shivan Dragon is definitely with you). The decks I build with that collection can barely stand up to Pre-Cons most days, but that's one of my favourite places to experience this format. When my old group started to push up, I sat down with them, and asked each player what they wanted out of the format. Of the 7 other players who regularly played in the group, two of them really wanted to push up and beyond what we were at, and experience a powerful EDH, some wanted to be near the middle of the pack, while one wanted nothing more than to play with their unchanged (minus foiling) pre-constructed Kaalia deck they'd bought years prior. Ultimately, there was no way to make every player happy every game, but collectively, we tried to make changes to help the group prosper, and not divide too much. The more spikey players built their decks up, while creating more casual decks to play with the group easily at a lower power. The middle group kept their decks, while trying to create something new that was a lot more powerful, and one even re-built a pre-con they'd previously bought, to play with the Kaalia player. Finally, that Kaalia player, who didn't have the means to build another deck, was happy to borrow a deck to play, as long as they'd still get games with Kaalia. At the end of the day, most of the players found that the alternate power levels interested them in different ways and I believe the group was stronger because of the experimenting.

That's not how real life works in most cases though. Some players don't enjoy certain levels of power, or strategies. Die-Hard Timmy players aren't easily going to give up their Craw Wurms for a Counterspell. But that's okay, because that's what makes the game great, in my opinion. There's something for everyone. EDH has a huge player base (I'd argue it might be the biggest in Magic, but of course there's no data to back that up!), and while I'd love if every player played at the same power level to allow pick up games with people you ultimately may not know to be fair, balanced and as fun as possible, that's impossible at the face of it. Instead, what I'd like this data to help provide, is a means for players to explore and experience new levels of play. Having one EDH deck for many players is all they can afford, and it's going to be difficult to have that deck operate against the multitude of others in a balanced fashion. But if this data can help even a single player to find interest in multiple levels of play, or help playgroups stay together and explore EDH as a whole, rather than feel pulled along by one or two players, then I'd consider it a win. However, I truly don't know how to do that myself. I've pushed the button to start this blender of ideas, but ultimately I do need help from all of you with ideas and strategies to make it work as well as possible.

Jeez that's a wall of text. Thanks everyone, though. Sheldon I'm sorry I'm derailing your thread a bit with all this data talk, but thanks for opening up the dialogue a lot, and asking the important questions of me. As for everyone else, seriously bite into me, and break this idea down. The better we can understand its flaws, the better we as a community can make it, right?

Right, I get what you're saying.

My main point is the types of people who typically will be the ones who see, consume, process, and adapt to the data provided are typically spikier players. I've noticed this trend on social media, that the more active players are the spikes. They are the ones that search out this type of data to improve. It seems like this will serve those spikes almost exclusively. Which is fine, it's just something to keep in mind. Data like this typically serves the purpose to "trim the fat" so to speak.

[pokken]

What Wizards has found out trying to hide data for modern is that 1) people still crave it, and 2) they will find a way (people have done massive amounts of self-reporting for instance modern GP results)

This started with the magazine printing winning decklists in the 90s. People want to know what actually works and what other people are doing.

I think most EDH players go through a phase where they're netdecking a lot and then start to develop their own ideas more. I think it's normal and fine.

[SkyeDragonQueen]

I definitely agree there. I hope to get this information to as many people as possible, not just the spikes, but it will be used by them no doubt. Hopefully we can find a way to convey the data in a way that benefits casual players and competitive ones alike. However I do agree with [mention]pokken[/mention] here, I think that many players move through the netdecking and into a more personalized approach, which is where perhaps the data will help lead them.

[Styrofoam]

What Wizards has found out trying to hide data for modern is that 1) people still crave it, and 2) they will find a way (people have done massive amounts of self-reporting for instance modern GP results)

This started with the magazine printing winning decklists in the 90s. People want to know what actually works and what other people are doing.

I think most EDH players go through a phase where they're netdecking a lot and then start to develop their own ideas more. I think it's normal and fine.

I don't disagree with it being normal and fine.

I'm talking about what the natural progression is, and that is homogenization and a sliding towards more optimized strategies.

[FireStorm4056]

I'd like to point out that WOTC estimates there are ~20 million MTG players worldwide. Meaning, even if you managed to get every single one of the 81,000 /r/EDH subscribers to use the form, you'd still be polling less than 0.5% of the playerbase. Considering the major sampling issues (i.e. the fact any players who use online resources / forums have already self-selected in some way), I don't see any way to control for the dozens of factors you'd need to consider to get accurate representative data for the larger population.

More importantly, I don't know how this data would provide actionable information that we couldn't already gather through conversation and open discussion.

I DO think this kind of data is very useful as a resource for individual players to research strategies and improve their own game, but it seems like a resource-intensive and error-prone approach towards moderating the format. WOTC employs an entire team whose job it is to analyze data and balance their tournament formats. Then, consider that none of the other formats are nearly as complex as EDH, that different EDH playgroups vary wildly in their expectations and goals (whereas tournament formats have one goal), and that WOTC has very reliable, high quality data to use in their analysis (tournament registrations, statistics from online play, etc). I simply don't know how you can apply this to EDH in any way that removes significant sampling bias and other statistical issues.

[pokken]

I don't disagree with it being normal and fine.

I'm talking about what the natural progression is, and that is homogenization and a sliding towards more optimized strategies.

If it's normal and fine why is it a problem? I get the concern I just think it's overblown. Everyone already knows how to make busted ass decks in commander. We've known since 5c hermit druid was born.

I think there is a possibility having more info out there causes some homogenization but I think it naturally leads to being better able to accomplish EDH's ends - which is being able to build decks at the appropriate power level to craft the experience you and your fellows want.

[pokken]

I'd like to point out that WOTC estimates there are ~20 million MTG players worldwide. Meaning, even if you managed to get every single one of the 81,000 /r/EDH subscribers to use the form, you'd still be polling less than 0.5% of the playerbase. Considering the major sampling issues (i.e. the fact any players who use online resources / forums have already self-selected in some way), I don't see any way to control for the dozens of factors you'd need to consider to get accurate representative data for the larger population.

More importantly, I don't know how this data would provide actionable information that we couldn't already gather through conversation and open discussion.

I DO think this kind of data is very useful as a resource for individual players to research strategies and improve their own game, but it seems like a resource-intensive and error-prone approach towards moderating the format. WOTC employs an entire team whose job it is to analyze data and balance their tournament formats. Then, consider that none of the other formats are nearly as complex as EDH, and that WOTC has very reliable, high quality data to use in their analysis (tournament registrations, statistics from online play, etc). I simply don't know how you can apply this to EDH in any way that removes significant sampling bias and other statistical issues.

Do you think we could get the information in EDHrec through "conversation and open discussion?"

My answer is no way. Not everybody wants to talk or is good at talking. The population that wants to talk about EDH is vastly smaller than the group that might submit a decklist to a random website.

Collecting data like this is a way to include even more people than those who currently have their voices heard - which is almost always people who are exceptional writers or content creators.

[FireStorm4056]

Do you think we could get the information in EDHrec through "conversation and open discussion?"

My answer is no way. Not everybody wants to talk or is good at talking. The population that wants to talk about EDH is vastly smaller than the group that might submit a decklist to a random website.

Collecting data like this is a way to include even more people than those who currently have their voices heard - which is almost always people who are exceptional writers or content creators.

I'm not saying data collection is inherently bad. I'm saying that data must be accurate and unbiased if you're going to use it to drive major decisions/actions that affect entire formats (namely, huge populations of people not sampled in your data collection). The key issue here is not, "Whether good data would be useful," it's, "How you even collect good data to begin with." All of the existing solutions only query a very tiny, biased, and self-selected slice of the population (namely, English-speaking players devoted enough to use online resources, log decklists, click on and fill out a form, etc.)

Data-driven decision-making CAN let you control for various factors and quantitatively eliminate biases, but it relies on having good sampling in the first place. If your sampling is bad then all of your quantitative validity goes out the window (and presumably, quantitative validity is the whole point of performing data analysis).

EDHRec and similar resources are very useful because they both sample from, and reach, their target audience. The idea is to offer information and statistics to players who go and seek them out. It can be VERY useful for improving your decklist, but not very useful for asking what ALL decklists look like in reality (and then, how players feel about all those decklists).

EDIT: You are right, more players submit decklists than post online. However, I'll mention again that decklists offer zero insight into what is actually happening during the games, how players feel, etc. Decklists offer tons of "statistics" but none of them really have anything to do with actual outcomes and experiences. For example, I could give you 100,000 elevation data points to make a topographical map, but if what you ACTUALLY care about is the temperature of the ocean, the data is meaningless. For this reason, I'd argue that online discussions such as this are still far more useful (and less work-intensive for the RC).

EDIT 2: Counterexamples - Flash is the "boogeyman" of CEDH, yet countless other blue cards are represented far more heavily (Brainstorm, Mana Drain, etc). You could look at all the decklists statistics you like and, without context, likely never conclude that Flash is a key offender. Yet, talk to basically any regular player of the format and they'll immediately give you a tirade on why Flash is unbalanced. The same is true for Paradox Engine - it was run in just 6% of all decklists. Compare this to ~30% for Lightning Greaves, Skullclamp, and others. Nothing about the data here illustrates the problems and issues with either of these cards in actual play... yet, these issues quickly come to light through just a handful of conversations with players.

[pokken]

Conversations are a very risky thing to rely on. Lots of people are very good at creating convincing cases that may be overblown. I've personally created pretty convincing arguments I look back on and find questionable now.

I'd rather have both lots of data and then make sure we have "conversations" with more than just those likely to talk at me, if possible.

Your analysis of the Paradox Engine thing doesn't go anywhere near as deep as we went with it, for the record. We broke it down and found that some commanders were as likely as 75% or so to play Paradox Engine and that it was played in a large number of non-competitive decks (the ratio for Urza wound up being about 2/3rds non-comp running PE).

So we were able to say that it is fairly likely (although not certain) that lots of casual decks were running paradox engine.

The data that they have is vastly more deep than we have access to as well since they don't expose an API to give us different ways to slice and dice it. For example we can't create heuristics for identifying if a deck is likely to be competitive and then say "Give me all the non-competitive decks that run Flash. We have to go one by one.

[Styrofoam]

I don't disagree with it being normal and fine.

I'm talking about what the natural progression is, and that is homogenization and a sliding towards more optimized strategies.

If it's normal and fine why is it a problem? I get the concern I just think it's overblown. Everyone already knows how to make busted ass decks in commander. We've known since 5c hermit druid was born.

I think there is a possibility having more info out there causes some homogenization but I think it naturally leads to being better able to accomplish EDH's ends - which is being able to build decks at the appropriate power level to craft the experience you and your fellows want.

It is only a concern if the gradual power creep of the format is what you sign up for. Considering Sheldon's most recent posts, I am not sure that is what they are shooting for.

I mean, anyone is free to use whatever resources they want when building decks. I just would imagine that the people who use the types of data that would be recorded and released would be spikes. Take that however you want, but it certainly supports that the format is shifting more towards tuned and optimized decks that favor spikes rather than timmys or vorthoses.

Just my 2c

[FireStorm4056]

Conversations are a very risky thing to rely on. Lots of people are very good at creating convincing cases that may be overblown. I've personally created pretty convincing arguments I look back on and find questionable now.

I'd rather have both lots of data and then make sure we have "conversations" with more than just those likely to talk at me, if possible.

Your analysis of the Paradox Engine thing doesn't go anywhere near as deep as we went with it, for the record. We broke it down and found that some commanders were as likely as 75% or so to play Paradox Engine and that it was played in a large number of non-competitive decks (the ratio for Urza wound up being about 2/3rds non-comp running PE).

So we were able to say that it is fairly likely (although not certain) that lots of casual decks were running paradox engine.

The data that they have is vastly more deep than we have access to as well since they don't expose an API to give us different ways to slice and dice it. For example we can't create heuristics for identifying if a deck is likely to be competitive and then say "Give me all the non-competitive decks that run Flash. We have to go one by one.

I definitely understand that conversations are not perfect - but as far as this format is concerned, the "data" is even more flawed. The biggest elephant in the room is the issue with sampling (mentioned previously). All the EDHRec data in the world still doesn't tell us what a huge percentage of players experience in their games, much less how they actually feel about the cards and games they play.

You've also effectively highlighted another problem with data - when you already know the answer you're looking for, it's very easy to retrospectively construct an "analysis" backing up any conclusion you want. There are dozens of ways to filter data and anecdotally "prove" any arbitrary argument you're looking to find. The example in this case was, "X card is played in Y% of 'casual decks' (however that's defined) with Z commander". The problem is, if you take the exact same analysis procedure and apply it to any other combination of X, Y, and Z, it doesn't work. Take Yisan decks: 92% run Llanowar Elves, 84% run Reclamation Sage, 73% run Priest of Titania, 64% run Craterhoof Behemoth. All good cards, but none needing a ban. Apply this to practically any other list and you find lots of good cards, but not "broken" cards.

So yes, the PE analysis "fits" the PE ban, but that procedure also ONLY fits the PE ban (and basically nothing else - a statistical error called "overfitting"). Practically every commander has cards with over a 50% inclusion rate, and yet very very few of them are considered broken, much less ban-worthy. If we were to take all the most played cards, assign them random names, then look at ONLY the random names, colors, and play rates, could we construct a good banlist? No. This tells us the analysis itself doesn't have any general predictive power. Rather, the analysis is artificially constructed to arrive at some answer we want it to "tell us."

EDIT: That's not to say PE shouldn't have been banned. There are good reasons to ban it... but none of them can be deduced from the EDHRec data. Consider the EDH Philosophy document's reasons for why cards could be banned:

- Cause severe resource imbalances
- Allow players to win out of nowhere
- Prevent players from contributing to the game in a meaningful way.
- Cause other players to feel they must play certain cards, even though they are also problematic.
- Are very difficult for other players to interact with, especially if doing so requires dedicated, narrow responses when deck-building.
- Interact poorly with the multiplayer nature of the format or the specific rules of Commander.
- Lead to repetitive game play.

None the data on EDHRec directly tells us about a single one of these bullet points, even though it's by far the best dataset we currently have.

[ev3rywhen]

EDIT 2: Counterexamples - Flash is the "boogeyman" of CEDH, yet countless other blue cards are represented far more heavily (Brainstorm, Mana Drain, etc). You could look at all the decklists statistics you like and, without context, likely never conclude that Flash is a key offender. Yet, talk to basically any regular player of the format and they'll immediately give you a tirade on why Flash is unbalanced. The same is true for Paradox Engine - it was run in just 6% of all decklists. Compare this to ~30% for Lightning Greaves, Skullclamp, and others. Nothing about the data here illustrates the problems and issues with either of these cards in actual play... yet, these issues quickly come to light through just a handful of conversations with players.

I'm willing to bet that this is because EDH / Commander is a singleton format, so there is a little bit of obscuration here.

In a magical world where EDH / Commander was just ... well, ED, you would probably see Flash stand out more like a sore thumb because of how critical it is, while greaves ... probably not as much. That's partially why you're seeing the discrepency above.