Town won 46% of Matrix 6 games

[MattP]

So unless I fucked my math all to hell (possible) that's statistically significant on most levels we give a shit about.

You didn't fuck up your math

[Plessiez]

That gives me p=0.01374, which I'd call significant any day of the week.

Pretty small effect size though.

It would be interesting (although it's probably not practical) to try to filter out all the newbies who played one game, replaced out early on and never signed up again. I'd hope that would account for a large part of the replacement rate, anyway.

[GreyICE]

That gives me p=0.01374, which I'd call significant any day of the week.

Pretty small effect size though.

It would be interesting (although it's probably not practical) to try to filter out all the newbies who played one game, replaced out early on and never signed up again. I'd hope that would account for a large part of the replacement rate, anyway.

Minimum effect size. However raw stats is 50% and we do have data it's significant.

[Toomai]

Okay I'd forgotten about this for a week, but here are the most current results, including data for all 89 Matrix6 Newbie games currently complete (1370-1454, 1456, 1457, 1466).

Spoiler: SIX SICK GRAPHS, YO

Notes:

• If a player didn't confirm, that replacement does not count (unless the mod acknowledges they read their role PM). Other replacements of that slot do count (unless they don't confirm either, etc).
• Force-replaces caused by another player or mod error do not count. Force-replaces caused by the target player do count.
• If a game had to be restarted due to mod error or something, the failed attempt is ignored.
• Players are assumed to have won regardless of modkills (unnecessary complication).
• The entire spreadsheet is located here. Redownload for the newest version.

My analysis:

• While I won't call it yet because we still don't have any setups that've been ran 30+ times, it's becoming increasingly clear that the overall 48%-52% winrate is not really representative of any of the individual setups, but more like a weird coincidence as to how the setups cancel each other out. Effectively:
  • 1 and 2 are heavily townsided.
  • 3 and C are scumsided, though a bit less.
  • A and B are even/very slightly scumsided.
• The difference between newbtown replacements and newbscum replacements is about 15%; smaller than the 20% of previous data, but still significant.
• Scumteams of two newbies are the most effective, surprisingly enough.
• Whichever faction gets lynched Day 1 has likely lost the game. This shouldn't come as too big a surprise, though lynching town D1 resulting in a 33% town winrate is less than I expected.

[saulres]

Scumteams of two newbies are the most effective, surprisingly enough.

I speculate that's due to meta not being available on newbies.

[Mr. Flay]

Yeah, newbies get both avoidable and reasonable passes for anti-town play.

[mastin2]

Relatedly,
-Town players massively underestimate the skill of most newb players, especially newbscum. (For instance, when most people think 'newb', they think "brand new to the game", when a fair share of the time, it's simply "brand new to mafiascum"; a newb with extensive experience elsewhere is fully capable of manipulating the most experienced of players. Heck, even a person who IS brand-new to the game doesn't necessarily mean bad player. Some players have a natural knack for the game--and if their naturally-talented, may still pull off a convincing ruse thanks to that.)
-Experienced players hold MASSIVE paranoia of other experienced players and almost none about newbs.

Between the two, it really isn't a surprise.

[fferyllt]

^^ That was a huge contributor to the scum win in my first MS game. I was perfectly happy to be underestimated. I didn't hide that I had played forum mafia before, but other players just assumed it was minimal experience and let me get away with playing an apparently naive game.

[Alduskkel]

Setup 1 being biased against the Mafia makes little sense to me; Jailkeepers are powerful but not that powerful.
Setup 2's stats are definitely screwed up. This thread shows that the F11 setup with Cop/Doc/M. RB was about 55-45 M-T
Setup 3 is obviously scumsided, but it's difficult to say if the statistics are portraying it as more scumsided than it actually is.
Setup A seems fine.
Setup B seems fine although those F11 statistics I linked gave a 65-35 breakdown instead, oddly enough.
Setup C is probably not as scumsided as it appears.

So, sadly, there's not much that can be concluded about balance I think.

[Alduskkel]

Actually, the two setups showing 70-30 M-T win percentages are definitely not as scumsided as that because the all-VT version of F11 was about 70-30. Since the two setups both have two PRs they clearly should trend better for the town than 70-30.

[callforjudgement]

Jailkeepers are very powerful in a Micro; less so in larger setups.

Also, the amount of information that players have about a setup can have a noticeable influence on win rates even if the setup itself is the same. (For instance, if a bastard mod lied and told scum that there was a cop and doctor in a mountainous setup, town would have a much better chance of winning.)

[Quilford]

I think we should X-shot the JK and possibly also the Cop.

[chamber]

How many cycles does the average cop or JK live? Its a reasonable idea to normalize their power, but may not be the actual core issue influencing win rates.

[Quilford]

Yeah, it'd be great to know the average PR lifetime in scum and town wins in each setup.

[Toomai]

I have the data necessary to figure that out, I just have to come up with a way to process it.

[BROseidon]

Jailkeepers are very powerful in a Micro; less so in larger setups.

This. If you day 1 lynch scum in a micro, a jail keeper becomes a doc+RB+cop.

[BROseidon]

Relatedly,
-Town players massively underestimate the skill of most newb players, especially newbscum. (For instance, when most people think 'newb', they think "brand new to the game", when a fair share of the time, it's simply "brand new to mafiascum"; a newb with extensive experience elsewhere is fully capable of manipulating the most experienced of players. Heck, even a person who IS brand-new to the game doesn't necessarily mean bad player. Some players have a natural knack for the game--and if their naturally-talented, may still pull off a convincing ruse thanks to that.)
-Experienced players hold MASSIVE paranoia of other experienced players and almost none about newbs.

Between the two, it really isn't a surprise.

This.

I don't think people knew I played elsewhere before here until penguin and I were in games together (not to mention the metric boatload of IRL mafia experience I had growing up). Getting underestimated in Xenoblade and NY165 were major factors in those wins.

[Toomai]

Well I don't know how to graph the data yet (any ideas?), but here's how it looks. The left tables show a count of how many times that role got killed in that phase, while the right ones show how many times that role was alive during that phase.



Observations:

• In setup 1, there doesn't appear to be much correlation between "JK life expectancy" and "town winrate".
• In setup 2, the only times scum won was when the Cop got lynched Day 1. The sample size is only 8 but that's a pretty stark division.
• By contrast, the Cop dies Night 1 pretty often in setup B.

[Quilford]

Yeah, I think this points towards nerfing the Cop some.