The Newbie NewD3 stats thread (upd. 2020-05-20, 51g)

[northsidegal]

Much thanks goes to Toomai, whose spreadsheet and formatting I have borrowed. Despite that, this is still the product of a great deal of work in collecting data and editing to accommodate a changed setup.

---

Spoiler: General/subsetup stats

Setup	Wins	Losses	Winrate	Games
A1	7	1	87.5%	8
A2	2	3	40.0%	5
A3	4	1	80.0%	5
B1	4	3	57.1%	7
B2	4	0	100.0%	4
B3	1	2	33.3%	3
C1	2	2	50.0%	4
C2	3	3	50.0%	6
C3	4	5	44.4%	9
A	13	5	72.2%	18
B	9	5	64.3%	14
C	9	10	47.4%	19
Total	31	20	60.8%	51

Spoiler: Result stats

Average player types per game: 5.9 Newbies, 3.1 SEs, 0.0 ICs

Type	Winrate	Total
Town Newbies	58.3%	211
Scum Newbies	36.9%	65
Total Newbies	53.3%	276
Town SEs	65.1%	106
Scum SEs	42.1%	38
Total SEs	59.0%	144
Town ICs	-	0
Scum ICs	-	0
Total ICs	-	0
All town	60.6%	317
All scum	38.8%	103
All players	55.2%	420

Spoiler: Replacement stats

Notes:

If a slot is replaced twice, that counts as one replaced slot and two replaced players. Therefore, the player rate cannot be any lower than the slot rate. A slot rate of 100% means that all 9 slots were replaced at least once. A player rate of 100% means that 9 players (from any slots) were replaced (so it can exceed 100%).

On average, games replace 2.6 slots with a standard deviation of 1.37, and 3.1 players with a standard deviation of 1.74.

Player	Slot replacement rate	Player replacement rate
Town Newbies	35.5%	41.2%
Scum Newbies	32.3%	36.9%
Total Newbies	34.8%	40.2%
Town SEs	17.0%	18.9%
Scum SEs	23.7%	34.2%
Total SEs	18.8%	22.9%
Town ICs	-	-
Scum ICs	-	-
Total ICs	-	-
All town	29.3%	33.8%
All scum	29.1%	35.9%
All players	29.3%	34.3%

Spoiler: Day 1 stats

• Town-scum-NL rate: 35-15-1 (random lynching would give 40-11-0; the difference is [-9.15%]-[7.19%]-[1.96%])
  
• If town is lynched D1, they go 16-19 (45.7%)
  
• If scum is lynched D1, they go 1-14 (6.7%)
  
• If there is no lynch D1, town goes 1-0 (100.0%)

Spoiler: Lynch accuracy stats

In the following table, the first three columns are the raw percentages, while the last two are the difference between that and random (positive = more, negative = less). So when town does better than random, it has a negative in lynching town and a positive in lynching scum (random never no-lynches, so the difference is the data).

Ratio	Town	Scum	No lynch		Town	Scum
7:2 (26 samples)	73.1%	23.1%	3.8%		-4.7%	0.9%
7:1 (2 samples)	50.0%	50.0%	0.0%		-37.5%	37.5%
6:2 (5 samples)	40.0%	60.0%	0.0%		-35.0%	35.0%
6:1 (6 samples)	33.3%	66.7%	0.0%		-52.4%	52.4%
5:2 (17 samples)	35.3%	64.7%	0.0%		-36.1%	36.1%
5:1 (5 samples)	20.0%	80.0%	0.0%		-63.3%	63.3%
4:2 (3 samples)	33.3%	66.7%	0.0%		-33.3%	33.3%
4:1 (13 samples)	61.5%	38.5%	0.0%		-18.5%	18.5%
3:2 (4 samples)	50.0%	50.0%	0.0%		-10.0%	10.0%
3:1 (0 samples)	N/A	N/A	N/A		N/A	N/A
2:1 (6 samples)	50.0%	50.0%	0.0%		-16.7%	16.7%

Spoiler: Game length stats

• No games have ended with Day 1 lynch (only possible via modkills)
  
• No games have ended with Night 1 kill (only possible via modkills)
  
• Games that end with Day 2 lynch take 15.1 days with standard deviation of 4.69 (11 games)
  
• No games have ended with Night 2 kill (only possible via modkills)
  
• Games that end with Day 3 lynch take 21.1 days with standard deviation of 5.86 (15 games)
  
• No games have ended with Night 3 kill
  
• Games that end with Day 4 lynch take 26.4 days with standard deviation of 6.23 (23 games)
  
• No games have ended with Night 4 kill
  
• One game has ended with Day 5 lynch; it took 35.4 days
  
• No games have ended with Night 5 kill
  
• No games have ended with Day 6 lynch
  
• One game has ended with Night 6 kill; it took 27.9 days
  
• No games have ended with Day 7 lynch
  
• Overall, games on average take 22.6 days with standard deviation of 7.44; the majority of games end on Day 4 (45.10%), with Day 3 and Day 2 being nearly tied for second-most common (29.41% and 21.57% respectively)

Spoiler: Replacement vs. length stats

• When 0 slots are replaced all game, games take 21.5 days with standard deviation of 4.66 (3 games)
  
• When 1 slots are replaced all game, games take 24.4 days with standard deviation of 6.48 (8 games)
  
• When 2 slots are replaced all game, games take 24.0 days with standard deviation of 7.74 (10 games)
  
• When 3 slots are replaced all game, games take 19.4 days with standard deviation of 6.99 (21 games)
  
• When 4 slots are replaced all game, games take 25.7 days with standard deviation of 3.31 (4 games)
  
• When 5 slots are replaced all game, games take 29.3 days with standard deviation of 7.42 (3 games)
  
• When 6 slots are replaced all game, games take 27.6 days with standard deviation of 7.03 (2 games)
  
• No games have had 7 slots replaced
  
• No games have had 8 slots replaced
  
• No games have had 9 slots replaced

• When 0 players are replaced all game, games take 21.5 days with standard deviation of 4.66 (3 games)
  
• When 1 players are replaced all game, games take 23.2 days with standard deviation of 6.02 (7 games)
  
• When 2 players are replaced all game, games take 24.2 days with standard deviation of 8.60 (8 games)
  
• When 3 players are replaced all game, games take 20.1 days with standard deviation of 7.24 (17 games)
  
• When 4 players are replaced all game, games take 22.6 days with standard deviation of 6.48 (6 games)
  
• When 5 players are replaced all game, games take 28.1 days with standard deviation of 8.49 (4 games)
  
• When 6 players are replaced all game, games take 23.4 days with standard deviation of 1.94 (5 games)
  
• No games have had 7 players replaced
  
• One game replaced 8 players; it took 34.7 days
  
• No games have had 9 players replaced

Spoiler: Replacement vs. winrate stats

For slots:

• When 0 slots are replaced all game, town goes 2-1 (66.7%)
  
• When 1 slots are replaced all game, town goes 4-4 (50.0%)
  
• When 2 slots are replaced all game, town goes 4-6 (40.0%)
  
• When 3 slots are replaced all game, town goes 16-5 (76.2%)
  
• When 4 slots are replaced all game, town goes 3-1 (75.0%)
  
• When 5 slots are replaced all game, town goes 0-3 (0.0%)
  
• When 6 slots are replaced all game, town goes 2-0 (100.0%)
  
• No games have had 7 slots replaced
  
• No games have had 8 slots replaced
  
• No games have had 9 slots replaced

For players:

• When 0 players are replaced all game, town goes 2-1 (66.7%)
  
• When 1 players are replaced all game, town goes 3-4 (42.9%)
  
• When 2 players are replaced all game, town goes 4-4 (50.0%)
  
• When 3 players are replaced all game, town goes 13-4 (76.5%)
  
• When 4 players are replaced all game, town goes 3-3 (50.0%)
  
• When 5 players are replaced all game, town goes 2-2 (50.0%)
  
• When 6 players are replaced all game, town goes 3-2 (60.0%)
  
• No games have had 7 players replaced
  
• When 8 players are replaced all game, town goes 1-0 (100.0%)
  
• No games have had 9 players replaced

Spoiler: Replacement vs. role stats

Vanilla Townie	264	15.2%	29.2%
Cop	12	16.7%	16.7%
Doctor	11	27.3%	27.3%
Jailkeeper	14	35.7%	35.7%
Tracker	16	37.5%	37.5%
Mason	28	21.4%	21.4%
Friendly Neighbor	11	54.5%	54.5%
Mafia Goon	71	26.8%	28.2%
Roleblocker	18	27.8%	27.8%
Rolecop	14	35.7%	35.7%

The following is a z-score graph. 0 means "is exactly average". 1 means "is 1 standard deviation above average"; -1 means "is 1 standard deviation below average". In other words, high bars mean "more replaces", while low mean "less replaces".

Notes

• For any stat that counts wins and losses, draws are ignored.
  
• The replacement stats recorded here are likely lower than in reality, as replacements are not recorded if the player never confirmed. If a player posts in any way (or has been acknowledged by the mod as having confirmed/read their role PM), replacing them counts.
  
• Slot-based stats might be a bit off due to cases where slots "level up" via replacement (e.g. an experienced player replaces into a newbie slot).
  
• Players are assumed to have won regardless of modkills (unnecessary complication). They are however still dead. So far there have been no complications involving multiple scum/PRs dying in the same phase.
  
• If the last scum concedes, it's recorded as a lynch on them. If both scum are alive when they concede, it's recorded as no lynch; the game simply ends in a town win with both scum alive. (This is how a mafia PR might still be alive at endgame in a town win.)

[/list]

[northsidegal]

Keep in mind that with 51 games played, many subsetup specific stats still have a very small sample size.

Overall, NewD3 has a roughly similar if closer to balanced winrate (60%) compared to 2d3's 65% town winrate. That being said, so far there is somewhat of a notable difference in winrates between the columns, with A, B and C going from most to least townsided.

A few curiosities:

• Setup A1 (Cop + Doc vs RB) is both the most played setup and has one of the highest town winrates. This stands in contrast to that same setup's winrate under the Matrix6 setup. There are many factors that could explain this, with the two that most immediately come to mind being the faster deadlines of modern newbie games as well as claiming considerations given the semi-open nature of setups.
  
• Over 50% of friendly neighbors have replaced out. Granted, it is tied for being the least-common role in newbie games, but the Doctor which it is tied with has a below-average rate of replacing out.
  
• There appears to be something going on with 3 slots/players replacing out?? Those numbers have the highest sample size as well as an unusually high town winrate. I'm not sure what to think of this. It's
  possible
  that it's a mistake in the spreadsheet, but I don't think that it is.

[northsidegal]

Rolling subsetup winrates:

[northsidegal]

This has been the case since Matrix6 (and possibly before, but I don't have the data on-hand for that). Personally, I might theorize that it an element across almost all micro setups.

[northsidegal]

There's an argument to be made that roles that aren't completely common across anywhere that plays Mafia reduce newbie retention. Neapolitan in 2d3 and so far Friendly Neighbor in NewD3 both have positive replacement z-scores in contrast to literally any other town roles. This is part of why I think that closed setups for newbie are probably not the best idea. You might say that the closed setups could stick with only the more basic roles, but I would say to that that the NewD3 setup basically

already is

a setup composed of subsetups with common roles which we consider to be roughly balanced. Basically, if we're going to have closed setups, those closed setups should really stick to the more basic roles to help newbie retention. And if we're only going to have closed setups with basic roles, it seems to me like a good idea to just come up with a list of setups like that and say "your newbie game is going to be one of these games". (Also, personally, I think that Beloved Princess is a terrible role. Maybe that's part of your point and I'm misunderstanding what you're saying)

The goal isn't necessarily a 50/50 split on winrate. I believe the previous consensus in threads discussing the newbie setup was that the primary goal is newbie retention, and sometimes I think that can mean keeping changes that might unbalance the winrate. For instance, I believe it was determined through an experiment a while back that daytalk in newbie games improves newbie retention (although I'm not sure how this was measured) despite increasing scum winrates significantly. Similarly—although I don't have evidence for this—I might imagine that the change to 10:7 deadlines has improved newbie retention despite increasing town winrates significantly, at least by Toomai's data for 2d3.

I think it's really concerning that a newbie who's a natural at mafia could roll scum alongside a newbie who was probably doomed to quit forum mafia no matter what setup was tried, and the latter can get lynched day 1 and the former could get turned off from forum mafia either because of general overall high power level that's unnecessary for assisting 3 SEs in catching a firsttime player or because of power roles that trigger bonuses off partner loss like in A2. Maybe that's a bad utilitarian solution, maybe it's better to have 7 newbies win and try to keep them that way, by volume, in which case we should go the other extreme and engineer town stomps, and weaker setups ought to go. I'd like to imagine the more hardfought wins retain more players than stomps, though.

I'm not really sure the best way to respond to this. I'm not sure I agree with anything you're saying here.

The situation you describe is possible, but it's also possible that the two best scum players on site could roll scum together in a newbie game and stomp the entire town. Should we change the setup so that that isn't possible? I think that fundamentally, setups are designed for the general rather than the edge case and so using a worst case scenario as a point against a setup is not really the best way to think about things. Certainly they deserve consideration as to how

likely

they are, but I think just the possibility isn't necessarily a point against something.

I'm not sure how serious you were with the last part, but I also think that it's the wrong way to think about the newbie setup. Even if it were proven to me that we got more newbie retention through town wins than scum wins, I would still say that engineering town stomps would be a bad idea, because I think a secondary goal of newbie games in addition to retention is to be an introduction to site meta / culture. With that in mind, it seems like a bad idea to give newbies an idea of setup balance that doesn't really align with the rest of the site.

[northsidegal]

And, as much as I may be making conclusive statements about the winrates in 2d3 and NewD3, there is still the fact that we just don't have all that much data for either of these setups. So our conclusions should be somewhat low-confidence.

For example: the setup Goons vs Doc + Tracker had a 44% town winrate in Matrix6 across 76 games. That same setup had a 70% winrate in 2d3 across 10 games. This would be an example of a really extremely swingy setup based on the death of a single Goon. Did something about site meta change from Matrix6 to 2d3 that caused town winrates to go up? Did town winrates actually even go up, or is there not enough data – given time, would it have trended towards 50%?

A less empirical and more philosophical question – even if the winrate would have been 50%, would the setup still bear changing, because of the swing? One might consider me stupid for even asking this question, but is it necessarily even a problem that scum tends to lose more when one of their members is lynched day 1?

[northsidegal]

but it's also possible that the two best scum players on site could roll scum together in a newbie game and stomp the entire town.

If any of the losing townies played like elli, even after a very dreadful townie got d1 mislynched they would have won. If the solo scum who lost their roleblocker partner day1 is playing against an unleashed follow-the-cop, it doesn't matter if they're Flavor Leaf or LLD, because mechanics mechanic. That might be me underrating the extent to which it's possible for good scumplayers to prevent a partner who makes poor decisions from getting lynched D1 in the first place; I think it has its limits, it's only an opinion. I'm not in the newbie queue a ton but the last time I was there it sure seemed hard.

I don't think your view is consistent. As far as I can tell, you seem to be saying "The losing townies in a game with the best scumplayers could have won if they played well, but that isn't the case for the losing scum in a game with an unfortunate PR situation." But that view seems to ignore that PR hunting is a skill in itself. Being Flavor Leaf or LLD or whoever the best scumplayer is

would

make a difference, because PR hunting is a trait of good scum players, and even if you're in a follow-the-cop N1 without your roleblocker you can make it to LyLo without a single confirmed innocent by making the right shots. Mathdino killed a power role literally every night he was ever alive as scum. Is that likely? Maybe not. Is someone playing like Elli in a game with the best scumplayers likely? Also probably not. My point is that you seem to ignore skill as a mitigating factor for scum, but not for town.


And yeah, I'm certain some people might be rolling their eyes at me specifically being the one to be "lecturing" on this, but I don't think it's too unreasonable to say that preventing your partner from getting lynched is a skill. As with all team based games there is only so much you can do to directly influence the outcome of a game, but all the same I would say that preventing scum lynches and pushing mislynches is something scum should be doing.

For example: the setup Goons vs Doc + Tracker had a 44% town winrate in Matrix6 across 76 games. That same setup had a 70% winrate in 2d3 across 10 games.

Cop-Doc-Roleblocker had over 70% winrate last time around. Mystifyingly it appears in this setup again, with an 88% winrate. Why does that one belong anywhere in the newbie matrix? The possibility of a day 1 roleblocker lynch is so, so absurd in that setup. I think even ignoring both stats people would generally agree RB-doc-cop is on the greensided end of an acceptable micro. I think it would be great if the newbie setup reflected the center or redsided end of acceptable micros given that it's likely newbie queue towns are strong.

It had a 77% winrate over 9 games. That same setup in Matrix6 had a 54% town winrate over 83 games. Do you see the point that I was trying to make in that post?

[northsidegal]

i agree that matrix6 was a lot easier to understand than any of the d3 setups. that being said, as long as we would be using the setups that we're currently using, the "matrix" of newD3 i think serves as a way to make things simpler – for instance, if the setups were just presented as a list, it might be harder to immediately understand that, say, the death of a rolecop means that you're in column B.

of course, that probably doesn't mean anything to someone who just wanted less subsetups in general, but still. back when a new newbie setup was being theorycrafted by a lot of people, people kept trying to come up with these really "cute" setups like Matrix6 that had some kind of graphical element, but i personally (along with some others) felt that this was unnecessary – just randomizing from a set of setups known to be good is enough. like i said though, as long as we do have the setups we do, the graphic element makes it easier to understand, i feel.