This Week at Bungie - 7/28/22

Raid the Date 

Making Matches 

Let's Talk About Skill and Connection 

We know this has been discussed with a lot of passion and goodwill in many places in the community (and inside Bungie), so we are going to give you a clear tl;dr before we get deeper into the how and why: 

• We’re striving towards a goal that all players—including New Lights!—can enter the Crucible and regularly get matches where they can feel competitive and have a reasonable chance of winning/competing. 
• Making fair matches using Skill-Based Matchmaking (SBMM) is going to be important to help meet that goal. 
• We are starting by implementing loose SBMM to the Control playlist at the start of Season 18. 
• Loose SBMM has a wider starting skill-similarity than Survival and should result in matching with a wider variety of players, while also eliminating some of the frustrations we see in our current system. 
• Expect loose SBMM to expand to other playlists in future Seasons as we tune what we consider a "high-quality match" by gathering real data and feedback from you. 
• We are not planning to add it to every matchmade Crucible playlist. 
• We will continue tuning until we are in a good place. 
• We will report tuning updates regularly. 
• We will be implementing a form of fireteam-size-preferred matchmaking in Season 19. 

A lot of what follows is pretty in-depth, feel free to skip to the Tuning section below if you aren’t interested in the details and just want a high-level view of what you will be experiencing! 

Goals of Creating High-Quality Matches

We developed some goals which we will be working on over the next few Seasons: 

• All players (including New Lights!) can enter the Crucible and regularly get matches where they can feel competitive. 
• All players, whether solo or with a fireteam, can find a place in the Crucible where they can play a variety of matches and have a reasonable chance of winning/competing. 
• We are defining reasonable as "expected win rate between 40 and 60% for most matches.” 
• Players are rewarded based on their skill, and proud of their skill. 
• Reserve a place for players who do not want to engage in the skill system. 

Generally speaking, any matchmaking in a competitive multiplayer game tries to put together high-quality matches. We consider three things when assembling a high-quality match: 

• Connection quality: 
• There are two types of connections that are important:  
• Connection to the game server. 
• Connection to all other players in the match. 
• Generally, bad connections to other players have a larger effect in Crucible than connection to the game servers, so when we talk about connection quality in Crucible, we are talking about that—connection from player-to-player.  
• Lower quality matches result in jerky movement by other players, missed shots, or getting unexpectedly damaged or killed. 
• When fireteams are spread across the globe, we pick a single player's latency to speed up finding matches. 
• Match fairness: 
• Ideally, all players in a match have a reasonable chance to win that match (i.e., have similar skill). 
• Matchmaking speed: 
• We always consider matchmaking speed as a key element—no one wants to wait 10 minutes between matches, no matter how perfect they end up being. 

When matchmaking, we must balance these three elements. If we want to lower matchmaking speed, we are either going to need matches that are less fair or matches with a lower connection quality. 

We’ll continue tuning to find the best balance possible. 

Skill 

Throughout this TWAB we will be using the term “skill.” In Destiny, that term refers to how we rate all players who participate in PvP on a scale of 2000. Player skill is reflected in a graph that looks like this: 

Internally, skill is a combination of stats made up of your performance (kills, deaths, captures, round wins, revives, dunks, etc.) that ranks you against all the other players in a match. Each player’s skill is compared against the skill of the other players in a match, and we make skill adjustments for all players at the end of a match where the two rankings differ. There is also a confidence rating—the more games the system has seen you in recently, the more confident the skill adjustment is. 

In addition to the stats mentioned above, skill encompasses all sorts of things: your reaction times/agility, how you approach fights, how well you know the map/mode, how well you know your character, how you build your character, the weapons, armor, and mods you use, and how you blend all of those into performance with other players. 

You won’t ever actually see a skill value in-game, and while we are currently only using it to try to get fairer matches in Survival and Elimination, we still track it for all modes (including Gambit!). This gives us a reasonable starting spot in new game types like Rift or Zone Control. 

Now, how do these skill numbers actually play out in-game? Here’s a good shorthand we use internally: 

• If someone is 200 skill above you, you can definitely tell they are better than you, and they will win ~75% of engagements against you. The opposite is true if someone is 200 or more below you. 
• By the time you get to a difference of 400, the better players are going to win ~90% of engagements and lower-skilled players need to get extremely lucky to pull off a win. 
• Once you get to a difference of 600 there is basically zero chance for the lower-skilled player to ever win a 1v1 conflict. 

Engagements should get fairer the closer you get to the same skill. This is our goal. 

The Problem Space 

As we started looking at the competitive landscape in Destiny, we noted a few things: 

Outside of Survival and Elimination, the ability to influence whether your team wins or loses is usually out of your personal control if you are average skill or below (half the population!). This can feel bad, as the match outcome feels essentially random, and you don't feel motivated to try to win. This has contributed to us de-emphasizing winning as a requirement to gain rewards in the Crucible.  

The current landscape also allows brand new players to match up with some of the highest skill veterans and are expected to compete. On the flip side, if you are highly skilled, you are often put on a team where it feels like you are carrying them and must constantly perform if you want to stand a chance at winning. This doesn't feel good for anyone. 

In Control, the skill disparities on a team can be stark—over 50% of matches have a skill disparity of 900 or more between best and worst player, which is so significant that the outcome is already known before a single shot is fired. On the other hand, in Freelance Survival, 60% of matches have a 250-skill difference or less. This is much more reasonable.  

These wide variances in skill also lead to more mercy games than you would expect. For example, Control: 

Wide disparities in skill also exacerbate other problematic elements: 

• With wide disparities in skill, trapping a single team in a spawn is significantly easier. 
• With wide disparities in skill, it's more likely that most of one team is dead at the same time, freeing the other side up to roam around and look for new targets without having to worry about danger. 

Because of these extreme factors, no matter what your skill, it becomes hard to tell if you are improving or not. "Was that a great play, or are they just a worse player than me?" You may, quite reasonably, look for other stats to demonstrate how good you are—kills, assists, and deaths (KDA) are great, but it's still unclear how good your opponents are. Given that matchmaking is dependent on lots of factors, a KDA in a low population situation can mean a very different thing (skill-wise) than a KDA in a normal or high population scenario.  

You can also check third party sites who track Elo (a ranking system originally designed for Chess with broad impact) Elo isn't something we track, use, or validate, so it's a "use at your own risk" data point.  

If you can't tell if you are improving, it's hard to be motivated to try to improve! 

We know we have to do something to solve these problems (and more) to get Crucible into a better place. We know we won’t be able to address everything in one fell swoop in Season 18, but this will be the start of an ongoing process to improve PvP over time. 

Match Balancing 

Once we match a group of players into a lobby, assuming we don't have any full fireteams, we try to split them up into balanced teams. If the player skills are somewhat random, the system has a tough time—we've tried several different algorithms here, to mixed results. For the time being, we are hoping that reducing the skill variability in any given lobby will make this easier. 

SBMM in Control 

At the start of Season 18, we are going to start turning match fairness back up in Control (and only Control) in the Crucible. We want to start slowly to limit the number of playlists we consider when tuning matchmaking with hundreds of thousands of people. We can do some testing, but nothing can fully simulate how the full population will be affected by these changes before we ship. We are going to be live-tuning the matchmaking parameters over the first few weeks until we land on something that provides a better balance between fairness, quality, and speed.  

We will not be touching any other Crucible playlists during Season 18. Trials has no planned changes to its matchmaking, Elimination and Glory will still use the same SBMM they have been using, and everything else will still use the connection-based matchmaking they have been using for years. We are currently planning to make further adjustments in Season 19 (based around the goals listed above), but rest assured, any major changes will be communicated in either a TWAB or a blog post, as well as patch notes. 

Connection-Based Matchmaking 

Connection-based matchmaking (CBMM) is what most of the Crucible playlists utilizes to find matches that have the best possible connection quality. 

• First, we identify a pool of available players with a good connection to you.
• Within that pool, we choose players with the very best connections. 
• If we can’t find players within that pool, we widen the variance in connection.
• We repeat until we find enough players, then we break them out into equally skilled teams. 

A key point about matchmaking in a fireteam: 

• The latency we measure to find a good match does not take into account a fireteam with disparate connection speeds. We only measure latency for one of the players in a fireteam. So, if you are in Tokyo, and you are in a fireteam with someone in New York and someone else in Johannesburg, you are in for some LAGGY Crucible matches no matter what lobby you join! 

Skill-Based Matchmaking 

Better known as SBMM, skill-based matching uses a similar model to connection-based matchmaking. In addition to latency, SBMM uses skill similarity when asking to join a lobby. Like latency, the acceptable skill similarities expand over time: 

• First, we identify a pool of available players with a good connection to you. 
• Within that pool, we choose players closely matched to your personal skill rating. 
• If we can’t find players within that pool, we widen the variance in skill. 
• If THAT doesn’t work, we expand the search again with more variance in connection quality. 
• Once enough players are selected, we break them out into equally skilled teams. 

Our current Glory matchmaking settings prioritizes connection quality and matchmaking speed  while still trying to find a fair match. The goal statement for our standard SBMM is: “We would rather sacrifice some match fairness in order to maintain connection quality and matchmaking speed.” 

Loose SBMM 

Our initial version of loose SBMM for the Control playlist is going to work a little differently. It starts with wider acceptable skill variance, and then expands very slowly on both acceptable skill and connection quality at the same time. 

The goal statement for this new loose SBMM is, "Start with a broad definition of match fairness and compromise on matchmaking speed in order to keep match fairness and connection quality high.” We expect overall matchmaking times to go up—moreso if you and your fireteam are on the eitherend of the skill curve—depending on the current population in your region. However, we are hoping the tradeoff for matches that aren't super sweaty or lopsided blowouts will be worth it. 

We have analytics set up to review overall matchmaking data each hour (especially critical over the opening few weeks of the Season) and will be monitoring and adjusting timings and thresholds above while we try to home in on good settings. Control is generally a nice high-population playlist, so it will be a good testbed for tuning like this. 

What are we going to be looking at as we tune? 

Amongst other things, we’re looking at:

• Matchmaking time
• Minimize players who cannot find a match in 10 minutes with a goal to keep the average under 2 minutes, and under 4 minutes for 95% of players.
• Mercy games
• Right now, mercy rates vary based on the map (as low as 5%, and as high as 25%).
• We believe the number of mercy games should be under 5% on all maps but not actually hit zero.
• Final score differential
• In general, games should be closer in score.
• Right now, 65% of matches end with one team hitting the score target, (15% going to time, and 15% ending with a mercy). Our goal is that >80% of matches end with one team reaching the score target, and most of the rest ending with a time limit. We are looking for most matches to have under 10-point difference between the two teams.
• Less variance between the top player and bottom player
• Right now, 5-10% of matches have the best player scoring 30-39 more kills than the worst player in the match, and 50% of matches have the best player getting 20-29 more kills. 
• We believe that 90% matches should have less than 20-kill difference between the best and worst players, and 50% should have no more than a 10-kill difference.

All of this is great, but there are some things it does not address that we will be looking toward in future Seasons: 

Skill Distribution 

As we discussed in the Skill section, player skill in Destiny (and most games) tends to follow a bell curve, centered around skill 100. That means half of the players are clustered between -100 and 300 skill, and just 1% above 800 or lower than –550. 

When you do skill-based matchmaking with skill windows, what ends up happening is players at either end of the bell curve have fewer potential players to match against, and thus potentially take longer to find a match with a good connection. This is one of the reasons we will start with a wider skill threshold and expand more slowly (to make sure we go through all available players). Like we’ve said, we expect this to cause longer matchmaking times initially, but it’s important to note that we are going to be looking at outlier skill thresholds and tuning the experience for them.  

In a future Season, we are hoping to introduce some technology that allows us to search with a wider skill variance based on your position in the skill curve and keep matchmaking times more consistent (with the downside of loosening some match fairness). 

Fireteam Size Mismatches 

It's no mystery that full fireteams often stomp six solo players who matched against them. Fireteams that are used to playing together may also be in voice chat with each other, allowing them to communicate more effectively than those who are not. Oddly enough, if we look at the average skill for solo players, it fits the bell curve from above clearly. If we look at the average skill for full fireteams in Control, we can see the bell curve centers around 400-500. So, not only do the fireteams have a communication advantage, but they also have a decisive skill advantage. The big question is, "Are high-skill players more inclined to play in fireteams? Or do regular fireteams make your skill go up?" 

Either way, we will be implementing a form of fireteam-size preferred matching in Season 19. We will be sharing details about how it works closer to release. Our goal is for it to be utilized like skill—sometimes as a strict requirement, sometimes as a loose one, (or sometimes not used at all!). Further, it will allow us to eventually replicate the benefits of playing in a Freelance playlist without having to split the population.  

That is our current plan going forward. We will keep you updated as we tune settings in Season 18 and beyond. 

Votes Are In 

Primed and Ready 

Player Support Report 

 

SEASHORE PACK UPDATE

EVENT CARD EXPIRATION 

KNOWN ISSUES 

Invasions and Sweeper bots 

 

It “Wimdy” But We Knittin’ Through It