Exam Professional Machine Learning Engineer topic 1 question 114 discussion

The game is more enjoyable - the better and "business metrics" points me to user engagement as best metric

This question doesn't specify how "additional revenue" is measured. Most businesses I've worked for would love "D" for all our models instead of anything else. That being said, C is the only measurable business metric there.

focusing on user engagement through the number of battles played daily provides a clearer indication of whether the model successfully creates balanced and enjoyable matches, which is the core objective. If players find battles more engaging due to fairer competition, they're more likely to keep playing. This can then translate to long-term benefits like increased retention and potential monetization opportunities.

Tempted by B but "user engagement" is the keyword.

Looking for "business metrics to track," I think C could be the most important metric. Although, option B is also a good choice.

C: Business metric i.e. outcome driven

"Business metrics" does suggest that the question is looking for metrics that are relevant to the business goals of the company, rather than purely technical metrics. In that case, C.could be a good choice. User engagement is an important metric for any online service, as it reflects how much users are enjoying and using the product. In the context of a multiplayer game, the number of battles played daily per user can indicate how well the model is doing in creating balanced teams that are enjoyable to play against. If the model is successful in creating balanced teams, then users are likely to play more games, which would increase user engagement. Therefore, C could be a suitable choice to track the performance of the model.

The focus is to obtain a model that assigns players to teams with players with similar level of skill (or average team 1 skill == average team 2 skill) A: A fast queue assignment may not focus on pearing players with the same levels of skills. A random assignment would work. B: This would be an option but is more difficult to measure than C, we don’t know If we have a measure of skill level. Also, for new players this metric would not be available at the beginning. I think “There are many new players every day.” is a key point important to discard answer B. C: Players play more games daily ← players enjoy the game more frequently and the other way round should also apply. Easy to measure also for new players. D:This focus on costs and revenue not on players matchmaking. I would go with C.

C because "user engagement" is a business metric https://support.google.com/analytics/answer/11109416?hl=en

Went with C

This is B, as it directly relates to our model's ability to predict player ability. There are many factors beyond our model which will impact user engagement (e.g. whether the game is actually enjoyable) so it's not a good measurement of the model performance

Answer C

The question is asking about "available players". Therefore, the business metric is the user engagement.

Asks for >business metric<, and problem states "user research indicates that the game is more enjoyable when battles have players with similar skill levels.", which means more battles per user if your model is performing well.

It's C. The question specifically asks for a business metric. Precision and recall are not business metrics, but user engagement is

The template uses the 'ability' to create teams. For this, we can conclude that the system measures the player's skill. So, nothing better than comparing the predict ability with the actual ability to understand the performance of the model.

A. Average time players wait before being assigned to a team B. Precision and recall of assigning players to teams based on their predicted versus actual ability These two metrics are the most relevant for measuring the performance of the model in assigning players to teams based on skill level. The average wait time can indicate whether the model is making efficient and quick team assignments, while precision and recall can measure the accuracy of the model's predictions. It's important to balance precision and recall since assigning players to a team with a large difference in skill level could have a negative impact on the players' gaming experience. C and D are also important metrics to track, but they may not be as directly tied to the performance of the team assignment model. User engagement can indicate the success of the overall gaming experience, but it can be influenced by other factors beyond team assignments. The rate of return is also an important metric, but it may not be a direct measure of the success of the team assignment model.