Your solution is producing performance bugs in production that you did not see in staging and test environments. You want to adjust your test and deployment procedures to avoid this problem in the future. What should you do?
A.
Deploy fewer changes to production
B.
Deploy smaller changes to production
C.
Increase the load on your test and staging environments
D.
Deploy changes to a small subset of users before rolling out to production
Question Statement: You want to adjust your test and deployment procedures to avoid this problem in the future
So based on this, I think the option "C" is correct, since it is the only one talking about doing changes in the test environment.
There is no indication given anywhere that the load is the problem or that the bugs are a result of load and not some other issue encountered when using a specific feature.
C. Increase the load on your test and staging environments.
As you have pointed out in "Question Statement", I do not see C covering "deployment procedures". Test and Staging environment is more on testing, but not about deployment procedure to production.
So, the only option that cover test and deployment is D. (Yes, kind of unacceptable to have the users to do "testing", but we make it "ok" by calling it "canary deployment")
With canary deployment we expose the new version to a small portion of users. With this approach maybe we don't see performance bugs in the canary release, since we don't have the 100% of traffic on the canary. But when we migrate the 100% of traffic to the new release (previous canary) we can see performance bugs.
A wouldn't prevent the bugs, it would just avoid them. B would help with root-cause analysis because it'd be a smaller change to review. C would test the performance of the system at its peak processing rates, so this assumes the bugs in production only occur because of usage. D would allow you to test the new code against smaller user sets to see if it occurs then, and if it still does you know it is not because of more user responses. So it's a tossup between C and D, D would be the cheaper/quicker answer so I'd choose D first then C if it's because of usage.
According to the question, [Your solution is producing "performance" bugs in production], so I think it is about the load. Plus canary test will not reproduce the bugs related to high load, I vote for C
The question is about the performance of the existing Code that they did not detect in Test environments . This is not about new API release . In order to test the performance they should increase the load in test environment and hence answer C.
Guys, you need to focus on the KEYWORDS in any question, it will help you to determine the best answer. The keyword for this question is "Performance", it is very clear that the load test on stage was not planned correctly (i.e/ lower than it should), so the performance bugs didn't appear, but when it comes to production with much bigger load the issues appear.
C because The performance issues in production might not have been seen in staging or test environments because the load (number of users, transactions, data volume, etc.) in those environments is not representative of the load in production. By increasing the load on your test and staging environments to match or exceed production levels, you can better simulate real-world conditions and catch performance issues before deployment
Without overthinking the wording, canary (and similar) deployment methodologies are often recommended in Google documentation, whereas increasing load in dev environments aren't. (My $0.02...)
Me too! I can't see how a "performance bug" might be mitigated via a canary deployment.
However, I see that C doesn't cover the "deployment" part of the question, then I deduce that the question is ambiguously formulated.
D. Deploy changes to a small subset of users before rolling out to production.
This approach, known as canary releasing or canary deployment, involves rolling out changes to a small group of users before deploying them to the entire user base. It is a very effective way to catch performance issues that might not have been apparent during testing.
C. Increase the load on your test and staging environments: This is definitely a good practice, as it can help simulate production-like conditions more closely. However, it may still not capture all real-world scenarios and user behaviors that can lead to performance issues.
Canary deployment is perfect to test new feature but to do stress testing, I do development for 25 years, when we want to resolve performance and scalability issues we do stress and load testing in pre prod environment, something you can't do by exposing the new feature to subset of users.
as for C: synthetic load may not cover all scenarios. For D obviously we need to have monitoring in place to see if e.g. system load or response times increased after canary deployment
I'm going for D. According to ChatGPT:
D. The best approach to avoid performance bugs in production that weren't detected in staging and test environments is to gradually roll out changes to a small subset of users before deploying them to production. This way, you can identify and address any issues that may arise in a controlled environment before affecting all users.
Why not C: Increasing the load on your test and staging environments, as suggested in option C, can be a valuable strategy for detecting certain types of performance issues related to scalability and load handling. However, it may not address all types of performance bugs or issues that are specific to the production environment.
A and B seem a bit obviously wrong. A is incorrect because it would be incompatible with CI/CD, and deploying fewer changes may still introduce bugs regardless of the low frequency. B would help root causing, but may also introduce new bugs that are exclusive to production.
A voting comment increases the vote count for the chosen answer by one.
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one.
So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
ghitesh
Highly Voted 4 years, 10 months agoSephethus
5 months, 1 week agoVedaSW
4 years, 1 month agofrancescogugliottagm
1 year, 1 month agoUrban_Life
2 years, 11 months agoRegisFTM
2 years, 10 months agoEroc
Highly Voted 5 years agonitinz
3 years, 8 months agomichael_m
2 years, 3 months agomichael_m
2 years, 3 months agoSreekey
4 years, 3 months agoAzureDP900
2 years, 1 month agoEkramy_Elnaggar
Most Recent 1 week, 4 days agonareshthumma
4 weeks, 1 day agodfizban
1 month, 1 week agomaxdanny
2 months, 2 weeks agoHungdv
3 months, 2 weeks agoHaigk
5 months agoa2le
5 months, 2 weeks agoRobert0
6 months agosantoshchauhan
8 months, 2 weeks agoPatrick2708
11 months, 2 weeks agoMahAli
11 months, 3 weeks agoMarekPL
11 months, 3 weeks agoPrakzz
1 year, 1 month agoyilexar
1 year, 1 month agojrisl1991
1 year, 2 months ago