I used to work with a QA person who really drove me nuts. They would misunderstand the point of a feature, and then write pages and pages of misguided commentary about what they saw when trying to test it. We'd repeat this a few times for every release.
This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing.
...eventually I realized that this person was somehow the best QA person I'd ever worked with.
how did misunderstanding a feature and writing pages on it help, not sure I follow the logic of why this made them a good QA person? Do you mean the features were not written well and so writing code for them was going to produce errors?
In order to avoid the endless cycle with the QA person, I started doing this:
> This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing.
Which is what I should have been doing in the first place!
That's not at all what they meant. They meant they ended up raising their own quality bar tremendously because the QA person represented a ~P5 user, not a P50 or P95 user, and had to design around misuse & sad path instead of happy path, and doing so is actually a good quality in a QA.
I worked with someone a little while ago that tended to do this; point out things that weren't really related to the ticket. And I was happy with their work. I think the main thing to remember is that the following are two different things
- Understanding what is important to / related to the functionality of a given ticket
- Thoroughly testing what is important to / related to the functionality of a given ticket
Sure, the first one can waste some time by causing discussion of things that don't matter. But being REALLY good at the second one can mean far less bugs slip through.
Most of the time QA should be talking about those things to the PM, and the PM should get the hint that the requirements needed to be more clear.
An under-specified ticket is something thrown over the fence to Dev/QA just like a lazy, bug-ridden feature is thrown over the fence to QA.
This does require everyone to be acting honestly to not have to belabor the obvious stuff for every ticket ('page should load', 'required message should show', etc.). Naturally, what is 'obvious' is also team/product specific.
I think noticing other bugs that aren't related to the ticket at hand is actually a good thing. That's how you notice them, by "being in the area" anyway.
What many QAs can't do / for me separates the good and the not so good ones, is that they actually understand when they're not related and just report them as separate bugs to be tackled independently instead of starting long discussions on the current ticket at hand.
so, QA should be noticing that the testers are raising tickets like this and step in and give the testers some guidance on what/how they are testing
I've worked with a clients test team who were not given any training on the system so they were raising bugs like spam clicking button 100 times, quickly resizing window 30 times, pasting War and Peace.. gave them some training and direction and they started to find problems that actual users would be finding
I didn't mean reporting things that you wouldn't consider a bug and just close. FWIW tho, "Pasting War and Peace" is actually a good test case. While it is unlikely you need to support that size in your inputs, testing such extremes is still valuable security testing. Quite a few things are security issues, even though regular users would never find them. Like permissions being applied in the UI only. Actual users wouldn't find out that the BE doesn't bother to actually check the permissions. But I damn well expect a QA person to verify that!
Was I meant though were actual problems / bugs in the area of the product that your ticket is about. But that weren't caused by your ticket / have nothing to do with that ticket directly.
Like to make an example, say you're adding a new field to your user onboarding that asks them what their role is so that you can show a better tailored version of your onboarding flows, focusing on functionality that is likely to be useful for you in your role. While testing that, the QA person notices a bug in one of the actual pieces of functionality that's part of said onboarding flow.
A good QA understands and can distinguish what is a pre-existing bug and what isn't and report it separately, making the overall product better, while not wasting time on the ticket at hand.
There's a typo in the URL here:
> If you have a topic in mind but are not sure if it is suitable for Paged Out!, check out the Writing Articles page or contact us
It links to `?page=writing.pho` rather than `.php`
(channeling Patrick McKenzie) If you have an S&P 500 index fund, you're a shareholder in Microsoft. Call their Investor Relations people, or send them a letter with this description. They will probably be of some help!
The same development studio, Sega AM2, recently had a developer reveal that he had put an Easter egg into Fighters Megamix for Saturn. However, he mistakenly introduced a crash bug in it.
This set me off looking for the Easter egg. After a couple days of reverse engineering, I finally found it [0]! I love looking for this stuff.
In this case it's probably smarter to resort to brute force.
Here's a C program that will run a lot faster than the Python. On my M1 Max MacBook Pro, I can evaluate all 9-button combos in 5.2 seconds. Each extra button should increase the runtime by a factor of 8. Allowing up to n repetitions should multiply the runtime by n. So you should be able to evaluate virtually all combinations in like 20 minutes without further acceleration.
If you feel like a challenge, Tony Hawk's Pro Skater 3 has some hashes with unknown inputs. The function is the same as the one in the article, and one target is 1eca8e89ad2dc1d6.
And another thing I noticed: because the hash is built Button by button, you can reuse part of the state when checking sequences. So if you’re checking a 10 button sequence, you get all subsequences of that almost for free (just need a comparison after every step). Getting to 18 buttons of length is still a lot of calculation though.
Good point - the dictionary attack produces some permutations that are too long, but it doesn't matter because you get the effect as soon as the final character of the correct code is entered.
I knew some from school, but stepping through a debugger with a video game that I remembered from childhood was a better education on computer engineering than anything I got in class.
This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing.
...eventually I realized that this person was somehow the best QA person I'd ever worked with.
reply