It would be nice if you could create a post regarding "testing DNA". How would a new engineer know if he/she has it in their blood. Many people think testing is just sitting behind the computer, triaging bugs or writing black box tests.
So how do we know if we have a testing DNA or not? Could it be that we have the hunger to write great automation tools, or perhaps interest creating some framework to help the development engineers to create unit tests easier? Is it when we find it enjoyable trying to find bugs in every corner of the code? Or perhaps it is when the testing engineer likes to prove the development engineer that his code is faulty.
Does Google hire exploratory testers? and those who dont write code to test code?
Writing code to test code is important and no doubt about it. Mostly those are checks that are performed.
While the test code would have not shown a problem with Buzz, a bunch of real good exploratory testers would have helped to find the problem.
Now, I am not one of those guys who would say "Ah! Buzz had problems so maybe things are wrong" but when a company as big as Google releases something and billions of eyeballs are watching, it might be a good idea to consider hiring skilled exploratory testers, too.
I have been tracking test openings of Google ever since I started looking for a job but there ain't anything that match the skills that most good testers I know of have. What kind of black box testing happens at Google, if it happens?
If you interview testers whose focus is to write code to test code and then ask them to perform exploratory testing, well, they might do it but you didn't hire them to do it and hence it might not be their cup of tea to focus on. With ever changing tools and technologies they are in a world of catching up with it and hence not their focus.
I know that it is not budget constraint that is actually stopping Google from hiring black box testers but something else that I don't know. Maybe you want to do things different but it is hard for me to convince based on my limited experience that it would be hard to remain different and successful without hiring black box exploratory testers for a longer time than this.
I have heard about Fedex Tour implementation from previous posts of James Whittaker but that was a part of the whole game than being the game itself.
Well, I am not proposing be black box only but I am trying to help myself understand how a company like Google doesn't have pure black box exploratory testers.
It's a little off topic, but Pradeep is pulling me to respond ..
"Now, I am not one of those guys who would say "Ah! Buzz had problems so maybe things are wrong""
I resemble that remark! :-)
I hope my public comments about buzz have been in the area of "We all have room for improvement."
We all have (some) problems, and it is easy to criticize. It's especially easy to criticize others and not talk about ourselves - so today's blog post, to fair, I wrote about testing issues a Socialtext:
Patrick, did your response to Pradeep's comment ("Good note. We will post a bit later with details about exactly this.") mean that the follow up post will explain why you don't use blackbox testers at Google, or is it going to explain how you use a blend of testers and test methods to achieve your results (or have I missed the point of your response altogether?)
Like Pradeep, I have been following job openings at Google for years and have yet to see anything that matches my experience (which has great breadth and depth, but no direct coding experience). Do these positions ever come up, or are we all going to have to get Computer Science degrees and learn Java to work at Google?
I Completely agree with Pradeep. It is not that I am a Black tester, I am also a automated tester. The reason i support this is, I felt the same a year ago. I always wanted to ask this question but dint know a source to ask? Do you guys have Black box testers? if so, I never seen a Single position since my career started nor my schooling. A good tester is one who can take the customer point of view and can analyze all the possible ways of errors and forthcoming errors but not just who can write pages of code. Of course I agree man power constrains accurate result with automation etc. I dont say automation is bad or against it.
But I strongly believe that core black box testing is also Mandatory just like automation.
The above mentioned are my idea and I thank the post owner to have such a wonderful blog, through which many of views wud come up just like me. Note: this is not a criticizum.
@Derek Harley...we have a blend of approaches and roles in testing: some manual/customer-focused, some semi-automated, and a significant focus on automated.
@the clairvoyant...yes, with strong testing ability, but not required experience for new grads.
We all test Google products- by using them everyday!
Oh do we? Lets assume that. How many bugs have you reported so far or how much of information you have passed on to Google that has helped them improve their products?
@Sriram - the staffing team is working hard to follow-up on everyone that sent me a resume. You should hear either way in the next 1-2 weeks. Thanks to everyone that is interested in Google!
@Sita -- it depends. If it's for the same role, I'd wait. If it's for a different role, it's ok to reapply. For instance, if you interviewed for SET and you want to interview for a Developer, yes it's ok to apply again.
I am very curious after reading this post, very nice. I like the way they treat testing.
Now I have a question, How come Google releases product like Wave and Buzz with severe functional defects and bugs?
Wave I can give a pass may be it is futuristic stuff, but how come Buzz release bought Gmail on its knees for example performance wise then off course privacy also was given a toss?
Hi everyone, my name in is Keith from SA and I would like make a comment on "testing DNA". This is a great idea to start with but lets face reality check...generally a human being is well capable of doing anything including acting...most candidates act up during evaluations/interviews, you can run ten interviews and they will pass but its only with time that you can get to know whether you have a right candidate with a right skill for your projects...often times candidates are desperate to join big companies to an extend that they are willing to do anything to join and on the other hand companies have projects in the pipeline to an extend that they don't have time to spend on the employment process thats where the problem is...you wouldn't have known my skills couple of years back but look now, I am recommended by all and I mean all employers I consulted with but it start with little knowledge and any candidate can crack it or loose it..."testing DNA" nope it won't pass the test of time...and thats reality, I conducted many interviews and I was interviewed many times, there are cases where you will know immediately that this candidate does not know what he/she is talking about but that does not mean the can't learn and become more than you want...the question is do you have a candidate with willingness to learn and can you identify a rough diamond...do you have time to invest in a junior...often time senior/skilled testers turn to be bigger than process because they know...nonetheless this is not always the case...Pat, over and above skill evaluate an attitude of a testers...surely you won't fail to get a tester, put your post online an hour is more than enough to have more than fifty applications...AMEN
@Basharat Wani -- we're not infallible. We spend a huge amount of energy on quality, but anyone who says they can prevent all bugs is fooling themselves. The reality is that we do our best, listen carefully to customers, and when we have issues fix them quickly.
@Keith Mashile -- I agree and disagree with parts of your comment. I agree that people sometimes (but that often, but sometimes) try to convince an interviewer of things they can't do. But, I don't think you can fool someone about your passion or skill if you dig deep enough. Unless the interview is quite superficial, I don't understand why you can't see someone's potential, passion, and ability. Testing DNA isn't magic. It's just shorthand for a deep passion and interest in testing. In my experience, you have it or you don't.
Hi Patrick, I am a test manager in Motorola. Actually we have tested lots of Android products, and I read lots of your posted papers and am inspired a lot by them, agile development/test model, really great.
However, I still have no idea on how Google organzie their product test through SA. Per my understanding, it is impossible for Google to depend more on Automation test, I agree that should be a long term goal for us, but so far, for most of companies, like, SE, Nokia, Moto, etc, they still relied on manual test more than Automation on their product test, because black box test is a good way to validate the requirements, that can answer if we comply our product requirements, and how much? Sometimes, we can use automation test for some stress, performance test.
Anyway, Could you share with me know what you think on Manual test?
Another desirable quality in test engineers is the innate ability to think of how people can do things the wrong way (and how to break things is good, too). I worked as a test engineer for several years and I can tell you thinking outside the "norm" is definitely a plus.
It would be nice if you could create a post regarding "testing DNA". How would a new engineer know if he/she has it in their blood. Many people think testing is just sitting behind the computer, triaging bugs or writing black box tests.
ReplyDeleteSo how do we know if we have a testing DNA or not? Could it be that we have the hunger to write great automation tools, or perhaps interest creating some framework to help the development engineers to create unit tests easier? Is it when we find it enjoyable trying to find bugs in every corner of the code? Or perhaps it is when the testing engineer likes to prove the development engineer that his code is faulty.
So how do new engineers know?
Good idea...but it sounds like you've already answered the question.
ReplyDeleteDoes Google hire exploratory testers? and those who dont write code to test code?
ReplyDeleteWriting code to test code is important and no doubt about it. Mostly those are checks that are performed.
While the test code would have not shown a problem with Buzz, a bunch of real good exploratory testers would have helped to find the problem.
Now, I am not one of those guys who would say "Ah! Buzz had problems so maybe things are wrong" but when a company as big as Google releases something and billions of eyeballs are watching, it might be a good idea to consider hiring skilled exploratory testers, too.
I have been tracking test openings of Google ever since I started looking for a job but there ain't anything that match the skills that most good testers I know of have. What kind of black box testing happens at Google, if it happens?
If you interview testers whose focus is to write code to test code and then ask them to perform exploratory testing, well, they might do it but you didn't hire them to do it and hence it might not be their cup of tea to focus on. With ever changing tools and technologies they are in a world of catching up with it and hence not their focus.
I know that it is not budget constraint that is actually stopping Google from hiring black box testers but something else that I don't know. Maybe you want to do things different but it is hard for me to convince based on my limited experience that it would be hard to remain different and successful without hiring black box exploratory testers for a longer time than this.
I have heard about Fedex Tour implementation from previous posts of James Whittaker but that was a part of the whole game than being the game itself.
Well, I am not proposing be black box only but I am trying to help myself understand how a company like Google doesn't have pure black box exploratory testers.
Its not a bad idea for sure.
Good note. We will post a bit later with details about exactly this. Watch for a post from James in
ReplyDeleteApril.
Thanks,
Pat
It's a little off topic, but Pradeep is pulling me to respond ..
ReplyDelete"Now, I am not one of those guys who would say "Ah! Buzz had problems so maybe things are wrong""
I resemble that remark! :-)
I hope my public comments about buzz have been in the area of "We all have room for improvement."
We all have (some) problems, and it is easy to criticize. It's especially easy to criticize others and not talk about ourselves - so today's blog post, to fair, I wrote about testing issues a Socialtext:
http://blogs.stpcollaborative.com/matt/2010/03/15/exhaustive-testing/
Good points Pradeep, and Patrick, I find that idea of testing DNA ... fascinating.
Why can't you run the select tests BEFORE the check-in, in order to prevent regressions?
ReplyDelete@Sriram, agree...you can.
ReplyDeletePatrick, did your response to Pradeep's comment ("Good note. We will post a bit later with details about exactly this.") mean that the follow up post will explain why you don't use blackbox testers at Google, or is it going to explain how you use a blend of testers and test methods to achieve your results (or have I missed the point of your response altogether?)
ReplyDeleteLike Pradeep, I have been following job openings at Google for years and have yet to see anything that matches my experience (which has great breadth and depth, but no direct coding experience). Do these positions ever come up, or are we all going to have to get Computer Science degrees and learn Java to work at Google?
I Completely agree with Pradeep. It is not that I am a Black tester, I am also a automated tester. The reason i support this is, I felt the same a year ago.
ReplyDeleteI always wanted to ask this question but dint know a source to ask? Do you guys have Black box testers? if so, I never seen a Single position since my career started nor my schooling.
A good tester is one who can take the customer point of view and can analyze all the possible ways of errors and forthcoming errors but not just who can write pages of code. Of course I agree man power constrains accurate result with automation etc. I dont say automation is bad or against it.
But I strongly believe that core black box testing is also Mandatory just like automation.
The above mentioned are my idea and I thank the post owner to have such a wonderful blog, through which many of views wud come up just like me.
Note: this is not a criticizum.
Madhuri
Are you guys hiring new grads without much test experience?
ReplyDelete@Derek Harley...we have a blend of approaches and roles in testing: some manual/customer-focused, some semi-automated, and a significant focus on automated.
ReplyDelete@the clairvoyant...yes, with strong testing ability, but not required experience for new grads.
@ Patrick
ReplyDeleteI have any come across, Manual/ Customer support in requirements. Anyways its good to know that you guys also have positions for Manual tester.
Coming to new grad positions, I hope Google consider resumes of only TOP-Notch Schools.
Do you guys have any Manual QA Positions at your company which are still open?
We all test Google products- by using them everyday!
ReplyDelete@Ajay Bhagwat,
ReplyDeleteWe all test Google products- by using them everyday!
Oh do we? Lets assume that. How many bugs have you reported so far or how much of information you have passed on to Google that has helped them improve their products?
:)
So, we don't right?
Patrick,
ReplyDeleteWhat is the typical turnaround time for processing incoming resumes for SET positions?
Thanks,
-Sriram
@Sriram - the staffing team is working hard to follow-up on everyone that sent me a resume. You should hear either way in the next 1-2 weeks. Thanks to everyone that is interested in Google!
ReplyDeletePat
I have been interviewed by Google in January.
ReplyDeleteCan I apply again.
@Sita -- it depends. If it's for the same role, I'd wait. If it's for a different role, it's ok to reapply. For instance, if you interviewed for SET and you want to interview for a Developer, yes it's ok to apply again.
ReplyDeleteI am very curious after reading this post, very nice. I like the way they treat testing.
ReplyDeleteNow I have a question, How come Google releases product like Wave and Buzz with severe functional defects and bugs?
Wave I can give a pass may be it is futuristic stuff, but how come Buzz release bought Gmail on its knees for example performance wise then off course privacy also was given a toss?
?????
Hi everyone, my name in is Keith from SA and I would like make a comment on "testing DNA". This is a great idea to start with but lets face reality check...generally a human being is well capable of doing anything including acting...most candidates act up during evaluations/interviews, you can run ten interviews and they will pass but its only with time that you can get to know whether you have a right candidate with a right skill for your projects...often times candidates are desperate to join big companies to an extend that they are willing to do anything to join and on the other hand companies have projects in the pipeline to an extend that they don't have time to spend on the employment process thats where the problem is...you wouldn't have known my skills couple of years back but look now, I am recommended by all and I mean all employers I consulted with but it start with little knowledge and any candidate can crack it or loose it..."testing DNA" nope it won't pass the test of time...and thats reality, I conducted many interviews and I was interviewed many times, there are cases where you will know immediately that this candidate does not know what he/she is talking about but that does not mean the can't learn and become more than you want...the question is do you have a candidate with willingness to learn and can you identify a rough diamond...do you have time to invest in a junior...often time senior/skilled testers turn to be bigger than process because they know...nonetheless this is not always the case...Pat, over and above skill evaluate an attitude of a testers...surely you won't fail to get a tester, put your post online an hour is more than enough to have more than fifty applications...AMEN
ReplyDeleteHi Was it too late to send you my resume?
ReplyDelete@Basharat Wani -- we're not infallible. We spend a huge amount of energy on quality, but anyone who says they can prevent all bugs is fooling themselves. The reality is that we do our best, listen carefully to customers, and when we have issues fix them quickly.
ReplyDelete@Keith Mashile -- I agree and disagree with parts of your comment. I agree that people sometimes (but that often, but sometimes) try to convince an interviewer of things they can't do. But, I don't think you can fool someone about your passion or skill if you dig deep enough. Unless the interview is quite superficial, I don't understand why you can't see someone's potential, passion, and ability. Testing DNA isn't magic. It's just shorthand for a deep passion and interest in testing. In my experience, you have it or you don't.
Hi Patrick, I am a test manager in Motorola. Actually we have tested lots of Android products, and I read lots of your posted papers and am inspired a lot by them, agile development/test model, really great.
ReplyDeleteHowever, I still have no idea on how Google organzie their product test through SA. Per my understanding, it is impossible for Google to depend more on Automation test, I agree that should be a long term goal for us, but so far, for most of companies, like, SE, Nokia, Moto, etc, they still relied on manual test more than Automation on their product test, because black box test is a good way to validate the requirements, that can answer if we comply our product requirements, and how much? Sometimes, we can use automation test for some stress, performance test.
Anyway, Could you share with me know what you think on Manual test?
Thanks a lot.
I am just wondering whether Google will use any third-party automation tool such as HP- Quick Test professional, or will it have its own tool?
ReplyDeleteIf Google's own tool is better than any other tool, will Google make it available for others also?
Thanks,
Rajamanickam
http://qualitypoint.blogspot.com
HI Patrick...
ReplyDeleteDo you guys hire the persons who are manual testers and don't have any knowledge of coding.......
Another desirable quality in test engineers is the innate ability to think of how people can do things the wrong way (and how to break things is good, too). I worked as a test engineer for several years and I can tell you thinking outside the "norm" is definitely a plus.
ReplyDelete