Adam White's Story

This blog has been moved…

Adam White — Sat, 22 Sep 2007 02:31:08 +0000

I decided to register the domain http://adamkwhite.com and set my blog up there.

Hope you like it!!!

When to Start Logging Bugs

Adam White — Fri, 14 Sep 2007 03:06:46 +0000

Testing starts as soon as we have a build to test. If it’s checked in – it gets tested even before the features are deemed “Feature Complete”

Now – how do we track the bugs found during this stage of development? Up until our most recent proejct we’ve always used the bug tracking system.

Now weve been asked to stop using the bug system because it adds too much overhead. The explanation given is “A developer is working on a feature and you log bugs on it when they know it isn’t working…then at the end they have to go back through the list and re-test everything you logged to make sure it’s fixed. It adds too much to their plate.”

My first reaction to this was one of fear and anger. Fear because our development culture appears to moving towards information hiding. To me, as a tester, not logging bugs is hiding information. One of the main goals of testing at PlateSpin is provide timely information to project stakeholders. Without having the bugs logged in a central place for all to see it becomes more difficult to get the information to everyone. Also, for most testers, not logging bugs is equivalent to developers not checking in their code for several days. I wonder what kind of reaction I would get if I asked development to stop checking code in after code freeze?

Why the sudden aversion from development to the bug system?

In the previous release we started testing a feature that was clearly not ready. By the time we were done we had accumulated about 200 bugs on a particular feature. This added a significant overhead for the developers responsible for that feature. The aversion is understandable to a degree. We made an “amateur” tester mistake. We used a tornado to knock over a straw house.

After being asked to not use the bug system I stated my case that I thought this was a bad idea and gave my reasons why. Other testers did the same. We decided to go the route the developers wanted and not use the bug tracking system. Instead we said we would send out a daily (running) list of issues.

During a pre-release status meeting somone said “It would be good if we could number these bugs so we could just refer to the numbers”.

“I know a system that we can use to do that – and it doesn’t require any effort to use.”

Someone else suggested that the testers could simply use Word instead of notepad to write the list of issues. This would give them access to the handy numbered list. (Notice who had to change their work)

Then someone suggested “It would be really nice if we could have these issues in one central place”

“I have just the system for that” – I replied.

Then it was requested testers put the bugs in a project tracking spreadsheet. This sheet had the exact same headings as the bug tracking system. Hmmm – very curious indeed.

This is where I drew the line. We told the developers that if they wanted the information in the spreadsheet that we didn’t mind. We just weren’t going to do it for them. If they wanted to enter it – they were more than welcome to.

How many issues do you think got copied into the spreadsheet?

We haven’t hit our feature complete date yet so I can’t say how this little experiment will work out. We’ve had some developers complain to us for not logging bugs on their code. When they ask for screenshots and logs we have to say we don’t have them becuase have no place to put them and reference them to a bug.

I’m really curious to see how things go with our experiment. Our backlog is already 100+ bugs. The day of feature complete is going to a fun bug logging day!

I mentioned above that we gave reasons for using the bug system instead of a spreadsheets – here are some of them.

Multiple people cannot edit the spreadsheet at one time. (Lets not tell them about shared workbooks in excel. It will be our secret – ok?)
One must continually look the spreadsheet to see changes in priority etc
No running log of notes and troublshooting steps. The bug database is often helpful when it comes time to write documentation and
knowledgebase articles
No automatic notification system when a bug status changes or when a new issue is opened.
No way to add screenshots, diagnostics, or other attachments. Having some, or all, of these items can significantly speed up the amount of time it takes to fix an issue.

Got any other ones we can use?

My Blog is moving

Adam White — Fri, 14 Sep 2007 03:09:21 +0000

I’ve decided to move my blog over to wordpress.

http://adamkwhite.wordpress.com/

Hope you will join me there for some new content – don’t forget to update your RSS feed

Reasons For Not Fixing Bugs

Adam White — Fri, 07 Sep 2007 23:13:23 +0000

I found myself arguing against fixing bugs today. This isn’t a situation I find myself in often. Our release is in 3 days and there were 80 or so bugs in total. From a pure tester point of view not fixing bugs feels like the worst possible sin in the world. As a manager who has to

help ship software it doesn’t feel so bad (anymore)

Here are some reasons I gave for us to stop fixing unimportant bugs.

- We have to ship software that we know works!
- We don’t want to assume the risk of causing regressions
- We don’t have enough time to go back and test the product thoroughly

- We need to focus in the areas that are currently broken without worrying about fixing unimportant areas.

- We don’t want to work overtime to test the fixes and spend time opening new bugs.

More Context – 6o of these bugs were the Priority 3 kind. The kind that when we get to the end of the release we usually say "We can live with that – move it to the next release". we have 3 priority levels 1, 2 and 3. We don’t have severity. The P3 bugs were the ones I didn’t want to be worked on and checked in as they weren’t crucial to the release

Can you think of more reasons?

A good thinking exercise would be to go through and make a list of reasons TO fix bugs. (This

is the side of the fence i’m usually on so this list should be easy for me )

Same or different?

Adam White — Thu, 10 Aug 2006 01:52:57 +0000

I found this while surfing the net the other day. Are the two sentences below the same or different? Explain

- Everybody is innocent until proven guilty

- Nobody is guilty until proven to be not innocent.

Feedback on my TASSQ presentation

Adam White — Wed, 26 Jul 2006 22:44:56 +0000

Here is the feedback that I received from the TASSQ board on my presentation. All the feedback is anonymous so I can’t track it back to the people who said it.

It is interesting to read some of the comments as they often tell you more about the values/ideas of the person commenting than on the subject that are actually commenting on – my presentation in this case.

Total attendees

Surveys completed

Additions to mailing

(including speakers)

Needs Exceeded

Improvement Expectations

	1	2	3	4	5	Null
Presentation was well organized		2	9	28	3
Appropriate level of detail		3	13	22	4
Relevant to your job		8	8	19	5	2
Clearly presented	1	1	16	19	5
Speaker’s knowledge on subject		2	7	24	9
Speaker’s presentation style		3	13	21	4	1
Overall rating for tonight’s meeting		2	12	22	3	3

Comments / suggestions

Each bullet represents comments from a single evaluation form, in no particular order,

with editorial comments in italics:

· Too junior (Comment on: Relevant to your job).
When Adam went to Disneyland, did he ever get off the Roller Coaster? He sounds like he loves to on the edge, like a train wreck waiting to happen! Some good ideas but it is high risk and/or more experience then required.

· One of the slides was useful, “What we are looking for”.

· Very knowledgeable and passionate speaker

· Question period was more relevant than the presentation content

· Very refreshing topic. Speaker needs to polish his presentation, especially the “Ummm, Ummm”

· Very informative

· Do your filing, paper work after the presentation. It was distracting when the reception desk was managing paper work

· Excellent Q&A session

· Very useful topic

· I enjoyed this. Even though I’ve done this before, I learned some new “tricks”

· Seemed uncomfortable or nervous (Comment on: Speaker’s presentation style)

· Very candid and personal experience driven discussion (Comment on: Speaker’s presentation style)

Presentation: Hiring The Next Generation Of Skilled Software Testers

Adam White — Sat, 22 Jul 2006 21:29:36 +0000

Back In March back I did a presentation called "Hiring The Next Generation of Skilled Software Testers". I presented to a testing group called TASSQ. I believe the attendance was between 50 and 70 people. This is the biggest crowd of strangers, that I didn’t know, that I’ve ever presented to.

You can find the presentation outline on the TASSQ website under dinner meetings. MSN spaces aren’t going to let me upload my PowerPoint slides. If you are interested in seeing the content of the presentation let me know and I’ll send it to you.

Overall, it was a great opportunity for me to build my personal brand. I learned much from it. I had a great turn out from my support group which included my whole team and Neetu (my girlfriend). They gave me some great feedback on my presentation and my style which I will incorporate into my next speaking opportunity.

I’m going to also post the feedback that I received on the presentation.

The popcorn heuristic

Adam White — Tue, 18 Jul 2006 01:49:01 +0000

At PlateSpin someone mentioned that shipping software is like making popcorn. Healthy testing progress follows similar patterns of bugs or "pops" that making a healthy bag of popcorn does.

I decided to run this heuristic by James Bach and Michael Bolton. After much back and forth here is the result (Written by James)

POPCORN HEURISTIC OF TEST PROGRESS

————————————-

KEY IDEA:

Finding bugs is something like popping popcorn. If the test process is healthy, expect the peak bug find rate to occur near the half-way point.

IMPLICATIONS:

1. If the peak find rate was nearer the beginning or the end of the test process than to the middle, then the test process probably was not healthy.

2. If the find rate is near its peak right now, then you probably aren’t close to shipping or you aren’t close to a healthy test process.

3. If you believe you are near the beginning of your healthy test process, then you should expect to find bugs more and more quickly.

4. If, despite every effort to find bugs, you are finding fewer and fewer of them, that probably means the software is reaching release readiness.

FALLIBILITY:

1. Unlike popcorn, bugs are very diverse. The same "heat" won’t make different bugs "pop". Therefore, the find rate may be slowing down simply because your testing isn’t good enough, not because the bugs aren’t there to be found. This may lead you to ship too soon.

2. Unlike popcorn, different bugs have different levels of importance.

Therefore, it’s the find rate of important bugs that we care most about.

However, there may not be enough important bugs to establish a clear trend.

You may ship too soon or too late.

3. Unlike popcorn, new "kernels" are constantly being added to the bag while it is popping. Sufficiently disruptive changes are expected to revive the find rate.

4. Unlike popcorn, you can’t hear all of them pop. If many bugs are going unreported, either because they are found by developers, or found by beta testers and not reported, or found by testers and not considered important enough, then the find rate will look artificially low. This is especially true near the end of a test project, as fatigue depresses sets in.

‘Would you tell me, please, which way I ought to go from here

Adam White — Tue, 04 Jul 2006 21:46:10 +0000

I have come across the quote below in two books - McLuhan for Managers by Mark Federman and Becoming a Techincal Leader by Gerry Weinberg.

‘Cheshire Puss,’ …’Would you tell me, please, which way I ought to go from here?’

‘That depends a good deal on where you want to get to,’ said the Cat.

‘I don’t much care where-’ said Alice.

‘Then it doesn’t matter which way you go,’ said the Cat.

‘-so long as I get SOMEWHERE,’ Alice added as an explanation.

‘Oh, you’re sure to do that,’ said the Cat, ‘if you only walk long enough.’

Lewis Carroll – Alice in Wonderland

The quote really ressonated with me. I’m a planner and goal setter. This quote sums up why I set goals and plans – I don’t want to wind up getting SOMWHERE where I don’t want to be.

Have you ever been in this situation?

Adam White — Thu, 29 Jun 2006 22:33:37 +0000

Here is an actual dialog between a tester and manager at PlateSpin. Have you ever been in this situation? What did you do? If you haven’t – what would you do if you were? I would love to hear your stories/comments/experiences with situations like this.

Manager: "How long will it take you to test Project X?"

Tester: 1 week

Manager: "How do you know that?"

Tester: "It always takes 1 week"

Tester: Do we have a requirements document?

Manager: Yes – we have a requirements document. It’s not up to date

Tester: Perfect. It’s still going to take 1 week

Manager: How can that be – I don’t understand

Tester: Neither do I. *sigh*

Tester: Do we have a technical spec?

Manager: No

Tester: That’s risky. It’s still going to take 1 week

Manager : I don’t understand.

Tester Interjects : You all ready said that.

Manager: How can it take you one week???? This project has 50% more features than the last one.

Tester : Why are you questioning me? Don’t you understand what we do?!?!?!

Manager: No

Tester: Ahhh – I see. So you what you are really saying is that can’t have confidence in my estimate because you don’t understand what we do. You also haven’t gone through a project with me yet so the level of trust isn’t established. Let me explain what we think we actually do.

*This is the part that i’m curious how have handled*

< — Tester explains. –>

<– Manager asks questions –>

<– Tester Clarifies –>

Manager: Great! Now I understand

Tester: Awesome – Can I go find more bugs now?

I will write more on what I explained but i’m really interested in knowing what you would do to explain to the manager to help them understand and have confidence in your estimates.