Emergencies

For this my thoughts are why does this make any difference.

Why is it that issues encountered in the live systems get fixes implemented with often little or no testing? Is it maybe a flaw in the testing we do or is that what everyone does?

this is what i would like to see:

Requirements - produced by the business as to what they expect

Coverage - amount of testing needed to provide assurance

Approach - Quick single page on the approach to providing appropriate testing.

Scripting - Test scripting for what is going to be covered likely to be confirmation testing and limited regression testing.

Execution

Sat once more in system test execution and once again having the same old problems. I am a week into the system test period and already around 2-3 days behind schedule. What is more due to the tightness of deadlines we have no contingency... or at least if it slips I wont be there cause I am doing something that I cannot move.. getting married!!

SIGIST conference - what did I take away?

The conference yielded a surprising array of different ideas from the tried and tested to the conceptual ideas. The items that caught my interest big style are highlighted below.

Codependency
So what do i mean? Why does testing accept the bum rap that it gets in brief as the title explains its called codependency.

Every time that we accept:-

Late handover of software to test
Software that is not unit tested
Missing requirements
Taking something not conforming to entry criteria
Signing off on something not meeting exit criteria
...to name a few

We are expected to:-

Test to crunched deadlines
Work overtime
Signoff on something with no confidence in it

Result in the long term:-

We take the rap when things go wrong that we had no time to test.
We end up expected to work overtime every release
We always get late releases
Testing is always crunched
Nothing is unit tested

This is written in my words but based directly on the talk presented today by Lee Copeland from Software Quality Engineering at the SIGIST conference

Requirements

My personal experience is with Requisite Pro so I will likely make any examples relate to this and MQC but it could also apply to any equivalent system.

If you hold your test requirements along side your business requirements within you requirements management system, then they can be synchronized so that areas of contention would be automatically identified should the business requirements change. These test requirements can then be fed through to MQC in a read only format for usage in providing the requirements linking.

For me one of the key things about doing this is that the test requirements (ie the translated business requirements) can be baselined alongside the business requirements. This would also allow the provision for metrics to be gathered on the number of requirement changes and how much those changes are directly effecting testing.

This is written in my words but based directly on the talk presented today by John Cheesman from Strat Software Ltd at the SIGIST conference


The potential issue with all this is that the requirements NEED to be maintained for this to function if the requirements are far beyond what is currently being worked on then this will be useless. But ask yourself a question “If we are testing to provide a measure of software quality back to the stakeholders, then how can we do this without requirements. The requirements are what they want”.

History of Testing

Few of us, me included know much about the history of software testing and those names that designed the core principals of software testing that we now use to base all of our methods on. Names such as Dave Gelperin, Glenford Myers, Boris Beizer, Bill Perry, Michael Fagan and Tom McCabe.

Further to this it is also startling how few testers have even picked up a book on software testing let alone read one. I think the continued learning and development of people in any profession is key to the knowledge of the work that those people are doing.

Many people don't view work and indeed life as a continued learning experience where by study is as important there as it ever was in school back when someone explained that “Two plus two equals four”.

This is written in my words but based directly on the talk presented today by Lee Copeland from Software Quality Engineering at the SIGIST conference

The people factor

Never underestimate the power of saying thankyou for all the work done to people especially staff under you when it is clear that they have done a good job.

Ensure that you reward good work with an appropriate reward this could be financial or leave or even just a thank you as above. The main thing is that this is one of the main things that makes doing these jobs worth while. It is also important to consider that things like talking to senior management to give them credit or getting it mentioned in a newsletter can be useful providing a relatively cheap but effective reward.

This is written in my words but based directly on the talk presented today by Martyn Caswell at the SIGIST conference.

system test or UAT

It seems to be an on going debate where I work as to who should cover exactly what...

System testers end up picking up alot of the unit testing as well as the functional testing of the system. They also end up covering much of what could be held traditionally under the banner of UAT. Testing items under the business processes etc all end up  being part of system test.

Guess I have heard some not so good things relating to UAT today and find myself thinking ... if they are running the same scripts often with the same people is this testing providing any business value. Should UAT not strictly be based on the requirements of what the business wants ... Back to the core training :

SYSTEM TEST := should test a system does what it was designed to do.

UAT := should test a solution does what it was required to do.

Following a recipe

I was sat cooking today and decided that it is a bit like testing when you think about it... sad i know.

When following the recipe you have an expected outcome, a set of ingredients (Inputs) and a collection of steps that you have to follow. The smallest of mistakes in the recipe or the following of the instructions will change the expected outcome (a pasty in this case). If a mistake is made then the pasty will be burnt (a defect) then a new pasty will need to be made correcting the mistake (bug fix)  the recipe will be amended (script corrections) and the new pasty will then be rechecked so that it is ok. At then end the pasty will be checked to see if it is acceptable (exit criteria).

Maybe i will revisit this one day and name it the ... Pasty Method ;)

The truth will out

Why is it that on so many projects that I have worked on, everybody knows the project is going down the toilet? What is more amusing is that everyone knows and still seems to be unwilling to shift the goal posts.

If a project is percieved as likely to be running late come up with a contingency plan and do it early. Not only that dont be unrealistic there is no point trying to force something that is not going to happen. I am sure everyone has found themselves hoping it will all go right now and then... but how often has that happened. 

I realise that sometimes legislation changes etc can force set in stone dates.... but locating what can safely not go in could reduce the time putting the vital changes in would take.

Log those defects

I have seen this again and again, and even been guilty of it myself a time or two... dont debate a defect just log it.

If you are running test scripts and the script doesnt match the system then fail it. This can be revisited to decide if the script or system is in error and corrected. Raise defects for all these errors encountered. This will mean that no one can later come back to you and say "This test will fail it is written wrong, so why have you passed it"

The vital thing is just logging all the defects and failing all the scripts to ensure that nothing is missed.  

Death by email

So I am sat in my chair when in comes an email with a possible issue for my testing. Great they are keeping me in on the loop. Then sadly I get another and another and another as i get emails from everyone responding to the email. 30 emails later!!!!, something that i was happy to hear was leaving me with a throbbing vein. 

I think that the lesson learnt is to read the emails that you recieve carefully and think clearly about what you are responding and to who. While what you say might be usefull to you if you are including a bunch of mailing lists chances are most of the people in the list wont find it important.

While mistakes happen consistently doing it is just going to upset people.

Helping or doing the job?

Couldnt stay quiet about it forever, how comes there are always people that think if they ask you nicely, you will do their job for them? Fair enough when the chips are down I am happy to help others to get work done on time. I would hope that most of my mates in work would do the same but in this case, I have been asked to do the work for a person who probably earns twice what I do. Not only that i also get little credit for the work that is done.

I would recommend the following in this case as it seems to work well for me.

1. Make sure where possible that a team leader/manager is aware of the work you are doing.
2. Keep a copy of all the work that you have produced locally as evidence.
3. Put your name to all the work that is produced.  If this is changed and you feel strongly about it you can take issue with this based on the copy that you have.
4. Get business agreement for the work you are doing. 

Nothings perfect

Nothing is perfect and the best laid plans will need to be modified over and over in the course of a project. The only thing that is inevitable is that change will happen...just to mention a few:

1. A software code drop could be delayed.
2. Members of staff could go off sick.
3. A third party tool could not work properly.

The critical thing is to plan in a measure of contingency so that should issues occur then the tasks can still be completed in time. The other thing to do is to ensure that all tasks are prioritised. If all the tasks are prioritised then it becomes easy to quickly decide which tasks can be run in the time available and which tasks can be left out with a limited risk.

Comes with the job

When you are doing any job then the stress and strains of both work and of life outside work can take their toll at times. I know many people who at some point in their career have needed leave related to this, It is nothing to be ashamed of. 

My thoughts on the matter are two fold:

1/ Looking after yourself is always more important than what is happening at work. If you struggle on working chances are you will just feel worse. Better to get sorted and come back to work fresh.

2/ Dont bottle things up let others know what is going on and that way issues can be addressed sooner.

Trust or Not

Trust in your developers and business analysts....but at the same time plan for when things go wrong. When you get opinions the nature of things is that developers give positive estimates of when things are going to be delivered. Plan for the software to be late and plan in contingency.

If you plan with that in mind, then should things go pear shaped you can pick up the slack and get things completed correctly on time.

Business analysts give the testers the basis for testing but confirm the information and dont be afraid to raise questions over areas that you dont agree with. Nothing is worse than getting a product that meets everything requested only to find that its not what is needed.

A good example of this is a manager requesting a computerised system for a hotel booking system but only having the budget for 1 computer. With 4 people working on the checkin desk this would mean 3 people sat around with nothing to do all day. A business analyst should pick up that the current paper based booking system would be more efficient and that the money could be better spent elsewhere. If they dont pick up this then who else is there?

Requirements

It always amazes me that requirements always seem to be left out and classic statements of project issues are:

"The requirements were defined incorrectly."
"The requirements changed late in the project."
"We delivered what you wanted and this differs from your expectations."

I know this is ideal world stuff but surely with formal review and a little extra time spent at the beginning of a project a great deal of issues would be solved at the end. This would stop the developers guessing what the system should do. This would also stop the testers having little as a basis for testing and mean that anything produced would likely be acceptable.

The other main thing to consider is to produce a workable product early where possible and continually get this reviewed with the potential users and stakeholders to ensure that they know exactly what is being produced and that it is acceptable.

I think in the real world the best way to help with requirements is to pick up the phone, make a meeting, send an email and chase, chase, chase. If you dont have the information that you need make a list of what is missing and then talk to the person that has been charged with the responsability of producing this information. This will both enable you to say that you have done everything that you could and if you obtain what you need it will let you do your job more effectively.

When is enough enough?

Something that I constantly hear is that there are no requirements and no supplementary specifications etc.

Quite to the opposite what I want to say here is "When is enough, enough?"

I have worked on several peices of work lately where I find myself feeling like some of the documentation is being produced for the sake of it rather than to aid in the system development lifecycle. Can it be that sometimes a formal process that describes rigid artifacts for testing and development can make extra work? The simple answer is "Yes" flexibility is needed to enable discretion to be taken on what needs to be produced and how the process should be adapted to suit all different situations.

ISEB Foundation

Guess this is just something that you have to do to get a job in testing. With all the jobs that are around stating ISEB foundation as a requirement you just have to do it.

Dont just do it for the exam though is my tip for this. If you embrace this course and its material and utilise it all for your testing and then develop this with continued learning your skills will build very quickly. It also highlights the areas that you need more practice in and points you in the right direction to improving these areas.

I know many people who would say that this qualification proves little about a testers aptitude but if you know the material then you have a level of understanding of the basics.

If you havnt done the ISEB Foundation in software testing then you can review the sylabus here:

ISEB Foundation

Practical Test Planning

If you have ever thought of taking a practical test planning course then truly I would recommend it. I wouldnt take it if you have no team leadership duties as that and project planning is what this course is aimed at. If you are managing a project though it opens up the whole process making it clear. What you should be producing and what documents will be ideal for resource and budget estimates.

I am going to decline saying who I went with for the training as there are plenty of training companies out there offering the course and all are probably equally as good.

Wasted an hour of my life!

Have you ever been sat in one of those meetings? You sit there listening to the endless drone of pointless conversation. If you want to pass on a report then pass on the report. For a meeting to be worth while then I believe that it should follow some set guidelines.

1. Set out a clear objective or set of objectives for what you aim to achieve from the meeting.
2. Define an agenda to structure the meeting.
3. Any documents should be issued before the meeting with specific areas to be review mentioned if required.
4. A chairman of the meeting identified to make keep the meeting on track.
5. A range of people identified to fill roles if required.

If none of the above are thought about then the likely outcome is a meeting where you sit either bored or rambling about the weather and what you did on holiday. Certainly not getting any constructive work done.