Definition - Delivery Management: the endeavor management task of properly managing the delivery of work products and services
Responsibilities - The typical responsibilities of Delivery Management are to:
Ensure that all deliverable work products and services are delivered as required by the binding legal agreements.
Ensure that all deliverable work products and services are delivered according to the budget and schedule.
Eliminate risks due to poor delivery.
Preconditions - Delivery management typically may begin when the following preconditions hold:
Either the: Endeavor has started or Center exists.
The management team is adequately:
Staffed.
Trained in delivery management.
Completion Criteria - Delivery management is typically complete when the following postconditions hold:
Either the:
Endeavor has been completed or the Center has been retired.
All deliverable work products and services have been successfully delivered to (and accepted by) the customer organization.
All commercial packages have been successfully delivered by the vendor organization(s) to the development organization.
All components have been successfully delivered by the subcontractor organization(s) to the development organization.
Steps - Delivery management typically involves the management team performing the following steps in an iterative, incremental, parallel, timeboxed, and ongoing manner:
Work with the development organization and customer organization to ensure that the deliverable work products and services are delivered as agreed.
Work with the vendor organization(s) to ensure that they successfully deliver acceptable commercial packages to the development organization.
Work with the subcontractor organization(s) to ensure that they successfully deliver acceptable components to the development organization.
Track deliveries against schedules and milestones.
Techniques - Delivery management typically involves the following techniques:
Close collaboration with external organizations.
Correspondance.
Phone conversations.
Face to face conversations.
Meetings.
Work Products - Delivery management results in the following work products:
Management Set:
Delivery Statement
Guidelines - Delivery management is critical because most endeavors fail to deliver all that was promised on time and within budget.
Relationship management and communication management are critical to successful delivery management.
Happy Testing !!!
Tuesday, January 29, 2008
Staging environment ???
Definition
The staging environment is any development environment that is primarily used to stage tested applications prior to their deployment to the production environment.
Objectives - The typical objectives of the staging environment are to provide a separate environment for the:
Content management team to incorporate approved content prior to publication to the content management production environment.
User experience team to perform system usability testing.
Customer representatives to evaluate the completeness, quality, and general status of the tested application prior to deployment.
Benefits - The typical benefits of the staging environment include:
Content problems can be found prior to publication in the production environment.
System usability testing can be performed in its own realistic environment without slowing down system and launch testing.
Customer representatives can evaluate the tested application without slowing down testing and deployment.
Hope this explains you about staging environment mail me if u need some more clarification on this.
Happy Testing !!!
The staging environment is any development environment that is primarily used to stage tested applications prior to their deployment to the production environment.
Objectives - The typical objectives of the staging environment are to provide a separate environment for the:
Content management team to incorporate approved content prior to publication to the content management production environment.
User experience team to perform system usability testing.
Customer representatives to evaluate the completeness, quality, and general status of the tested application prior to deployment.
Benefits - The typical benefits of the staging environment include:
Content problems can be found prior to publication in the production environment.
System usability testing can be performed in its own realistic environment without slowing down system and launch testing.
Customer representatives can evaluate the tested application without slowing down testing and deployment.
Hope this explains you about staging environment mail me if u need some more clarification on this.
Happy Testing !!!
Wednesday, January 16, 2008
Cricket Vs Software testing :)
Cricket is a bat-and-ball sport contested by two teams, usually of eleven players each. Let us see if we can correlate the sports cricket with that of software testing! I am going to identify and list out few points that seem similar between software testing and cricket:
1. (Un)Predictability: Experts say, ‘unpredictability of cricket is its greatest charm’! It is very difficult to predict a win or loss before the last ball is bowled. And it is the unpredictability, which makes cricket so interesting. Teams might look strong or weak on a piece of pen and paper. But in cricket, unpredictability reins, not the statistics.
Coming to software testing, there could be testers who believe that it is easy and straightforward to expect some fixed output after executing a series of test steps, but I only wish; testing software was as simple as that! Echoing some experts in software testing: trying to predict the result of a test in terms of PASS or FAIL criteria can be one of those dangerous traps in software testing world, where a tester can shoot at his own feet! Irrespective of the number of test scripts (either manual test cases or automated test scripts) a tester has written, until the tester gets the application module to test, nothing can be told for sure about the state of the application and its behavior. Unpredictability is one of those things that make software testing so much fun.
2. Skills: Cricket is a game where only skillful players can make their team a winner. I am not sure about other countries, but if you have ever been to India, chances are high that you might have seen cricket being played on a street behind your hotel room! They say, cricket is a fever here in India. Here people are so obsessed with the game that they not only play and watch cricket but also eat, sleep and even drink cricket! I have played fair amount of cricket in my school and college days. You might have too. But the reason, why players like Sachin Tendulkar, Brian Lara and Ricky Ponting are considered as one of the finest cricketers and not us, lies in the cricketing skills they possess.
Likewise, in software testing too, it is the testing skill that differentiates a good tester from a mediocre one. The one, who has the better testing skills in his arsenal, can find more important defects quickly in the software that he is testing. Learning, practicing and applying are three golden rules to acquire any skill. With determination and strong will power, nothing is impossible to learn. Fortunately, software testing is no exception!
3. Game Planning (Knowing the Opponent): Professional cricket is all about knowing the strengths and weaknesses of the opponent team and devising a game plan in order to combat their strengths and to exploit their weaknesses.
In software testing, knowing the testing mission is the first step in determining the goal of the testing effort. Without being clear about the goal, it might turn out fatal to go about testing straightaway. Once the tester is clear about the testing mission, then he can analyze his chances of success or failure depending on the availability and expertise of the resource in hand and the complexity of the testing problem. e.g. imagine a situation where the tester has to test the application to find out it’s robustness to guard against hackers and other malicious users. Knowledge of a fair deal of details about the level of attack that can be attempted against the application, can give the tester a better chance to plan out a strategy to emulate the attack and to test how the application guards against it.
4. Handling Pressure: Winning a game of cricket is all about handling the pressure well. The team that is able to handle the tremendous amount of pressure of the game finally wins the game.
For those who think or have been told that software testing is an easy job and can be done by every Tom Dick and Harry, let me warn you, you have been terribly misguided! Testing is a career, which demands lot of intellectuality and stability of mind to work under variety of pressures (like technical pressure, pressure due to workload, managerial pressure, pressure of deadline, pressure arising from the nature of the job etc). As testers, the basic requirement of our job demands us to deliver bad news (presence of defects, buggy modules that fail miserably on testability grounds etc) to different stakeholders (the programmers, management staffs, clients) of the product under development. Nobody likes to hear bad news. Unfortunately, gone are the days when messengers were not hanged just because they brought in a bad news to the king! Hence, unless the tester is quite good at handling the pressure arising as a byproduct of his work, and is not so good at being diplomatic, he might find it tough to carry on with his job for long. On the other hand, the tester who has got the capability to handle the pressure till the end, has every chance of winning the Testing World Cup!
5. Adaptability: If you have been watching cricket for sometime now, then you must be experienced enough to know that whenever a team visits a different continent for playing cricket, it often finds it difficult to play up to its usual standards, of course unless we are talking about a team like Australia. Most Asian teams find it difficult to play down under and the vice versa. And the reason lies mainly in the difference in the pitch condition in the different continents. Soil texture, clay quality, humidity, temperature, amount of grass, dust etc can affect the behavior of a cricket pitch. The players who can adapt themselves quickly with the new environment can gain an edge in the game.
Just as no two cricket pitches can be exactly identical, no two AUTs (Application Under Test) can be same. Hence, I often get surprised when I see testers trying to clean themselves off testing assignments with excuses such as:
» “I have not tested anything like this before!”
» “This platform is completely new to me!”
» “I am new to this technology. I can not test it!”
» “I am a manual tester. I can not use this test automation tool!”
» “I need the domain knowledge before I can test this application!”
» “Hey, you don’t have any base documents (URS, SRS, BRS blah blah…)! How can I start testing your application?”
To me, this simply translates as a bad attitude of a tester, who is not ready to adapt and more importantly not ready to learn something new. For such testers, I have just one thing to say:
"Don’t forget that, even big and powerful animals like the Dinosaurs and the Wooly Mammoth became extinct just because they FAILED to adapt to their changing surrounding environments; we are just testers!"
When I say so, I DON’T mean that I am a great tester who has been excellent at adaptability throughout his testing career. On the contrary, I have been through several such instances when I was stupid enough to oppose such changes and was in fact, hostile to proposed changes. And don't be too surprised when I tell you that, some of the excuses that I have mentioned above were from my own mouth! However, I have learnt my lesson of adaptability in the hard, bitter way. So I just wanted to warn others who are new and passionate about testing but might commit the same mistake of lack of adaptability that I have committed in my earlier years of testing career.
6. Patience: Cricket is a game of patience. At times, a batsman might be finding it tough to steal a run due to the sharp bowling attack, tough ground conditions etc. But if the batsman has got enough temperament and patience, he might soon find it easy to not only steal a single but also score boundaries against the same bowler under same pitch conditions. Similarly with patience and consistent good bowling, a bowler could turn around the fate of a match by picking up quick wickets.
Similarly, a lot of patience is required while testing an application. There might be days when nothing seemed to work right for you and you would end up without catching a single defect in the whole testing session. There might be times when you would find yourself hitting your head on the keyboard while trying to find ways to reproduce a hard-to-reproduce defect. But you ask any expert in software testing, and he would suggest you to have patience. Putting in extra effort and having patience can help you come out of such dry days of testing.
7. Team Game: Cricket is a team game. Here the team wins the match, which displays excellent exhibition of team play. And so is software testing. All the team members of a testing team can’t be with equal skill sets and with equal level of expertise. Having a testing team with members with different specialization can add to the versatility of the team’s overall performance. I do understand that there can be testing geniuses, who are jack of all testing trades and are all-rounders when it comes to testing. But finding such a tester can be a tough task for any test manager. So as a workaround a good test manager might look forward to build a team which has testers specializing in different aspects of testing (functionality testing, performance/stress testing, database testing, usability testing, automation engineers, risk-based testing etc).
8. Lack of Concentration can Doom you: Cricket requires a high level of concentration. As a batsman you loose your concentration and you could loose your wicket. As a bowler you loose your concentration and you could be hit for boundaries all over the ground! As a fielder you loose your concentration and you could drop a catch (remember! catches win matches!) or give extra runs by mis-fielding.
Coming back to testing, imagine a tester who misses to identify a defect that just happens in front of his eyes, as he was looking for some other defect at that moment. Experts call this phenomenon as inattentional blindness. This can also happen if your mind is too tired due to continuous testing and you start loosing concentration, and in turn, start loosing defects!
9. Wearing safety Helmets and Testers: Batsmen (and even the close-in fielders) wear helmets to safeguard their head from the fast coming deliveries from the bowler. In case of software testing, unfortunately, the testers are like the safety helmet! They act as the representative of the client/end users of the project. And act as the last line of defense of the client/end users against the possible ways in which the software might fail. So as testers, start feeling proud of your profession, as you are safeguarding somebody by taking the hits yourself.
10. Learning Lessons from Failures: Even world champions like Australia was once defeated in a cricket match against the under-dogs like Bangladesh. Success and failures are two integral part of any game. The team that wins today can loose the next match and vice versa. But what differentiates a world champion from an average team is the extent to which they learn their lessons from their failures.
On a similar note, a software testing project can go terribly bad due to various reasons. I have witnessed and have been part of few such testing projects that went nowhere nearer where they were intended to go! But each experience, good or bad, has some lessons in it. A good tester tries to learn lessons from his past failures to make sure they are not repeated in future. The failure might not have been as a result of a mistake on the part of the tester, but still, there might be some lessons in that failure that could help the tester in becoming better at his testing in future.
If you have reached this paragraph after patiently reading through my post, then I must appreciate you for your patience. You have certainly shown a desirable skill of a good tester – patience. :) I would like to hear your ideas about my analogy between cricket and software testing. Can you think of more such points, which can link them together? Let me know your ideas by commenting. Howzat!!!
Happy Testing…
1. (Un)Predictability: Experts say, ‘unpredictability of cricket is its greatest charm’! It is very difficult to predict a win or loss before the last ball is bowled. And it is the unpredictability, which makes cricket so interesting. Teams might look strong or weak on a piece of pen and paper. But in cricket, unpredictability reins, not the statistics.
Coming to software testing, there could be testers who believe that it is easy and straightforward to expect some fixed output after executing a series of test steps, but I only wish; testing software was as simple as that! Echoing some experts in software testing: trying to predict the result of a test in terms of PASS or FAIL criteria can be one of those dangerous traps in software testing world, where a tester can shoot at his own feet! Irrespective of the number of test scripts (either manual test cases or automated test scripts) a tester has written, until the tester gets the application module to test, nothing can be told for sure about the state of the application and its behavior. Unpredictability is one of those things that make software testing so much fun.
2. Skills: Cricket is a game where only skillful players can make their team a winner. I am not sure about other countries, but if you have ever been to India, chances are high that you might have seen cricket being played on a street behind your hotel room! They say, cricket is a fever here in India. Here people are so obsessed with the game that they not only play and watch cricket but also eat, sleep and even drink cricket! I have played fair amount of cricket in my school and college days. You might have too. But the reason, why players like Sachin Tendulkar, Brian Lara and Ricky Ponting are considered as one of the finest cricketers and not us, lies in the cricketing skills they possess.
Likewise, in software testing too, it is the testing skill that differentiates a good tester from a mediocre one. The one, who has the better testing skills in his arsenal, can find more important defects quickly in the software that he is testing. Learning, practicing and applying are three golden rules to acquire any skill. With determination and strong will power, nothing is impossible to learn. Fortunately, software testing is no exception!
3. Game Planning (Knowing the Opponent): Professional cricket is all about knowing the strengths and weaknesses of the opponent team and devising a game plan in order to combat their strengths and to exploit their weaknesses.
In software testing, knowing the testing mission is the first step in determining the goal of the testing effort. Without being clear about the goal, it might turn out fatal to go about testing straightaway. Once the tester is clear about the testing mission, then he can analyze his chances of success or failure depending on the availability and expertise of the resource in hand and the complexity of the testing problem. e.g. imagine a situation where the tester has to test the application to find out it’s robustness to guard against hackers and other malicious users. Knowledge of a fair deal of details about the level of attack that can be attempted against the application, can give the tester a better chance to plan out a strategy to emulate the attack and to test how the application guards against it.
4. Handling Pressure: Winning a game of cricket is all about handling the pressure well. The team that is able to handle the tremendous amount of pressure of the game finally wins the game.
For those who think or have been told that software testing is an easy job and can be done by every Tom Dick and Harry, let me warn you, you have been terribly misguided! Testing is a career, which demands lot of intellectuality and stability of mind to work under variety of pressures (like technical pressure, pressure due to workload, managerial pressure, pressure of deadline, pressure arising from the nature of the job etc). As testers, the basic requirement of our job demands us to deliver bad news (presence of defects, buggy modules that fail miserably on testability grounds etc) to different stakeholders (the programmers, management staffs, clients) of the product under development. Nobody likes to hear bad news. Unfortunately, gone are the days when messengers were not hanged just because they brought in a bad news to the king! Hence, unless the tester is quite good at handling the pressure arising as a byproduct of his work, and is not so good at being diplomatic, he might find it tough to carry on with his job for long. On the other hand, the tester who has got the capability to handle the pressure till the end, has every chance of winning the Testing World Cup!
5. Adaptability: If you have been watching cricket for sometime now, then you must be experienced enough to know that whenever a team visits a different continent for playing cricket, it often finds it difficult to play up to its usual standards, of course unless we are talking about a team like Australia. Most Asian teams find it difficult to play down under and the vice versa. And the reason lies mainly in the difference in the pitch condition in the different continents. Soil texture, clay quality, humidity, temperature, amount of grass, dust etc can affect the behavior of a cricket pitch. The players who can adapt themselves quickly with the new environment can gain an edge in the game.
Just as no two cricket pitches can be exactly identical, no two AUTs (Application Under Test) can be same. Hence, I often get surprised when I see testers trying to clean themselves off testing assignments with excuses such as:
» “I have not tested anything like this before!”
» “This platform is completely new to me!”
» “I am new to this technology. I can not test it!”
» “I am a manual tester. I can not use this test automation tool!”
» “I need the domain knowledge before I can test this application!”
» “Hey, you don’t have any base documents (URS, SRS, BRS blah blah…)! How can I start testing your application?”
To me, this simply translates as a bad attitude of a tester, who is not ready to adapt and more importantly not ready to learn something new. For such testers, I have just one thing to say:
"Don’t forget that, even big and powerful animals like the Dinosaurs and the Wooly Mammoth became extinct just because they FAILED to adapt to their changing surrounding environments; we are just testers!"
When I say so, I DON’T mean that I am a great tester who has been excellent at adaptability throughout his testing career. On the contrary, I have been through several such instances when I was stupid enough to oppose such changes and was in fact, hostile to proposed changes. And don't be too surprised when I tell you that, some of the excuses that I have mentioned above were from my own mouth! However, I have learnt my lesson of adaptability in the hard, bitter way. So I just wanted to warn others who are new and passionate about testing but might commit the same mistake of lack of adaptability that I have committed in my earlier years of testing career.
6. Patience: Cricket is a game of patience. At times, a batsman might be finding it tough to steal a run due to the sharp bowling attack, tough ground conditions etc. But if the batsman has got enough temperament and patience, he might soon find it easy to not only steal a single but also score boundaries against the same bowler under same pitch conditions. Similarly with patience and consistent good bowling, a bowler could turn around the fate of a match by picking up quick wickets.
Similarly, a lot of patience is required while testing an application. There might be days when nothing seemed to work right for you and you would end up without catching a single defect in the whole testing session. There might be times when you would find yourself hitting your head on the keyboard while trying to find ways to reproduce a hard-to-reproduce defect. But you ask any expert in software testing, and he would suggest you to have patience. Putting in extra effort and having patience can help you come out of such dry days of testing.
7. Team Game: Cricket is a team game. Here the team wins the match, which displays excellent exhibition of team play. And so is software testing. All the team members of a testing team can’t be with equal skill sets and with equal level of expertise. Having a testing team with members with different specialization can add to the versatility of the team’s overall performance. I do understand that there can be testing geniuses, who are jack of all testing trades and are all-rounders when it comes to testing. But finding such a tester can be a tough task for any test manager. So as a workaround a good test manager might look forward to build a team which has testers specializing in different aspects of testing (functionality testing, performance/stress testing, database testing, usability testing, automation engineers, risk-based testing etc).
8. Lack of Concentration can Doom you: Cricket requires a high level of concentration. As a batsman you loose your concentration and you could loose your wicket. As a bowler you loose your concentration and you could be hit for boundaries all over the ground! As a fielder you loose your concentration and you could drop a catch (remember! catches win matches!) or give extra runs by mis-fielding.
Coming back to testing, imagine a tester who misses to identify a defect that just happens in front of his eyes, as he was looking for some other defect at that moment. Experts call this phenomenon as inattentional blindness. This can also happen if your mind is too tired due to continuous testing and you start loosing concentration, and in turn, start loosing defects!
9. Wearing safety Helmets and Testers: Batsmen (and even the close-in fielders) wear helmets to safeguard their head from the fast coming deliveries from the bowler. In case of software testing, unfortunately, the testers are like the safety helmet! They act as the representative of the client/end users of the project. And act as the last line of defense of the client/end users against the possible ways in which the software might fail. So as testers, start feeling proud of your profession, as you are safeguarding somebody by taking the hits yourself.
10. Learning Lessons from Failures: Even world champions like Australia was once defeated in a cricket match against the under-dogs like Bangladesh. Success and failures are two integral part of any game. The team that wins today can loose the next match and vice versa. But what differentiates a world champion from an average team is the extent to which they learn their lessons from their failures.
On a similar note, a software testing project can go terribly bad due to various reasons. I have witnessed and have been part of few such testing projects that went nowhere nearer where they were intended to go! But each experience, good or bad, has some lessons in it. A good tester tries to learn lessons from his past failures to make sure they are not repeated in future. The failure might not have been as a result of a mistake on the part of the tester, but still, there might be some lessons in that failure that could help the tester in becoming better at his testing in future.
If you have reached this paragraph after patiently reading through my post, then I must appreciate you for your patience. You have certainly shown a desirable skill of a good tester – patience. :) I would like to hear your ideas about my analogy between cricket and software testing. Can you think of more such points, which can link them together? Let me know your ideas by commenting. Howzat!!!
Happy Testing…
“If the software product has some defects, can it still be called a quality product?”
This is a question I was asked during a recent conversation with one of my blog readers (who is also a software tester). It appeared that the tester friend was asked this question in some testing interview and the interviewer had expected a simple YES/NO type answer! Unfortunately the tester, who was unsure about the answer, also expected a similar (YES/NO) answer from me for the above question. To me, this actually seems like a paradoxical question, which can be answered only after breaking it down further into few individual questions, which can throw more light on the context of the subject. But before I attempt to answer this question, let me assume few definitions of the keywords used in the question.
Software Product: A software product is typically a single application or suite of applications built by a software company to be used by *many* customers, businesses or consumers. The mass-market notion differs from custom software built for the use of a single customer by software development firms. – Wikipedia
Defects: A software defect (or "bug") is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended (e.g., producing an incorrect result). Most software defects arise from mistakes and errors made by people in either a program's source code or its design, and a few can also be caused by compilers producing incorrect code. – Wikipedia
Quality: Quality is value to some person, who matters. – Gerry M. Weinberg
Coming back to the original question, when someone asks, “if the software product has some defects, can it still be called a quality product”, I think a possible answer is: “I can’t tell you for sure unless you are ready to answer few of my questions first”!
If you are a software tester, I am almost sure that you are yet to see software, which does not have any defects in it (even after thorough testing). Even if the testers are not able to identify any significant amount of defects with further testing activity, still no tester with a stable brain on his shoulder would ever dare to state that “the software is defect-free after the testing effort”! In that sense, all software is with some defect (either known defects or yet-to-be-found defects). Since no software is without defect, what is the point in asking something like: "if the software product has some defect …"? What do you think?
I am not sure if the interviewer actually wanted to ask: “if the software product has some *known* defects *after it’s release*, can it still be called a quality product?”
This sounds to me like a more meaningful and tactable question which can be attempted for answering! As a software tester, I understand that there can be situations when a product is known to have some unfixed defects and yet the management can decide to release the product. This can particularly happen when,
a) The known defects are of low severity and are less devastating. Even if the end-user encounters such defects, it is speculated that the defects would not cause much trouble/nuisance to the end-user.
b) These are corner case defects and the chance of the end-user being hit by such defects is less.
c) The stakeholders of the project don’t feel that these defects are of much importance!
Whatever might be the reason behind the decision to release the product with those known defects, I am not sure if we can surely tell that the shipped product is of high quality or not, simply on the basis of the defects shipped with the product! Echoing some experts in software testing, quality of software is much more than presence or absence of defects. Imagine a software product, which is shipped with a belief that it does not have any known defects and is quite stable. But wait! What if the end-user does not like it, as it is difficult to learn and operate? What if, it is difficult to train others how to use the software? In a single sentence, what if the product fails severely from usability aspect? Think of a video game, which is very robust (does not crash easily) and lacks any visible defect, and yet the persons who play this game don’t find it interesting enough! Can we still call it a quality software/video game? I guess not!
Coming back to our original question, even though a software product has defects, it could still be called a quality product. At the same time, even though a software product has lesser known-defects, it might not be of good quality! Quality of software is multi-dimensional and can't be decided by the presence or absence of defects alone! Hence, it might be little tricky to answer the above question, without being clear on the understanding of quality by the particular organization. As a tester, do you think you can answer the above question? I am eager to hear your views on the subject.
Happy Testing…
Software Product: A software product is typically a single application or suite of applications built by a software company to be used by *many* customers, businesses or consumers. The mass-market notion differs from custom software built for the use of a single customer by software development firms. – Wikipedia
Defects: A software defect (or "bug") is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended (e.g., producing an incorrect result). Most software defects arise from mistakes and errors made by people in either a program's source code or its design, and a few can also be caused by compilers producing incorrect code. – Wikipedia
Quality: Quality is value to some person, who matters. – Gerry M. Weinberg
Coming back to the original question, when someone asks, “if the software product has some defects, can it still be called a quality product”, I think a possible answer is: “I can’t tell you for sure unless you are ready to answer few of my questions first”!
If you are a software tester, I am almost sure that you are yet to see software, which does not have any defects in it (even after thorough testing). Even if the testers are not able to identify any significant amount of defects with further testing activity, still no tester with a stable brain on his shoulder would ever dare to state that “the software is defect-free after the testing effort”! In that sense, all software is with some defect (either known defects or yet-to-be-found defects). Since no software is without defect, what is the point in asking something like: "if the software product has some defect …"? What do you think?
I am not sure if the interviewer actually wanted to ask: “if the software product has some *known* defects *after it’s release*, can it still be called a quality product?”
This sounds to me like a more meaningful and tactable question which can be attempted for answering! As a software tester, I understand that there can be situations when a product is known to have some unfixed defects and yet the management can decide to release the product. This can particularly happen when,
a) The known defects are of low severity and are less devastating. Even if the end-user encounters such defects, it is speculated that the defects would not cause much trouble/nuisance to the end-user.
b) These are corner case defects and the chance of the end-user being hit by such defects is less.
c) The stakeholders of the project don’t feel that these defects are of much importance!
Whatever might be the reason behind the decision to release the product with those known defects, I am not sure if we can surely tell that the shipped product is of high quality or not, simply on the basis of the defects shipped with the product! Echoing some experts in software testing, quality of software is much more than presence or absence of defects. Imagine a software product, which is shipped with a belief that it does not have any known defects and is quite stable. But wait! What if the end-user does not like it, as it is difficult to learn and operate? What if, it is difficult to train others how to use the software? In a single sentence, what if the product fails severely from usability aspect? Think of a video game, which is very robust (does not crash easily) and lacks any visible defect, and yet the persons who play this game don’t find it interesting enough! Can we still call it a quality software/video game? I guess not!
Coming back to our original question, even though a software product has defects, it could still be called a quality product. At the same time, even though a software product has lesser known-defects, it might not be of good quality! Quality of software is multi-dimensional and can't be decided by the presence or absence of defects alone! Hence, it might be little tricky to answer the above question, without being clear on the understanding of quality by the particular organization. As a tester, do you think you can answer the above question? I am eager to hear your views on the subject.
Happy Testing…
Friday, December 7, 2007
Testers can get and share information with developers, to improve collaboration and make better programs with fewer struggles.
While I'm not sure that the common saying "Testers and developers think differently" is accurate, I think it's safe to say that testers and developers certainly have different motivators for their work—and, more importantly, different pressures and perspectives that guide the directions their work takes. Testers are motivated by management to find and report problems within the system as quickly as possible, while developers are motivated to complete code as quickly and accurately as possible in order to move on to the next problem.
Somewhere in this jumble of speed versus quality, someone should be motivating testers to find better, more meaningful bugs, but I rarely see that motivation happening. And someone should be motivating developers to improve the quality of the code they create or modify, but that prompting often takes a backseat to "More code, faster."
Because of these different motivations of testers and developers, and the time pressures that everyone on the team is under, communication suffers. It's easy for a tester to think of a simple test that the developer should have run to find a particular problem before releasing the code—but the tester doesn't appreciate the pressures imposed on the developer. And it's easy for the developer to be dismayed by an overwhelming number of low-value or meaningless defects being reported—but the developer doesn't appreciate the expectations and metrics against which management measures the tester.
As more defects are submitted, developers have less time to work on them, and communication may break down. Members of the project team commonly start to rely on short, assumptive descriptions and comments entered into a defect-tracking system as the fastest and most effective means of communication. Little time is spent face to face, where the most effective communication can take place and dialogues can offer insights to both the tester and the developer.
In this article, I'll discuss how recent projects used the following simple techniques to fix problems faster and improve communication between testers and developers:
· Sharing test scripts between teams
· Distributing the ability to execute smoke tests
· Performing runtime analysis together
· Using log files to isolate problems
· Using defect-tracking systems effectively
· Speaking face to face
Not all of these techniques added "face time" to our project communications, but all of them helped everyone involved to gain a better understanding of the pressures and constraints under which both developers and testers work.
Share Test Scripts Between Teams
Problems encountered by test scripts can be lengthy or difficult to reproduce. In past projects, I've submitted defects that took hours to reproduce due to the sequence of events in the scripts being executed. More often than not, this setup requirement made it impossible for the developer to reproduce the issue based on the information in the log. If you make all of your test scripts available to the entire team, however, you give developers the ability to look at the script code, look at the script logs, re-run the scripts and watch them execute, re-run them in local environments with debug information written to logs, or re-run them in conjunction with other tools.
In addition to sharing the test scripts, provide the development team with a remote automated test-script execution box. Most enterprise tools allow for distributed test execution. If you provide one of your test lab machines to run your scripts, developers can execute the tests they need while simultaneously using their own computers to keep developing. This technique allows developers to work on your problem; without the testing box, the developers might not be able to run a full test due to time and equipment constraints. To make this strategy most effective, reference your scripts in the submitted defect list. The development team can run the scripts without checking with you first, removing a manual step.
Sharing test scripts with developers also enables everyone on the team to use the same tools used to develop the scripts. When team members use the same tools, a likely side-effect is developers taking the time to offer improvements to the scripts. After running one of my scripts, a developer once told me that he already had a unit-test script that did something similar "behind the GUI." Together we reviewed both scripts, and ultimately we transferred the data from my regression script to work with his unit-test script. His unit-test script executed in a fraction of the time of my regression script, and the results were easier to read. The more feedback from developers you get on your scripts, the more powerful they'll become.
In addition, the more you collaborate with developers, the more likely they'll be to fix your problem; after all, it may have been their optimization that found the bug. By distributing the ability to execute any of your scripts, you increase communication between developers and testers.
Testers vs. Developers
Assertions can be derived as soon as a specification exists, however in this code-based methodology, assertion localization does not occur until after code testability is measured. Thus since testability measurements must be performed before the assertions are derived, this process cannot be performed until later in the software development life-cycle.
However, knowing that, for example, variable a needs to be tested after assignment statement 100, does not indicate what constraint the assertion should be testing for (to determine whether a problem with a has occurred). Hence our methodology has only addressed one part of the oracle/assertion problem, specifically placement, not derivation. Someday we would like to link this tool up with a formal specification language, and automatically generate the assertions, but that remains a research goal for the future.
As mentioned earlier, testers are likely to be incapable of deriving correct assertions, while developers are more likely to derive assertions that mimic the semantics of the code already there. If the developer does not understand the specification, their assertions will be incorrect. That is to say that if a developer's code is faulty, their assertions will almost certainly be faulty.
This brings us to our final recommendation for how to team developers and testers in a partnership to strategically derive and embed assertions. Let the testers find where assertions are needed, and leave it to the developers to determine what the assertions should be. Note that this is different than the case where the developer determines both where to place the assertions and what they should be. Here, the developer is being forced to derive assertions that the tester needs for places in the code where the developer might not be as sure as to what the assertion should be. If this happens, it forces developers to dig deeper into the code and requirements than they might have already done. This plays a similar role as code inspections, except that the person digging into the code is also the person that is likely to have written the code. Although this is not a foolproof solution, it is the best recommendation that we can provide at this time. Having developers spend more time comparing their previous understanding of the code to the existing requirements can only serve to improve the code's quality. Even if the assertion that the developer derives does not detect an error, the fact the the developer is forced to derive the assertion will increase the likelihood that the developers themselves find errors, because they are forced to get more familiar with less familiar internal computations in the code.
Sunday, November 25, 2007
CMM-Capability Maturity Model
Capability Maturity Model:The Capability Maturity Model (CMM) is a methodology used to develop and refine an organization's software development process. The model describes a five-level evolutionary path of increasingly organized and systematically more mature processes. CMM was developed and is promoted by the Software Engineering Institute (SEI), a research and development center sponsored by the U.S. Department of Defense (DoD). SEI was founded in 1984 to address software engineering issues and, in a broad sense, to advance software engineering methodologies. More specifically, SEI was established to optimize the process of developing, acquiring, and maintaining heavily software-reliant systems for the DoD. Because the processes involved are equally applicable to the software industry as a whole, SEI advocates industry-wide adoption of the CMM.
The CMM is similar to ISO 9001, one of the ISO 9000 series of standards specified by the International Organization for Standardization (ISO). The ISO 9000 standards specify an effective quality system for manufacturing and service industries; ISO 9001 deals specifically with software development and maintenance. The main difference between the two systems lies in their respective purposes: ISO 9001 specifies a minimal acceptable quality level for software processes, while the CMM establishes a framework for continuous process improvement and is more explicit than the ISO standard in defining the means to be employed to that end.
The CMM is similar to ISO 9001, one of the ISO 9000 series of standards specified by the International Organization for Standardization (ISO). The ISO 9000 standards specify an effective quality system for manufacturing and service industries; ISO 9001 deals specifically with software development and maintenance. The main difference between the two systems lies in their respective purposes: ISO 9001 specifies a minimal acceptable quality level for software processes, while the CMM establishes a framework for continuous process improvement and is more explicit than the ISO standard in defining the means to be employed to that end.
What is CMM (SEI Capability Maturity Model)? The Capability Maturity Model for Software (CMM) is a framework that describes the key elements of an effective software process. There are CMMs for non software processes as well, such as Business Process Management (BPM). The CMM describes an evolutionary improvement path from an ad hoc, immature process to a mature, disciplined process. The CMM covers practices for planning, engineering, and managing software development and maintenance. When followed, these key practices improve the ability of organizations to meet goals for cost, schedule, functionality, and product quality. The CMM establishes a yardstick against which it is possible to judge, in a repeatable way, the maturity of an organization's software process and compare it to the state of the practice of the industry. The CMM can also be used by an organization to plan improvements to its software process. It also reflects the needs of individuals performing software process, improvement, software process assessments, or software capability evaluations; is documented; and is publicly available.
Testing Life Cycle Team Structure
- An effective testing team includes a mixture of members who has Testing expertise/Tools expertise.
- Database expertise/Domain/Technology expertise.
- Consultants/End users.
- The testing team must be properly structured, with defined roles and responsibilities that allow the testers to perform their function with minimal overlap.
- There should not be any certainty regarding which team member should perform which duties.
- The test manager will be facilitating any resources required for the testing team.
Subscribe to:
Posts (Atom)
