Tuesday, May 5, 2015

Quality Assurance Vs Testing

A couple of weeks back, we were doing training for a team. We talked at length about the benefits of team members who specialize in one discipline, moving around to help other disciplines. As I was explaining the role the test engineers play in this regard, Frank Vega(https://twitter.com/fxvega) mentioned something that put it a lot more succinctly. Frank, who was helping me out with the training mentioned that what we are talking about is the difference between testing and quality assurance. Let us draw the distinction more clearly.

Testing as defined by Wikipedia(yes, the source of all truths) is - Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Testing has the connotation that most or all the code for the piece of software being developed is already written. Piece of software might mean the entire project in a waterfall world or just one story in the agile world. In essence, the act of testing can begin only after the code has been written. There are many techniques that make this process effective. Test case design techniques like pairwise testing, equivalence classes, automated tests at different levels can make this activity a very profitable one. Many testers can even open up the code and write tests based on the code they see. White box testing is very effective at finding flaws and writing well guided test cases.

On the other hand Wikipedia defines Quality Assurance as a way of preventing mistakes or defects in manufactured products and avoiding problems when delivering solutions or services to customers. Quality assurance makes sure that quality is built into every artifact and every part of the process. Quality assurance begins from the writing of requirements and does not end till the shipping of the code. Reviewing/collaborating on requirements, code, test cases and automated tests are all activities that provide assurances at multiple levels that bugs are being avoided. Not saying that the actual execution of test cases is not a great source of quality assurance, but often we waste our best quality assurance minds on pure testing activities, rather than elicit their help in other quality assurance activities.

While testing is a product oriented activity, quality assurance is a process oriented activity. Testing is meant to find defects so that the problems can be resolved, while quality assurance is meant to prevent defects so that the problem never arises.

Similar to the concepts mentioned in the last post on this blog, QA folk have a much larger role to play than just testing code for the product. Most of us are familiar with the exponential increase in the cost of bugs the later it is in the development process. Getting your quality engineers to review and/or contribute to requirements could help better define stories and more often than not avoid bugs from being introduced even before any code is written. Getting QAs to sit with engineers when they are writing  code can help the developer understand the testing strategy and avoid writing the defects in the first place. Very often I have observed QAs be the bridge between the business language of the product analysts and the technical jargon of the software engineers.

This is very different from the traditional perspectives of "testers should test". How many times have we discovered that the requirements were written with assumptions that were invalid?  Having a QA review these even before the coding for a story started would have not only prevented the defect/rework from being introduced, but would also have prevented context switching for the devs and the analysts who might have moved on to the next story by this time. I am in no way suggesting that actually testing the coded piece of software is a useless activity. What I am suggesting, is that if all your QA folks are doing is writing and executing test case and automation, that is a waste of their talent.

There is still specialization to be craved in the discipline of developing automated test suites. You still need the more technical type of tester that is great at developing test suites that test the product well and prevent regression defects from being introduced by future stories. These folks can still review requirements, open up the code to develop test strategies and most importantly they can fix the defects that they find without ever sending the story back to the engineer(s) that worked on the story. Once again, preventing context switching and keeping the flow of stories going. The test engineers in this case provide Quality Assurance by being a lot more involved in the coding of the product than in the writing of the requirements.

What about the other roles on the team and their contribution to quality assurance? Product Analysts can and should help test stories as much as testers should help review/write requirements. Engineers should be involved in activities like code reviews, writing unit tests, developing other automation test suites and using their skills in the best possible ways to ensure that a quality product is being produced. The engineers sometimes have the best handle at assuring and testing quality as far as performance, deployment and security concerns go. They might be able to contribute to, requirements, code and test creation, especially in these disciplines.

None of this is revolutionary, we have always asked our teams to produce the highest quality products possible. The problem is that we have tried to do this via a heavy emphasis on testing our products rather than performing quality assurance activities on our products. I feel very lucky to work with some great quality assurance folks on my team alongside product analysts and developers who do their part in ensuring that they have built quality into the product rather than throwing things over the wall at the "tester" and making it their problem.

Wednesday, March 11, 2015

Not My Job And The New Definition Of The Superstar

Recently, I attended a training course in "Fierce Conversations". The course is based on the book by by Susan Scott that bears the same name. A very worthwhile course for anyone managing and/or leading a team. There was a lot to learn about the different types of conversations and how to go about having them in order to achieve the best results. The conversations under the coaching and delegation sections were specifically of great interest to me.

As a part of the course, we had to identify our trigger words. Words that make us get upset when having a conversation with someone. Beyond identifying these words, we were asked to pair up, share our trigger words with our partner and then use them with each other in conversation. My partner got through almost our entire conversation without reacting, but his last trigger word almost made him flinch. When it was my turn, I found out how much my trigger words have changed since I started working with teams that are trying to embrace Kanban and flow. I always thought the trigger word that affected me the most was "Calm Down", but I quickly discovered that the phrase "That is not my job" evokes a stronger reaction.

"Not my job" is the enemy of flow. What it means is that you are more interested in living in your box and working on what is your area of expertise rather than helping the team out where the flow of work is getting interrupted. This means programmers helping out the QAs, if the testers are getting backed up. This means QAs helping out the BAs if there is not a lot to test, but requirement definition is moving slowly. People moving around till everyone becomes a utility player who can plug and play in any area is, in my opinion, the future of teams.

This does not mean that we completely get rid of specialization. You will have folks who are primarily developers and those that are primarily UX designers and those that are primarily QA, but everyone on the team picks up skills in every other area. This means that resources can be moved around to free up any blockages that might be propping up in the team's flow. If an individual works in only one discipline, they are great at what they do. These same folks are rewarded very well for being good at the one discipline they have, which in turn leads to creation of a certain type of Superstar. Unfortunately this definition of a Superstar, while great for a team working in a waterfall model, is counterproductive to a flow-centered agile team. It seems that a lot of organizations have moved to being agile shops, but still reward waterfall Superstars.

What we need is a new definition of the Superstar. The superstar in a flow/agile world is not a master of just one discipline but an expert or almost expert in multiple disciplines. An amalgamation of multiple skills that can help the team regardless of what specialization the team needs help in. The new superstar is not a tester, an analyst or a programmer, he/she is just a developer. A great one at that. One that can help out the team to effectively maintain flow and remove blockages. A Superstar who not only helps the team finish work, but also pairs with every other team member to help them learn and to learn from them. I have been lucky to work with a few individuals that have been Superstars and the team is just rocking and rolling when they are in stride. Rewarding this behaviour and encouraging it is the key to making sure that it becomes more prevalent.

Think of it as a video game where your character's stats determine it's effectiveness. Lets say our players(developers) have three types of statistics - Analysis, Programming and Testing. Now imagine a player whose stats are along the lines of Analysis = 6/10, Programming = 7.5/10, Testing = 7/10. For me, this individual is clearly more valuable than the Superstar from the Waterfall era. They can help the team in the majority of the work the team deals with in all respects. If the business analysts are stuck then our all-rounder can help get some stories ready for the team. If the developers are stuck and the testers are idle, the all-rounder can code some stories and make them available for testing.

The stats of a waterfall era superstar would look more alone the lines of Analysis = 1/10, Programming = 9.5/10, Testing = 2/10. This person can code a lot of stuff, which can lead to a backlog in testing, but cannot help remove that backlog. This can make it hard for the team to achieve the ideal flow efficiency.  These experts are not necessarily all bad news. Experts are great for guidance and knowledge transfer. Pairing them with individuals who need their skills further developed, might be the best way to get those two things accomplished. Experts can be very bad for the team as they can create knowledge and skill silos. The more we let them work in their ares of expertise by themselves, the deeper those silos get. The trick is to not reward them for their individual contributions in their areas of expertise but for how much they have been able to bring the team's average statistic in their area of expertise up.

The idea is for everyone on the team to hone their all-round skills. There is a great amount of flexibility, agility and flow efficiency to be gained here. Some of us are better at gathering requirements, others at programming and others at testing. We all have abilities in every other discipline and in an ideal team, we will all be at level 10 in all disciplines. That might be a pipe dream, but, just like everything else we can strive for the ideal. As long as we keep raising our levels in all disciplines, we are making progress towards the new agile Superstars.

It all starts with hiring the right people and training the existing ones. Your job postings should look for developers who work on teams, rather than BAs or Programmers or QAs. Hire people who love to learn and who love to be involved in all aspects of the story. Train, encourage and reward your existing team for cross functional expertise and dissuade silos. It is time we start hiring team players and not superstars. People with the urge to grow, and the desire to help everyone else grow. There will always be the very few who would go into the expert role, but those should also be team players who are willing to teach.

The old definition of the Superstar needs to be removed from the books. It belongs in the waterfall era where churning out work in just your discipline was the key.  We need the new definition of the superstar to become the norm. Reward people for their versatility and their learning and teaching abilities. It is time for us to move our personnel and our hiring practices to the next stage, where we have moved our teams. We need to hire people that increase the agility and cross functionality of the team, not those that might hinder it. Hire, encourage and create the Superstars of the agile era teams.

Thursday, February 26, 2015

When Process Junkies Are Parents - Personal Kanban

It all started a fine Thursday evening when my wife and I got home. Our daughter was in trouble with her grades and was falling behind on her schoolwork. We sat down and started reviewing where she stood with everything and what problems she was having and quickly figured out the issues. She had too many things going on at the same time. She was not sure about what to prioritize. There was no certainty whether the items she believed to be done, were actually completed.

Earlier in the day there was a Kanban training for a team that I was helping conduct and a lot of the issues/solutions we were talking about in the training seemed to be similar to the ones that we were talking about with our daughter. My wife and I, both running Kanban teams, we had an obvious solution to all problems - "You need a Kanban board". The following was the result.

Miranda(my daughter) seems to love it. Her WIP limit on the Doing column is 2. This helps her concentrate on fewer items at a time and also helps her pull the most urgent ones forward. Everything goes through a parent sign-off before it is considered closed. This helps ensure that everything that we consider done has been reviewed for completeness and quality. Miranda gets a constant visualization of her progress on tasks and so do we. There is no mis-communication on whether a task was finished or not.

Watching Miranda achieve productivity with her board made me think of my own problems with prioritization of work items. I have numerous things pushed onto my backlog and I try to get too many of them done at the same time. The solution again was obvious - I needed my own board. I would love a physical board, but since I work on items both at the office and at work, that was not going to work. Physical boards are great information radiators(exactly what Miranda needed), but are just too cumbersome to carry around if you need them in two places. So, I found an app that worked with my favourite browser, Chrome - Kanban-chi and have been using it since. As you can see this post is, as of right now in my Doing column.

I have to say, it has come the full circle. From the things I have learned working with my team to what Dan Vacanti,  Mike Longin and myself talk about in Kanban training to the working needs of my daughter back to how I can best manage my work. Sometimes, it seems, we know the answers, it just takes questions that are closer to home to help see the right context. I am sure I will still have some issues managing flow on my work items, but making consistent progress should not be a problem. I recommend personal Kanban to anyone who is willing to give it a try. If it can work for a 20-things-thrown-at-me-a-day-middle-manager and for a school going teenager, it can work for anybody. Also having a Unicorn and a couple of Pandas(I think thats what they are) drawn on your board helps as well.

Tuesday, December 30, 2014

Retrospectives - Objective Vs Subjective

Traditionally Agile teams run retrospectives to facilitate learning, development and improvement of the team. My teams have been doing the same over the past ten years. We have tried multiple different formats to get the right outcomes. We have been through it all and we have all been through it - lack of participation, finger pointing, irrelevant topics, lack of prioritization of issues, many a time one of these problems or another hinder the effectiveness of a retrospective. Recently my team moved to a more objective retrospective from a completely subjective form of retrospective.

Subjective Retrospectives

The most common form of retrospective employed is the subjective form. This shows up in numerous variations. It consists of collection and listing of ideas/topics, prioritization, either by moderator or by the participants via voting, discussion and finally extraction of action items. This method, as you can see can range from almost autocratic to a fully team driven setup. There is a middle way, which I like to call guided setup where the facilitator coaxes the team along through the process.

Autocratic Guided Team Driven
Team Lead/Scrum Master/Coach comes up with list of items to be reviewed during the retrospective. Team Lead/Scrum Master/Coach helps seed ideas for discussion and encourages team members to add topics for discussion. Team creates list of items to be discussed with every one having equal access to introducing topics.
Team Lead provides the priority of topics to be covered in the retrospective.Team Lead helps the team prioritize the topics to be discussed.Team votes on the list that they have prepared and generates a prioritized list.
Team Lead presents his/her thoughts on the topics and prompts discussion/feedback on these topics. Team goes through the topics in priority order and Team Lead helps seed discussion if no discussion seems to emerge organically. Team goes through the prioritized topics and discusses/reviews each item to generate next steps.

Regardless of which of these subjective retrospective methods are used the most important piece is to derive action items and to have individuals be responsible for the completion of these. From a casual reading of the above table, it would seem that a team should never use the Autocratic model, should sometimes use the Guided and almost exclusively use the Team Driven approach. That maybe ideal, but unfortunately, is not real. Depending on how mature the team is, any of these approaches would be appropriate. A team with little agile experience might actually benefit a lot more by employing the Autocratic model as they might need more externally imposed discipline and help identifying the issues and the priority of those issues affecting their agility.

Objective Retrospectives

My team has started using metrics for retrospectives. We look at our scatterplot which plots our stories along the axes of dates and days taken to close the story. We then, talk about the stories that are the outliers on the higher end and figure out why they took longer than other stories. This ends up becoming a very direct discussion about why things are taking longer than they should and exposes process inefficiencies very explicitly. I found that this helped the team identify direct impact action items that improve the team's efficiency. For example in the plot below, the stories in the red circles are the ones we would concentrate on in the retrospective.

Objective retrospectives do have the side-effect of the team trying to game the system to make the numbers look good. That, in my opinion is not a bad thing if you choose the metric you are using well. I find days to finish a story to be a good metric as in order to make it look better, your direct options are to either create smaller stories or reduce the number of items you are working on. Both of these are positives in any agile system. Lets say you chose a different metric, velocity for example, gaming the system there could be achieved by overestimation of stories. The idea is to pick a metric that reveals inefficiency and making that metric better improves the team overall.

We did notice an issue with the objective retrospectives. There are certain things, that dont show up as data points on the scatterplot. There are both positive and negative subjective comments that are left over to be covered after the objective retros are done. The team going forward is now doing a split review where we start with the objective and cover what is slowing down progress and then do a shortened subjective Team Driven retrospective to cover any other topics that need to be discussed.


Regardless of the method you employ, it is the willingness of the participants to improve as a team that is critical. That is why sometimes the Autocratic or Guided subjective retrospectives might be the right place to start for teams new to retrospectives as a tool for improvement. Objective retrospectives make sure that you talk about everything that is slowing down the team's work. The priority goes to the issues that create the greatest slowdowns and you can easily see the results the next time you review the data. Subjective retrospectives still have their place, but a mature team should be able to derive most of the value out of a retrospective via the objective method.

Wednesday, September 24, 2014

Are You Experienced - How does UX fit in with an Agile team?

UX has in many cases been an afterthought on fast moving projects. There are distinct ways to approach the integration of the user experience process into the product development process and we will explore a couple of these here. To get more insight, I interviewed two UX designers that I have worked with to get feedback on how two different styles of integration affect them, the teams and the product as well. Details of what seems to work and what does not are presented below.

The least effective approach towards UX that I have observed is the retroactive approach, where the rest of the development process is agile, but interaction with the user experience group is in a waterfall manner. Team produces a product as the business analyst and/or product owner specifies it and then it is sent over to the User Experience group for evaluation. UX makes recommendations and based on the time available in the project or whether the dev team has moved on to other features, the recommendations are implemented or put on the backlog for "future".

The second approach is also that of UX as a service but more in step with dev teams. The UX group advises the analysts and works with them during the initial design phase of the project. There is a primary resource assigned to the team that can be reached out to at any point of the development process. The primary resource attends sprint reviews and gives feedback to the product analysts, if they see any need for a design change. This is a much more "agile" approach as it makes it possible for the team to get feedback, at least at the end of every iteration.

The third approach is the one where dedicated user experience resources are a part of the team. They are embedded and function just like every other member of the team, being involved in stories from kickoff to reviews. They are able to provide constant feedback and get an idea of the technical constraints the team is working under. At the same time the dedicated resources are able to provide a slight paradigm shift in the mindset of the team. The entire team can benefit by becoming slightly more user focused as that is the perspective the user experience folks use to look at the product.

I talked to a couple of folks that I have worked with to get their feedback on the last two approaches outlined here. Lauren Martin and Bruno Torquato agreed to participate and help me out in drawing out the distinction in outcomes from the two approaches. Here are the advantages of both approaches as outlined by them.
  • Advantages of UX as a service
    • Creative Inspiration  - Lauren points out that being in the same space as other UX designers helps promote an environment where since everyone is involved in a similar creative activity, creative inspiration flows a lot more freely. This is harder to attain in an agile team setting where you are separated from the broader UX team/department.
    • Mentoring and Sharing - Working as one group providing a UX service, allows for opportunities to mentor less experienced team members and also the ability to share ideas and discoveries. "Cross pollination of ideas" according to Bruno is more natural and less forced as it has to be with UX being embedded on agile teams.
    • Planning and Execution - Lauren feels that it is easier to create plans and help with the long term execution of these plans. Also there are fewer ad-hoc requests from the team that could potentially derail the longer term plans,
    • Objective Perspective - "Outside perspective can give you space to provide insight", says Bruno. He believes that the deliverables that he provides are not "colored by the immediate needs" of the team when working in a UX as a service mode. It allows the designers space to provide insight that might be lost with the embedded perspective.
  • Advantages of Embedded UX
    • Greater Visibility and Understanding - There is increased visibility into the processes used by the designers. Lauren believes this helps the "team have a broader understanding" of what designers do and how they get to the resulting deliverables.
    • Team Ownership - Embedding user experience designers in the teams helps the team own the user experience as a collective. This enables Bruno to disseminate his "perspective and have the rest of the team take ownership" of the design as well.
    • Realistic Expectations - With the increased visibility and understanding come more realistic expectations from the team. It becomes easier for the team to understand whet the designer is shooting for. Rather than finished full scale designs being passed to the team, designers can work on a small scale where expectations are clearer.
    • Immediate Iterations - Desk checks and story reviews help make sure that design can be iterated upon quickly and at times almost as soon as it is implemented. Sometimes, this is before it is implemented, if the team has technical limitations or any other obstacles that come up, the designer can help remove those or modify the design as needed.

Both Lauren and Bruno made excellent points that can not be summarized simply as advantages of these systems. They both at this point work as embedded designers an agile teams. Lauren remarked how when being embedded, at times "creative flow is almost impossible to attain". On her team, the two designers have had to split responsibilities between "just in time and work ahead" responsibilities to make sure they can maintain creative flow and respond to team requests.

Bruno makes a point about the being within the team lays "more importance on developing relationships within the team" and the fact that the dynamic within the team  can help with having more momentum with the project. Another advantage of team specificity mentioned by Bruno is being able to adopt process to product , i.e. use the ideal process based on team and product type.

Last bit though, my favourite quote from the interviews -
"It still depends on the team, and the system does not matter as much"- Lauren.
Rings very true, raising the team's consciousness is more important than the system the UX folks work in. 

Wednesday, August 20, 2014

Leading Agile Teams : Effective Process Improvements On Teams

This is the last post in the Leading Agile series based on the four interviews with team leads at my organization.  Below are the responses to the question "What changes have you made in processes on the team that have helped the team become more agile and/or productive?".  There were multiple improvements pointed out by the participants, but the graphic below shows the ones that they considered to be the ones that had the biggest impact.

Other changes that the interviewees mentioned as having an impact -
  • Involving UX designers earlier in the process
  • Switching to a purely electronic board from a physical board
  • Having cells own story creation
  • Developing leadership within the team
  • Moving merging of code from separate departments much earlier in the process.
  • Blurring the lines between teams that share the codebase.

Finally, a heartfelt thank you to the interviewees for their participation. It was a lot of fun putting this together and I have learned a lot through the interviews and the reevaluation of our conversations. Thanks Again.

Tuesday, August 12, 2014

Leading Agile Teams : Conditions Teams Thrive In - II

In the last post we saw the infographic that included comments from four of the Team Leads that were interviewed for this series of posts. Here are some thoughts on the responses that were gathered.

Yvette Gaitan

"When the team has the proper tools available to do its work. Adequate communication from upstream teams about changes so that we are better prepared. Communication and team-wide awareness of the correct priorities in a space where priorities are constantly changing."

Yvette's team is often downstream from numerous other development teams. That explains the emphasis on communication both within and without the team. Agile environments are characterized by change and the ability of the team to adapt to such changes. Yvette's team seems to do this well as long as the priorities have been well communicated. They seemed to have understood that change is inevitable, but getting a heads up puts them ahead of the game.
Specialized tools that take the tasks that the team performs and automates them are always beneficial and welcome, regardless of team, methodology or industry. If the team is in unique situations and is given the bandwidth to come up with tools and processes that make them more efficient, the effects can be long lasting.

Michael Longin

"Informal attitude in the team room, balanced with structure through formal plannings, reviews and retrospectives.  Knowing that we are set up for success, but failure is OK.  Provide a safe failure environment and help people while they are failing.  Feeling of influence in the product."

A team working together day in and day out develops its own personality. For the team to develop both informal interactions to cope with the high pace and formal attitudes to achieve structure as Mike points out. Mike talks about a safe failure environment and contrasts it with being set up for success. This is important for the team and the organization as a whole. This means that plans are set up so that the team succeeds and there is no organizational backlash if the team happens to fail. Another important snippet here is "help people while they are failing", as signs of failure and opportunities to avoid/readjust are often present long before failure actually happens. 
Most development teams I have talked to mention this last point of having influence in the product. This is probably the single most empowering notion I have come across when talking to my colleagues. Influence leads to ownership, which helps make the team excited about the problems they are solving and the solutions they are creating.


"We work well under stress, specially when something has gone wrong. Team closes a lot of stories towards the end of the release. It helps the team to see where it stands in comparison to other teams."

Here we see a team that is under pressure and as a result has learned to deal with it by coming together and pushing hard. When things go wrong, teams come together and tackle the problem head on. This team seems to do that. What might be missing though, is the same coming together to come up with processes to prevent things from going wrong. The team here also seems to have a competitive spirit and takes pride in excelling especially in comparison to other teams in the organization. 
While this team is obviously working hard and producing results, there is something amiss in release planning for the team if end of release is when the pace picks up. This is not ideal. Teams should not need stress to perform well, in fact, stress, although sometimes unavoidable should be prompting reflection and change in team behaviour.

Mariana Sabogal

"Having a collaborative Product Owner is a great help. Team's feedback taken into account during planning, specially the technical perspective. This gives the team feeling of ownership."

Mariana reiterates the point that Mike has also made. Team having an influence in the product empowers them and gives them ownership.  Making technical feedback a part of the planning is a great way to get technical debt prioritized as well. Both Mariana and Mike talked about being closer to the customer and having input into the product as being a great components of getting the most out of teams. It would be good to clarify here that these are independent observations.
The importance of a collaborative Product Owner cannot be undervalued as Mariana notes. There are ample examples of a non-involved or no-collaborative Product Owner running teams off-course and being an impediment instead of a problem solver. Mariana feels that whenever she is working with a Product Owner that works well with her team, performance and productivity of the team is very high for that time period.