Sambhav Singh

Agile way of things – How should QAs test?

How many of my fellow QAs out there have freaked out working in fast paced AGILE projects? If I am not wrong, most of the time you’re being burdened with incremental builds to test for the upcoming release.

Gone are the days when things used to happen in waterfall development fashion. With hyper-competitive markets, stakeholders can’t wait for long to launch a working prototype of their product.
Customer habits and usability are changing rapidly with the advancement in technology, the same reason why end-user feedbacks have become prominent in initial stages of product development.


This is where Agile comes into picture. (And, it’s not just limited to IT, check references 1,2,3)

As David Kelley (Founder of IDEO) said: “Failure Sucks, but Instructs“, so the problem is not with failing, it’s always good if you can grab some positives out of it. Agile Development let us perceive how our product fares in the market in a very short span of time. And if we do fail, we fail quick and improvise to get an improved version out instantly. But, with all this hullabaloo of Agile ways of development, QAs are having a hard time dealing with it.

What we intend to achieve from this blog?

To figure out the best practices to be followed by QAs working in Agile environment.

What kind of products are we talking about?

The products wherein the requirements are fuzzy or when we’re helping a visionary build an MVP in which we all are ideating things for the first time to test and see what works.

What are the common problems faced by testers in Agile?

  • QAs are over-burdened with tightly scheduled releases/builds planned every week/fortnight in Agile.
  • Change requests are a common thing, which come in through feedback from target audience. Since Stakeholders are also continuously learning and improvising in AGILE, QAs are in turn catered with minimal requirements which makes it hard to test the stories under consideration.

Why do these problems occur?

As mentioned earlier, since the products we’re talking about have volatile requirements, and hence multiple iterations, we QAs have work galore with as many testing cycles and unfortunately limited amount of time.
Although, if we’re dealing with projects which have rigid and in-shape requirements, the to and fro testing cycles for QAs are somewhat reduced.

How to help QA(s) overcome these problems?

1. Be Proactive & enthusiastic
  • Fill your Coffee mugs and get to work, there’s no place for drowsiness anymore.
  • Unlike traditional ways of development, you’ll rarely have any buffer time with you in Agile (new features to be coded, tested, bugs fixed and deployed in a week or so)
  • WORK HARD PLAY HARD should be your mantra.
2. Be part of the process from the start (POCs/ MVPs)

As per Wiki, the term MVP was made popular by Eric Ries (Author: The Lean Startup)

  • Be involved in the Design meetings (to check for usability concerns)
  • Be involved in the Stakeholder meetings or any Agile planning meetings (to come up with all impacted areas in software and showcasing roadblocks through these in the initial phases itself)
  • Ensure that while Sprint Planning, the UI and UX of picked stories have already been decided upon, approved and signed off by Product owners. This helps avoid delays due to design changes later in the sprint cycle.
3. Be able to decide upon a good Acceptance Criteria
  • Good Acceptance Criteria will help get your Agile project from “It Works as Coded” to “It Works as Intended.” (Read more at: What Characteristics Make Good Agile Acceptance Criteria?)
  • We should articulate the acceptance criteria together with Devs and Stakeholders keeping in mind the intention of the story.
  • QAs Beware. Don’t let the White box tester in you overlook the Black box aspects of testing.
4. Be Precise while Testing

As nobody’s perfect, there’s always scope for improvement. Stakeholders agree that at the price of delivering builds so quickly, we can’t assure that they’ll be perfect.
So, once we’re done with Smoke testing, we should focus on testing stories strictly as per acceptance criteria and try and knock off happy flows asap (a must).

5. Be ready for Quick Automation
  • Since we are learning and improvising in AGILE intermittently and builds are shot to QAs quite frequently, we need to incorporate automation for Regression/High Level scenarios (at least).
  • One of the best ways is to have a BDD Framework like Cucumber set up for your project.
    Cucumber –> Fast, Easy, Understandable by all (English like language (Gherkin) is used), Feature Files (serve as Test Cases and let us track coverage)
  • For POCs we can also use Record & Play tools (like Selenium IDE, IMacros browser addons) to create small test suites quickly.
  • If your application aims at serving loads of users concurrently, you can try exploring some upcoming performance test tools like Taurus, which gets up and running in minutes using simple YAML files for configuration of load tests.
6. Be one level ahead in Testing
  • We need to catch issues fast- to do so, we should go a layer beneath UI Testing i.e. APIs. We can’t wait for issues to show up on UI and hence should thoroughly test APIs for all kinds of inputs and outcomes. (Useful tools: Postman, JMeter, Swagger)
  • Most data/rendering errors can be avoided just by testing APIs and APIs are going to be uber famous in future because of Cloud Computing and IOT trends.
  • This practice reduces the testing effort by a large percentage as defects are identified way before the code renders on UI.
7. Raise your voice
  • Give justified story points as per actual QA workload (tracking all the dependencies for a new story) rather than agreeing with fellow teammates blindly.
  • Better estimates result in timely releases.
8. Suggestions are most welcome
  • We can suggest doing AB testing of modules if we are unsure of how end users might react to a campaign.
    A/B testing (sometimes called split testing) is comparing two versions of a web page to see which one performs better. You compare two web pages by showing the two variants (let’s call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, wins! (Read more at: The Complete Guide to A/B Testing)
  • This let us do a market research of what’s the most favorable of two things using the method of: Try & Decide.
9. Be the Build Man

Or you may get so tired, tired of waiting, tired of waiting for builds…) ~ The Kinks 🙂

  • QA roles are re-shaping very quickly; we should also know about the Continuous Integration (CI) tools (like Jenkins, TFS, Chef) because builds are improvised upon very quickly and we are working on so many platforms at the same time: Web, IOS, Android, Chromium in Chromebooks.
  • Devs also should make a habit to use CI tools to broadcast builds throughout the team.
10. Be smart
  • For products aiming cross browser releases, we can make use of tools (like Browsersync, Saucelabs) to achieve parallel testing on multiple browsers.
  • For Cookies Testing or to detect Security compromises in our product, we should be familiar with debugging tools (like Charles, Fiddler) to track the Request/Response pairs.
  • Always keep looking for ways to save your valuable time by utilizing tools and automation.

Related Articles


Our Development Process

Can great products be built without having any solid development processes I don’t know but I can confidently say that it’s a definitive topic for a thesis Well, we decided to do away with all the development... Read more

Some Misunderstandings about Agile Process

I’m an absolute fanatic about delivering products via an Agile Process That wasn’t always the case I wasn't a believer earlier but I’m now We primarily follow Scrum in our organization and can totally vouch for... Read more
  • Shalvika Sood

    Very good observations Sambhav. QA role is definitely morphing into multiple aspects instead of being limited to just testing. In Product companies they are fast becoming the Product knowledge champions. Just wanted to add that QA folkes should be careful of one thing while defining (or refining) Acceptance Criteria. One must always watch out for ‘over documenting’ – it should not morph into Test cases. I mentioned QA specially because that is an occupational hazard

    • Thanks for acknowledging Shalvika.
      And I second your opinion on Acceptance Criteria (AC), we should know that Acceptance Criteria and Acceptance Tests are 2 different things.
      I can tell this from personal experience, for there have been instances when while refining the AC, in the flow, I ended up documenting the Test Cases in the story itself (the things that I had to regress around but they were in no way directly related to the story).
      We should refrain from doing this and AC should just be a minimal documentation defining when can we mark a story as “Done”. Although, it might include edge cases/ scenarios for any impacted areas while implementing the story.

      • Shalvika Sood

        Yes. There is another term for developers – brain dead! If you write down everything, everything, then where is developer intelligence?! AC is usually the basic highlights for success, sometimes a little about the user interaction if it’s required. Failure conditions are rarely mentioned, unless the failure condition is a primary feature point. So ‘screen should show only to Admin’ is an acceptance criteria, but ‘Screen should not show to user, manager, supplier’ are test cases

  • Vivek Tewari

    Sambhav this is awsome and helpful !! Great going

    • Thanks Vivek.
      Good to know you found this useful.
      Subscribe to Humble Bits for more such blogs and insights. 🙂

  • Aman Srivastava

    Thoroughly put Sambhav. Nice work.

    • Thanks Aman.
      Hope you were able to grab some positives out of this. =)

  • Regarding the Record & Play that you’ve mentioned, Selenium IDE is by far not the best option. Selenium IDE has quite a few problems, and, besides, there are quite a few worthy Selenium IDE alternatives that you can choose from:

    • Hey Ilya,
      Thanks for stopping by and giving this a read. 🙂
      Actually, I mentioned Selenium IDE for creating small test suites quickly for POCs wherein we are short on time and just need to verify happy flows of whatever we’re presenting to the stakeholders. I had used this a couple years back for a Collage site and it served it’s purpose.
      However, I know of it’s shortcomings as mentioned in the blog you shared.
      And, if we’re concerned with “No Coding”, “Automatic Management of Waits” and “UI Automation” features, those options seem promising. I would surely try Screenster sometime, I’m assuming it works on image objects somewhat like Sikuli. And the fact that it has AI incorporated in it, to deal with result comparisons is intriguing. Thanks for bringing this up. 🙂