Chris
Chris Software engineer. WePay, LinkedIn, PayPal. Co-author Apache Samza, PMC Apache Airflow.

Yes, you still need a QA team (but not for writing your tests)

Yes, you still need a QA team (but not for writing your tests)

I was having lunch with a friend of mine recently where we were talking about a discussion he’d been having with a QA manager that he was interviewing. The discussion was about the role of QA in a modern engineering organization. The candidate had responded that they viewed their role like that of security. I found this really thought provoking, and I want to unpack it a little bit.

The best place to follow me is on my mailing list. Get new posts and recommended reading every Friday.

Security does a lot, but there are three areas in particular that I think are relevant to QA:

  • Setting and enforcing policy
  • Security testing
  • Tooling

The security wing of an organization is often responsible for setting and enforcing security policies. Everything from supported encryption algorithms to data access policies to supported third party software. QA should be doing the same thing, but with a focus on quality. What are the appropriate policies to get a product to an acceptable level of quality? SLA times for defects, code coverage policies, code smell policies, testing requirements and the like are all in scope.

The second area with some relevance is in testing. Note here that I pre-pended the word security. Security engineers do not write your unit tests. Nor do they write your integration tests. They do, however get involved with black box (or white box) penetration testing and security validation. They run scans on your network, work with third party penetration testers, and such. QA should be doing something similar, but for the products being built: functional testing. Functional testing should be done at the product-level (as opposed to service level). This looks very similar to black box penetration testing, except we’re validating that a product works as expected, instead of checking whether it’s secure (as expected).

Finally, tooling. In the realm of security, this might include libraries developers can use to encrypt their data, or services that are responsible for vending secrets. In QA-land, tooling should include things like comparing logs or metrics when rolling out a new canary or blue/green deployment against the older version(s) of a service. It might include performance tests suites for running tests against internal infrastructure, or test frameworks (such as ducktape) for running distributed tests. Production infrastructure such as Chaos Monkey can also be run by QA.

Tooling can also include portions of the CI/CD pipeline. For example, QA might own test infrastructure such as Jenkins, CircleCI, TravisCI, TeamCity, etc. They shouldn’t own the actual tests being run (that should be the dev teams), but providing first-class CI tooling for validating software quality can have a dramatic impact on an engineering organization’s velocity.

As you can see, this all leaves QA with a lot to do!