Roundtable Forum on Generative AI Presents Unprecedented Opportunities & Risks.
Elevate Xchange April 17 th 4:30p-8:30p
How Gen AI Helps Us Deliver Value-Based Testing
2024-04-24 5 min read (all content provided by Keysight’s website except the video)
At the heart of this blog is my simple concern that testing is too often seen as a cost to the business, not an enabler that adds value. I’ll explore why that’s the case – and show why and how gen AI holds the answer to improving the QA function’s reputation.
But let’s start by considering how much the IT landscape has shifted because I think that holds the key to understanding where it’s all gone wrong.
From IT as King to User as King
It’s a truism because it’s true: the world of IT regularly changes out of all recognition. At the start of my career, the web was just getting going, and IT in business largely consisted of mainframe applications, client service, and thick client systems.
It was a completely different landscape in another sense too. Users were expected to follow the workflow of the application not vice versa. If you didn’t follow the workflow and something didn’t work, it was your fault, not the system’s fault.
Testing built up around similarly rigid architectures, with exacting governance, standards, and organizations around them.
Fast forward to today, and we have multiple devices, multiple applications, and multiple connectivities. The importance of user experience (UX) means applications must work how users want them to, not vice versa.
It’s all made testing a thousand times more complicated than it used to be. Yet in the same time, testing hasn’t really evolved. We still have relatively rigid testing architectures, governance, and standards.
And it gets worse too.
Choose Two of Better, Faster, Cheaper
Competition is fiercer than ever. Whether it’s your internal IT systems or your customer-facing systems, your organization needs faster release cadences to enhance customer satisfaction and drive competitive advantage. It also needs you to deliver at lower cost with better quality.
Yet, as we all know, when it comes to the well-known product triangle of better, faster, cheaper, you can only choose two.
Agile and DevOps helped us to speed up release cadences and introduced the concept of the minimum viable product.
But testing struggles to keep up with DevOps speeds, and that’s a problem that isn’t easily solved.
You can increase speed by lowering testing requirements, but this reduces quality. You can increase quality by maintaining testing requirements, but this reduces speed. You can throw resource at testing to increase speed and maintain quality, but this increases cost. In the first two scenarios, you damage the user experience. In the third, you cost the business more than it wants.
Test automation goes some of the way to helping, but not far enough, as I explain in Meet Eggplant GAI, the AI-Powered Tool That Harnesses the All Possibilities of Test Automation.
Ultimately, the result is that the business starts to lose faith in the IT function’s ability to deliver what’s required – solutions that are better, faster, and cheaper. At the heart of this reputational issue is whether the business views the QA function as a cost or an enabler.
Too often, it’s seen as a cost. Aside from the difficulty in delivering on better, faster, and cheaper, there’s an even more fundamental presentational problem. Put simply, you spend time and money running a thousand tests. You find 20 defects – which will need time and money to fix. Testing is a Critical Business Enabler
In fact, QA should be viewed as an enabler. We’re doing what the business needs us to do – driving the quality that will enhance the UX, increase customer satisfaction, boost reputation, and deliver competitive advantage.
The question is, then, how do we deliver testing better, faster, and cheaper so we can restore testing’s reputation and have it viewed as a critical business enabler?
The answer is that you use Gen AI tools such as GAI. GAI turbo-charges the human-in-the-loop so the QA function becomes the value-adding team that gives the business what it wants. A Quick Intro to GAI Eggplant Generative AI (GAI) is a fine-tuned large language model (LLM) specifically designed for testing.
The base model is trained on ISO/IEEE/BSI/ISTQB® testing material from trusted sources, so you can be confident of the quality of the inputs.
It’s also trained on industry verticals such as healthcare, telecommunications, or aerospace and defense, so it brings industry-specific insight.
It gives you its sources too, so you can apply your own knowledge to confirm the quality of
the response.
You can improve the base model to your precise requirements by feeding it all your proprietary knowledge. And you can do it with complete confidence because GAI is offline and therefore both 100% secure and EU AI Act complaint.
GAI can generate all the automation assets, all the models, and all the tests that you need in any scenario. It can also streamline requirements and provide test case optimization for example by eliminating duplicates.
It can go further too, with the upcoming release of Sentient Test Expert (STE) from Keysight’s Eggplant. You can give STE any digital interface, and it will utilize next-generation cognitive reckoning technology powered by Large Action Models (LAM) and Large Vision Models (LLaVA) to create a Universal Language Test Translator (ULTT) that allows STE to autonomically test it. Human testers can assess the test suggestions, apply their own knowledge to discard the ones that add no value and ask STE to run the ones that have possibilities.
These capabilities mean that test automation can finally realize its potential.
But the ability to write, run, and refine tests is just the start of the power and potential these tools give you.
Supercharge the Human-in-the-Loop The critical component of a truly successful DevOps approach is someone who is a specialist in the business side of things and a specialist in the technical requirements of implementation and testing. There aren’t many of those around! Most people are either a specialist in the business or a specialist in the technical side of things.
GAI is effectively a specialist in both. It has business knowledge and it has technical testing knowledge. When you layer humans-in-the-loop on top, you add in the knowledge worker with wisdom to be able to validate and verify that everything expressed by every stakeholder and every permutation is correct. It gives you the ability to translate the subject matter of the business into something the IT and the technical teams can use to build an application and automate its testing.
It speeds up the pipeline and helps people on both sides of the equation to collaborate and work together effectively.
In short, it means we can properly shift left, bring testing right into the requirements phase of development, embed UX from the start – and elevate the value of testing. It’s a game-changer that helps us deliver what the business wants and needs.