QA & testing glossary
Plain English definitions for the software testing and AI-powered QA terms you'll run into while working with Klarent.
A
Acceptance testing
Acceptance testing checks whether a finished system actually meets what the customer or end user needs, rather than just what was technically specified. It's usually the last testing stage before release, done by or on behalf of the people who'll actually use the software, not only the team that built it.
Accessibility testing
Accessibility testing checks whether an application can actually be used by people with disabilities, including those relying on a screen reader, keyboard-only navigation, or assistive technology. It usually measures against a published standard, most commonly WCAG, rather than a subjective sense of whether something feels usable.
Agentic AI testing
Agentic AI testing is the specific pattern of an AI agent planning its own testing steps, taking action, observing the result, and deciding what to do next, applied to QA instead of a general-purpose workflow. It's the technical mechanism that usually sits underneath a highly autonomous testing system, not a separate capability on its own.
Agentic workflows
An agentic workflow is a multi-step process carried out by one or more AI agents that can plan their own steps, use tools, and adjust course based on what each step actually returns, with limited or no human input along the way. It differs from a single prompt-and-response interaction by chaining reasoning and action across several steps toward a goal.
AI hallucination
AI hallucination is when a language model generates output that sounds confident and plausible but isn't grounded in fact, in its training data, or in the context it was given. In AI-driven testing, that might mean a generated test case references a UI element or a field that doesn't actually exist.
AI QA
AI QA is the application of artificial intelligence across the quality assurance discipline as a whole, not just running tests, but planning what's risky enough to test, predicting where a defect is likely to appear, and reporting on quality without a person assembling it by hand. It's QA with AI woven into the process, not only the execution.
AI reasoning
AI reasoning is a model working through a problem in intermediate steps before producing a final answer, rather than jumping straight from a question to a response. In testing, that's what lets a model explain why it judged a result a pass or a fail, instead of just returning a verdict with no visible logic behind it.
AI software testing
AI software testing is the general practice of applying artificial intelligence anywhere across the testing lifecycle, including generating tests, prioritizing which ones matter most, analyzing results, and predicting where a defect is likely to show up. It overlaps with AI test automation, which applies AI to designing, running, and maintaining automated tests.
AI test analytics
AI test analytics is the use of models to interpret the data a test suite produces, rather than just charting it. It clusters related failures, separates real defects from flaky ones, tracks where a suite is losing reliability, and turns thousands of run records into a short list of things worth acting on.
AI test automation
AI test automation is the practice of using artificial intelligence, including machine learning and generative models, to design, run, and maintain software tests with little or no manual scripting. AI agents interpret how an application behaves, turn plain-language descriptions into working test cases, and adjust those tests on their own as the software underneath them changes.
AI test execution
AI test execution is the run phase of an AI-driven test suite, where agents carry out test steps against a live application, decide what to do when the screen does not match expectations, and judge whether the result counts as a pass or a failure. It replaces a fixed replay of scripted commands with a run that can reason mid-flight.
AI test maintenance
AI test maintenance is the use of AI to keep an automated test suite working as the application under test changes. Instead of an engineer editing broken scripts after every release, models detect what moved in the interface or API, repair the affected steps, and flag the failures that point at real defects.
AI test prioritization
AI test prioritization is the use of models to decide which tests to run first, or at all, for a given change. It ranks a suite by how likely each test is to catch a defect in the code that just moved, so the highest-value cases run early and a pull request gets a useful signal in minutes instead of hours.
AI testing tool
An AI testing tool is software that uses AI, rather than only fixed scripts, somewhere in how it creates, runs, or maintains tests. The label covers a wide range in practice, from a traditional tool with one AI feature bolted on to a product built around AI from the ground up, worth checking before trusting the label.
AI workflow automation
AI workflow automation is the use of AI models and agents to carry out a multi-step process end to end, deciding what to do next based on the result of the previous step rather than following a fixed script. It differs from traditional automation by adapting its path when a step's outcome isn't what was expected.
API mocking
API mocking is standing in a fake version of a real API that returns realistic, predictable responses, so a team can develop or test against it without the real dependency actually being available. It lets work proceed when the real API is slow, unfinished, unreliable, or something a team would rather not hit repeatedly in a test.
API testing
API testing checks an application's programming interface directly, sending requests and verifying the responses, status codes, and data it returns, without going through a user interface at all. It exists to catch a broken or incorrect response at the layer where it actually happens, faster and more reliably than testing the same behavior through the UI.
Appium
Appium is an open-source framework for automating native, hybrid, and mobile web apps on iOS and Android, along with some desktop platforms. It extends the WebDriver protocol to mobile, so a team can write mobile tests using the same approach, and often much of the same code, as a Selenium-based web suite.
Automated testing
Automated testing is running tests with software instead of a person, letting a script execute the same steps and compare results every time instead of someone doing it by hand. The real decision most teams face isn't whether to automate, but which tests are worth automating and which are better left manual.
Autonomous testing
Autonomous testing describes how much of the testing process runs without a person actively directing it, from AI assisting with one step, like generating a test case, up to a system that independently decides what to test, executes it, and triages the result. It's best understood as a spectrum of autonomy, not a single fixed capability.
Azure DevOps
Azure DevOps is Microsoft's suite of services for managing the software development lifecycle, spanning planning, source control, builds, testing, and releases. It bundles five services, Azure Boards, Repos, Pipelines, Test Plans, and Artifacts, that teams can adopt together or individually, in the cloud or self-hosted on-premises.
C
CI/CD pipeline
A CI/CD pipeline is the automated sequence of steps that takes a code change from commit to a tested, deployable build, and often all the way to production. It typically runs builds, tests, and deployments in order, so a change earns its way to release instead of shipping on trust alone.
Compatibility testing
Compatibility testing checks that an application works correctly across the different environments it might actually run in, operating systems, hardware, software versions, and network conditions, not just the one environment it was built and tested on. Cross browser testing is one specific, common form of it, focused on web browsers.
Continuous testing
Continuous testing means running automated tests continuously throughout the software delivery pipeline, not just at one fixed stage, so a team gets fast, constant feedback on whether a change is actually safe to move forward. It's less a specific test type and more a practice of weaving testing into every step of delivery.
Cross browser testing
Cross browser testing checks that a web application looks and behaves correctly across the different browsers, and browser versions, real users actually visit with. It exists because browsers don't render every layout, script, or CSS feature identically, so code that works perfectly in one can look broken or behave differently in another.
Cypress
Cypress is an open-source, JavaScript-based testing framework for web applications, used mainly for end-to-end and component tests. Unlike tools that drive the browser from the outside, Cypress runs inside the browser alongside the application, which gives it direct access to the DOM and a fast, visual debugging experience that front-end teams tend to favor.
D
Device testing
Device testing checks how an application behaves on real or virtual physical devices, phones, tablets, wearables, accounting for differences in screen size, hardware, operating system version, and device-specific interruptions like an incoming call. It goes beyond what a browser or OS version alone can reveal, since real hardware introduces constraints a simulator doesn't always capture.
DevOps testing
DevOps testing is the practice of testing as a shared, continuous responsibility across the entire DevOps loop, plan, build, test, release, deploy, and operate, rather than a separate phase owned by one team near the end. It includes testing infrastructure and monitoring production itself as a source of quality signal, not just testing application code before release.
Digital quality assurance
Digital quality assurance is QA applied across an organization's entire digital portfolio, web, mobile, APIs, and other digital channels, rather than one application at a time. It's less about a single team's testing process and more about governance, standards, and metrics that keep quality consistent across many products and teams at enterprise scale.
E
End-to-end testing
End-to-end testing verifies a complete user journey through an application from start to finish, exactly the way a real person would use it, rather than checking one component or one connection in isolation. It sits at the top of the testing pyramid, the broadest and most realistic check, and also the slowest and most expensive to run.
Enterprise test automation
Enterprise test automation is automated testing scaled and governed across a large organization, many teams, often several legacy and modern systems at once, rather than one team automating its own product. It adds concerns a single team's automation effort rarely has to deal with, shared standards, compliance, and return on investment measured across dozens of applications.
Exploratory testing
Exploratory testing is unscripted testing where a person investigates an application in real time, deciding what to try next based on what they just observed, rather than following a predefined test case. It exists to catch the kind of problem nobody thought to write a specific check for in advance.
F
False negative
A false negative is a test that passes even though the thing it's checking actually has a real defect, letting a problem through undetected. It's the opposite of a false positive, where a test fails despite nothing actually being wrong, and of the two, a false negative is generally the more dangerous mistake.
False positive
A false positive is a test that fails even though the thing it's checking actually works correctly, pointing at a problem that isn't really there. It's the opposite of a false negative, where a test passes despite a real defect, and both quietly erode how much a team can trust its test results.
Flaky test
A flaky test is one that gives inconsistent results, passing sometimes and failing other times, against the exact same code, with nothing changed in between. It's a specific, common cause of a false positive, and a suite with too many of them stops being something a team can actually trust.
Functional testing
Functional testing checks that a piece of software does what it's supposed to do, verifying its features and behavior against stated requirements rather than how fast or secure it is. It answers one plain question for each feature: given this input, does the application produce the result it's expected to.
G
GitHub Actions
GitHub Actions is GitHub's built-in automation platform for continuous integration and continuous delivery, running builds, tests, and deployments in response to events in a repository. Workflows are defined as YAML files stored in the codebase, so a pipeline lives and versions alongside the application it builds.
GitLab CI
GitLab CI is GitLab's built-in continuous integration and delivery system, running builds, tests, and deployments defined in a single YAML file stored in the repository itself. Because it lives inside GitLab alongside the code, a pipeline can react directly to a push, a merge request, or a tag without a separate CI system bolted on.
GitOps
GitOps is a way of managing infrastructure and deployments where a Git repository is the single source of truth for what a system should look like, and an automated process continuously syncs the live environment to match it. Instead of running commands against a server by hand, a team merges a change and lets automation apply it.
I
Integration testing
Integration testing checks separate pieces of a system, like a service and the database it queries, or two internal APIs, work correctly once they're connected. It exists to catch the class of bug that only shows up when components interact, the kind a unit test, run in isolation, was never going to find.
Intelligent automation
Intelligent automation combines rule-based automation, like RPA, with AI capabilities such as machine learning and natural language processing, so a system can both execute a process and make judgment calls within it. It handles the routine, repetitive parts of a task the way older automation always did, plus the parts that used to need a person's judgment.
L
Large language model
A large language model, or LLM, is a type of AI model trained on a massive amount of text to predict the next piece of text given what came before. That simple mechanism, repeated token by token, is what lets it write, summarize, answer questions, and generate code, without being explicitly programmed for any one of those tasks.
Load testing
Load testing checks how an application behaves under an expected amount of concurrent traffic or activity, measuring things like response time, throughput, and error rate as demand increases. It exists to answer a question functional testing can't: not just whether a feature works, but whether it still works when many people use it at once.
M
Manual testing
Manual testing means a person directly executing test steps and observing the result, rather than a script doing it. It's still how a lot of exploratory and usability testing gets done, since it's the one approach that catches a problem nobody thought to write a specific check for.
Microservices testing
Microservices testing is the practice of testing an application built as many small, independently deployable services instead of one unified codebase, which means verifying each service on its own and also how it behaves once it's talking to a dozen others. It needs techniques, like contract testing, that a single-service application rarely has to reach for.
P
Page object model
The page object model is a design pattern for browser test automation that wraps each page, or a distinct part of one, in its own class holding that page's elements and available actions. Tests call those methods instead of touching selectors directly, so a UI change means updating one class, not every test that used it.
Performance testing
Performance testing checks how an application behaves in terms of speed, responsiveness, and resource use under a given set of conditions, rather than whether its features work correctly. It covers a family of related checks, including load, stress, and scalability testing, each asking a slightly different question about how the system performs.
Playwright
Playwright is an open-source framework for end-to-end web testing and browser automation, created by Microsoft. It drives Chromium, Firefox, and WebKit through a single API, so one test suite can check an application across the engines behind Chrome, Edge, Firefox, and Safari, written in JavaScript, TypeScript, Python, Java, or C#.
Prompt engineering
Prompt engineering is the practice of writing and refining the instructions given to an AI model so it produces a specific, reliable output. In AI-driven QA, that means shaping the wording, examples, and constraints in a prompt so a model generates accurate test cases, automation scripts, or test data instead of vague or invented ones.
Puppeteer
Puppeteer is an open-source Node.js library that gives developers a high-level API to control Chrome, Chromium, and Firefox. Maintained by Google's Chrome DevTools team, it automates browser actions for testing, web scraping, screenshots, and PDF generation, running headless by default so it fits neatly into scripts and pipelines.
R
Regression testing
Regression testing is re-running existing tests after a code change to confirm that features which worked before still work. It exists to catch the case where a fix or new feature quietly breaks something unrelated, and it becomes more important, and more expensive to skip, the more often a team ships.
Release testing
Release testing is the set of checks a team runs specifically around a release going out, confirming the exact build being shipped actually works, the release process itself executes cleanly, and a rollback is ready if something goes wrong. It's organized around the release event itself, not a particular testing technique.
Retrieval-augmented generation
Retrieval-augmented generation, or RAG, is a technique where a model looks up relevant information from an external source before generating an answer, rather than relying only on what it learned during training. It grounds a response in specific, current, or private information the model was never trained on, which cuts down on confidently wrong answers.
S
Sanity testing
Sanity testing is a quick, narrow check run after a small change or a bug fix, confirming that the specific area it touched actually works, without retesting the whole application. It's a fast, focused pass, not meant to catch everything, just to confirm a fix did what it was supposed to before moving on.
Security testing
Security testing checks an application for vulnerabilities that could let someone access data, functionality, or systems they shouldn't be able to reach. It asks a different question than most other testing: not whether a feature works as intended, but whether it can be made to do something it was never intended to do.
Selenium
Selenium is an open-source suite for automating web browsers, used mainly to write and run functional and regression tests for web applications. It drives a real browser the way a user would, clicking, typing, and navigating, and lets teams script those actions in Java, Python, C#, JavaScript, or Ruby against Chrome, Firefox, Edge, and Safari.
Shift left testing
Shift left testing means moving testing earlier in the development process, checking code as it's written rather than waiting until a feature is finished or a release is near. The name comes from a left-to-right timeline of development stages, testing moves toward the start instead of staying parked at the end.
Smoke testing
Smoke testing is a quick pass through an application's most critical functions, run right after a new build, to confirm it's stable enough to test further. It exists to catch a build that's badly broken before a team spends hours running a full test suite against something that was never going to pass.
Software quality assurance
Software quality assurance, or QA, is the set of processes and activities a team uses to prevent defects and build confidence that a product meets its requirements, before it ever reaches a user. Testing is one activity within QA, actually running the software and checking it, but QA is the broader discipline around it.
Software quality metrics
Software quality metrics are the measurable indicators a team or organization tracks to understand how good its software actually is, things like defect density, test coverage, and escaped defect rate, rather than relying on a gut feeling. A single metric rarely tells the whole story, which is why most teams track several together.
Software testing
Software testing is the process of running a program, or examining its code, to find defects and confirm it behaves the way it's supposed to before users ever see it. It ranges from checking a single function in isolation to verifying how an entire system holds up under real usage, and it happens throughout development, not only at the end.
Stress testing
Stress testing pushes a system past the traffic or load it's actually expected to handle, on purpose, to find the point where it breaks and see how it fails. Unlike load testing, which checks realistic conditions, stress testing is deliberately unrealistic, mapping out a system's real limits rather than its everyday operating range.
Synthetic test data
Synthetic test data is fake data generated to match a real schema or business rules, used in place of actual user information so a test can run against realistic input without ever touching anything real. It's one specific technique within the broader practice of test data management, and the one that avoids exposing real user data entirely.
System testing
System testing checks a complete, fully integrated application as a whole, against its overall requirements, rather than checking one piece or one connection at a time. It's the level right before user-facing acceptance testing, confirming the entire system behaves correctly once every component is actually assembled together.
T
Test automation
Test automation is using software to run tests, compare actual results against expected ones, and report the outcome, instead of a person carrying out those same steps by hand. It exists so a team can run the same checks repeatedly, quickly, and consistently, something manual testing struggles to do at any real scale or speed.
Test automation framework
A test automation framework is the shared structure a team builds its automated tests on top of, covering how a test is organized, how it talks to the application, how results get reported, and how common actions get reused. It's the scaffolding that keeps a growing suite from turning into a pile of unrelated scripts.
Test automation platform
A test automation platform is a broader ecosystem of testing capability, creation, execution, management, and reporting, built to work together rather than a single standalone product. The distinction from test automation software is mostly one of scope: a platform is something other tools and workflows connect to and build on, not a tool used in isolation.
Test automation software
Test automation software is a packaged, typically commercial product that provides the tooling to automate tests without a team building that tooling from scratch, covering some mix of test creation, execution, and reporting out of the box. It's the alternative to assembling a custom test automation framework from open-source pieces.
Test case
A test case is a single, specific set of steps, inputs, and expected results used to check one particular piece of a system's behavior. It's the smallest concrete unit of testing, precise enough that anyone, or any script, can follow it and get the same pass or fail result every time.
Test coverage
Test coverage is a measure of how much of an application, its code, its features, or its requirements, is actually exercised by a test suite. It's usually expressed as a percentage, but the number only means as much as what it's actually measuring, and a high percentage doesn't automatically mean the testing itself is good.
Test data management
Test data management is the practice of creating, maintaining, and provisioning the data a test actually needs to run, realistic enough to exercise real behavior without exposing real user information. A test is only as trustworthy as the data behind it, and bad or missing data breaks a test for reasons that have nothing to do with the code.
Test environment
A test environment is a separate, controlled setup where an application runs for testing purposes, isolated from production so a test can't affect a real user and real traffic can't interfere with a test. It typically mirrors production closely enough that a result found there actually predicts how the application will behave once it's live.
Test environment management
Test environment management is the ongoing practice of provisioning, configuring, refreshing, and coordinating access to test environments, keeping them stable and available for the teams that actually depend on them. It's the operational discipline behind a test environment, not the environment itself.
Test management
Test management is the practice of planning, organizing, and tracking a team's testing effort, deciding what needs to be tested, who's testing it, and how the results get recorded and reported. It ties together test cases, test plans, and execution results into one coherent view of how well a release has actually been covered.
Test management software
Test management software is a dedicated tool for organizing test cases, tracking execution results, and reporting coverage across a release, replacing a spreadsheet once a team's testing effort outgrows what one can handle. It typically integrates with a bug tracker and a CI/CD pipeline, so manual and automated results live in one place.
Test plan
A test plan is a document that lays out the scope, approach, and schedule for testing a specific release or feature, deciding what will be tested, how, by whom, and what counts as done. It's the plan a team works from, distinct from the individual test cases that actually get executed against it.
Test reliability
Test reliability is how consistently a test produces the correct result, pass when the application genuinely works, fail when it genuinely doesn't, run after run. It's the property a team is actually protecting when it worries about flaky tests, false positives, and false negatives, which are each a specific way reliability breaks down.
Test run
A test run is one specific instance of executing a test suite, a single, timestamped record of which tests were run, when, against which build and environment, and what each one's result was. It's the unit that actually gets tracked over time, not the suite definition itself, which stays the same between runs.
Test script
A test script is the coded, executable form of a test case, the actual instructions a tool runs to perform the steps, provide the input, and check the result, without a person carrying it out by hand. It's what a test case becomes once it's automated, rather than followed manually.
Test strategy
A test strategy is a high-level document describing an organization's overall approach to testing, the principles, methods, and tools it defaults to across projects, rather than the specifics of any single release. It sits above a test plan, which applies that strategy to one particular feature or release.
Test suite
A test suite is a collection of test cases grouped together because they share a purpose, a feature, a type of testing, or a point in the release process when they're meant to run. It's the organizational unit above a single test case, letting a team run, manage, and report on a group of related checks as one thing.
U
UI testing
UI testing checks an application's user interface itself, whether elements render correctly, respond to interaction, and look right, rather than the logic running underneath it. It's often confused with functional testing, but UI testing is specifically about the interface layer, buttons, layouts, text, not whether the business logic behind it is correct.
Unit testing
Unit testing is the practice of testing the smallest pieces of code, typically a single function or method, in isolation from the rest of the application. It exists to catch a defect in one small piece before it's buried under the layers of code and other tests built on top of it.