{"slug": "sdets-changing-tide", "title": "SDETs Changing Tide", "summary": "A qualitative report from Quash, based on five public QA community discussions, finds that test automation maintenance—not test creation—is the primary burden for QA engineers, with locator brittleness as the sharpest mobile pain and flaky tests eroding trust in automation. The report also shows that QA engineers view AI as a helpful junior assistant rather than a replacement, and that the real ROI of automation is release confidence, not just hours saved.", "body_md": "# The State of Test Automation Maintenance: What QA Engineers Actually Say\n\n[TL;DR](#tldr)[Methodology](#methodology)[Finding 1: Maintenance Is the Killer](#finding-1-maintenance-is-the-killer)[Finding 2: Locator Brittleness Is the Acute Mobile Pain](#finding-2-locator-brittleness-is-the-acute-mobile-pain)[Finding 3: Flaky Tests Break Trust Before They Break Pipelines](#finding-3-flaky-tests-break-trust-before-they-break-pipelines)[Finding 4: AI Is Treated Like a Junior QA, Not a Replacement](#finding-4-ai-is-treated-like-a-junior-qa-not-a-replacement)[Finding 5: ROI Is Confidence, Not Just Hours Saved](#finding-5-roi-is-confidence-not-just-hours-saved)[What Engineering Leaders Should Take From This](#what-engineering-leaders-should-take-from-this)[What QA Engineers Should Take From This](#what-qa-engineers-should-take-from-this)[Where Mobile Test Automation Needs to Go Next](#where-mobile-test-automation-needs-to-go-next)[A Transparent Note From Quash](#a-transparent-note-from-quash)[Conclusion: Maintenance Is the Real Automation Test](#conclusion-maintenance-is-the-real-automation-test)\n\nTest automation was supposed to make QA teams faster.\n\nIn practice, many QA engineers describe a messier tradeoff: automation gives teams coverage, repeatability, and release confidence, but it also creates another system that has to be maintained.\n\nScripts need updates. Locators drift. Framework versions change. CI failures need triage. Test data expires. A flaky test gets rerun until it becomes background noise.\n\nThat is the real cost of test automation maintenance.\n\nThis report is based on qualitative voice-of-customer research across five organic public QA community discussions. The goal is not to present a statistically representative survey. The goal is to capture the language, complaints, objections, and priorities QA engineers keep repeating when they talk about test maintenance, flaky tests, AI testing tools, and mobile test automation.\n\nThe clearest finding is simple:\n\nQA engineers do not hate automation. They hate maintaining automation that breaks for reasons users never care about.\n\n## TL;DR\n\n**Test automation maintenance is the real ceiling.** Teams do not struggle only with writing tests. They struggle with keeping tests useful after the app changes.**Locator brittleness is the sharpest mobile pain.** IDs, XPath, accessibility labels, waits, and device differences create constant upkeep.**Flaky tests destroy trust.** Once teams stop believing red builds, automation loses authority.**AI is welcomed as assistance, not replacement.** QA engineers describe useful AI as a draft, a helper, or a junior QA that still needs review.**ROI is confidence, not just hours saved.** The best automation gives teams confidence to release, not just a bigger test count.\n\n### Get the Mobile Testing Playbook Used by 800+ QA Teams\n\nDiscover 50+ battle-tested strategies to catch critical bugs before production and ship 5-star apps faster.\n\n## Methodology\n\nWe reviewed five organic public QA community discussions focused on test automation, AI-assisted QA, locator brittleness, flaky tests, and the day-to-day reality of maintaining automated tests.\n\n**The reviewed discussions covered:**\n\nQA tools that are useful day to day\n\nAI for test case creation\n\nAI tools that help write and run automated UI tests\n\nAutomation feeling like another full-time job\n\nBuilding AI-assisted testing workflows with coding agents\n\n**Each discussion was coded for:**\n\nRepeated pain points\n\nSpecific practitioner language\n\nObjections to automation tooling\n\nObjections to AI testing tools\n\nMobile test automation maintenance signals\n\nThemes around QA ownership, judgment, and trust\n\n**A note on quotes:** the quote wall below uses short public-community quotes or lightly cleaned fragments from the reviewed discussions. Usernames are intentionally omitted. The quotes are not private interviews, and they should not be treated as survey responses.\n\nThis is best read as a qualitative field report: what QA engineers say when they are talking to each other, not filling out a vendor form.\n\n## Finding 1: Maintenance Is the Killer\n\nThe strongest recurring signal was not “we need more automation.”\n\nIt was: maintaining automation becomes the problem.\n\nOne thread captured the wound directly:\n\n“Maintenance is the number one killer of automation.”\n\nAnother described the day-to-day pain:\n\n“One locator change and you're fixing tests for hours.”\n\nAnd another variation hit the framework side:\n\n“New framework version? Half your pipeline breaks.”\n\nThat is the core tension in test automation maintenance. The first version of a test suite is not the real test. The real test comes after the product changes.\n\nA new onboarding screen appears. A button label changes. A permission dialog behaves differently on Android. A payment flow gets a new loading state. The test fails, but the product still works.\n\nNow QA has to answer the question every automation team eventually faces:\n\n**Did the product break, or did the test break?**\n\nThat investigation is the hidden labor of automation.\n\n### The Maintenance Loop\n\n**A brittle suite creates a loop:**\n\nApp changes\n\nLocator, data, or environment drifts\n\nTest fails\n\nQA investigates\n\nTest is updated, quarantined, or ignored\n\nTrust either recovers or erodes\n\nThat loop is normal in small amounts. But when the loop becomes constant, automation stops feeling like leverage. It starts feeling like another product the QA team has to maintain.\n\nThis is why test maintenance should not be treated as an afterthought. It is the real long-term cost of automation.\n\nFor teams already dealing with inconsistent failures, the issue is often bigger than one [flaky test](/blog/how-to-fix-flaky-tests-complete-diagnosis-guide). It is the full maintenance surface around the suite: selectors, waits, device state, test data, backend dependencies, CI stability, ownership, and triage discipline.\n\n## Finding 2: Locator Brittleness Is the Acute Mobile Pain\n\nLocator brittleness was the most specific technical pain in the research.\n\nTraditional UI automation depends on implementation-level references: resource IDs, accessibility labels, XPath, text selectors, class names, or UI hierarchy. This works when the UI is stable and the engineering team consistently maintains test-friendly identifiers.\n\nMobile apps rarely stay that clean.\n\nButtons move. Labels change. Components get refactored. Native dialogs interrupt flows. Android and iOS expose different automation surfaces. Device sizes vary. Loading states appear at slightly different times. Accessibility identifiers are missing, inconsistent, or not treated as product-critical.\n\nOne recurring complaint was simple:\n\n“developers are not maintaining ids leading to breaking tests.”\n\nAnother practitioner pushed back with an experienced counterpoint:\n\n“why is a locator change taking hours? skill issue.”\n\nThat pushback matters. Skilled QA engineers are right that locator brittleness can be reduced with better discipline: stable test IDs, Page Object Models, reusable selectors, strong waits, and developer-QA coordination.\n\nSo the honest conclusion is not: “locators are impossible.”\n\nThe honest conclusion is:\n\nLocator-based automation demands permanent discipline.\n\nFor well-staffed teams with mature automation engineers, that may be acceptable. For small QA teams shipping mobile changes every sprint, it becomes a serious maintenance tax.\n\n### Why Mobile Makes Locator Maintenance Worse\n\n**Mobile test automation adds extra instability because tests must deal with:**\n\nNative permission dialogs\n\nKeyboards\n\nGestures\n\nDevice state\n\nDifferent screen sizes\n\nAndroid and iOS behavior differences\n\nOS version differences\n\nSlow or inconsistent network conditions\n\nReal-device quirks\n\nApp backgrounding and foregrounding\n\nThat is why locator maintenance is not just a test-code issue. It becomes a release-confidence issue.\n\nIf you are using Appium, locator strategy deserves deliberate planning. The upcoming guide on [Appium iOS vs Android locators](/blog/appium-ios-vs-android-locators) should cover this in more depth. Do not publish this internal link until that page is live. Until then, link to the or [Appium alternatives](https://quashbugs.com/blog/appium-alternatives).\n\n## Finding 3: Flaky Tests Break Trust Before They Break Pipelines\n\nA **flaky test** is usually defined as a test that passes and fails inconsistently without a relevant product change.\n\nThat definition is technically correct, but it undersells the real damage.\n\nA flaky test does not only waste time. It trains the team to distrust automation.\n\nThe first time a test fails randomly, someone reruns it. The fifth time, people start ignoring it. Eventually, a red build no longer means **“something broke.”** It means **“probably CI again.”**\n\nThat is the moment automation loses authority.\n\nA good automation suite should create a trusted signal. When it fails, the team should care. When it passes, the team should have more confidence in the release.\n\nFlaky tests destroy both sides of that equation.\n\n### How Flakiness Erodes Automation Value\n\n|\n|\n|\nFirst flaky failure | A test fails inconsistently | Someone reruns it |\nRepeated flakiness | The same test keeps failing randomly | Team starts discounting the signal |\nSuite-level flakiness | Multiple tests fail for unclear reasons | CI loses authority |\nHidden regression | A real issue appears among noisy failures | The team responds late |\nLost confidence | Automation becomes unreliable | Manual QA pressure returns |\n\nThis is why flaky test work is not cleanup. It is trust repair.\n\nA flaky suite says: **“We have automation, but we do not fully believe it.”**\n\nThat is a dangerous place to be. The team still pays the cost of maintaining automation, but the business no longer gets dependable release confidence from it.\n\nFor a deeper diagnosis workflow, use the [flaky tests guide](/blog/how-to-fix-flaky-tests-complete-diagnosis-guide).\n\n## Finding 4: AI Is Treated Like a Junior QA, Not a Replacement\n\nQA engineers are not universally anti-AI.\n\nThe research showed a more practical view: QA teams are open to AI when it helps with drafts, repetitive execution, summarization, debugging, or regression support.\n\nBut they reject the idea that AI can fully replace QA judgment.\n\nThe cleanest practitioner framing was:\n\n“treat AI output like junior QA output, review it.”\n\nAnother repeated idea was:\n\n“AI is a draft, not a replacement for thinking.”\n\nThat is the right model.\n\nA junior QA can be useful. They can draft test cases. They can execute flows. They can notice issues. But they need context, review, and guidance from someone more experienced.\n\nAI testing tools should be framed the same way.\n\n### The AI Model QA Engineers Actually Accept\n\n**QA engineers are more likely to accept AI for:**\n\nDrafting test ideas\n\nGenerating first-pass test cases\n\nSummarizing failures\n\nRunning repetitive regression\n\nIdentifying changed screens\n\nHelping with test maintenance triage\n\n**They are less likely to accept AI for:**\n\nFinal release judgment\n\nExploratory testing strategy\n\nBusiness-risk prioritization\n\nContext-heavy edge cases\n\nReplacing QA headcount\n\nFully autonomous PRD-to-production testing\n\nThe distinction is not small. “AI replaces QA” is radioactive because the buyer and champion is often the QA engineer you are insulting.\n\nThe better message is:\n\nAI should remove drudgery so QA can spend more time on judgment.\n\nThat is the lane practitioners are actually open to.\n\n## Finding 5: ROI Is Confidence, Not Just Hours Saved\n\nA lot of test automation ROI content starts with hours saved.\n\nThat is not wrong, but it is incomplete.\n\nYes, automation can reduce repeated manual regression work. Yes, it can speed up feedback. Yes, it can reduce repetitive testing effort.\n\nBut the strongest practitioner framing was about confidence:\n\n“Automation ROI isn't about hours saved, it's about confidence gained.”\n\nAnother phrasing made the same point:\n\n“the benefit isn't stopwatch savings, it's system trust.”\n\nThis is the better frame for test automation maintenance.\n\nThe real value of automation is not that it lets you say “we automated 500 tests.” The value is that your team can release knowing the highest-risk flows still work.\n\n### The Better ROI Frame\n\n|\n|\n“We saved tester hours” | “We know critical flows still work” |\n“We automated 500 cases” | “We covered the highest-risk regression paths” |\n“We reduced manual effort” | “We reduced release uncertainty” |\n“We run tests faster” | “We can trust failures when they happen” |\n“We increased coverage” | “We increased confidence in the release” |\n\nThis matters even more in **mobile**.\n\nA mobile app can fail in ways that are highly visible and expensive: broken login, failed OTP, checkout bugs, payment failures, location issues, notification failures, permission dead ends, or device-specific layout problems.\n\nNobody cares that QA saved time if the wrong bug escapes.\n\nThat is why mature teams should measure automation ROI through confidence signals:\n\nCritical flows covered\n\nFlakiness rate reduced\n\nFalse failures reduced\n\nFailure diagnosis time reduced\n\nEscaped defects reduced\n\nRelease blockers caught earlier\n\nRegression cycles completed reliably\n\nFailures backed by clear evidence\n\nFor the business case side, see the [test automation ROI calculator](/blog/test-automation-roi-calculator-business-case).\n\n## What Engineering Leaders Should Take From This\n\nThe mistake is buying automation as if test creation is the whole problem.\n\nIt is not.\n\nTest creation is the easy demo. Test maintenance is the long-term proof.\n\n**Before scaling any automation platform, ask:**\n\nWhat happens when the UI changes?\n\nWho owns test maintenance?\n\nHow often do tests fail for non-product reasons?\n\nCan failures be diagnosed quickly?\n\nDoes the suite depend on fragile locators?\n\nAre we automating high-risk flows or chasing vanity coverage?\n\nDoes the tool reduce maintenance, or create a new kind of maintenance?\n\nWill QA engineers still trust this system three months from now?\n\nThat last question is the real one.\n\nA tool that looks impressive in week one but creates maintenance drag by month three is not solving the real problem. It is moving the problem.\n\n## What QA Engineers Should Take From This\n\nThe practitioner consensus is not **“automation is bad.”**\n\n**It is sharper than that:**\n\nAutomate repetitive regression where it clearly reduces pain.\n\nDo not automate low-value flows just to increase test count.\n\nTrack flaky tests as trust risks, not minor annoyances.\n\nPush for stable testability hooks if using locator-based frameworks.\n\nTreat AI output as a draft.\n\nKeep humans responsible for risk, judgment, and exploratory coverage.\n\nMeasure automation by release confidence, not just execution speed.\n\nThe best QA teams are not anti-automation. They are anti-waste.\n\nThey want automation that removes boring work, catches real regressions, and makes releases safer.\n\nThey do not want another brittle system that breaks every sprint and then gets blamed on QA.\n\n## Where Mobile Test Automation Needs to Go Next\n\nMobile testing is where these maintenance problems become most visible.\n\nMobile apps have more moving parts: devices, emulators, OS versions, gestures, keyboards, permissions, network states, app backgrounding, backend dependencies, and fast-changing UI.\n\nThat makes mobile test automation valuable, but also fragile when it is built on brittle layers.\n\nThe next generation of mobile automation needs to reduce the maintenance burden, not just create tests faster.\n\nThat means:\n\nLess dependence on implementation-level selectors\n\nBetter handling of dynamic screens and app states\n\nClear failure evidence with screenshots, logs, and step context\n\nTests grounded in actual app behavior\n\nAI assistance that stays reviewable\n\nWorkflows QA teams can understand and control\n\nRegression automation that improves confidence without bloating the suite\n\nThis is also why tooling language needs to change.\n\nQA engineers are tired of vague claims like “AI-powered,” “self-healing,” and “fully autonomous.” They want to know what actually happens when the app changes.\n\nDoes the test survive?\n\nDoes the tool explain what failed?\n\nDoes QA stay in control?\n\nDoes it reduce maintenance, or does it create another maintenance layer?\n\nThose are the questions that matter.\n\n## A Transparent Note From Quash\n\nQuash is our product, so we have a point of view here.\n\nWe built Quash around many of the same patterns this research surfaced: mobile tests should be easier to create, less tied to brittle locators, grounded in real app behavior, and useful to QA teams rather than positioned as a replacement for them.\n\nThat does not mean every team should drop its current framework. Appium, native frameworks, and scripted automation still make sense for teams with strong automation engineering support.\n\nBut if your mobile automation suite keeps breaking because the UI changes, locator maintenance is eating QA time, or flaky tests have made CI hard to trust, it may be time to rethink the maintenance model instead of adding more scripts.\n\nYou can explore Quash’s [mobile test execution](/test-executor) workflow if that problem sounds familiar.\n\n## Conclusion: Maintenance Is the Real Automation Test\n\nThe first test is not the real test of an automation strategy.\n\nThe real test comes later.\n\nIt comes after the UI changes. After the locator disappears. After the framework updates. After CI fails randomly. After the team has ignored enough flaky tests that a red build no longer creates urgency.\n\nThat is when you find out whether automation is giving the team confidence or quietly creating more work.\n\nQA engineers are not asking for magic. They are asking for automation that respects reality:\n\nApps change.\n\nMobile is messy.\n\nLocators drift.\n\nAI needs review.\n\nFlaky tests destroy trust.\n\nHumans still make the hard quality calls.\n\nROI means confidence, not just speed.\n\nThe teams that understand this will build smaller, stronger, more trusted automation suites.\n\nThe teams that ignore it will keep adding tests to a system nobody fully believes.\n\nAnd that is the **real state of test automation maintenance.**", "url": "https://wpnews.pro/news/sdets-changing-tide", "canonical_source": "https://quashbugs.com/blog/state-of-test-maintenance", "published_at": "2026-08-27 08:12:17+00:00", "updated_at": "2026-08-27 08:49:08.281091+00:00", "lang": "en", "topics": ["ai-tools", "ai-products"], "entities": ["Quash"], "alternates": {"html": "https://wpnews.pro/news/sdets-changing-tide", "markdown": "https://wpnews.pro/news/sdets-changing-tide.md", "text": "https://wpnews.pro/news/sdets-changing-tide.txt", "jsonld": "https://wpnews.pro/news/sdets-changing-tide.jsonld"}}