A Buyer's Checklist for Hiring a Custom Software Agency

Nearly every agency you talk to this year will call itself "AI-native." A buyer's guide from MatrixLabX says those self-assigned labels can't be verified at the evaluation stage and are useless as selection criteria.

Gartner's October 2026 Critical Capabilities for Custom Software Development Services gives you a better starting point. It breaks agency quality into ten capabilities. This post turns the ones that matter most to a buyer into questions you can ask, with the weak answers to watch for.

What Gartner's October 2026 Evaluation Covers

Gartner defines custom software development services as the professional services organizations hire to design, build, modernize or iterate custom applications for unique business needs. These solutions are typically not available as commercial off-the-shelf products. The October 2026 Critical Capabilities document judges providers on how well they build digital products by combining design, generative AI, APIs, preassembled assets and specialized technical expertise.

It is a companion to Gartner's Magic Quadrant for the same market and covers 20 providers: Accenture, Capgemini, Coforge, Cognizant, Deloitte, DXC, Endava, EPAM, GlobalLogic, Globant, HCLTech, Hexaware, IBM, Infosys, NTT DATA, Persistent Systems, Softtek, Thoughtworks, Virtusa and Wipro.

Three Use Cases, Ten Capabilities

Gartner scores those providers against three use cases:

• Unique user experience. Software built for highly specific interfaces and multiexperience touchpoints.
• Unique operational processes. Software that automates and runs workflows specific to the buyer's organization.
• Unique products. Software that generates revenue directly or opens alternative growth channels for the buyer.

The scoring uses ten Critical Capabilities:

1. Business Acumen
2. Design (User and Customer Experience)
3. AI, GenAI and Agentic AI Expertise
4. API and Integration Expertise
5. Analytics and BI Experience
6. Software Engineering Approaches
7. Multiexperience Development
8. Talent Operations
9. Technical Architecture and Cloud
10. Quality Engineering

You may never shortlist one of the 20 firms. The list still works as a map of what a serious agency should be able to explain. The sections below cover the areas that cost buyers the most: AI maturity, code and model ownership, integration and architectural debt, security, and the team that does the work.

Read Analyst Badges With Care

Vendors use analyst reports as sales material, and not always accurately. If an agency shows you a "licensed reprint" with a score, check that its name and its use cases appear in the Gartner document itself. The October 2026 report covers only the 20 providers and three use cases above.

Even accurate claims are positioning. Deloitte says it has held a leadership position for the fifth consecutive year. Thoughtworks promotes its "Visionary" status and ties it to its AI/works platform. HCLTech highlights a Customers' Choice recognition. Treat each as a starting point for questions about how the agency works day to day.

AI Maturity Questions That Expose Marketing Claims

Gartner projects that by the end of 2026, 40% of enterprise applications will include task-specific AI agents by default, up from fewer than 5% in 2025. An agency that can wire an LLM API into a search box is no longer showing much. The question is whether it can build and govern systems that act.

Chatbots Answer, Agents Act

A chatbot built on retrieval-augmented generation answers a question and stops. An agentic system perceives its environment, processes information, selects actions and carries out multi-step workflows across live enterprise systems. It connects to something like a CRM or ERP and writes to it. It also needs to know when to hand a decision to a person, especially in regulated work.

The payoff is real. Rule-based automation applied to complex enterprise workflows delivers 10 to 15% cycle time reductions, according to Alice Labs. Agentic deployments on the same process categories deliver 30 to 50%, roughly three times the gain. Alice Labs credits exception handling. Agentic systems can reason through process variations that rule-based systems escalate to a human.

Named deployments show what "scoped" looks like:

• Morgan Stanley. In early 2025 it deployed a GPT-based code review agent, DevGen.AI, that analyzed over 9 million lines of legacy code. The estimated savings were 280,000 developer hours, and 15,000 developers were freed from manual code translation.
• Klarna. One customer service AI agent handled the workload of 853 full-time employees and saved an estimated $60 million by the third quarter of 2025. Even so, its CEO told Bloomberg in May 2025 that AI chatbots were cheaper but gave lower quality, and Klarna began hiring people again so customers can always reach a human.
• JPMorgan. It runs over 450 distinct AI use cases in production daily.
• AMD. Its agentic HR system cut HR resolution time by 80%, resolved 50% of queries through autonomous self-service and raised employee satisfaction by 70%.

Most of these are the companies' own figures, so hold them to the same test as an agency's proposal and ask for the baseline.

The Rework Tax and the Governance Gap

Those numbers come with a cost that vendor decks tend to skip. Forbes reports that organizations implementing AI tools lose nearly 40% of expected productivity gains to employees fixing low-quality outputs. Larridin cites Workday's January 2026 research, which found that 37% of the time saved through AI is offset by rework.

When a task is automated, the work attached to it shifts toward oversight, validation and exception handling. That work tends to land on more senior, higher-paid staff, which adds wage pressure. Keyhole Software adds that token consumption, real-time monitoring, safety controls and continuous model tuning become first-class cost drivers in agentic systems.

Governance is where many organizations fall short. In an Ivanti study, 85% of IT professionals said there is a named, accountable owner for every AI agent and workflow in their IT organization, but only 42% said that accountability is actually clear. An agency's value, then, depends less on which models it uses than on the governance it builds around them.

How to Test an Agency's AI Operating Model

MatrixLabX suggests looking past the label to the operating model. Ask who does the work, what a human approves before an action runs, and whether the vendor can produce an immutable audit trail of what happened. On the buyer's side, a proper AI maturity assessment looks at the capacity to absorb, use and sustain a solution in production. Larridin lists four dimensions to measure together: adoption breadth (share of users), fluency depth (how well people use it), workflow integration (whether it is embedded in the work) and impact measurement (quantifiable business outcomes).

Three red flags come up in proposals:

• Outcome figures with no baseline or attribution method. MatrixLabX calls these marketing claims, not results.
• A list of the LLM platforms the agency subscribes to, offered in place of an explanation of its human-in-the-loop model.
• Autonomy pitched as a selling point with no defined control surface. MatrixLabX describes this as a liability transfer to you.

There is also a pace test. Alice Labs, citing TechTarget's April 2026 reporting, says GenAI implementations at LegalZoom and Samsara achieved measurable productivity gains within 90 days by scoping tightly to one workflow before expanding. If an agency's first phase is a sprawling platform with no single workflow to measure, ask why.

Checklist 1: AI Maturity and Agentic Capability

Code Ownership, Model Ownership and Contract Gaps

Ownership is where AI-assisted development creates new legal exposure. An AI procurement guide from Arjun Jaggi argues that AI systems need a different approach than traditional SaaS or software buying, because outputs are probabilistic, underlying models change, and data ingestion works in ways standard contracts do not cover.

What Went Wrong in Recent AI Contracts

Jaggi's guide reviewed enterprise AI contracts signed between 2022 and 2025 and found two recurring holes:

• Model deprecation. 78% of the contracts had no explicit requirement for notice when a model is deprecated. If a vendor retires the model version your pilot was built on, the software can break with no warning.
• Training on your data. 43% did not say whether the customer's proprietary data could be used to train the vendor's general models.

The guide's advice for regulated deployments is to make compliance documentation a contract prerequisite. If the agency fine-tunes a model on your data, the contract should say plainly that the resulting IP belongs exclusively to you. That keeps the agency from reselling your custom logic to your competitors.

So the ownership clause should cover more than source code. It should name the custom source code and any fine-tuned AI models built during the engagement. Add the notice period for model deprecation and the cost of getting your data out. Data portability fees and early termination clauses are costs you will want to see before you sign.

Checklist 2: Code and Model Ownership

Integration Debt and Architectural Debt

Most buyers think of software cost as a build cost. It is mostly a keep-it-running cost. Adevs reports that roughly 60% of all software lifecycle costs go to maintenance, and says some studies push that figure to 80% for long-lived enterprise systems. A good agency prices that in from the start and explains what drives it up.

Budget for Maintenance Before You Sign

For standard applications, annual maintenance runs 15% to 25% of the original development budget for every year the software is in active use. Intigate Technologies gives the arithmetic:

• A $50,000 project needs $7,500 to $12,500 a year.
• A $100,000 project needs $15,000 to $25,000 a year.
• A $500,000 enterprise platform needs $75,000 to $125,000 or more a year.

That range assumes reasonably clean code, adequate documentation and modern architecture. When technical debt is high, Adevs says maintenance climbs to 30% to 40% of the original development cost each year. Adevs adds that heavily regulated industries such as healthcare and finance run higher because of compliance overhead.

Ask the agency for its five-year total cost of ownership estimate, including operating system updates and security patching. Weak answers are "modern software needs no maintenance" or no budget for routine dependency updates.

What Technical Debt Costs

The Software Improvement Group defines technical debt as the result of software and architecture decisions that are expedient in the short term but add cost or effort later. It shows up daily as slower releases, rising maintenance effort, higher operational risk and less room to innovate. A structured legacy code audit is often the first step to measuring it. The scale is large:

• Deloitte's 2026 Global Technology Leadership Study estimates technical debt absorbs 21% to 40% of total IT spending. By that estimate, of every €100 spent on IT, €21 to €40 services debt instead of delivering new value.
• The Consortium for IT Software Quality estimated the cost of poor software quality in the United States at $2.41 trillion annually, with a significant share tied to technical debt.
• Rockstar Developer University reports that developers spend more than 17 hours a week, about 42% of the work week, on maintenance issues like debugging, refactoring and navigating bad code. A Swedish longitudinal study found developers lose 23% of their time to technical debt.
• Sonar's 2023 research, as cited by Rockstar Developer University, estimates code-level debt costs roughly $306,000 a year for a project of 1 million lines, about 5,500 developer hours of remediation, and up to $1.5 million over five years.

The upside is measurable too. According to State of Software 2026, moving one system from a maintainability level that flags a delivery risk (2-star) to the recommended threshold (4-star) frees roughly 5.8 full-time equivalents, about €870,000 per system per year in recovered capacity.

The Debt AI Coding Tools Do Not Fix

AI coding assistants are good at code-level debt like duplication and overly complex functions. They generally cannot resolve architectural debt, which takes system-wide context and domain understanding an autonomous agent does not currently have. Gartner predicts architectural debt will make up 80% of all technical debt by 2027, and warns that by 2028 AI will create more architectural debt than it solves, as developers use generative tools to ship poorly integrated components quickly. An AI-assisted legacy modernization approach can keep those tools from adding to the problem.

For a buyer, this means an agency that boasts of AI-driven speed needs a second answer about who owns the architecture. Ask who makes system-level design decisions, how they are documented, and how they are reviewed when AI-generated code arrives in volume.

Softtek's own commentary on Gartner's Magic Quadrant offers a useful test. If a provider treats DevSecOps, cloud-native delivery or quality engineering as optional add-ons or premium tiers, that is a definitive red flag. These should be built into the delivery system from day one.

Integration Is Where Estimates Break

The typical enterprise runs hybrid and multicloud environments with multiple identity providers, modern APIs and legacy monolithic systems, according to Wadline. Integration decides time-to-value and reliability. Clearframe Labs says data integration failures are the number one reason enterprise AI initiatives stall, and cites O'Reilly research in which 81% of enterprises say API quality directly predicts implementation success.

That is why the timing of the integration assessment matters. Each connected system brings its own API limits, permission structures and documentation quirks, and this is where optimistic estimates tend to fail. Artic Sledge advises asking how an agency assesses integrations before it gives you a timeline and cost. A competent agency verifies third-party documentation, confirms API access, tests sample calls and sets fallback plans if data access is restricted. One that assumes any API will just work, without checking, is adding risk nobody has priced.

Multiexperience Delivery

Gartner also scores providers on multiexperience development. Modern products span web, mobile, conversational interfaces, IoT and augmented reality. A multiexperience development platform gives teams an integrated set of front-end tools and "backends for frontends" capabilities for building across touchpoints.

At a minimum, real capability means cross-platform support for iOS, Android and responsive web, plus progressive web apps, chatbots, wearable applications and IoT integrations. For internal apps and multi-team portfolios, these platforms reduce duplicated work by standardizing identity flows, API scaffolding and deployment patterns. If an agency proposes separate codebases for each channel, you are signing up for the fragmentation this approach is meant to avoid.

Checklist 3: Engineering and Integration Debt

Security Practices You Can Verify

Security claims are easy to make and easy to check. IBM found that 71% of enterprises now rank data privacy as their top selection criterion for AI and software vendors, according to Clearframe Labs. Ask for evidence in four areas.

Certifications, Encryption and Access

• Certification. Pertama Partners says to require SOC 2 Type II certification or a recognized equivalent, and to look for ISO 27001 certification as evidence of a mature information security management system. Ask to see the audit report. An agency that only points to its cloud provider's certifications has no audit of its own operations.
• Encryption. Pertama Partners recommends encryption in transit with TLS 1.2 or higher and encryption at rest with AES-256 or equivalent.
• Access control. The delivered software should support single sign-on, multi-factor authentication and granular role-based access control. Hardcoded credentials, or no native integration with standard enterprise identity providers such as Active Directory, should end the conversation.
• Data residency and audit trails. You need clear boundaries on where data is stored, for regulations like GDPR or HIPAA, along with immutable audit logs and transparent compliance reporting.

The AI side adds its own security questions. If the agency builds agents that touch your systems, ask what an auditor would see, as in Checklist 1, and how hallucinations are tracked across multi-step workflows.

Checklist 4: Security and Compliance

Talent Operations and Team Continuity

Gartner counts talent operations as a critical capability because an agency's ability to source, upskill and retain engineers drives project continuity and the preservation of institutional knowledge.

What Turnover Does to a Project

Rockstar Developer University puts technology turnover at 13% to 25% a year, depending on role and region. Replacing one developer costs 50% to 200% of that developer's annual salary. For a senior engineer earning $180,000, that works out to $90,000 to $360,000 per departure, covering recruiting, onboarding, lost productivity during ramp-up and disruption to client deliverables.

Scale it up. At a 500-person engineering organization with 13% turnover, 65 developers leave each year. On a long engagement, each replacement has to relearn your business logic, which Rockstar Developer University says slows delivery and raises the risk of critical defects.

A Market Short on Senior Engineers

A Hashbyt article on Medium argues that talent metrics often look at the oversupply of junior and mid-level developers and miss the real shortage. That shortage is in senior, production-ready engineers who can run AI in production, own complex architectures and make sound long-term decisions.

Contract Terms That Protect the Team You Were Sold

Ask for the agency's engineering turnover rate and what protects your institutional knowledge if a lead developer leaves. Written handover and documentation processes are a reasonable thing to ask for. Then put the team in the contract. Artic Sledge and DesignRush both recommend key personnel clauses so the senior architects and business analysts who pitched you are not rotated off after discovery.

Business acumen is the capability that is easiest to skip. Ask how success will be measured, such as cycle time reduction or a drop in manual data entry, and who is responsible for collecting those numbers. "Shipped on time" is a weak definition of success, and so is putting the measurement burden entirely on you.

Checklist 5: Team and Operational Rigor

Running the Selection Process

A good checklist still fails inside a poor process. Viewpoint Analysis says enterprise software selections rarely go wrong because the market lacked good options. They go wrong because the process was too slow, too complicated or missing validation steps.

A Faster RFI

Viewpoint Analysis also says traditional requests for information produce sprawling questionnaires that take weeks to write and draw copy-pasted answers. It recommends a leaner "Rapid RFI" that moves from a longlist to a shortlist of four to six relevant vendors. Its guidance:

1. Lead with a problem statement. Describe the business challenge in plain language, with enough context for the vendor to calibrate its technical response.
2. Keep it to five to ten pages. Ask only for what shows whether the vendor has a credible, experience-backed answer.
3. Keep the decision team to three to five people. State the timeline and expect full engagement from vendors.

The five checklists above fit into this format. Put the questions that matter most to you in the RFI, and save the contract-level ones for the finalists.

Price the Whole Relationship

Clearframe Labs says vendor selection should include a total cost of ownership analysis, since the development quote is never the full cost. It lists the hidden items to account for:

• Migration fees. Data transfer and re-platforming of legacy databases.
• Integration labor. Internal IT hours needed to support the agency's integration work.
• Compliance upgrades. New certifications and security audits.
• Change management. Retraining for team upskilling and new workflows.
• Exit penalties. Data portability fees and early termination clauses.

Add the maintenance range from the engineering section and the AI running costs Keyhole Software lists: tokens, monitoring, safety controls and model tuning. A proposal that omits these has an incomplete price.

Conclusion

Gartner's ten capabilities are most useful as a way to ask better questions. Start with the five checklists. Send the AI, security and team questions with your RFI, and hold the ownership and exit questions for the finalists. Score each answer against the weak responses listed, and ask for documents, not descriptions.

If you want a second set of eyes on an agency proposal or a contract draft, Baytech Consulting is glad to talk it through with you.

FAQ

What does Gartner's October 2026 Critical Capabilities report evaluate?

It evaluates 20 custom software development providers against three use cases: unique user experience, unique operational processes and unique products. It scores them on ten capabilities, including AI, GenAI and agentic AI expertise, API and integration expertise, software engineering approaches, talent operations and quality engineering.

How can I tell whether an agency's AI maturity is real?

Ask about the operating model. MatrixLabX recommends finding out who does the work, what a human approves before an action runs and whether the vendor can show an immutable audit trail. Treat outcome figures with no baseline, a list of subscribed LLM platforms and autonomy with no control surface as red flags. Also ask for a single scoped workflow with measurable results at 90 days.

How much should I budget for maintenance after launch?

Plan for 15% to 25% of the original development cost every year, according to Intigate Technologies. That means $15,000 to $25,000 a year on a $100,000 project. Expect 30% to 40% if the code carries high technical debt, and more than the standard range in regulated industries such as healthcare.

What should the contract say about code and AI model ownership?

It should give you exclusive ownership of the custom source code and any fine-tuned AI models built during the engagement. It should bar the agency from training general models on your data and set a notice period for model deprecation. In the contracts Arjun Jaggi reviewed, 78% had no deprecation notice requirement and 43% were silent on training use.

Which security certifications should an agency hold?

Pertama Partners says to require SOC 2 Type II certification or a recognized equivalent, and to look for ISO 27001 certification as evidence of a mature information security management system. Ask to see the audit report. Also ask about encryption in transit and at rest, SSO, MFA and role-based access control in the software you receive, and data residency and audit log practices. Certifications held only by a cloud provider do not cover the agency's own operations.

Why does agency turnover matter to my project?

Each departure forces a replacement to relearn your business logic, which slows delivery and raises defect risk. Replacing one developer costs 50% to 200% of annual salary, and industry turnover runs 13% to 25% a year. Ask for the agency's own turnover rate and put key personnel into the contract.

How long should an RFI be, and how many vendors should I shortlist?

Viewpoint Analysis recommends an RFI of five to ten pages that opens with a problem statement, leading to a shortlist of four to six vendors. Keep the internal decision team to three to five people and share the timeline up front.

Supporting Links

• Gartner Critical Capabilities for Custom Software Development Services, October 2026
• Alice Labs on AI implementation case studies and cycle time gains
• Ivanti research on scaling AI in IT operations
• Arjun Jaggi's AI procurement checklist
• Software Improvement Group on the cost of technical debt
• Artic Sledge on questions to ask a software development agency

About Baytech

At Baytech Consulting, we specialize in guiding businesses through this process, helping you build scalable, efficient, and high-performing software that evolves with your needs. Our MVP first approach helps our clients minimize upfront costs and maximize ROI. Ready to take the next step in your software development journey? Contact us today to learn how we can help you achieve your goals with a phased development approach.

About the Author

Bryan Reynolds is an accomplished technology executive with more than 25 years of experience leading innovation in the software industry. As the CEO and founder of Baytech Consulting, he has built a reputation for delivering custom software solutions that help businesses streamline operations, enhance customer experiences, and drive growth.

Bryan’s expertise spans custom software development, cloud infrastructure, artificial intelligence, and strategic business consulting, making him a trusted advisor and thought leader across a wide range of industries.