How Enterprise IT Operations Are Transitioning from Human Staffing to Agentic AI Intelligence

The modern enterprise operates at a pace that has long outstripped human response capabilities. As hybrid cloud infrastructures, microservices, and continuous integration and continuous delivery (CI/CD) pipelines accelerate business-critical software delivery, they simultaneously generate an overwhelming flood of monitoring alerts. This velocity of system data has turned traditional IT operations (ITOps) into a crisis management function defined by alert fatigue, skyrocketing remediation costs, and severe operational vulnerability.

The consequences of this operational strain are no longer theoretical. In July 2024, a single faulty software update triggered a global IT outage that paralyzed transportation networks worldwide. Following the incident, a major U.S. airline disclosed in a filing with the U.S. Securities and Exchange Commission (SEC) that it suffered approximately $380 million in lost revenue alongside $170 million in added operational costs. The catastrophe stemmed from roughly 7,000 flight cancellations over a span of five days, highlighting how fragile enterprise software ecosystems have become when exposed to automated, systemic failures.

Beneath these high-profile catastrophic failures lies a daily, grinding erosion of productivity driven by "alert storms." Empirical research presented at the 2020 International Conference on Software Engineering analyzed a large commercial bank and uncovered a troubling weekly pattern: the institution routinely experienced alert storms ranging from hundreds to thousands of alerts per minute. Each storm required an average of six engineers spending 55 minutes apiece just to triage. Of the 44 engineers surveyed in the study, 77% stated that the sheer volume of alerts made it practically impossible to manually identify true system problems. Compounding this, researchers observed a dangerous psychological defense mechanism among engineering teams—most engineers tended to ignore the alerts entirely until customer complaints finally exposed the underlying failure.

To examine how forward-thinking enterprises are breaking this cycle, Emerj recently hosted a comprehensive series on the AI in Business Podcast. The program explored how organizations can modernize IT operations by moving away from reactive firefighting toward predictive prevention, alert noise correlation, and the automation of repetitive Tier 1 (L1) tasks. The podcast featured insights from industry leaders including Assaf Resnick, Founder and CEO of BigPanda; Luke Rotta, Director of Site Reliability Engineering at Charles Schwab; Dave Glick, Senior Vice President of Growth Technology at Walmart; and Dhrubojyoti Das Deb, Senior Vice President of Technology at JPMorganChase.

The overarching consensus among these leaders is clear: as enterprise architectures grow exponentially more complex, the economics of IT operations must shift from scaling human headcount to scaling intelligent automation.

The Anatomy of an IT Crisis: From Data Overload to Knowledge Graphs

For decades, the dominant corporate response to escalating technical complexity has been to scale human resources, often through outsourcing to global systems integrators. However, Assaf Resnick argues that enterprises are fundamentally misdiagnosing the issue. IT operations is no longer a staffing problem; it is a data problem.

Outsourcing models have routinely fallen short of solving this structural issue. With average engineer tenures hovering around seven months in outsourced environments, institutional knowledge frequently departs just as workers become familiar with proprietary system architectures. Furthermore, maintaining 2nd- or 3rd-shift coverage necessitates multiple handoffs per day, creating inherent vulnerabilities where critical operational context is lost. Dhrubojyoti Das Deb echoes this risk, noting that deep technical knowledge of legacy applications often departs alongside the specific subject matter experts who built them.

Resnick advocates for a paradigm shift: enterprises must onboard autonomous AI agents the same way they onboard human employees—by embedding them within an enterprise-owned IT knowledge graph. Unlike vendor-locked monitoring systems, an internal knowledge graph captures three vital pillars of operational context: architectural topology (how applications and infrastructure components connect), historical telemetry (past alerts, incidents, and resolutions), and tribal knowledge (change tickets, documentation, and expert notes).

To understand the practical impact, consider a hypothetical network failure affecting a commercial bank’s automated teller machine (ATM) network. In a legacy environment, thousands of disconnected alerts flood an L1 support team unfamiliar with the specific sub-network. A triage process targeted for 15 minutes stretches across two hours, followed by an emergency bridge call lasting six additional hours before engineers finally trace the root cause to a minor security configuration update deployed three weeks prior.

With an intelligent knowledge graph and specialized AI agents deployed, the system identifies the anomaly within 30 seconds, instantly matches it against a structurally similar incident from seven months earlier, and recommends a verified rollback procedure. Where the agent’s confidence score dips below a predefined threshold, it does not convene a chaotic bridge call; instead, it intelligently routes the ticket directly to the specific human domain expert responsible for that infrastructure layer.

Resnick notes that this operational shift alters the core cost structure of enterprise IT, replacing a rotating corps of 300 outsourced operators with a streamlined agentic layer overseen by two or three site reliability engineers (SREs). He points to a Fortune 500 healthcare enterprise that successfully utilized this approach to eliminate incidents and self-inflicted outages valued at half a billion dollars. Crucially, Resnick warns organizations against attempting to overhaul all tools, processes, and outsourcing contracts simultaneously. Drawing a parallel to how chief information officers successfully migrated enterprise architectures to the public cloud, he advises shifting workloads one workflow at a time.

Cross-Application Alert Correlation and the Hunt for Root Causes

According to Dhrubojyoti Das Deb, the primary bottleneck in modern incident response is no longer a scarcity of monitoring data, but rather the inability to pinpoint where a fault originated. While logging, distributed tracing, and telemetry tools have proliferated across enterprise ecosystems, each application’s diagnostic data remains trapped in isolated silos.

In a chain of five interconnected microservices, Das Deb explains, the fourth service will frequently display the visible error message, while the actual point of failure occurred in the first service. That initial service may have thrown no direct alerts while silently passing corrupted data through the second and third components. Consequently, engineering teams waste precious hours investigating the downstream symptom rather than tracing the fault back to its upstream source.

Luke Rotta reframes the cost of this investigative delay through the lens of business opportunity cost rather than raw downtime alone. When a customer-facing transactional service experiences latency or failure, the enterprise instantly loses the ability to capture revenue, with the financial penalty compounding during peak business hours. Dave Glick observes a parallel drag on internal engineering productivity. When an outage ticket arrives, highly paid software engineers are forced to act as "digital detectives," spending hours trying to reconstruct events and reproduce errors—a tedious operational toil that both Glick and Rotta aim to eliminate.

Das Deb identifies three primary frictions that prevent teams from separating signal from noise during an outage: tool sprawl, fragmented telemetry data, and a lack of standardized logging protocols. Rotta argues that simply layering unintegrated observability tools on top of one another exacerbates the problem without giving responders the rapid contextual clarity they require. To combat this, Das Deb’s engineering teams enforce a strict limit, constraining analysts to between one and three core observability tools across as many applications as possible to ensure seamless data correlation.

Rotta outlines the essential questions that any automated context layer must answer instantaneously during an incident: What changed? What is specifically broken? Why is it failing? Who is the designated point of contact? And what happened the last time this exact signature occurred?

To answer these questions, Das Deb envisions AI operating in a structured, two-stage workflow. First, machine learning models ingest thousands of raw log lines and error messages, aggregating them to reveal that 5,000 distinct alerts are actually minor variations of just two underlying root-cause incidents. Second, specialized AI agents—spanning databases, network infrastructure, and cloud environments—collaborate to investigate the anomalies in parallel.

This shifts the daily reality for engineers. Before AI adoption, analysts manually waded through thousands of alerts, often barking up the wrong tree by investigating symptomatic applications. Post-adoption, AI groups the alerts, correlates them with deployment logs and historical incident data, and presents human analysts with a highly probable root cause for rapid validation.

Volume-First Automation and the Elimination of Operational Toil

When determining where to deploy automation, Luke Rotta advises enterprises to prioritize high-volume, low-risk operational toil over high-risk, low-frequency events. High-risk failures typically have robust contingency plans and fail-safes already built around them. The true drain on corporate culture and team morale is death by a thousand cuts—repetitive, manual tasks that persist despite being decades-old problems. For instance, Rotta points out that many major financial institutions still employ human operators to manually clear out servers running out of disk space.

By applying AI-driven automation to these high-volume patterns, enterprises can autonomously resolve known anomalies, log the remediation actions, and escalate only those exceptions that require human judgment. Assaf Resnick notes that this eliminates the archaic operational scramble where Tier 1 operators must frantically phone network, cloud, and development teams just to confirm whether an outage is localized or systemic.

At Walmart, Dave Glick utilizes weekly AI innovation sessions where roughly 75% of demonstrated automation projects focus on eliminating routine toil—such as change request processing and release management—rather than building brand-new customer-facing features. Tasks that once required dedicated administrative teams, such as compiling weekly incident ticket summaries, are now completed by AI in a matter of minutes while surfacing systemic behavioral patterns across the business.

However, organizations frequently encounter what Rotta calls the "hamster wheel" phenomenon: engineering teams are so overwhelmed by incoming operational volume that they literally lack the time required to build the automation that would free them from it. To break this cycle, Rotta famously advocates for radical experiments, such as deliberately disabling non-critical alerting for a single day to measure the impact. In many cases, business operations remain entirely unaffected, while the few genuine issues that surface reveal precisely which alerts possess true operational value.

For highly regulated institutions that cannot risk disabling alerts, Rotta recommends a pragmatic alternative: maintain raw alert logging for compliance, but mandate that engineering teams react exclusively to breached Service Level Objectives (SLOs) and Service Level Indicators (SLIs). Furthermore, Dhrubojyoti Das Deb suggests building operational confidence incrementally by cataloging all daily friction points, prioritizing them by remediation ease, and successfully automating the two easiest items to build organizational momentum for broader AI deployments.

Shifting Focus from Ticket Mitigation to Complete Ticket Elimination

Adding headcount to Tier 1 and Tier 2 support desks to absorb rising ticket volumes is viewed by industry leaders as a fundamentally flawed strategy that treats surface symptoms while ignoring structural root causes. Dave Glick champions an aggressive operational philosophy centered on a clear hierarchy of action: "Eliminate, automate, optimize. The best ticket is no ticket."

Glick encourages engineering organizations to aggressively hunt down what he terms "paper cuts"—the minor, long-standing system friction points that continuously frustrate both internal enterprise users and external customers. In one instance, enterprise sellers were frequently blocked by an arbitrary image upload limit that no one could actively justify. Implementing a simple, one-line code modification permanently erased that entire class of support tickets, simultaneously driving down error rates and eliminating seller friction. Left unaddressed, Glick warns, these small operational annoyances metastasize into major business roadblocks.

To scale this approach, Glick’s organization relies on three foundational practices: embedding automated testing protocols directly into CI/CD pipelines, establishing continuous feedback loops between customer service teams and core software engineers, and deploying generative AI agents to shoulder the burden of routine software validation. For example, during a recent careers site launch, an autonomous coding agent successfully wrote the entire test suite in approximately one week—a labor-intensive engineering task that would have otherwise required months of manual developer effort to complete before a hard product deadline.

Assaf Resnick reiterates that true IT modernization begins with prevention rather than faster reaction times. By deploying proactive change-management agents that evaluate proposed system modifications before they reach production environments, enterprises can intercept risky updates before they trigger catastrophic outages. Dhrubojyoti Das Deb adds a final, foundational rule for engineering leaders: when an incident repeats itself, the repetition is not merely an operational inconvenience—it is an explicit organizational failure to solve the underlying architectural flaw.

By transitioning from human-heavy firefighting to intelligent, agentic IT operations, modern enterprises can finally reclaim team capacity, protect crucial revenue streams, and build resilient digital infrastructures capable of scaling securely into the future.