Real or sci-fi: Robotic software automation

Software robotics Robotics automation CSC Blogs

Is it possible to automate new business processes without IT integration?

How old are you? Remember “screen scraping”? You plugged a lead into a computer terminal’s auxiliary port in order to intercept the characters displayed. This information could then be fed to another program. Screen scraping was a crude way to integrate two systems and so make a new process.

Over time, screen scraping grew in sophistication. Eventually it became possible to reliably drive one system, via its green screen UI, purely by simulating keystrokes generated by another program. In effect, software was pretending to be the user. When real users were employed to do this it was called “swivel chair integration.” The idea of screen scraping was to avoid manual re-keying when several systems that had not been designed to work together were involved in a process or transaction.

Surely we don’t need screen scraping today? Don’t we now have enterprise service buses, service oriented architectures and business process engines? Aren’t all of our enterprise applications designed to work seamlessly together via well-defined, clearly published, loosely coupled and RESTful interfaces?

Dream on. Imagine this:

A large enterprise has tens or hundreds of applications. They sit on various platforms. They share some, but not all, of the enterprise data. The CIO is seeking to reduce the complexity, and cost, of his IT estate. A transformation (modernization) program is under way, but many business-critical applications are a long way down the queue for treatment. A top-down strategic initiative is under way to “service enable” the enterprise architecture (EA), but it seems to be making slow progress. Now hop over to the C-Suite:

The business exists in a rapidly moving market. The CMO is trying to build new end-to-end customer experiences. It comes as a shock to her that the IT team claim that before this is possible several systems will  need to be re-factored, re-platformed and then integrated to support the new “customer experience” that the “digital” team with their walls full of Post-It notes has envisaged. While the business case for these complex IT changes is being prepared, the call center is coming under pressure. They are reporting that they cannot cope with escalating workloads. They sense the opportunity to automate some processes. Surely, those manual steps in customer-facing processes – copying information from one system to another, reconciling databases, handling each case – could be automated. A lot of the mess seems like non-value-added work to the new “digital” team. And just to make matters more complicated, two business units have decided to outsource routine customer transaction processing (the clerical, well-defined, repetitive work not requiring much human judgement) to third-party BPO specialists, including offshore.

What’s to be done?

In this fast-moving business environment, does it make sense to wait for neatly automated processes in the IT queue? It would be cute to believe that the EA team can create the clean IT environment in which building a new process is nothing more than a point-and-click exercise. In reality this is rarely the case, perhaps other than in green-field Web scale enterprises.

For all of these reasons, screen scraping has not gone away, but may have grown up. The industry is now calling it “Robotic Process Automation.” It may not be regarded as a strategic solution by purists, but it will find a role in the IT mix. Whether your IT strategy can support it is a question only individual CIOs can answer.

In the past there was no way on earth that a traditional screen-scraping solution could provide what a modern major enterprise now needs: transactional integrity across their systems when data changes, inherent data protections, audit trails and other systems management functions. Put simply: screen scraping of old was “Lipstick on a Pig.” The same is true of what some call “Web scraping.” Changing the surface UI format alone does not make a modern automation solution.

As screen scraping technology evolved, it needed a new name. Start-ups bringing new tools to market did not want to be associated with the negative connotations of the past.

Robots. Robots everywhere!

The phrase “robotic software automation” sometimes “robotic process automation” (RPA) was coined as an illusion to the use of robots in manufacturing to replace humans on the factory line. The idea is more subtle than it might first appear.

Robotic software is designed to be used by business users to allow them to build and deploy new processes across systems that were never designed or intended to work together. Suppose you are the manager of a BPO unit or call center operation. You are being asked to do more with less. You are facing targets for head count reduction. Or perhaps your business is growing, and more customer interactions need to be supported yet the business cannot afford to increase front-office capacity. With the right tool, perhaps a software robot could offload repetitive work from humans?

In one case study, 10 software robots replaced 20 human FTEs. The processes that were automated (partially or fully) included client data changes, client policy changes, complaints handling and renewals. Using robot software, no costly IT integration was required. Even consistency and accuracy improved. How?

I should explain that I am not in the business of advocating the replacement of knowledge workers with robots. Far from it. Decisions on these matters are for those who understand their own their business priorities. The possibility to automate away non-value-added work, however, exists. One manufacturer used software robots to replace 200 people from its order-taking process. Pressure on front or back office may drive such difficult decisions. And there could be other factors that demand automation. Companies operating in a global environment must provide 24-hour customer response.

Putting aside debates about “The Future of Work,” how does robotic software automation actually work?

The key is to place the automation capability directly into the hands of those who know what to do with it – the staff directly involved in the process they deliver to customers. They know the IT systems inside-out, not from the perspective of systems architecture, but by how they support (or don’t support) a good customer experience and – most importantly – productivity of the front-office and back-office teams. These “on-stage” and “backstage” customer workers also understand the intricate details of the application UIs and what is required to make their job easier. Trying to pin down a functional specification for a new IT project is not going to work for them, because the business is simply moving too fast!

Just as a spreadsheet user knows how to plug in formulas to get a job done, the user of robotic automation software knows what rules and interactions to configure to create a new process that works for them, in the context of what they do – day-to-day, hour-to-hour.  As long as the automation software is user-friendly they can do the job themselves, and they often do. That’s not to say that consultants (internal or external) might not also be needed to shape the project, to find the business cases and high-impact opportunities to automate. Process redesign expertise is often required.

And IT will need to be involved. After all, they are being asked to support a new tool. The CIO will want to know how the new tool works, whether it can be supported, whether it is resilient, what load it places on existing applications, and whether it could compromise corporate compliance. They will also want to see a business case in advance of software purchase.

Sceptics abound.

When software robots are discussed one question always arises: If existing applications are themselves subject to change, will the robots continue to work, or will the rules and workflows break as the application UIs change?

The best robotic automation solutions are designed to minimise such impacts. Remember how the “model-view-controller” principle applies to the design of a good UI. Well, robotic automation software can also isolate itself (within limits) from changes in the application it orchestrates. Newly coded business logic can be written in a way that is transparent and does not hold any temporary data of its own. Robots can even be written to deal with exception cases arising from unexpected application behavior. Permissions can be added to ensure that new robot scripts are only modified by those authorized to do so.

It is helpful to think about the potential of robotic automation using the standard model of any automation. Does the process replace a human, or assist a human? Does it automate all tasks, or a subset? Does it extend the human work, or limit it? Where is boundary between non- and value-added work?

The best robotic automation software will assist the business analyst to identify and evaluate opportunities to automate, and provide features to ensure that those processes evolve thereafter with the business. Yes, this is a form of business process management (BPM)!

Robotic software is a fuzzy field.

Despite the efforts of industry analysts to define the field of “robotic automation,” it remains somewhat ill-defined. The functionality of products in the marketplace varies considerably.

irpa2The grandly titled Institute for Robotic Process Automation (IRPA) has said that the automation of knowledge work will be “this decade’s engine of growth.” Robotic solutions will “free up human labor to provide additional capacity for more strategic work.”

The new industry group points to solutions that allow non-technical employees to reconfigure software “robots” and “assistants” to capture and interpret existing applications for processing transactions not envisaged by the IT teams that created those applications: manipulating data, triggering responses and communicating with other digital systems. The primary benefits will accrue to “those companies whose people perform high-volume, highly transactional functions.” Use cases cited include IT support processing, workflow processes, remote infrastructure management and back-office work of all kinds.

The technology of robotic automation is not well-defined.

It certainly includes presentation-layer automation software. Solutions of this kind mimic the steps of a rules-based, non-subjective process, without compromising the existing IT systems. Examples exist in finance operations, procurement, supply chain, customer service, data entry and purchase order processing. In this way, human teams can use software robots to scale up to meet demand, perhaps in response to a spike in workloads, or permanently control labor costs. Such solutions can either be bolted into an existing application to enhance its ability to capture routine work, or across multiple applications in “swivel chair” fashion.

focusThe problem with a fuzzy field like this is that it can lose its focus.

Some proponents of the term “software robot” extend its use to all kinds of diverse approaches, including workflow engines, intelligent agents and voice response systems. IBM was even invited to speak about its Watson technology in this context at Automation Innovation 2014, the inaugural conference of IRPA.

The interest in software robots appears to have grown out of the outsourcing industry’s interest in all kinds of automation, and their constant quest for relevance and efficiency. Presentation-layer automation (a.k.a. the evolution of screen scraping) is said to be a mysterious transformation technology, disrupting many outsourcing players. I think this may be somewhat of an exaggeration, but the fact remains that automation is playing a greater and greater role in all business processes.

Vendors that took to the stage at Automation Innovation 2014 included BluePrism, Automation Anywhere and a raft of BPO providers, including HCL, Cognizant and Tata. Other solutions that have appeared in media and analyst coverage include OpenSpan, IPSoft, AyehuGenFourOpenConnect, Verint Systems, Celaton inSTREAM, Arago, ThoughtonomyInnovise and Pegasystems. What a crazy mix!

It is interesting to see Pegasystems mentioned. This “BPM” solution is built on a rules, not a workflow, foundation. Decision-making is a large part of what’s required in robotic automation. Expect other BPM suite vendors to consider jumping on the marketing momentum created by the “robotic automation” moniker.

I should clarify one final point. Software robots – however defined – will not replace traditional IT systems integration. It will not replace message queues, SOA, process design and process execution engines. Far from it. Surface-layer robots will always act in a complementary role. Whether you regard this modern form of “screen scraping” as lipstick on a pig, or a strategic business weapon, only you can decide.

A tool is a tool. And a fool with a tool is still a fool. Long live business process thinking.


  1. Great piece, Howard.
    I agree there is fuzziness in the definition of RPA. I am gobsmacked to see Pega in the mix – to me that is traditional IT: BPMS. I admire Pega very much, as a company and its software, but any analyst that is grouping this in the robotics quadrant as missed the point.
    RPA is trying, above all things, to be new way of integrating, automating and orchestrating applications to complete business processes that have escaped the IT program. Of course, in a perfect world, we would have enough IT resources (and cash) to automate the enterprise. In this perfect world, there wouldn’t be thousands of clerical staff manning back offices to manage all the exceptions from the sunny day scenarios; and the BPO and ITO industries would not exist.
    But there will never be enough IT to go around, and the cost of IT programs does not make economic sense for the granular processes in the long tail of automation that every operational manager knows all too well.
    So, any new approach will only work if it is business led. This is the only way to make it fast enough to respond to real business events: a bulk processing error, a competitor product launch, a new market opportunity, or simply a BAU headache that will never make it up the IT priority list.
    But, previous business led solutions like scripting, macros or spreadsheets for example, have fallen into the trap of becoming “Rogue IT” – the support for which is often dependent on a lone local employee with unique technical skills. So, there is a need for proper governance, resilience, security, stability, regulatory compliance, scalability and analytics if RPA is going to become a credible enterprise solution. This comes not only from the software but from the operating model and methodology that surrounds it.
    Of our customers, the ones that have achieved transformational benefits are those who have successfully embedded RPA as a persistent capability, a center of excellence that relentlessly reacts to business priorities and serves the organization with a virtual workforce.
    This is about digital labor as an alternative to human beings, not a replacement for traditional IT; finding the best resource (human, robotic, or industrial IT automation) for each business challenge is the key to operational excellence.

  2. Great post, way better than my usual pitch.

    I believe the “battle” between RPA and BPM is as old as the technology itself. It reflects the two ways of automation, a transformative one (BPM) and an adaptive one (RPA). Steam engine technology has transformed the landscape to railroad us thru industrial revolution. Fast forward to self-driving cars. Little doubt it might have the same order of magnitude impact as the steam engine. Would it work if it would require us to rip up all paved roads and install infrastructure for the new cars to move around on their own? On the contrary, the self driving robot acts like the presentation layer software robot. It senses the surroundings as the software robot senses the buttons and links and fields on the screen. It is the right approach for making self-driving cars a reality and RPA is a right approach to automate those many low-value, low-frequency business processes that fall thru BPM sieve.

    I second Alastair saying that the BPO industry makes the most compelling case for RPA to become a mainstream technology. How silly would be to have millions of people re-keying data in a self-driving car world.

  3. Gianluca Sini says:

    This post deserve more likes, I’m a fan of this approach.

    Let’s say that “Screen Scraping” is still needed because lack of Open Data; I found myself writing xls code to scrape data out of notes mails, out of other tools, because the report I receive are changing every now and then, and because sometimes they are mistakenly manually/copied/pasted from other data sources.

    Less human interactions on raw data means less faults in reports and actions flows.

  4. Koushik Radhakrishnan says:

    Enjoyed the read – I completely agree it is the way of the future. It needs critical thinking so that we don’t automate incrementally but make a more radical shift of non cognitive work to smarter robots. The leadership efforts for vision, awareness and acceptance part of this particular change management for automation is going to be exciting.

  5. hsmith50130 says:

    Thank you all for your comments. I think we shall hear more of this topic going forward. Here is a recent piece in Infoworld:

  6. Howard,
    A very thorough and insightful piece.
    What it also serves to reinforce is the perspective we take when evaluating opportunities for automation. The majority of those in the market, and those who analyse, advise and comment on it, have focused on the benefits of RPA in volume repetitive tasks performed by (typically entry level) administrative staff.
    And of course with good reason, since the nature of that work supports both the technology use case and economics of a business case for its deployment.

    But we also see wider applications in more complex work where more in-depth subject matter expertise is required.
    Consider the daily tasks an expensive, highly skilled database administrator might undertake across multiple environments or systems, or the actions of a service manager in extracting, manipulating and formulating information to compile monthly reports, or the “ready for trading” checks across systems and applications a bank’s IT staff might need to undertake each day before trading can commence. Neither volume processes nor delivered by entry level resources, but structured, manual and requiring interaction with a variety of systems, tools and applications nonetheless – and solid candidates for RPA.

    There are a myriad more examples I could provide, but suffice to say this; the ability to use the Virtual Workforce to automate human activities extends well beyond the volume back office team, and it’s place in an IT strategy alongside BPM, ERP, ITSM and other tool suites, and the transformation programs to which you refer is both warranted and proven.

  7. nigel barron says:

    HfS has been extensively research and writing on robotic process automation (RPA) / Autonomics technologies for the last several years and now in mid 2015 we take stock of the maturity of that technology in the mind of enterprise buyers and where we see the future of process automation innovation in the 2H of 2015 and beyond.

  8. Hola estuve observando durante unas 4 horas en la nube hoy y no
    he llegado a ver nada tan ejemplar como tu articulo. Me ha exclamado mucho el modo que posees al escribir imagino que es lo que me ha
    llegado. Para mi, si un porcentaje alto los propietarios de webs tuvieran este tipo de comentarios, Internet seria mucho mas practico.
    Espero que ofrezcas mas por estos lares y siga disfrutando posts tan buenos como este.


  1. […] Real or sci-fi: Robotic software automation […]

  2. […] Real or sci-fi: Robotic software automation […]

  3. […] Real or sci-fi: Robotic software automation […]

Speak Your Mind


This site uses Akismet to reduce spam. Learn how your comment data is processed.