How Control-M Automation API and Control-M Workload Change Manager accelerate the delivery of your digital business services
The software Copernican Revolution: customer in the center, not code
I’ve spent many years working in a software development laboratory, enough to see the software development process transitioning from waterfall to agile, and I’ve seen how much the agile process provides benefits to customers.
The waterfall model, where all software development phases from conception, to analysis and design, through implementation, testing, and production, are sequentially performed, and the software is released only at the end of the project, raises a serious problem. This approach can barely keep up with the pace at which businesses today need to deliver new offerings and enhancements to remain competitive. And more critical, by the time the code is delivered, the customer requirements may have changed. If you are using the waterfall development methodology, by the time you know the requirements need to be adjusted you are likely too far down the path to affect any changes until the next development cycles begin. Over time, you will lose any hope of gaining the competitive advantage over others who have adopted the agile approach to software development and delivery.
The agile methodology captures customer feedback earlier in the development lifecycle. Additionally, by going through all phases in smaller, continuous cycles, it ensures faster delivery of code drops, gradually delivering value to the market, and providing more chances to meet customer needs.
In the software development world, agile won the race, and established itself as the best way to deliver faster and higher quality application software. But in organizations, application code still has a long way to go to get to production.
Most of the applications, especially data manipulation or business analysis, are made up of batch components. In order to deliver value, they need to be managed through job flows. That means they need to be automatically launched and linked at specific times, under specific conditions. But people who develop the applications’ business logic do not typically dedicate time to automating job flows. They just stick applications together using scripts, basic schedulers, or by stretching Continuous Integration/Continuous Delivery capabilities of tools like Jenkins.
That means that once the software is ready (implemented, tested, run), applications are passed by developers to operations (schedulers). Even if your developers have adopted an agile approach to application development, all too often this handoff to operations begins another waterfall process. In order to run applications according to organizational standards, schedulers will likely rework the scripts and then design, implement, test, and promote all job flows through the standard workload automation tool. But again, by the time the job flows are deployed, initial needs for the software may have changed and at this point those changes are difficult if not impossible to absorb.
Agile itself may no longer be enough: you need to shift automation earlier in the application release process
What good is agile development to application delivery if the race stops at the handoff between developers and operations?
Developers may rush, but operations must care about stability and compliance more than time-to-market – the DevOps dilemma emerges again and slows the application delivery lifecycle down.
Control-M Automation API is specifically designed to eliminate these issues. Control-M Automation API allows jobs to be shifted left, so that job implementation, test and promotion occur earlier in the application release lifecycle. All of the job delivery phases can be implemented in a self-service manner by the developers, and in the best way possible: through coding.
This concept is referred to as Jobs-as-Code. Exactly like it happens with Java, or Python for example, Jobs-as-Code can be tested and debugged on developers local machines, even with no network connection, using Control-M Workbench.
Jobs and job flows are developed, tested and debugged all together with the code. The jobs can then be stored and promoted through existing Source Control Management tools, such as GIT, or Continuous Integration/Continuous Delivery tools, such as Jenkins. Managing Jobs-as-Code is a powerful and efficient way to embed jobs as another artifact in the application release model, foster experimentation, increase testing, simulate the production environment, and ultimately, to ensure better quality, faster and safer deployment of final services.
But there’s more to consider when it comes to DevOps
What if the organizational structure doesn’t encourage developers’ autonomy? In many organizations, developers are not allowed – or not willing – to access the workload automation environment in any way. Nevertheless, in those situations, Control-M can still help accelerate the application delivery process.
How? Let’s focus on organizations where developers and schedulers have distinct roles, with well-defined boundaries, and the scheduler role starts where the developer role ends. With this handoff approach it’s more critical than ever that the developers and operations communicate effectively. Without constant contact, the application delivery process is destined to slow down simply due to the inevitable back and forth when operations finds issues that must be addressed before promoting code to production. When moving application code into the schedulers hands, developers need to pass along information on when, where, and how to run applications.
Often this communication happens verbally, through emails, or through tools like Visio, resulting in a number of iterations until developers’ requirements are clearly mapped into job flows by schedulers. What could have taken hours or minutes has now likely taken two weeks or more depending on the code being reviewed and rolled out to production.
Control-M Workload Change Manager is a job flows “graphical designer,” that helps developers and operations communicate through a standard process – reducing the time from development requirements to job flow implementation. Further, it reduces the risk of errors by enforcing site standards during implementation.
Accelerate business service delivery tailored for your organization
Net-net: Control-M provides flexible solutions to support any level of DevOps adoption and accelerates application delivery. Organizations are still exploring ways for developers and schedulers to better collaborate, and today the level of DevOps adoption is different across organizations and – in many cases – across an organization’s departments. It depends on the organization’s project size, internal processes, security standards, risk toleration, skills, and cultural attitudes. There’s no need for organizations to change anything in their structures or processes in order to take advantage of Control-M. On the contrary, Control-M Automation API or Control-M Workload Change Manager, or a combination of both can bring value into any organization, whatever your level of DevOps adoption may be.