An innovative look at the scheduling challenges of the 21st century

The Role of IT in the Enterprise

While profitability, market share, growth and competitive advantage remain top business objectives, the role of IT has kept changing dramatically in the 21st century. Originally, IT was limited to supporting internal functions, performing basic back-office tasks and was detached from business decisions. Today, IT plays an integral part of every business process and decision, and is often the vehicle for organizational success.

The importance of IT in the enterprise continues to increase as a company's competitive advantage, time to market, profitability, scalability and ability to react to change are determined by its IT capabilities. Companies gain a competitive edge when they quickly implement sophisticated enterprise resource planning (ERP), customer relationship management (CRM), and supply-chain management (SCM) systems, and integrate these systems with homegrown as well as legacy applications.

Consequently, one may conclude wrongly that selecting the right application is sufficient. In reality, the selection process is not enough; application deployment and operations are what produce business results. Thus, to ensure extensive implementation and meet enterprise goals, a comprehensive best-of-breed solution is required that not only addresses technological needs bust also meets business needs.

The Growing Importance of Batch Scheduling

The emergence of transaction processing (TP) monitors in the 1970s, distributed computing in the 1980s, and enterprise resource planning (ERP) systems in the 1990s, each by its turn was supposed to eliminate the need for batch processing and batch integration. Nevertheless, the demand for batch integration has increased with the surfacing of each new technology, and not without good reasons:

Contrary to the common assumption that batch processing is a thing of the past, it is actually on the rise and becoming increasingly prevalent. The constant increase of online transactions, fueled by the integration of electronic commerce with business processes, magnifies the need for batch processing since each online transaction can generate multiple batch transactions. In fact, due to the complex nature of busines-to-business (B2B) processes that can take hours or even days to complete, many B2B solutions use batch integration heavily.

In the past, job schedulers were used to automate batch activities in order to meet data center needs. Nowadays, batch processing continues to fulfill its traditional role, however operating in an increasingly complex environment that comprises multiple platforms and applications.

Most production environments consist of both online and batch transactions. While online, message-oriented transactions are addressed by enterprise-application integration (EAI) solutions, such as IBM®'s MQSeries, a new type of scheduler is required to manage the batch integration of business transactions. In addition to having enterprise-wide scheduling capabilities, this new scheduler is required to manage dynamic business processes across platforms and applications, while considering workflow logic, end-user interaction and on-the-fly workflow changes. Without these capabilities, enterprise batch integration cannot be performed.

An innovative scheduling solution capable of delivering core-scheduling capabilities, while providing the infrastructure and technologies necessary for enterprise integration is vital. Such a scheduler is best described as a business-integrated scheduler.

Defining Business Integrated Scheduling

Business-integrated scheduling describes an enterprise scheduler that combines various activities into one business process, thus providing not only a technology framework, but also a business solution. It is in the interest of all IT decision-makers, at all management levels, that the enterprise scheduling environment be truly integrated with their business.

To create value, a business-integrated scheduling solution must address the following:

Scheduling Need
Business Benefit
The ability to guarantee that service levels of all managed objects are met 24x365, including rapid problem detection, escalation and resolution
 Profitability, superior customer relations
The ability to scale from a departmental solution to a complete enterprise solution, using the same technology
 Time to market, cost savings, leveraged knowledge
The ability to support a wide variety of platforms and applications in their native form
 Cost-savings, leveraged knowledge, superior customer relations
The ability to provide an integration framework for batch processes inside and outside the enterprise (i.e., B2B) with open, standard technologies (e.g., XML)
 Time to market, support for new or changed business processes, profitability
The ability to interact with various types of users, such as operators, scheduling analysts, application experts and end users in their own "language" (terminology)
 Cost-savings, ROI, leveraged knowledge, improved partner relations

To better analyze the need for business-integrated scheduling, we have categorized the business challenges into three main areas:

The applications challenge

In the past, IT departments had to create their own homegrown applications to manage their business and meet the needs of their users. To reduce costs, achieve standardization and attain a competitive advantage, companies began to implement packaged solutions such as ERP, and CRM. With the establishment of e-businesses, additional unique B2B applications were introduced.

In order to optimally satisfy diverse business needs and to attain and maintain a competitive edge, an enterprise is obliged to choose best-of-breed applications. Each application is self-contained and is developed independently using different data models and semantics. Each is written and designed with different technologies, on different operating systems and using different communication protocols. While every application addresses a specific business need, all applications operate within one IT business environment. As a result, modern enterprises are encountering the heterogeneity challenge.

Today, batch environments consist of cross-application data transfers and multi-step batch transactions, running on different operating systems or within self-contained application environments. Since each application has its own requirements, capabilities and functions, it must be treated individually. Moreover, new emerging systems such as Linux and Microsoft Windows XP are rapidly becoming production platforms. To meet the business goals of the enterprise, these heterogeneous applications and operating systems must be properly utilized, monitored and managed by a tool capable of understanding the unique workings of each system and application, while still offering a high degree of automation.

A business-integrated scheduler allows IT departments to support a large portfolio of dissimilar applications such as ERP, CRM, new operating systems and legacy systems, all of which must be integrated into one cohesive business environment.

The integration challenge

In order to create a unified business environment, it is vital to fully integrate the various applications and operating systems that are implemented across the various departments of the organization. Without this integration, it is impossible to achieve complete, efficient business processes.

B2B applications are increasing the importance and need for batch integration as they remove the boundaries between organizations. Business processes can be triggered by events either within the enterprise or from the arrival of a B2B transaction originating outside the organization. In order to achieve complete integration, the various enterprise applications should be integrated seamlessly.

According to GartnerGroup, "Through 2005, more than 50% of an enterprise's inter-application interfaces will continue to use a batch data exchange design" . Batch processing indeed remains the most efficient method for cross-application integration.

Accordingly, the critical role of a business-integrated scheduler is to combine the batch processes of diverse sources such as end users, applications and operating systems into one unified, manageable business process. Due to the complexities of information exchange and interdependencies, only a business-integrated scheduler offering specialized modules that interact with each application and operating system actively and directly can carry out this task.

B2B systems have not eliminated the need for human intervention, and have created the need for the integration of end-user batch flows into the business process. As portals become the de facto standard in many organizations, integration must be achieved between the portal users and the processes within the production environment, while hiding the complexities from the end user.

From a technological perspective, an enterprise business-integrated scheduler can be viewed as the backbone of the enterprise batch-integration efforts. Moreover, by offering an SDK that incorporates homegrown or industry-specific applications, a business-integrated scheduler provides unlimited support and unparalleled growth opportunities.

Through fully open and standard API technologies (e.g., XML) the solution should provide external applications the ability to perform a full range of operations, such as submitting a scheduling request directly into the production environment and tracking the status of its execution. This capability eliminates the need for developing a scheduling engine for each application and allows developers to focus on the core competency of the application.

Based on these capabilities, the scheduler should be able to perform the process flows needed for true batch integration.

The management challenge

With diverse applications and platforms, increasing volumes of ad hoc activities and ever-changing business needs, managing enterprise-wide production environments efficiently is a challenge. The lack of proper management for such a complex environment may lead to production delays or even downtime, causing immediate damage to the organization's reputation, ability to provide quality service, and profitability.

According to GartnerGroup, application failures and operator errors cause an average of 80% of unplanned application downtime. To significantly reduce downtime, Gartner Group identifies a need for organizations to invest in automation tools such as job scheduling and event management. While event management, as well as network and system management (NSM) solutions are important for reporting and escalating system-related problems, a batch-specific management solution is still required to proactively ensure batch processes run properly and flawlessly.

A batch-specific management solution must provide efficient enterprise-wide problem detection and resolution tools in order to prevent and recover from production delays. Additionally, the solution must provide enhanced management capabilities through an advanced user interface whose abilities surpass those of a traditional GUI. Such a user interface must be an integral and embedded component of the solution's architecture, and must simplify the complexity of the managed environment.

To assure maximum business availability it must be possible to easily and intuitively detect problems, analyze their source, resolve them and verify their resolution from a focal point of control, via a unified interface. The implementation of such a solution must minimize the occurrence of human errors, provide superior management capabilities and therefore reduce production delays and downtime.

Smooth operations and advanced service-level management should be guaranteed with the implementation of a scalable, best-of-breed business-integrated scheduling solution. Such a solution must increase automation and provide advanced batch process management tools in a diverse and complex environment.

Addressing the Business Challenges

To meet business challenges, organizations require a solution that has the specific capabilities to confront and answer their business demands. CONTROL-M® by BMC Software is a business-integrated scheduler that provides a comprehensive solution for the heterogeneous enterprise. This vigorous solution offers complete platform and application coverage with support for over 20 platforms and various off-the-shelf application-specific plug-ins. CONTROL-M also offers a toolkit that enables users to tailor their own plug-ins for homegrown applications or applications not yet covered by CONTROL-M.

CONTROL-M specialized control modules provide application and operating system-specific interfaces. Each module connects and relates closely to the data structures and terminology for a particular application. Every module is designed to support a specific application and has the ability to submit and manipulate batch activity within that application. All the processes required for batch processing within the applications are performed through a CONTROL-M interface. This interface is aligned with the look and feel of each application's interface, and serves its needs.

Since integration is a key challenge, all plug-ins connect to a central engine that can manipulate business processes from all of the various applications while integrating cross-application business flows. CONTROL-M can handle all business processes, regardless of their origin or scope.

In order to attain full integration, a scheduling solution must be able to properly integrate processes triggered by end users with other production environment processes. CONTROL-M/eTrigger provides an interface that seamlessly integrates end-user ad hoc processes with the back office while ensuring that they are an integral part of the production environment.

To allow this high level of integration, CONTROL-M provides a truly open environment where scheduling services are available for every application using APIs and software development kits. XML and JAVA technologies are used to provide user access and application access to scheduling services. Apart from integrating various applications, the open interface can interact with external systems, such as network system management products and more, to ensure a tightly integrated environment.

CONTROL-M provides the most advanced management capabilities to efficiently orchestrate the entire environment. It ensures smooth and flawless business flow submission and offers the highest possible level of automation. Through its ability to analyze job submission results and perform the appropriate remedial actions, CONTROL-M automates business flow submission scenarios. Hence, manual intervention is reduced to a bare minimum, while 24x365 availability is guaranteed.

In cases where manual intervention is required, CONTROL-M provides a comprehensive management tool – CONTROL-M/Enterprise Manager. This interface displays all existing applications and platforms in the environment and enables the initiation of management actions via one screen for the entire enterprise. CONTROL-M/Enterprise Manager supplies an enhanced Web-based interface that enhances productivity by allowing control and monitoring from any location at any time. CONTROL-M/Enterprise Manager makes sure no events are overlooked, while it increases productivity and performs corrective actions to guarantee that problems are solved at maximum speed.

CONTROL-M/Enterprise Manager provides the means for creating and planning business flows and ensures that there are no endless "loops" and that flow is ordered in the most efficient manner. This exceptional tool speeds up implementation processes.

The ability to predict how various future scenarios will affect your environment is vital. For instance, the capacity to predict what would happen if a certain number of transactions or an additional server were added to the environment, is very valuable. CONTROL-M enables the creation of what-if scenarios via CONTROL-M/Preview.



Summary

Managing modern enterprises is a challenging task. With competition fierce, organizations are forced to employ and integrate new applications and platforms into their already complex environments. The simultaneous implementation of numerous applications, as well as modules within applications, is inevitable. Moreover, much like TP monitors and ERP systems, B2B applications have rapidly increased the need for tight batch integration. To attain business benefits from the implementation of multiple applications, organizations must manage their entire environment under a single unified umbrella.

In order to realize business success, organizations must create a unified production environment using a business-integrated scheduling solution that can centrally and efficiently manage numerous diverse applications. In addition, a business-integrated scheduler must be able to integrate additional applications into the environment quickly and smoothly, as well as enable communication with applications from external organizations. Only a business-integrated scheduling solution can tackle such versatile and critical business requirements.

CONTROL-M is a business-integrated scheduling solution that addresses all the above needs as well as provides legacy support and rich scheduling capabilities. For years, CONTROL-M has served organizations reliably as their core business application batch scheduler. Now, with its new and innovative architecture, CONTROL-M is the optimal batch business-integration solution on the market.



GartnerGroup, D. Scott, July 29, 1998