Workload Automation in a High-growth, Nonstop Environment at Raymond James Financial

BY

Working Group analyzing Reports

Ten years ago, Raymond James Financial (RJF) was a regional investment firm serving clients primarily in the Southeastern U.S. Today, it’s an international financial services organization with a network of 6,700 financial advisors serving more than 2.7 million client accounts across North America, South America, and Europe.

As we’ve merged with and acquired companies and experienced organic growth, our IT environment has become incredibly sophisticated. In response to expansion, complexity, and the demands of the digital economy, our IT organization has undergone a dramatic transformation that positions us to help drive growth and deliver new technology services that help clients achieve their financial objectives. It’s a high-pressure environment to work in, but it’s also an exciting and rewarding one.

Like all groups within IT, the workload engineering team constantly focuses on faster, more effective ways to get things done. Our mandate is to ensure that batch processes run reliably every night so that critical systems are ready for the start of each business day. We couldn’t make that happen without a sophisticated workload automation solution that:

  • Simplifies monitoring of the batch environment
  • Provides tools and predictive analytics to help us solve issues before they disrupt job flows
  • Uncovers inefficiencies that slow processing times

Control-M is the workload engineering team’s first line of defense in identifying bottlenecks and escalating and remediating batch processing issues. In the past year, we’ve absorbed a 42 percent increase in monthly job executions.

Control-M in Action

One of the RJF’s key competitive differentiators is our financial advisor desktop. This integrated desktop gives advisors streamlined access to client data, account information, news, and research, plus tools for financial planning, investment proposals, portfolio modeling, and search.

Our data warehouse and consolidated data store (CDS) are the two primary systems that power the desktop. Hundreds of applications are involved in providing data to these two systems and drawing data from them. Control-M manages the thousands of daily interdependent jobs—including more than 400 file transfers to and from external sources—that keep the data warehouse, CDS, and other systems up to date.

Control-M began delivering value right away by simplifying and automating the monitoring of our nightly batch processes for this network of interconnected and interdependent systems and applications. It provides insight into the workload on a minute-by-minute, real-time basis, which is invaluable because we can spot small problems before they become big ones. In addition, it leverages escalation rules we’ve specified to keep us focused on the issues that are most important.

Control-M Batch Impact Manager and predictive analytics give us a clear and accurate estimate of how long it will take for nightly processes to run and lets us know when we’re at risk of failing to meet our service level agreements (SLAs). We’ve established thresholds so that any time a job runs 30 minutes longer than is typical, our first-level computer operators receive an alert. If they can’t resolve it, the issue is automatically escalated to the next level.

Control-M tools speed troubleshooting and remediation so that we can meet SLAs. If a job for one application is affecting downstream jobs, we can run reports that pinpoint where the bottlenecks are. Then we can work with application owners and developers to solve the problem. Predictive analytics also give us visibility into what might happen when we introduce new jobs into the nightly workload, so we can avoid disruptions when we implement changes.

Control-M supports our constant focus on identifying and eliminating inefficiencies. It presents job flows graphically so that developers and application owners can see the interdependencies among applications and jobs. Here’s an example: Over one four-month period, we saw that, at month end, a couple of data warehouse jobs were running as much as an hour longer than previously. Using Control-M, we identified the jobs in the stream that were the biggest contributors to the slowdown. We then met with developers to find the cause and come up with a fix. The developers modified their code, we tested it, and we rolled it into production. At the next month end, the jobs ran even faster than they had before our remediation efforts, which was a big win.

When people can see the big picture with respect to overall batch processing, they understand how their jobs fit into the overall flow. They know what has to happen before their jobs can even begin and they can visualize how a small problem in one job flow can create big problems downstream. Visibility is fostering a spirit of cooperation and collaboration that has allowed us to regularly reduce the nightly processing time—even though the number of jobs we run and the number of applications and users we support keeps growing. Today, we’re not just meeting SLAs; we’re exceeding them. We often have services available two hours earlier than expected, which makes our users very happy.

Positioned for Continued Growth

We’re proud of our record for exceeding SLAs, especially considering RJF’s remarkable growth. The fact that we’ve absorbed that growth so easily demonstrates the scalability of Control-M. A year ago, we were executing an average of 1.4 million jobs a month. Last month, we hit the two-million mark for monthly job executions. We’re squeezing more and more jobs into a narrow batch window.

As we look ahead, we see more opportunities for improvement. We’re rolling out mobile Self Service, so people will be able to keep tabs on their jobs on the go using tablets and smartphones. We also plan to implement the Workload Archiving and Managed File Transfer capabilities of Control-M. Workload Archiving will enable us to maintain and log data for a longer period of time and query the data without affecting production. Managed File transfer will enhance the management of the hundreds of internal and external file transfers we do every day.

We’re pleased to be working with a partner like BMC Software because of the company’s commitment to our success. It was very impressive to have BMC ask for our input on production requirements for Workload Archiving and Managed File Transfer and then see that our recommendations helped drive the direction of those solutions. It’s a collaborative relationship that benefits us as well as BMC.

Related posts:

EMA: Issues and Priorities in Modern Workload Automation


Modernize your approach to workload automation based on the latest industry trends and priorities

Read the Report ›

These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.

Share This Post


Chris Haynes

Chris Haynes

Manager, Workload Engineering
Over the last 5+ years, I worked with Control-M in the Scheduling, Administrating and now in a leadership role at Raymond James Financial. My team (Workload Engineering) has implemented Control-M AFT, SQL and Messaging on multiple platforms and currently implementing Archiving. I have lead two upgrades from our initial implementation of V7 and we are currently on V9 FP3. Since implementing Control-M, we have collaborated with BMC on the Archiving and the recently released MFT module which has provided invaluable experience for my team. My team is responsible for fully leveraging workload automation as applications become more complex and integrated. We continue to push the limits of Control-M with these complexities and maintain the highest performance levels in support of our business users.