Does your Workflow Automation software meet your new business requirements?
Staying competitive in today’s ever growing digital world will challenge businesses and increase pressure on IT in the digital transformation process. With a $4 trillion internet based economy and growing at 10% per year, CEO’s are looking at how customers use technology and asking themselves what that means for their company.
The 3V’s (Volume, variety and velocity) are defining properties or dimensions of big data in which businesses are trying to capitalize on by extrapolating consumer information to increase revenue. It’s safe to say your company has some type of Big Data project in the works. It could be just in its infancy or maybe even knocking at the door of production.
The fact that big data is likely a critical initiative delivering complex workflows, presents a variety of challenges for workflow engines like Oozie or any other silo workflow engine tool in your environment. Your Hadoop programs ingest data from a variety of sources. Without seamless integration, how do you know if workflows upstream are or will cause a problem downstream?
Example of a Business Process flow:
Oozie maybe ok for testing your workflows in Hadoop, but once in production these time proven attributes are really needed to ensure success of your CEO’s investment.
- Effective job control
- Forecasting and Predictive analysis
- Version Control
- A repeatable process
- DevOps capabilities
- SLA management
- End user visibility & notification
Something else to consider once in production. Who will run and monitor those workflows? Is it cost effective to have your company’s expensive resources (i.e., Data Scientist, Application Developers /Programmers, etc.) to run and monitor batch workflows?
There are reasons why companies abandoned the use of CRON and Windows task scheduler to run their critical applications, even though they are free. Oozie falls into that same category with Hadoop. When considering which vendor to run your business workflows throughout the enterprise, including Hadoop, make sure to ask them the following questions.
- Does your current workflow automation solution run and monitor workflows in Cloudera, MapR or Hortonworks? Are they certified?
- What is your yearly development investment on workflow product xxx?
- What portion of that investment helps support my big data initiative?
- What workflow innovations have you delivered to the market in the last 3 years?
- What has your vendor developed to assist in your digital transformation around workflow automation?
- Does your vendor have a conversion program for Oozie? If yes, does it require Professional Services or is it yours to use repeatedly if needed?
- MTTR is critically important and can have a direct effect on revenue.
- How many Customer Support representative are dedicated to my Workflow automation solution?
- Do you have follow the sun support?
- What is your customer satisfaction rating? (CXI of 85 is considered world class)
Obviously there are many more questions you should ask of your vendor, which I would highly encourage. As you start to investigate which workflow engine vendor you wish to align with, I would recommend adding BMC Control-M to your list.
- Best Practices in Big Data Automation: Thinking Beyond Oozie to the Enterprise Requirements
- Apache Spark: Working with Streams
- Apache Hive Beeline Client, Import CSV File into Hive
- Machine Learning and AI Frameworks: What’s the Difference and How to Choose?
- Reading Streaming Twitter feeds into Apache Spark