Here we explain what is a Spark machine learning pipeline. We will do this by converting existing code that we wrote, which is done in stages, to pipeline format. This will run all the data transformation and model fit operations under the pipeline mechanism. The existing Apache Spark ML code is explained in two blog posts: part one and part … [Read more...]

In part one we explained how to create a training model. In this part we show how to make predictions to show which machines in our dataset should be taken out of service for maintenance. First, here is how to submit the job to Spark with spark-submit: jar file that contains com.bmc.lr.makePrediction what file to read, i.e., the one you … [Read more...]

Here we explain a use case of how to use Apache Spark and machine learning. This is the classic preventive maintenance problem, one of the most common business use cases of machine learning and IoT too. We take the data for this analysis from the Kaggle website, a site dedicated to data science. This is sensor data from machines, specifically … [Read more...]

Here is the last part of our analysis of the Tripadvisor data. Part one is here. In order to understand this, you will need to know Python and Numpy Arrays and the basics behind tensorflow and neural networks. If you do not, you can read an introduction to tensorflow here. The code from this example is here and input data here. We create a … [Read more...]

  Code refactoring means to take a working program and change it to make some improvements. It changes the code but not the outcome. These improvements can: make it easier for other programmers to read make aesthetic improvements such as implement a clever idea make the program run faster or use less resources adhere to … [Read more...]

The Google Cloud Machine Learning Engine is almost exactly the same as Amazon Sagemaker. It is not a SaaS program that you can just upload data to and start using like the Google Natural Language API. Instead, you have to program Google Cloud ML using any of the ML frameworks such as TensorFlow, scikit-learn, XGBoost, or Keras. Then Google spins up … [Read more...]

In the last blog post we showed you how to use Amazon SageMaker. So read that one before you read this one because there we show screen prints and explain how to use the graphical interface of the product, including its hosted Jupyter Notebooks feature. We also introduced the SageMaker API, which is a front end for Google TensorFlow and other … [Read more...]

Amazon SageMaker is a managed machine learning service (MLaaS). SageMaker lets you quickly build and train machine learning models and deploy them directly into a hosted environment. In this blog post, we’ll cover how to get started and run SageMaker with examples. One thing you will find with most of the examples written by Amazon for SageMaker … [Read more...]

Here we show how to use Amazon AWS Machine Learning to do linear regression. In a previous post we explored using Amazon AWS Machine Learning for Logistic Regression. To review, linear regression is used to predict some value y given the values x1, x2, x3, …, xn. in other word it finds the coefficients b1, b2, b3, … , bn plus an offset c to … [Read more...]