A Step-by-Step Guide To Build ML Models For Research
Building machine learning models for research can be challenging, especially if you are developing them from scratch. Several factors come into play before you successfully deploy ML models for production. Some of these aspects include data collection, training data in a limited amount of time, non-representative training data, poor quality of data, irrelevant/unwanted features, overfitting the training data, underfitting the training data and offline learning, development of the model, etc. Last year, Gartner reported that only 53 per cent of projects made it from prototype to production — and that’s at organisations with some level of AI/ML experience. On the other hand, that number is likely to be much higher, with some failure-rate estimates touching nearly 90 per cent. In the McKinsey report, out of 160 companies, 88 per cent did not progress beyond the experimental stage. Chris Chapo, vice president, advanced analytics at Amperity, said one of the biggest reasons is that people believe by throwing money at a problem or putting technology in, success comes out the other end. “That just doesn’t happen,” said Chapo. So, what is the secret sauce here? Is there an easy way out or a step-by-step guide to successfully deploy machine learning models? Michael A. Lones, in a research paper titled ‘How to avoid machine learning pitfalls: a guide for academic researchers,’ explained the common mistakes that occur when using machine learning techniques and what can be done to avoid them. Lones is an associate professor in the School of Mathematical and Computer Science, Heriot-Watt University, Edinburgh. Data is everything Before diving into the sophisticated techniques, you probably should perform some fundamental data clearing methods on every machine learning project. Unfortunately, these are overlooked by even seasoned machine learning professionals and researchers yet are so critical that if skipped, models may break or report overly optimistic performance results/outcomes. Lones says that it is normal to want to rush into training and evaluating models, but it is important to take the time to think about the goals of a project, to understand the data, its limitations, previous works, etc. He says, if you don’t do these things, you might end up with results that are hard to publish or models that are not suitable for their intended purpose. So, before you start building your models, here are some things that are important to consider. Take time to understand your data Look at all your data Make sure you have enough data Talk to domain expertsSurvey the literature Think about how you want your model to be deployed Building reliable models Building models is one of the most important parts of machine learning. With modern ML frameworks, it is easy to throw all techniques at your data and see what works. This, sometimes, leads to a disorganised mess of experiments that is hard to justify and record. Lones believes that approaching model building in an organised manner is the way forward. You make sure to utilise data correctly and put adequate consideration into the choice of models. He suggests the following steps to ensure you build a reliable model: Do not allow test data to leak into the training process Try out a range of different models Use inappropriate models Optimise your model’s hyperparametersBe cautious where you optimise hyperparameters and select Evaluating the robustness of models Lones says one needs to have valid results that you can draw reliable conclusions from. Without this, it becomes difficult to assess the outcomes. He says one should think carefully about how they are going to use data in their experiments, measure the true performance of their models, and how they are going to report this performance in a meaningful way. Here are the tips to consider while evaluating your machine learning model Use an appropriate test set Use a validation set Evaluate a model multiple timesSave some data to assess your final model instance Do not use accuracy with imbalanced datasets Comparing models fairly Comparing models is one of the most common practices in AI/ML research work, but it is also challenging to get it right. Lones says that if a researcher carries out a comparison unfairly and publishes it, other researchers may subsequently be led astray. Therefore, he suggested that researchers evaluate different models within the same context, explore multiple perspectives, and utilise statistical tests appropriately. Here are the things to consider while comparing your machine learning models: Do not assume a bigger number means a better model Use statistical tests when comparing models Do correct for multiple comparisonsDon’t always believe results from community benchmarks Consider combinations of models Reporting results Lones says that the aim of academic research should not be self-aggrandising but rather an opportunity to contribute to knowledge. Therefore, one needs to provide a complete picture of their work, showcasing what worked and did not. “Machine learning is often about trade-offs, and you should try to reflect this with a nuanced and considered approach to reporting results and conclusions,” says Lones. Here are some of the critical things to consider before reporting the results of your model: Be transparent Do report performance in multiple ways Do not generalise beyond the data Be careful when reporting statistical significance Do look at your models Conclusion Lones believes that the documents do not tell you everything you need to know, the lessons sometimes have no firm conclusions, and some of the things discussed here are debatable. “This, I am afraid, is the nature of research. The theory of how to do ML almost always lags behind the practice; academics will always disagree about the best ways of doing things, and what we think is correct today may not be correct tomorrow,” says Lones. He said that one has to approach machine learning the same way they would approach any other research area — with an open mind, willingness to stay updated with the latest developments, and the humility to accept that you do not know everything.
In the Mckinsey report, out of 160 companies, 88 percent of them did not progress beyond the experimental stage
Building machine learning (ML) models for research can be challenging, especially if you are developing them from scratch.
["AI (Artificial Intelligence)", "AI latest", "Guide", "learn machine learning", "Machine Learning", "Machine Learning Latest", "Machine Learning New", "machine learning research", "ML models", "ML research"]
["RPA", "Machine Learning Latest", "Ray", "R", "Machine Learning New", "analytics", "Guide", "machine learning research", "Go", "machine learning", "AI", "ML", "ML models", "Machine Learning", "GAN", "ML research", "learn machine learning", "Aim", "AI latest", "AI (Artificial Intelligence)"]
["AI", "machine learning", "ML", "analytics", "Aim", "Ray", "R", "Go", "GAN", "RPA"]
Amit Raja Naik, known as the 'AI Human,' serves as the editor at AIM Media House, where he leads a team of talented tech journalists who are driving and shaping technology conversations across India and around the world.