From Weather Forecasting To ‘Nowcasting’: How Google Is Using ML To Predict Precipitation
From predicting the weather by analysing cloud patterns in the Babylon era to using electric telegraph and telephone in the 1800s to analysing satellite imagery in the 20th century, the process of weather forecasting has evolved significantly over the years. And, now, in this digitised era, predicting weather and simulating long term climate trends is being done with the help of machine learning models by analysing volumes of data by computer models. With the evolving era, nowcasting and forecasting weather have come on the edge of a significant paradigm shift. In a blog post by Jason Hickey, Senior Software Engineer, Google Research, he stated how Google’s new research would enable speedy local weather predictions using satellite images, with no lag. In the blog, he confirmed that in an earlier paper — ML for Precipitation Nowcasting from Radar Images — the group, including Shreya Agrawal, Luke Barrington, Carla Bromberg, John Burge, Cenk Gazen and Jason Hickey have presented their new research of developing machine learning models for forecasting precipitation. According to Hickey, making accurate weather prediction has always been challenging because of the unpredictable storms and natural events that can happen on an hourly time scale. And therefore, this ML model has been designed to “addresses this challenge by making highly localised physics-free predictions using the data.” He further stated that considering the inference is computationally cheaper, given an already trained ML model, it allows high-resolution weather forecasting, almost instantaneously. The model which used to take hours can now be able to generate forecasts that have a kilometre of resolution with a total latency of only a few minutes, without any delay in data collection. According to researchers, as the world is now pressured by extreme weather conditions, this sort of trained ML model will come handy in extreme weather forecasting models. And, in turn, can also be used as a tool for crisis management during climate change disasters. The Tech Behind Usually, to predict weather forecasting, a massive amount of data is being fed into the algorithm that uses deep learning techniques to learn it and then make predictions based on the past data. However, the trained ML model works on a physics free approach for the forecasting process. The model has been designed to learn from the atmospheric examples daily without applying any prior data fed on to the system. The underlying convolution neural network (CNN) ‘U-Net’, which comprises a sequence of layers — a set of mathematical operations — takes the input satellite imagery and then transforms them into output images. The sequence of layers in the convolutional neural network are usually arranged in an encoding phase, which, in turn, decreases the resolution of the output images. However, with Google AI, the separate decoding phase has been added to expand the low-resolution images. Top: Image showing location of clouds as measured by geosynchronous satellites. Bottom: Radar image showing location of rain as measured by Doppler radar stations. (Credit: NOAA, NWS, NSSL // Google Blog) To start with, the engineering team trained the model by feeding historical data from 2017 to 2019 collected from the US for evaluation, and then compared the same to three baselines models — High-Resolution Rapid Refresh (HRRR); an Optical Flow (OF) algorithm; and a persistence model. According to researchers, once compared, Google artificial intelligence outperformed all the traditional methods by using precision and recall graphs. The model would be treating weather prediction as an image-to-image translation problem and believed in leveraging state-of-the-art CNN. Moving forward, for the best results, Google is planning to integrate the HRRR mechanism with its ML model to have accurate forecasts. According to Google, the organisation is also looking to apply machine learning directly to 3D observations in the future. The blog stated, “The numerical model used in the HRRR method can make better long term predictions because it uses a full 3D physical model. Cloud formation has always been harder to observe with 2D images, and therefore, it is harder for machine learning methods to learn convective processes.” Also Read: India To Use AI To Improve Weather Prediction Google ML To The Rescue The Butterfly Effect which stated, any small change in a complex phenomenon can have a significant impact on the outcome, stays true to the process of weather forecasting, where a slight amount of change in one variable could drastically change the future climatic events. Therefore a lot of unpredictability lies in forecasting weather. And, that’s where artificial intelligence plays a huge role in bringing in accuracy and reliability in the process of forecasting weather. A visualization of predictions made over the course of roughly one day. Left: The 1-hour HRRR prediction made at the top of each hour. Center: The ground truth, i.e., what we are trying to predict. Right: The predictions made by our model. (Credit – Google Blog) One of the most significant advantages of this trained ML model is the speed at which it can forecast the weather. With traditional models, the system had to read a vast amount of data from several weather stations, regularly and used to take hours and supercomputers to predict the weather accurately. However, according to the Hickey, this new model by Google would only take 5-10 minutes to generate a forecast, which used to carry around six hours earlier. This is because the newly trained model doesn’t try to model a complex weather system; instead, it makes predictions on simple radar data. “As weather patterns get easily altered by climate change, which can lead to increase in extreme weather, it becomes crucial for weather departments to attain actionable insights at high spatial and temporal resolutions,” stated by the paper. Also Read: Centre Joins Hand with IBM AI To Help Farmers Outlook As society is now making bold strides in creating actionable environmental insights for the betterment of society and nature, there is an increasing need for exploiting newer technologies for solving such issues. American Meteorological Society (AMS), once mentioned in one of their papers, that the use of modern AI technology would drastically help in weather forecasting by sifting through the overwhelming amount of data to find the accurate insight. The paper also stated, AI innovation can also help in predicting the effects of extreme weather and can help in energy consumption. Although this whole Google approach to artificial intelligence hasn’t seen its full potential, several other tech companies already started to investigate this area with newer innovations. In March, last year, IBM purchased The Weather Company to use their data on their in house AI development — Watson, which led to the development of Deep Thunder to provide hyper-local weather forecasts within a 0.2 to 12 miles resolution. Monsanto is another company that has been investing in artificial intelligence for weather forecasting to give predictions for agricultural reasons. Apart from these, social media giant, Facebook has also stepped up in this field by developing a method to analyse satellite imagery to determine the disaster-prone areas and the number of damages it has suffered. With such innovations in hand, users would now be able to get accurate minute-to-minute weather forecast updates.
From predicting the weather by analysing cloud patterns in the Babylon era to using electric telegraph and telephone in the 1800s to analysing satellite imagery in the 20th century, the process of weather forecasting has evolved significantly over the years. And, now, in this digitised era, predicting weather and simulating long term climate trends is […]
Google's new machine learning research would enable speedy local weather predictions using satellite images, with no time lag.
["AI (Artificial Intelligence)", "Artificial General Intelligence", "Artificial Intelligence India", "Artificial Neural Network", "Machine Learning", "Machine Learning India"]
["Go", "artificial intelligence", "machine learning", "TPU", "AI", "neural network", "Artificial Neural Network", "ML", "Artificial Intelligence India", "Machine Learning", "Machine Learning India", "RAG", "Artificial General Intelligence", "deep learning", "R", "AI (Artificial Intelligence)"]
["AI", "artificial intelligence", "machine learning", "ML", "deep learning", "neural network", "RAG", "TPU", "R", "Go"]
Sejuti currently works as Associate Editor at Analytics India Magazine (AIM). Reach out at sejuti.das@analyticsindiamag.com