Machine Learning Enhances Assam Government’s Disaster Response Amid Floods
Every year, floods displace millions of people in Assam, forcing them to seek shelter in relief camps. In fact, devastating floods were some of the watershed events in the state’s history. Each year, the Indian government allocates hundreds of crores towards relief efforts during Assam floods. But, a major challenge the government faces is ensuring that these funds are effectively utilised and that the right resources reach the appropriate locations promptly. This issue persists due to the fragmented nature of government data, which is often stored in isolated silos. In the case of Assam government, the data related to disaster management is stored across 18 different departments and other central agencies. But a research lab working at the intersection of data, technology, design and social science has come to their aid. Called CivicDataLabs (CDL), the startup is working closely with the Assam government to unify its data and analyse it in a much better way. ( Credits: Reuters) Setting Data Standards “The Government of Assam is investing heavily in disaster risk reduction efforts. However, they have faced challenges in tracking the allocation of funds to ensure they are directed to the correct regions and at the appropriate times, and whether that’s translating into increased risk reduction,” Gaurav Godhwani, co-founder and executive director at CivicDataLab told AIM. The finance department in Assam and the Disaster Management Authority knocked on the startup’s door, and they decided to answer. Godhwani revealed that various state departments actively collaborated with the startup, especially with data sharing as required. The startup then worked towards cleaning and standardising the datasets. “We worked with the department to see what procurements were happening related to floods– When did the last embankment repair happen after damage happened? When was the restoration of that particular geography conducted? Pretty soon, we realised that data, because of lack of data standards, was not able to capture this critical information,” he said. Hence, CDL introduced an international data standard known as the Open Contracting Data Standard, which is adopted in over 50 nations. As a result, Assam became the first state in India to adopt this international standard. Moreover, Godhwani and his team created a platform where all the datasets come together. They also created a data exchange layer that enables all the publishers, including the 18 departments and other central agencies, to manage disaster-related data. “Most of them have their own databases, their own APIs, not talking to each other. The challenge was to bring all of those together in one place. So we streamline the datasets, standardise them based on common geographical indicators, like state, district, and revenue circles and give an ID to all of those data sets so that they are all uniquely identified,” Godhwani explained. (Stakeholder Consultation with ASDMA, Source: CivicDataLabs) Early success Once the datasets were streamlined and standardised, CDL used simple machine learning techniques to derive valuable insights from the data. “We presented our initial findings, demonstrating hazard analysis and disaster modelling using GIS techniques and simple machine learning techniques. We highlighted districts and revenue circles at higher risk, emphasising areas needing greater attention,” Godhwani said. In response, the Assam Government requested the integration of disaggregated data for periodic analysis spanning a five-year period. CDL developed a platform which allows comprehensive assessment of past trends and facilitates planning for the upcoming monsoon cycle and subsequent restoration efforts. Godhwani revealed that the Assam Government has already started results by leveraging the data model CDL built. The authorities used the model to determine the allocations and verified them with assistance from on-ground staff. “They found that the algorithm effectively identified areas that had previously been overlooked. As a result, revenue circles that historically received insufficient funds have now begun to receive more support based on our model,” Godhwani pointed out. Secondly, the department’s capabilities have significantly expanded. Previously, generating analytics reports during disasters was a time-consuming process. They lacked knowledge of suitable software and effective data visualisation techniques. “Now, they are actively pursuing online courses on geospatial data management and AI. We have observed a renewed enthusiasm for data science and what it can do.” ( Capacity building workshop with Assam state and district officers conducted by CDL, Source: Civic Data Labs) Machine Learning Techniques Come to Aid Godhwani said this was done using three three machine learning algorithms. The initial algorithm employed was Random Forest, aimed at understanding the potential for inundation. This was achieved by analysing satellite imagery sourced from multiple sources to observe the extent of inundation during the monsoon season. Secondly, the Data Envelopment Model was used to extract variables from textual information, like maternal and child health indicators, public procurement etc. “The algorithm helps in understanding how many times procurement has happened before the monsoon season, during the monsoon season and after the monsoon season. The algorithm creates a model on top of all of these, giving them scores based on the past data, and when the past data is missing, it creates a baseline that whenever the new data is coming, we can measure whether we are up that baseline or below the baseline.” Lastly, CDL then utilises the TOPSIS scoring method to consolidate all data into a comprehensive risk index, enabling measurement on a scale of one to five, where one indicates lower risk and five indicates higher risk. This scoring methodology computes all variables in one central location and provides timely updates to the department on which areas to prioritise each month. This ensures that updates can be made on a monthly basis. “Earlier, it used to take a couple of months to compute just one timestamp for the department. So that’s the kind of automation which we’re seeing with these machine learning algorithms,” Godhwani pointed out. Leveraging generative AI While machine learning algorithm has sufficed so far, Godhwani believes Large Language Models (LLMs) can be leveraged to scale the platform. Currently, all the data generated goes to a public platform, but Godhwani believes that if the same data has to be disseminated to the public at large, something like a rule-based engine would not suffice. Information could be disseminated to the public through a chatbot or through social media applications which the public is already using. “Information such as, for instance, if you’re going in this particular area, please be careful there is a likely inundation, which may happen in the next 24 hours, 48 hours because of heavy rains. That’s the kind of opportunity we have with large language models,” Godhwani said. CDL has already conducted a pilot and shown the results to the Assam government. But in terms of deployment, it’s still a few months away. “The government agencies need to become more comfortable in piloting this at scale. The other thing is ensuring the ethics, responsible development and modelling is done in a collaborative manner. It can’t be hallucinating, as most these models today still hallucinate. If they are, the risks are extremely high.” About Civic Data Labs Godhwani founded Civic Data Labs with Deepthi Chand. Interestingly, the duo has registered Civic Data Labs as a startup and not as a non-profit. The team is 52 members strong consisting of disaster risk reduction experts, climate scientists, data scientists, data engineers, technologists, designers,  user researchers, and technological architects. Even though the company is bootstrapped, it raises funds for projects. For instance, the project with the Assam government is funded by the Rockefeller Foundation and the Patrick J. McGovern Foundation in collaboration with Open Contracting Partnership, a global think tank. The startup works closely with non-profits, academic institutions, volunteer groups, government agencies, and departments.
The algorithm effectively identified areas that had previously been overlooked. As a result, revenue circles that historically received insufficient funds have now begun to receive more support
Civic Data Labs (CDL) is standardising the Assam Government's datasets and using machine learning techniques to analyse the data efficiently.
[]
["data science", "Go", "machine learning", "AI", "ML", "RAG", "Aim", "analytics", "generative AI", "R"]
["AI", "machine learning", "ML", "data science", "analytics", "generative AI", "Aim", "RAG", "R", "Go"]
I have a keen interest in creative writing and artificial intelligence. As a journalist, I deep dive into the world of technology and analyse how it’s restructuring business models and reshaping society.