Meet The New Marathi RoBERTa
In July 2019, the Facebook research team introduced the Robustly Optimized BERT Pretraining Approach (RoBERTa)–an improvement over the Bidirectional Encoder Representations from Transformers (BERT), a self-supervised method for NLP tasks released by Facebook in 2018. Two researchers, Nipun Sadvilkar and Haswanth Aekula, have now pretrained the RoBERTa model on Marathi language using a masked language modelling (MLM) objective in a self-supervised manner. The duo unveiled the model at Hugging Face’s community week. Completed with @huggingface + Jax/Flax community week and happy with so much learnings, outcomes.✨ Introducing RoBERTa Marathi Language Model with its own huggingface logo in Maharashtrian style (Marathi language spoken state in India)Github Repo:https://t.co/MarHTo8FiE pic.twitter.com/xxLcoDwSjs— Nipun Sadvilkar (@NipunSadvilkar) July 17, 2021 The model is primarily aimed at tasks that use the whole sentences (potentially masked) to make decisions, such as sequence classification, token classification or question-answer. The duo used this model to fine-tune text classification tasks like iNLTK and indicNLP. Since the Marathi mc4 dataset is made up of text from Marathi newspapers, it might involve biases that can affect all fine-tuned versions of the model, the team warned. Credit: GitHub
The duo unveiled the model at Hugging Face’s community week.
Two researchers, Nipun Sadvilkar and Haswanth Aekula, have now pretrained the RoBERTa model on Marathi language using a MLM
["BERT", "Hugging Face", "RoBERTa"]
["Hugging Face", "text classification", "RoBERTa", "AI", "ML", "Transformers", "BERT", "NLP", "Aim", "JAX", "NLTK", "R"]
["AI", "ML", "NLP", "Aim", "JAX", "Hugging Face", "Transformers", "NLTK", "text classification", "R"]
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at shraddha.goled@analyticsindiamag.com.