Hands-On Guide to Small Text: A Python Tool for Active Learning
Today’s scenario where the data is more accessible and increasing in amount has increased the efforts to analyse and preprocess the data for a data scientist. There are many kinds of processes that need to be followed by a data scientist in preprocessing the data, cleaning the data, validating the data, and labelling the data. Labelling the data can be very much effort taking and time taking. In supervised learning, we can understand how much labelled data is important for a model to be trained to perform well. For example, in the NLP process, we have sentiments of the people, but the sentiments are not categorized; we can’t perform our learning in this case. Before the development of the model, we need to provide tags or labels to the sentiments, and here we can understand how difficult it can be to go through all the data for providing labels to the sentiment. To reduce the number of efforts taken by labelling raw data, active learning comes into the picture. What is active learning? In an active learning environment, learners try to become experienced in making decisions in the field for which learners are assigned meaning-making. For example, in checking multiple-choice questions, the paper checker gives marks for the right answers and negative or zero marks for wrong answers. So the learner will be the paper checker, and he will try to be an expert in giving marks from the category of marks. In machine learning processes, it is a subset of the whole learning process, as we know that this happens many times when we have to analyze data that is unlabeled, and in the whole process of machine learning, we need to provide our data points labels so that we can easily complete our machine learning process. And also, it is very helpful for models to provide well-labelled data for completing the learning process, but in cases where we need to provide levels to the data, active learning takes part. The concept of active learning is to make a machine-learning algorithm to reach a higher level of accuracy by providing them with small numbers of training datasets. At the same time, models are allowed to choose the dataset from which they want to learn. To provide labels to the unlabeled data, we usually require a human data annotator, which is a powerful way to annotate the data but is it efficient. For a data scientist, there are several tasks like analyzing the data, visualizing it, understanding it and finding patterns in the data. Aren’t these tasks pretty time-consuming? And if with all these we add one more task for them to provide labels to data accurately, the whole procedure will become too harsh, hectic and time-consuming. As A Result, data scientists will need to focus more on the procedure of labelling data. To reduce those efforts, active learning helps. Active learning works differently in different situations. Roughly we can categorize active learning into three categories. Stream-based selective sampling.Pool based sampling.Membership query synthesis. Stream-based selective sampling In this category, we try to make an algorithm that can determine the benefits to query for the labels for a specific unlabeled dataset while the model is training. The algorithm works with the models and provides labels to the dataset if labelling sequenced data sufficiently benefits training. Pool based sampling In this type of active learning algorithm, algorithms try to evaluate data samples for modeling and label the best fit data samples from a data source before performing the modeling part of the machine learning. Often algorithms learn from the already labelled datasets to provide labels to the unlabeled data and these are the most commonly used algorithms for active learning. Membership query synthesis These algorithms are not useful for all the cases because we try to generate the synthesis data. The algorithm is allowed to generate the data for labelling. This is useful where we can easily generate the data instances. Small text is an easy way to apply active learning in different kinds of machine learning procedures. Next, in the article, we are going to discuss the Small text for active learning. Small-Text As the article’s name suggests, the article is more focused on the small text library for active learning, which provides active learning algorithms for text classification and allows mixing and matching many classifiers and query strategies to build active learning applications. Using its features, we can easily develop classifiers using sklearn libraries, and also, in addition, we can use PyTorch classifiers with transformer models. This article starts with small-text by implementing a binary classification model on sklearn provided 20 newsgroup data. Before starting the model, we will know how we can install this package in python. pip install small-text Output: If anyone wants to go ahead with the PyTorch classifiers and transformers, this is an extra requirement we need to download. We can download using the following command. pip install small-text[transformers] Since I am using google Colab, I will need to clone the whole package presented in the link. So next, I will clone the package. !git clone https://github.com/webis-de/small-text.git Output: After cloning it, we can install the package by giving the exact address where it got installed. Also, before giving the address, you will need to mount your drive for the Colab notebook. Using the following command, we can mount our drive. from google.colab import drive drive.mount('/content/drive') After the mounting, we can install the package which we have cloned using the following command. pip install /content/small-text Output: After restarting the runtime, we can use the whole package. Now we can start our procedure of developing a binary classifier using small-Text library packages. Importing the required library: import numpy as np from small_text.active_learner import PoolBasedActiveLearner from small_text.classifiers import ConfidenceEnhancedLinearSVC from small_text.classifiers.factories import SklearnClassifierFactory from small_text.query_strategies import PoolExhaustedException, EmptyPoolException from small_text.query_strategies import RandomSampling from small_text.data import SklearnDataSet Defining active learning parameters: clf_template = ConfidenceEnhancedLinearSVC() clf_factory = SklearnClassifierFactory(clf_template) query_strategy = RandomSampling() Defining a function to load the data using sklearn: from sklearn.datasets import fetch_20newsgroups def get_twenty_newsgroups_corpus(categories=['rec.sport.baseball', 'rec.sport.hockey']): train = fetch_20newsgroups(subset='train', remove=('headers', 'footers', 'quotes'),categories=categories) test = fetch_20newsgroups(subset='test', remove=('headers', 'footers', 'quotes'),categories=categories) return train, test def get_train_test(): train, test = get_twenty_newsgroups_corpus() return train, test Defining a function to preprocess the data: from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.preprocessing import normalize from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.preprocessing import normalize def preprocess_data(train, test): vectorizer = TfidfVectorizer(stop_words='english') x_train = normalize(vectorizer.fit_transform(train.data)) x_test = normalize(vectorizer.transform(test.data)) return SklearnDataSet(x_train, train.target), SklearnDataSet(x_test, test.target) Defining a function to evaluate the results using sklearn’s  metrics: from sklearn.metrics import f1_score def evaluate(active_learner, train, test): y_pred = active_learner.classifier.predict(train) y_pred_test = active_learner.classifier.predict(test) print('Train accuracy: {:.2f}'.format( f1_score(y_pred, train.y, average='micro'))) print('Test accuracy: {:.2f}'.format(f1_score(y_pred_test, test.y, average='micro'))) print('---') Merging all these functions into the main function: def main(): # Active learning parameters clf_template = ConfidenceEnhancedLinearSVC() clf_factory = SklearnClassifierFactory(clf_template) query_strategy = RandomSampling() text_train, text_test = get_train_test() train, test = preprocess_data(text_train, text_test) active_learner = PoolBasedActiveLearner(clf_factory, query_strategy, train) labeled_indices = initialize_active_learner(active_learner, train.y) try: perform_active_learning(active_learner, train, labeled_indices, test) except PoolExhaustedException: print('Error! Not enough samples left to handle the query.') except EmptyPoolException: print('Error! No more samples left. (Unlabeled pool is empty)') Defining a function to perform active learning. def perform_active_learning(active_learner, train, labeled_indices, test): for i in range(15): # ...where each iteration consists of labelling 20 samples q_indices = active_learner.query(num_samples=20) y = train.y[q_indices] active_learner.update(y) labeled_indices = np.concatenate([q_indices, labeled_indices]) print('Iteration #{:d} ({} samples)'.format(i, len(labeled_indices))) evaluate(active_learner, train[labeled_indices], test) This loop will perform 15 iterations of active learning; in each iteration, 20 samples of the news are queried and then updated. The update step will reveal the true label to the active learner. Defining a function to initialize the active learner; this function is required for model-based query strategies. def initialize_active_learner(active_learner, y_train): indices_pos_label = np.where(y_train == 1)[0] indices_neg_label = np.where(y_train == 0)[0] ([np.random.choice(indices_pos_label, 10, replace=False), np.random.choice(indices_neg_label, 10, replace=False)]) x_indices_initial = x_indices_initial.astype(int) y_initial = [y_train[i] for i in x_indices_initial] active_learner.initialize_data(x_indices_initial, y_initial) return x_indices_initial Running the main function: if __name__ == '__main__': main() Output: Here we can see that the accuracy of the model has increased. The size of the output was big, that is why in the image, the whole output is not put in the article, but it started with the training accuracy of 1.00 and the test accuracy of 0.76 with the sample size of 40. Here in the article, we have seen what active learning is and how we can perform it easily with small Text. This library has many features like it provides full functionality to work on data labelling with transformer models. Moreover, since it is totally developed on python packages, it is very easy to understand with a basic knowledge of python. In the case of studies, we can also use reinforcement learning in place of this. Reinforcement learning is inspired by behavioural psychology, where active learning is closer to supervised learning. The development of the machine learning model is more rigid and accurate in the results than reinforcement learning. References Small text – read the docs.Github- small-text.Google colab for codes.
the article is more focused on the small text library for active learning, which provides active learning algorithms for text classification and allows mixing and matching many classifiers.
small text consists active learning algorithms for text classification and matching many classifiers and query strategies to build models
["Active Learning", "Automation", "data labelling", "data preparation", "data preprocessing", "Guide", "Machine Learning", "NLP", "text classification", "text dataset"]
["NumPy", "text classification", "machine learning", "TPU", "data preprocessing", "AI", "PyTorch", "Automation", "data preparation", "Machine Learning", "Transformers", "RAG", "Active Learning", "data labelling", "NLP", "Colab", "text dataset", "Guide"]
["AI", "machine learning", "NLP", "PyTorch", "Transformers", "Colab", "NumPy", "RAG", "text classification", "TPU"]
Yugesh is a graduate in automobile engineering and worked as a data analyst intern. He completed several Data Science projects. He has a strong interest in Deep Learning and writing blogs on data science and machine learning.