How To Use XGBoost To Predict Housing Prices In Bengaluru: A Practical Guide
XGBoost, an abbreviation for eXtreme Gradient Boosting is one of the most commonly used machine learning algorithms. Be it for classification or regression problems, XGBoost has been successfully relied upon by many since its release in 2014. It is a library for implementing optimised and distributed gradient boosting and provides a great framework for C++, Java, Python, R and Julia. The Reason Behind Its Popularity XGBoost provides a number of features that can greatly impact the efficiency and performance of a model. Parallelisation, distributed computing, cache optimisation, automatic handling of missing data are some of its features that stand out compared to other algorithms and libraries. Another set of factors include the speed of execution and the performance of the model in both regression and classification type problems. The algorithm is most effective in producing a model with lesser variance and a more stable prediction. Your First Program To Implement XGBoost In Python We will be using a subset of the dataset given for the hackathon “Predicting House Prices In Bengaluru” at MachineHack.com. (Click here to download the complete dataset). You can also take a look at our latest hackathon Uber City Traffic Visualisation Challenge. Installing XGBoost Use the python pip installer to install the XGBoost library from your terminal. pip3 install xgboost Let’s get coding: Importing the libraries import numpy as np import pandas as pd Importing the dataset dataset = pd.read_excel('House_prices.xlsx') The data consists of features of Houses in locations across Bangalore. The problem is to predict the ‘price’ of the houses from  ‘total_sqfeet’, ‘size’, ‘bath’, ‘balcony’, ‘area_type’ and ‘location’. The data has been preprocessed to some extent. Click here for instructions on Data preprocessing in python. Extracting the values and categorising the features to dependent and Independent Variables X = dataset.iloc[:,[0,1,2,3,4,6]].values y = dataset.iloc[:, 5].values X: Set of Independent features(‘total_sqfeet’, ‘size’, ‘bath’, ‘balcony’, ‘area_type’ and ‘location’). Y: Dependendent Feature (price) Handling categorical Variables The first three features ‘size’, ‘area_type’ and ‘location’ in the dataset consist of categorical values and hence is required to encode them to numbers. We will use the label encoder to encode the features into numerical values. from sklearn.preprocessing import LabelEncoder le_X_0= LabelEncoder() le_X_1= LabelEncoder() le_X_2= LabelEncoder() X[:, 0] = le_X_0.fit_transform(X[:, 0]) X[:, 1] = le_X_1.fit_transform(X[:, 1]) X[:, 2] = le_X_2.fit_transform(X[:, 2]) X after Label Encoding: Splitting the data set into test and training set from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size = 0.2, random_state = 0) The test set will consist of 20% of the data in the dataset while the training set will contain 80%. Creating and initialising  XGBoost Regressor from xgboost import XGBRegressor regressor  = XGBRegressor() Fitting the regressor with data After initialising the regressor we need to fit the regressor with training data so that it can learn the correlations between the features to give an accurate prediction for new inputs. regressor.fit(X_train, y_train) Predicting the prices Predicting for training set: Y_pred_train = regressor.predict(X_train) The above code will give the X_tain (training set with independent features) as input to the regressor and predicts values for prices. The predicted values are stored in the numpy array Y_pred_train. Predicting for test set: y_pred = regressor.predict(X_test) The above code will give the X_test (test set with independent features) as input to the regressor and predicts values for prices. The predicted values are stored in the numpy array Y_pred Evaluating accuracy with RMSLE Calculating RMSLE def rmsle(y_pred,y_test) : error = np.square(np.log10(y_pred +1) - np.log10(y_test +1)).mean() ** 0.5 Acc = 1 - error return Acc print("Accuracy attained on Training Set = ",rmsle(Y_pred_train, y_train)) print("Accuracy attained on Test Set = ",rmsle(y_pred,y_test)) Output
XGBoost, an abbreviation for eXtreme Gradient Boosting is one of the most commonly used machine learning algorithms. Be it for classification or regression problems, XGBoost has been successfully relied upon by many since its release in 2014. It is a library for implementing optimised and distributed gradient boosting and provides a great framework for C++, […]
XGBoost, an abbreviation for eXtreme Gradient Boosting is one of the most commonly used machine learning algorithms. Be it for classification or
["XGBoost"]
["NumPy", "machine learning", "TPU", "AI", "distributed computing", "Python", "Ray", "XGBoost", "R", "Pandas"]
["AI", "machine learning", "Ray", "XGBoost", "Pandas", "NumPy", "distributed computing", "TPU", "Python", "R"]
A Computer Science Engineer turned Data Scientist who is passionate about AI and all related technologies. 
Contact: amal.nair@analyticsindiamag.com