Running Jupyter Notebook on a remote server

With a command-line interface to the server, it is often hard to quickly scan through the contents on a server. This can be circumvented using jupyter-lab (or jupyter notebook) running on the server and accessing it using a client machine. I presume you have already installed jupyter-lab (or jupyter-notebook) on server. Jupyter-lab is a better option as it comes with a file-navigator, spread-sheet viewer (faster than excell, reminds me of sublime text) and an image-viewer. Check out this video for the latest feature updates in jupyter-lab.

Continue reading “Running Jupyter Notebook on a remote server”

Extracting top feature names for a trained classifier in order for sci-kit learn

Post describes how to extract top feature names from a supervised learning classifier in sklearn.

Note: The training dataset X_train and y_train are pandas dataframe with column names.

After fitting/training a classifier clf, the scoring for features can be accessed (method varies depending on the classifier used).

  • For example, for logistic regression it is the magnitude of the coefficients and can be accessed as clf.coef_
  • For DecisionTree, it is clf.feature_importances_

Sort the scores in descending order using np.argsort() and pass it as an index to the column names of X_train.columns.

# For Decision Tree classifier

from sklearn.tree import DecisionTreeClassifier
import numpy as np

clf = DecisionTreeClassifier(random_state=42), y_train)

importances = clf.feature_importances_

# printing top 5 features of fitted classifier
print (X_train.columns[(np.argsort(importances)[::-1])][:5])
print(sorted(zip(X_train.columns,importances),key=lambda x: x[1])[::-1]