KNN Classifier build from scratch with numpy, what is wrong with the code?

Question

Why my KNN Classifier build from scratch with numpy gives different results than the sklearn.KNeighborsClassifier? What is wrong with my code?

# create a function that computes euclidean distance and return the most common class label 
    # for given k.
    def k_neighbors(self, x):
        lengths = [self.euclidean_length(x, x_train) for x_train in self.X_training] 
        k_index = np.argsort(lengths)[: self.k] 
        k_nearest_labels = [self.y_training[i] for i in k_index] 
        counts = np.bincount(k_nearest_labels) 
        most_common_label = np.argmax(counts) 
        return most_common_label

# running KNN classifier with K=5 to fit the data and make predictions.
classifier1 = KNN_Classifier(k=5) 
classifier1.fit(X_training, y_training)
predicted1 = classifier1.predicting(X_test)

They both apparently do the same but I have different outcomes. Where is the bug in my code?

from sklearn.neighbors import KNeighborsClassifier

classifier2 = KNeighborsClassifier(n_neighbors=5, algorithm='brute', p=2)
classifier2.fit(X_training, y_training)
predicted2 = classifier2.predict(X_test)

aminrd · Accepted Answer · 2021-11-10 18:20:25Z

1

Based on sklearn documentation, there are multiple reasons:

Distance metric: you are using Euclidean distance metric, while sklearn by default uses minkowski which in X,Y make differences
To find k nearest neighbours, sklearn, by default, choose one of the kd_tree, BallTree and BruteForce methods, however, in your k_neighbours() function, you use BruteForce.
Last but not least, k value in your test is 5, while you're using 4 for skleran equivalent

answered Nov 10, 2021 at 18:20

aminrd

5,2605 gold badges34 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Yordan Иванов Over a year ago

Thanks man, that was helpful! Kind regards.

Yordan Иванов Over a year ago

I changed the sklearn.KNeighborsClassifier parameters to match with what you said but still not the same results. You can see in the code above. Any other suggestions?

Collectives™ on Stack Overflow

KNN Classifier build from scratch with numpy, what is wrong with the code?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related