Machine Learning KNN (IT & Computer Science Coursework)

Machine Learning KNN
Q1: Does a larger feature space increase or decrease the chance (extend) of overfitting? Explain your answer in 1 or more sentences.
A: A larger feature space would increase the chances of overfitting than a smaller feature space. This is because the model’s complexity increases when the number of features also increase. Accordingly, since overfitting refers to a function that is very much fit to a limited series of data points, then the increase in complexity (and features of course) will lead to this case.
Q3: Assume that your k-nearest neighbor algorithms suffers from overfitting, so you increase the size of the training set size. You find that the predictive accuracy on new data has improved. However, what are downsides of increasing the training dataset size.
A: One of the downsize of increasing dataset size is a slight drop or performance, especially when it reaches the point where additional data points can lead to diminishing returns. Another downside is the possibility of over-representation of the training data. This would also reduce the test accuracy.
Q4: Does the (computational) time complexity of k-Nearest Neighbor classifier grow linearly, quadratically, or exponentially with the number of samples in the training dataset? Explain your answer in 1 or more sentences.
The computational complexity of classifying new samples increases linearly with the increase in the number of samples in the training data set
A: When the number of samples in a training dataset increases, so does the (computational) time complexity of classifying new samples. Specifically, th...