Python-sklearn-近邻方法

Sklearn 近邻方法sklearn.neighbors提供 KNN 分类/回归、最近邻搜索、核密度估计等。️ K-近邻分类KNeighborsClassifier⭐fromsklearn.neighborsimportKNeighborsClassifier modelKNeighborsClassifier(n_neighbors5,# 邻居数weightsuniform,# uniform或distance(距离加权)algorithmauto,# 搜索算法# auto: 自动选择# ball_tree: BallTree(高维慢)# kd_tree: KDTree(低维快)# brute: 暴力搜索(小数据)leaf_size30,# BallTree/KDTree 的叶节点大小p2,# Minkowski 距离的幂(1曼哈顿, 2欧氏)metricminkowski,# 距离度量metric_paramsNone,# 额外度量参数n_jobsNone)model.fit(X,y)# 关键属性print(model.classes_)print(model.effective_metric_)# 实际使用的度量print(model.effective_metric_params_)# 度量的参数print(model.n_features_in_)# 拟合时看到的特征数print(model.n_samples_fit_)# 拟合时的样本数print(model.outputs_2d_)# 训练时 y 是否为二维# 预测y_predmodel.predict(X)y_probmodel.predict_proba(X)# 各类别概率y_log_probmodel.predict_log_proba(X)# 查询 k 个最近邻distances,indicesmodel.kneighbors(X,n_neighbors5,return_distanceTrue)# distances: (n_samples, n_neighbors) — 最近邻的距离# indices: (n_samples, n_neighbors) — 最近邻的索引# 获取指定半径内的邻居indices,distancesmodel.radius_neighbors(X,radius1.0,return_distanceTrue)# 邻居图的稀疏表示graphmodel.kneighbors_graph(X,n_neighbors5,modeconnectivity)# mode: connectivity(0/1 矩阵) 或 distance(距离矩阵) K-近邻回归KNeighborsRegressor⭐fromsklearn.neighborsimportKNeighborsRegressor modelKNeighborsRegressor(n_neighbors5,weightsuniform,# uniform或distancealgorithmauto,leaf_size30,p2,metricminkowski,metric_paramsNone,n_jobsNone)model.fit(X,y)y_predmodel.predict(X)# 与分类器相同的方法distances,indicesmodel.kneighbors(X) 半径近邻RadiusNeighborsClassifierfromsklearn.neighborsimportRadiusNeighborsClassifier modelRadiusNeighborsClassifier(radius1.0,# 查询半径weightsuniform,# uniform或distancealgorithmauto,leaf_size30,p2,metricminkowski,outlier_labelNone,# 半径内无邻居时的标签# None: 导致错误# most_frequent: 用最常见的标签# int/str: 固定标签metric_paramsNone,n_jobsNone)model.fit(X,y)y_predmodel.predict(X)# 注意: 没有 predict_proba因为邻居数不定# 不同样本的邻居数可能不同RadiusNeighborsRegressorfromsklearn.neighborsimportRadiusNeighborsRegressor modelRadiusNeighborsRegressor(radius1.0,weightsuniform,algorithmauto,leaf_size30,p2,metricminkowski,metric_paramsNone,n_jobsNone)model.fit(X,y)y_predmodel.predict(X)NearestCentroid— 最近质心分类器每个类用其质心代表预测样本分配到最近的质心。fromsklearn.neighborsimportNearestCentroid modelNearestCentroid(metriceuclidean,shrink_thresholdNone# 收缩参数特征选择作用)model.fit(X,y)print(model.centroids_)# (n_classes, n_features)y_predmodel.predict(X) 最近邻搜索结构KDTree/BallTree直接使用底层的数据结构进行最近邻搜索。fromsklearn.neighborsimportKDTree,BallTreeimportnumpyasnp# KDTree低维数据高效通常 D 20treeKDTree(X,leaf_size40,metriceuclidean)# BallTree高维数据更优treeBallTree(X,leaf_size40,metriceuclidean)# 查询 k 最近邻dist,indtree.query(X_query,k5,return_distanceTrue)# 查询半径内邻居返回数组个数可能不同ind,disttree.query_radius(X_query,r1.0,return_distanceTrue)# 获取两棵树的核密度# tree.kernel_density() — 在 BallTree 中 核密度估计KernelDensity— 核密度估计fromsklearn.neighborsimportKernelDensityimportnumpyasnp modelKernelDensity(bandwidth1.0,# 带宽核宽度algorithmauto,# auto,ball_tree,kd_treekernelgaussian,# 核函数# gaussian — 高斯核# tophat — 均匀核# epanechnikov — Epanechnikov 核# exponential — 指数核# linear — 线性核# cosine — 余弦核metriceuclidean,atol0,# 结果的绝对容差rtol1e-8,# 相对容差breadth_firstTrue,# 广度优先或深度优先leaf_size40,metric_paramsNone)model.fit(X)# 评分: 返回每个样本的对数概率密度log_densitymodel.score_samples(X)densitynp.exp(log_density)# 转换为概率密度# 采样X_sampledmodel.sample(n_samples100,random_state42)# 总对数似然total_llmodel.score(X)# 所有样本的平均对数似然带宽选择:fromsklearn.model_selectionimportGridSearchCV params{bandwidth:np.logspace(-2,1,20)}gridGridSearchCV(KernelDensity(kernelgaussian),params,cv5)grid.fit(X)print(fBest bandwidth:{grid.best_params_[bandwidth]}) 近邻图kneighbors_graph()/radius_neighbors_graph()fromsklearn.neighborsimportkneighbors_graph,radius_neighbors_graph# K 近邻图Akneighbors_graph(X,n_neighbors5,modeconnectivity,n_jobs-1)# 半径近邻图Aradius_neighbors_graph(X,radius1.0,modedistance,n_jobs-1)# 可用于谱聚类fromsklearn.clusterimportSpectralClustering modelSpectralClustering(n_clusters3,affinityprecomputed)model.fit(A)NeighborhoodComponentsAnalysis— 邻域成分分析监督式距离度量学习。fromsklearn.neighborsimportNeighborhoodComponentsAnalysis ncaNeighborhoodComponentsAnalysis(n_componentsNone,# 降维后的维度initauto,# auto,pca,lda,identity,randomwarm_startFalse,max_iter50,tol1e-5,callbackNone,verbose0,random_stateNone)nca.fit(X,y)X_embeddednca.transform(X)# 配合 KNN 使用knnKNeighborsClassifier(n_neighbors3)knn.fit(nca.transform(X_train),y_train)y_predknn.predict(nca.transform(X_test)) 调参指导选择 K 值fromsklearn.model_selectionimportcross_val_scoreimportmatplotlib.pyplotasplt k_rangerange(1,31)scores[]forkink_range:knnKNeighborsClassifier(n_neighborsk)cv_scorescross_val_score(knn,X,y,cv5,scoringaccuracy)scores.append(cv_scores.mean())# 可视化plt.plot(k_range,scores)plt.xlabel(k)plt.ylabel(Cross-validated accuracy)plt.title(Elbow for KNN K)plt.show()best_kk_range[np.argmax(scores)]print(fBest k:{best_k})距离加权 vs 均匀加权# uniform: 所有邻居等权# distance: 越近的邻居权重越大 (1/distance)# 当 K 较大时distance 加权通常更好KNeighborsClassifier(n_neighbors15,weightsdistance) 特点总结优点缺点简单直观无需训练预测慢需计算所有距离无假设、非参数维度灾难高维失效天然支持多分类对特征尺度敏感需标准化可增量添加数据内存消耗大需存储全部数据[[sklearn-总览|← 返回总览]]