数据融合>> vimpy>> 返回
项目作者： bdwilliamson

项目描述 ：
Perform inference on algorithm-agnostic variable importance in Python
高级语言： Python
项目地址: git://github.com/bdwilliamson/vimpy.git
创建时间： 2017-04-04T21:46:16Z
项目社区：https://github.com/bdwilliamson/vimpy
开源协议：MIT License
下载


" class="reference-link">Python/vimpy: inference on algorithm-agnostic variable importance

PyPI version
License: MIT

Software author: Brian Williamson

Methodology authors: Brian Williamson, Peter Gilbert, Noah Simon, Marco Carone

R package: https://github.com/bdwilliamson/vimp

Introduction

In predictive modeling applications, it is often of interest to determine the relative contribution of subsets of features in explaining an outcome; this is often called variable importance. It is useful to consider variable importance as a function of the unknown, underlying data-generating mechanism rather than the specific predictive algorithm used to fit the data. This package provides functions that, given fitted values from predictive algorithms, compute nonparametric estimates of variable importance based on $R^2$, deviance, classification accuracy, and area under the receiver operating characteristic curve, along with asymptotically valid confidence intervals for the true importance.

For more details, please see the accompanying manuscripts “Nonparametric variable importance assessment using machine learning techniques” by Williamson, Gilbert, Carone, and Simon (Biometrics, 2020), “A unified approach for inference on algorithm-agnostic variable importance” by Williamson, Gilbert, Simon, and Carone (arXiv, 2020), and “Efficient nonparametric statistical inference on population feature importance using Shapley values” by Williamson and Feng (arXiv, 2020; to appear in the Proceedings of the Thirty-seventh International Conference on Machine Learning [ICML 2020]).

Installation

You may install a stable release of vimpy using pip by running python pip install vimpy from a Terminal window. Alternatively, you may install within a virtualenv environment.

You may install the current dev release of vimpy by downloading this repository directly.

Issues

If you encounter any bugs or have any specific feature requests, please file an issue.

Example

This example shows how to use vimpy in a simple setting with simulated data and using a single regression function. For more examples and detailed explanation, please see the R vignette.

  1. ## load required libraries
  2. import numpy as np
  3. import vimpy
  4. from sklearn.ensemble import GradientBoostingRegressor
  5. from sklearn.model_selection import GridSearchCV
  7. ## -------------------------------------------------------------
  8. ## problem setup
  9. ## -------------------------------------------------------------
  10. ## define a function for the conditional mean of Y given X
  11. def cond_mean(x = None):
  12.     f1 = np.where(np.logical_and(-2 <= x[:, 0], x[:, 0] < 2), np.floor(x[:, 0]), 0)
  13.     f2 = np.where(x[:, 1] <= 0, 1, 0)
  14.     f3 = np.where(x[:, 2] > 0, 1, 0)
  15.     f6 = np.absolute(x[:, 5]/4) ** 3
  16.     f7 = np.absolute(x[:, 6]/4) ** 5
  17.     f11 = (7./3)*np.cos(x[:, 10]/2)
  18.     ret = f1 + f2 + f3 + f6 + f7 + f11
  19.     return ret
  21. ## create data
  22. np.random.seed(4747)
  23. n = 100
  24. p = 15
  25. s = 1 # importance desired for X_1
  26. x = np.zeros((n, p))
  27. for i in range(0, x.shape[1]) :
  28.     x[:,i] = np.random.normal(0, 2, n)
  30. y = cond_mean(x) + np.random.normal(0, 1, n)
  32. ## -------------------------------------------------------------
  33. ## preliminary step: get regression estimators
  34. ## -------------------------------------------------------------
  35. ## use grid search to get optimal number of trees and learning rate
  36. ntrees = np.arange(100, 500, 100)
  37. lr = np.arange(.01, .1, .05)
  39. param_grid = [{'n_estimators':ntrees, 'learning_rate':lr}]
  41. ## set up cv objects
  42. cv_full = GridSearchCV(GradientBoostingRegressor(loss = 'ls', max_depth = 1), param_grid = param_grid, cv = 5)
  43. cv_small = GridSearchCV(GradientBoostingRegressor(loss = 'ls', max_depth = 1), param_grid = param_grid, cv = 5)
  45. ## -------------------------------------------------------------
  46. ## get variable importance estimates
  47. ## -------------------------------------------------------------
  48. # set seed
  49. np.random.seed(12345)
  50. ## set up the vimp object
  51. vimp = vimpy.vim(y = y, x = x, s = 1, pred_func = cv_full, measure_type = "r_squared")
  52. ## get the point estimate of variable importance
  53. vimp.get_point_est()
  54. ## get the influence function estimate
  55. vimp.get_influence_function()
  56. ## get a standard error
  57. vimp.get_se()
  58. ## get a confidence interval
  59. vimp.get_ci()
  60. ## do a hypothesis test, compute p-value
  61. vimp.hypothesis_test(alpha = 0.05, delta = 0)
  62. ## display the estimates, etc.
  63. vimp.vimp_
  64. vimp.se_
  65. vimp.ci_
  66. vimp.p_value_
  67. vimp.hyp_test_
  69. ## -------------------------------------------------------------
  70. ## using precomputed fitted values
  71. ## -------------------------------------------------------------
  72. np.random.seed(12345)
  73. folds_outer = np.random.choice(a = np.arange(2), size = n, replace = True, p = np.array([0.5, 0.5]))
  74. ## fit the full regression
  75. cv_full.fit(x[folds_outer == 1, :], y[folds_outer == 1])
  76. full_fit = cv_full.best_estimator_.predict(x[folds_outer == 1, :])
  78. ## fit the reduced regression
  79. x_small = np.delete(x[folds_outer == 0, :], s, 1) # delete the columns in s
  80. cv_small.fit(x_small, y[folds_outer == 0])
  81. small_fit = cv_small.best_estimator_.predict(x_small)
  82. ## get variable importance estimates
  83. np.random.seed(12345)
  84. vimp_precompute = vimpy.vim(y = y, x = x, s = 1, f = full_fit, r = small_fit, measure_type = "r_squared", folds = folds_outer)
  85. ## get the point estimate of variable importance
  86. vimp_precompute.get_point_est()
  87. ## get the influence function estimate
  88. vimp_precompute.get_influence_function()
  89. ## get a standard error
  90. vimp_precompute.get_se()
  91. ## get a confidence interval
  92. vimp_precompute.get_ci()
  93. ## do a hypothesis test, compute p-value
  94. vimp_precompute.hypothesis_test(alpha = 0.05, delta = 0)
  95. ## display the estimates, etc.
  96. vimp_precompute.vimp_
  97. vimp_precompute.se_
  98. vimp_precompute.ci_
  99. vimp_precompute.p_value_
  100. vimp_precompute.hyp_test_
  102. ## -------------------------------------------------------------
  103. ## get variable importance estimates using cross-validation
  104. ## -------------------------------------------------------------
  105. np.random.seed(12345)
  106. ## set up the vimp object
  107. vimp_cv = vimpy.cv_vim(y = y, x = x, s = 1, pred_func = cv_full, V = 5, measure_type = "r_squared")
  108. ## get the point estimate
  109. vimp_cv.get_point_est()
  110. ## get the standard error
  111. vimp_cv.get_influence_function()
  112. vimp_cv.get_se()
  113. ## get a confidence interval
  114. vimp_cv.get_ci()
  115. ## do a hypothesis test, compute p-value
  116. vimp_cv.hypothesis_test(alpha = 0.05, delta = 0)
  117. ## display estimates, etc.
  118. vimp_cv.vimp_
  119. vimp_cv.se_
  120. vimp_cv.ci_
  121. vimp_cv.p_value_
  122. vimp_cv.hyp_test_

The logo was created using hexSticker, lisa, and a python image distributed under the CC0 license. Many thanks to the maintainers of these packages and the Color Lisa team.