Fairlearn #6539
Merged
Fairlearn #6539
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added `inPlace:true` to the append to fix the bug that no columns are added; Added unit testing to test the basic funcitonality of Metrc.Regression.ByGroup()
Added MSE and MAE (MeanAbsoluteError), and included tests for MSE and RMS
RMS and MSE fully supported by Fairlearn.Metric.Regression now accross all three functions
created the moment class and utilityParity class. Initial commit
Passed the initial unit tests for Demographic Parity. Every class is made public which needs to be changed in the future. Utility Parity still needs to be changed for other parities to work
…yClassificationSearchSpaceGenerator update
…lculating the signed weights
There is still a lot of development needed to be done with the moment class
…search getting sensitive feature column names directly from a getter function, adjusted default value for the option to a random value
built the first prototype for the gridSearchTrialRunner. We have enabled a seperate training set for training and testing set for validating the result, which is a different approach from the original implementation.
Added a new feature so that users can look up row item by the name of the column instead of the raw row index
…ridsearch" This reverts commit a9f07a9.
… fairlearn In fairlearn AutoML, we have to add in a singleton to the serviceCollection called moment, which will be later extracted to calculate the fairness parity.
Added an if statement to output fairlearn metric if using fairlearn
…nsion in fairlearn" This reverts commit 328f718.
the experiment is able to add a moment to its serviceCollection which is later used to calculate fairlearn parity.
Created a tuner so that we can go through the search space through the gridsearch algorithm
| // get all the columns of the schema | ||
| DataViewSchema columns = _eval.Schema; | ||
|
|
||
| // TODO: is converting IDataview to DataFrame the best practice? |
There was a problem hiding this comment.
Its not the best practice, especially when input files (depending on the trainer) can be streamed in and can be larger than the memory on the machine. Its probably fine to start out this way, but it will be something that is needed to change later.
| using Microsoft.ML.SearchSpace; | ||
| using Microsoft.ML.SearchSpace.Option; | ||
|
|
||
| namespace Microsoft.ML.Fairlearn.reductions |
There was a problem hiding this comment.
Did you mean to have the namespace be reductions with a lowercase r? Would be better to be a capital.
There was a problem hiding this comment.
@LittleLittleCloud looks like its still lowecase r. Is there a reason for it to be that way?
There was a problem hiding this comment.
Nope there's no reason for that but a mistake. I thought I fixed it but maybe not. Let me check
michaelgsharp
requested changes
May 4, 2023
michaelgsharp
approved these changes
May 10, 2023
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #6539 +/- ##
==========================================
+ Coverage 68.59% 68.78% +0.19%
==========================================
Files 1201 1215 +14
Lines 250296 251523 +1227
Branches 26094 26227 +133
==========================================
+ Hits 171681 173006 +1325
+ Misses 71802 71705 -97
+ Partials 6813 6812 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
|
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We are excited to review your PR.
So we can do the best job, please check:
Fixes #nnnnin your description to cause GitHub to automatically close the issue(s) when your PR is merged.Might be helpful to resolve #1912