Skip to content

Issue 4047, improve error message give expected type #5189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 2, 2020

Conversation

frank-dong-ms-zz
Copy link
Contributor

address #4047

@frank-dong-ms-zz frank-dong-ms-zz marked this pull request as ready for review June 2, 2020 06:20
@frank-dong-ms-zz frank-dong-ms-zz requested a review from a team as a code owner June 2, 2020 06:20
@frank-dong-ms-zz frank-dong-ms-zz requested a review from harishsk June 2, 2020 06:20
@codecov
Copy link

codecov bot commented Jun 2, 2020

Codecov Report

Merging #5189 into master will decrease coverage by 4.25%.
The diff coverage is 39.20%.

@@            Coverage Diff             @@
##           master    #5189      +/-   ##
==========================================
- Coverage   73.33%   69.07%   -4.26%     
==========================================
  Files        1007      762     -245     
  Lines      188388   144527   -43861     
  Branches    20286    18374    -1912     
==========================================
- Hits       138145    99833   -38312     
+ Misses      44723    39510    -5213     
+ Partials     5520     5184     -336     
Flag Coverage Δ
#Debug 69.07% <39.20%> (-4.26%) ⬇️
#production 69.07% <39.20%> (+0.05%) ⬆️
#test ?
Impacted Files Coverage Δ
...rosoft.ML.Data/DataView/BatchDataViewMapperBase.cs 68.67% <0.00%> (-0.84%) ⬇️
...soft.ML.Data/DataView/DataViewConstructionUtils.cs 85.48% <0.00%> (-0.15%) ⬇️
...rc/Microsoft.ML.Data/Scorers/RowToRowScorerBase.cs 85.95% <0.00%> (-0.36%) ⬇️
...rosoft.ML.Data/Transforms/LabelConvertTransform.cs 15.38% <0.00%> (-0.14%) ⬇️
...rosoft.ML.Data/Transforms/PerGroupTransformBase.cs 76.33% <0.00%> (-0.46%) ⬇️
...Microsoft.ML.Transforms/OptionalColumnTransform.cs 77.68% <0.00%> (-0.46%) ⬇️
src/Microsoft.ML.Transforms/ProduceIdTransform.cs 0.00% <0.00%> (ø)
...Microsoft.ML.Transforms/SvmLight/SvmLightLoader.cs 95.45% <0.00%> (-0.25%) ⬇️
src/Microsoft.ML.Data/Data/DataViewUtils.cs 72.45% <25.00%> (-0.24%) ⬇️
src/Microsoft.ML.Data/DataView/Transposer.cs 83.92% <25.00%> (-0.27%) ⬇️
... and 271 more

Comment on lines 227 to 228
throw Ch.Except($"Invalid TValue: '{typeof(TValue)}', " +
$"expected type: '{_getter.GetType().GetGenericArguments().First()}'.");
Copy link
Member

@antoniovs1029 antoniovs1029 Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: shouldn't this be _getter.GetType().GetGenericArguments().First().GetGenericArguments().First() just as you did in Transposer.cs since this expects a VBuffer? #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is bug


In reply to: 433884538 [](ancestors = 433884538)

Comment on lines +735 to +740

var originGetter = _getters[_colToActivesIndex[column.Index]];
ValueGetter<TValue> getter = originGetter as ValueGetter<TValue>;
if (getter == null)
throw Ch.Except("Invalid TValue: '{0}'", typeof(TValue));
throw Ch.Except($"Invalid TValue: '{typeof(TValue)}', " +
$"expected type: '{originGetter.GetType().GetGenericArguments().First()}'.");
Copy link
Member

@antoniovs1029 antoniovs1029 Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(pinned to code to allow for threaded conversation)

In general, I think that the approach you used to fix this issue is fine, and it's actually a very good addition to make our exception messages more clear.

The only potential problem I see with it is that maybe this will end up throwing another exception: for example, the _getters in this method is an array of Delegates (i.e. it's not an array of ValueGetter<>'s) so the cast as ValueGetter<TValue> could potentially fail not only if the the generic parameter is mistaken, but also in the case that _getters[i] isn't a ValueGetter<> to begin with. If it isn't a ValueGetter<>, and if it's not a delegate with generic types, then the .First() call in here will throw an exception because the .GetGenericArguments() method would have thrown an empty array.

I haven't checked the other places were you've made changes, but some of them do have an original _getter which is already a ValueGetter<> (so this wouldn't be a problem in those cases) but in other cases the original _getter is a Delegate instead of ValueGetter<> (so potentially they might not have GenericArguments, and then throw another exception).

I'm not sure if I'm explaining myself.

In any case, I also don't know if we should worry about this problem... that other exception would only be thrown in some very fringe case (which I'm not sure if it's actually possible to reach that case), and in any way we were already throwing an exception. So maybe this is unimportant. #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand your concern, please check out the implementation of this _getters, mainly method CreateGetterDelegate in same class and you will find below implementation, so it is guaranteed you will get ValueGetter for this Delegate but the real problem is you are using wrong TValue type.

        private ValueGetter<TValue> CreateGetterDelegate<TValue>(ShufflePipe pipe)
        {
            Ch.AssertValue(pipe);
            Ch.Assert(pipe is ShufflePipe<TValue>);
            var pipe2 = (ShufflePipe<TValue>)pipe;
            ValueGetter<TValue> getter =
                (ref TValue value) =>
                {
                    Ch.Assert(_pipeIndex == _pipeIndices[_circularIndex]);
                    pipe2.Fetch(_pipeIndex, ref value);
                };
            return getter;
        }

In reply to: 433925403 [](ancestors = 433925403)

Copy link
Member

@antoniovs1029 antoniovs1029 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from a nit comment I left, and an observation that might be unimportant, it LGTM.

Comment on lines 694 to +697
ValueGetter<VBuffer<TValue>> getter = _getter as ValueGetter<VBuffer<TValue>>;
if (getter == null)
throw Ch.Except("Invalid TValue: '{0}'", typeof(TValue));
throw Ch.Except($"Invalid TValue: '{typeof(TValue)}', " +
$"expected type: '{_getter.GetType().GetGenericArguments().First().GetGenericArguments().First()}'.");
Copy link
Member

@antoniovs1029 antoniovs1029 Jun 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In here we expect a VBuffer, and that's why you're getting .GetGenericArguments() twice, which is fine to get a descriptive exception.

But what happens if _getter is, let's say, of type ValueGetter<int> whereas we expect one of type ValueGetter<VBuffer<float>> ? since the getter we're getting isn't a VBuffer<> then another exception will be thrown when calling the second .First() because the second .GetGenericArguments() would return an empty array. This other exception would be more confusing than the original one, so please add the necessary logic to handle this case correctly in the places where we expect a VBuffer. 😄 #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check out the definition of _getter: private readonly ValueGetter<VBuffer> _getter;


In reply to: 433952241 [](ancestors = 433952241)

@antoniovs1029 antoniovs1029 self-requested a review June 2, 2020 15:12
Copy link
Member

@antoniovs1029 antoniovs1029 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add the changes of the last comment I made. Also, I wonder if it would be a good idea to add a Test to check that the changes work correctly for the ValueGetter case.

@harishsk
Copy link
Contributor

harishsk commented Jun 2, 2020

You can include the issue number in your PR, but please give a more meaningful title to your PR. #Resolved

@frank-dong-ms-zz frank-dong-ms-zz changed the title Issue 4047 Issue 4047, improve error message give expected type Jun 2, 2020
@frank-dong-ms-zz
Copy link
Contributor Author

type for ValueGetter is determined by the type definition so it is safe to give exception


In reply to: 422781310 [](ancestors = 422781310)

@frank-dong-ms-zz
Copy link
Contributor Author

Thanks, I changed the title, issue number is already put at PR comments


In reply to: 637723803 [](ancestors = 637723803)

@antoniovs1029 antoniovs1029 self-requested a review June 2, 2020 21:00
@frank-dong-ms-zz frank-dong-ms-zz merged commit 1ea2b47 into dotnet:master Jun 2, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants