IDataView Type System needs to be extensible to support Image scenarios properly

[glebuk]

The Problem

Currently it is impossible make prediction from images if they do not come from file. PredictionEngine<IT,OT> and IEnumerable to IDV is impossible if T contains images/

The Root Cause

The fundamental issue is that in several places in our system, our types are hardcoded and limited to a small set. Actual IDataView type system IS open, so it should be possible to support any type. However, some of our machinery is not so extensible. For example, if we need to add full support of images, we would need to update:

• converters between IEnumerable <-> IDataView,
• PredictionEngine<>
• API Utils,
• Schema utils
• etc.

What we need is a dependency-injection mechanism to allow component developers to inject definitions of types supported by our IDV machinery. Such mechanism should be flexible and extensible. For an example of hard-coded implementation, check #3263. It is illustrative but incorrect as it is not flexible. Note @TomFinley's comment on different options on how to do this properly. We need to define a mechanism that is extensible, so that a dev can add new transforms to work on new data types, such as sound and enjoy all benefits of ML.NET

Definition of Done:

• Mechanism to register additional IDV types so that they are supported by PredictionEngine and IEnumerable converters.
• Add such registration for images and date types
• Ensure that the registration (at least for images) occurs automatically whenever appropriate component is used (for example, if image resizing transform is used, the image type is registered automatically )
• Ensure that prediction engine, conversion to and from IEnumerable work correctly with tests.

The following issues will also be solved by this issue:

#3369, #3460, #3582, #2121, #2495, #3582, #3274
In addition, this work is needed to properly support ONNX and TF scenarios for structured data such as image, speech, video, or Audio.

[CESARDELATORRE]

Agree. I've been looking for in-memory images support since #2121. I think this is "a must" feature (pre-requisite) in ML.NET (support for in-memory images) in order to be able to release the TensorFlow and ONNX packages as v1.0 (Currently 0.12).
Note that this proposed implementation would be centralized (not specific within the TensorFlow or ONNX packages), though, so it'll need a new version for ML.NET itself, too. Minor version?

[wschin]

It should be a minor version because we won't break BC. Once I had a solution locally but got removed when doing git reset... Let me see if I can reconstruct it.