Context.Data.CreateTextLoader<T> throws error Can't determine the number of source columns without valid data

[PeterPann23]

System information

• OS version/distro: Windows 10
• .NET Version (eg., dotnet --info):
  AutoML 0.3.0
  Microsoft.ML 1.0.0.0

Issue

I thought one can map a class and load it with the annotation but get an error when calling it on a parameter that's not available for me.

• What did you do?
  I wrote a little test to play with Microsoft.ML.AutoML and test it against a label and a vector
  I add my litle Program
  where I call in the ticket

basically I cal:

public static IDataView GetDataView<T>(MLContext mlContext, FileInfo trainingFile)
{
    var loader = mlContext.Data.CreateTextLoader<T>(separatorChar: '|', hasHeader: false);
    return loader.Load(trainingFile.FullName);
            
}
public class Data
{
     [LoadColumn(0)]
     public string Label { get; }

     [LoadColumn(1, 40_731)]
     [VectorType(40_730)]
     public float[] Features { get; }
}

• What happened?
  I get an error stating

System.ArgumentOutOfRangeException: 'Can't determine the number of source columns without valid data
Parameter name: Source'

Stack

   at Microsoft.ML.Data.TextLoader.Bindings..ctor(TextLoader parent, Column[] cols, IMultiStreamSource headerFile, IMultiStreamSource dataSample)
   at Microsoft.ML.Data.TextLoader..ctor(IHostEnvironment env, Options options, IMultiStreamSource dataSample)
   at Microsoft.ML.Data.TextLoader.CreateTextLoader[TInput](IHostEnvironment host, Boolean hasHeader, Char separator, Boolean allowQuoting, Boolean supportSparse, Boolean trimWhitespace, IMultiStreamSource dataSample)
   at ConsoleMLWizard.Program.GetDataView[T](MLContext mlContext, FileInfo trainingFile) in C:\Users\W2307\source\repos\ConsoleMLWizard\Program.cs:line 99
   at ConsoleMLWizard.Program.Main(String[] args) in C:\Users\W2307\source\repos\ConsoleMLWizard\Program.cs:line 37

• What did you expect?

I expected the anotations to work, I can load the data fine using:

var loader = context.Data.CreateTextLoader(options: new TextLoader.Options()
{
    Columns = new[] {
        new TextLoader.Column(name:"Label", dataKind: DataKind.String, index: 0),
        new TextLoader.Column(name:"Features",dataKind:DataKind.Single,minIndex:0,maxIndex:40731)
    },
    HasHeader = false,
    Separators = new[] { ',' },
                
});
var data = loader.Load(dataFile.FullName);

Source code / logs

[srsaggam]

Could it be because of _ in your
[LoadColumn(1, 40_731)]

Try [LoadColumn(1, 40731)] ??

Update: My bad! this is a new notation for digit separator.

[PeterPann23]

Why would it, it's a valid integer and part of the new notation named digit separator are you not able to repoduce the error?

[artidoro]

So I think the solution to your issue is that your Data class should have both get and set auto-properties. So it should be defined as follows:

public static IDataView GetDataView<T>(MLContext mlContext, FileInfo trainingFile)
{
    var loader = mlContext.Data.CreateTextLoader<T>(separatorChar: '|', hasHeader: false);
    return loader.Load(trainingFile.FullName);
            
}
public class Data
{
     [LoadColumn(0)]
     public string Label { get; set; }

     [LoadColumn(1, 40_731)]
     [VectorType(40_730)]
     public float[] Features { get; set; }
}

[artidoro]

Let me know if that solves the problem, and in the meantime I'll try to see if we can catch the error earlier, or improve the message.

[PeterPann23]

LOL what a dumb mistake to make, thanks!