Skip to content

Buffer re-use using ArrayPool and a few more checks #4293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Oct 4, 2019
Merged

Buffer re-use using ArrayPool and a few more checks #4293

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
798fee9
commit b468adbd7d3b625154bc991cb6f0931198303e25
harshithapv Oct 2, 2019
62036b7
fixed merge conflicts
harshithapv Oct 2, 2019
18e6922
Fixed some unit tests that were failing after the merge. Addressed a …
harshithapv Oct 2, 2019
49dc4a0
Fixed TensorFlow unit tests
harshithapv Oct 2, 2019
75eaa81
Changed the buffer re-use logic for ReadToEnd
harshithapv Oct 2, 2019
3c82762
Changed ReadToEnd function to read using span instead of unsafe blocks
harshithapv Oct 2, 2019
20d6660
removed unnecessary commits
harshithapv Oct 3, 2019
9e68a93
Added version check with backward compatability. Addressed Zeeshan's …
harshithapv Oct 3, 2019
c3f8d9d
Kept my changes. Need those tests.
harshithapv Oct 3, 2019
1c0e81c
Fixed tab and synced to master
harshithapv Oct 3, 2019
9ebd4f5
Addressed comments. Checkpoint commit
harshithapv Oct 3, 2019
1b893b3
Merged Mustafa's changes. Synced with master
harshithapv Oct 3, 2019
b30daf5
changed the solution files and version check in ImageLoader.cs
harshithapv Oct 3, 2019
f7d68dd
Added changes for StableApi.csproj
harshithapv Oct 3, 2019
dcb2583
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
harshithapv Oct 3, 2019
a94d56e
Added ArrayPool for buffer re-use
harshithapv Oct 3, 2019
c1dbd41
Handled the case when MakeGetter src is empty we need to send an empt…
harshithapv Oct 3, 2019
56e1e25
Merge branch 'master' of https://github.com/dotnet/machinelearning in…
harshithapv Oct 3, 2019
fe0de45
Addressed comments
harshithapv Oct 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed merge conflicts
  • Loading branch information
@harshithapv
harshithapv committed Oct 2, 2019
commit 62036b7b5db0f193436afbf94c7e7b6fbda27943
21 changes: 19 additions & 2 deletions ...rosoft.ML.Samples/Dynamic/ImageClassification/ResnetV2101TransferLearningEarlyStopping.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ public static void Example()

IDataView trainDataset = trainTestData.TrainSet;
IDataView testDataset = trainTestData.TestSet;

/*
var pipeline = mlContext.Model.ImageClassification(
"ImagePath", "Label",
// Just by changing/selecting InceptionV3 here instead of
Expand All @@ -68,7 +68,24 @@ public static void Example()
learningRate: 0.01f,
earlyStopping: new ImageClassificationEstimator.EarlyStopping(minDelta: 0.001f, patience:20, metric:ImageClassificationEstimator.EarlyStoppingMetric.Loss),
metricsCallback: (metrics) => Console.WriteLine(metrics),
validationSet: testDataset);
validationSet: testDataset);*/
var validationSet = mlContext.Transforms.LoadImages("Image", fullImagesetFolderPath, false, "ImagePath") // false indicates we want the image as a VBuffer<byte>
.Fit(testDataset)
.Transform(testDataset);

var pipeline = mlContext.Transforms.LoadImages("Image", fullImagesetFolderPath, false, "ImagePath") // false indicates we want the image as a VBuffer<byte>
.Append(mlContext.Model.ImageClassification(
"Image", "Label",
// Just by changing/selecting InceptionV3 here instead of
// ResnetV2101 you can try a different architecture/pre-trained
// model.
arch: ImageClassificationEstimator.Architecture.ResnetV2101,
epoch: 50,
batchSize: 10,
learningRate: 0.01f,
metricsCallback: (metrics) => Console.WriteLine(metrics),
validationSet: validationSet,
disableEarlyStopping: true));


Console.WriteLine("*** Training the image classification model with " +
Expand Down
4 changes: 2 additions & 2 deletions ...osoft.ML.Samples/Dynamic/ImageClassification/ResnetV2101TransferLearningTrainTestSplit.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ public static void Example()
batchSize: 10,
learningRate: 0.01f,
metricsCallback: (metrics) => Console.WriteLine(metrics),
validationSet: testDataset,
disableEarlyStopping: true);
validationSet: validationSet,
disableEarlyStopping: true));


Console.WriteLine("*** Training the image classification model with " +
Expand Down
1 change: 1 addition & 0 deletions pkg/Microsoft.ML.ImageAnalytics/Microsoft.ML.ImageAnalytics.nupkgproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<PropertyGroup>
<TargetFrameworks>netstandard2.0;netstandard2.1;netcoreapp2.1</TargetFrameworks>
<PackageDescription>ML.NET component for Image support</PackageDescription>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>

<ItemGroup>
Expand Down