You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This sample demonstrates using the Intel® Neural Compressor, which is part of the Intel® AI Tools with the with Intel® Optimizations for TensorFlow* to speed up inference by simplifying the process of converting the FP32 model to INT8/BF16.
3
+
This sample demonstrates using the Intel® Neural Compressor, which is part of the AI Tools with the with Intel® Extension for TensorFlow* to speed up inference by simplifying the process of converting the FP32 model to INT8/BF16.
4
4
5
-
| Property | Description
6
-
|:--- |:---
7
-
| Category | Getting Started
8
-
| What you will learn | How to use Intel® Neural Compressor tool to quantize the AI model based on TensorFlow* and speed up the inference on Intel® Xeon® CPUs
9
-
| Time to complete | 10 minutes
5
+
| Property | Description|
6
+
|:--- |:--|
7
+
| Category | Getting Started|
8
+
| What you will learn | How to use Intel® Neural Compressor tool to quantize the AI model based on TensorFlow* and speed up the inference on Intel® Xeon® CPUs|
9
+
| Time to complete | 10 minutes|
10
10
11
11
12
12
## Purpose
13
13
14
-
This sample shows the process of building a convolutional neural network (CNN) model to recognize handwritten numbers and demonstrates how to increase the inference performance by using Intel® Neural Compressor. Low-precision optimizations can speed up inference. Intel® Neural Compressor simplifies the process of converting the FP32 model to INT8/BF16. At the same time, Intel® Neural Compressor tunes the quantization method to reduce the accuracy loss, which is a big blocker for low-precision inference.
14
+
This sample shows the process of building a convolution neural network (CNN) model to recognize handwritten numbers and demonstrates how to increase the inference performance by using Intel® Neural Compressor. Low-precision optimizations can speed up inference. Intel® Neural Compressor simplifies the process of converting the FP32 model to INT8/BF16. At the same time, Intel® Neural Compressor tunes the quantization method to reduce the accuracy loss, which is a big blocker for low-precision inference.
15
15
16
16
You can achieve higher inference performance by converting the FP32 model to INT8 or BF16 model. Additionally, Intel® Deep Learning Boost (Intel® DL Boost) in Intel® Xeon® Scalable processors and Xeon® processors provides hardware acceleration for INT8 and BF16 models.
17
17
18
18
You will learn how to train a CNN model with Keras and TensorFlow*, use Intel® Neural Compressor to quantize the model, and compare the performance to see the benefit of Intel® Neural Compressor.
19
19
20
20
## Prerequisites
21
21
22
-
| Optimized for | Description
23
-
|:--- |:---
24
-
| OS | Ubuntu* 20.04 (or newer) <br> Windows 11, 10*
| Software | Intel® Neural Compressor, Intel® Extension for TensorFlow*|
27
+
> **Note**: AI and Analytics samples are validated on AI Tools Offline Installer. For the full list of validated platforms refer to [Platform Validation](https://github.com/oneapi-src/oneAPI-samples/tree/master?tab=readme-ov-file#platform-validation).
27
28
28
29
### Intel® Neural Compressor and Sample Code Versions
29
30
30
31
>**Note**: See the [Intel® Neural Compressor](https://github.com/intel/neural-compressor) GitHub repository for more information and recent changes.
31
32
32
-
This sample is updated regularly to match the Intel® Neural Compressor version in the latest Intel® AI Tools release. If you want to get the sample code for an earlier toolkit release, check out the corresponding git tag.
33
+
This sample is updated regularly to match the Intel® Neural Compressor version in the latest AI Tools release. If you want to get the sample code for an earlier toolkit release, check out the corresponding git tag.
33
34
34
35
1. List the available git tags.
35
36
```
@@ -43,22 +44,6 @@ This sample is updated regularly to match the Intel® Neural Compressor version
43
44
git checkout 2022.3.0
44
45
```
45
46
46
-
### For Local Development Environments
47
-
48
-
You will need to download and install the following toolkits, tools, and components to use the sample.
49
-
50
-
-**Intel® AI Tools **
51
-
52
-
You can get the AI Kit from [Intel® oneAPI Toolkits](https://www.intel.com/content/www/us/en/developer/tools/oneapi/toolkits.html#analytics-kit). <br> See [*Get Started with the Intel® AI Tools for Linux**](https://www.intel.com/content/www/us/en/develop/documentation/get-started-with-ai-linux) for AI Tools installation information and post-installation steps and scripts.
53
-
54
-
Intel® Extension for TensorFlow* is included in the Intel AI Tools Offline Installer package.
55
-
56
-
-**Jupyter Notebook**
57
-
58
-
Install using PIP: `$pip -m install notebook`. <br> Alternatively, see [*Installing Jupyter*](https://jupyter.org/install) for detailed installation instructions.
59
-
60
-
-**TensorFlow\* 2.2** (or newer)
61
-
62
47
## Key Implementation Details
63
48
64
49
The sample demonstrates how to:
@@ -72,131 +57,101 @@ The sample demonstrates how to:
72
57
- Test the performance of the FP32 model and INT8 (quantization) model.
73
58
74
59
## Environment Setup
60
+
61
+
You will need to download and install the following toolkits, tools, and components to use the sample.
62
+
75
63
If you have already set up the PIP or Conda environment and installed AI Tools go directly to Run the Notebook.
76
64
77
-
### On Linux*
65
+
### 1. Get AI Tools
66
+
67
+
Required AI Tools: Intel® Neural Compressor, Intel® Extension for TensorFlow* (CPU)
78
68
79
-
#### Setup Conda Environment
69
+
If you have not already, select and install these Tools via [AI Tools Selector](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-tools-selector.html). AI and Analytics samples are validated on AI Tools Offline Installer. It is recommended to select Offline Installer option in AI Tools Selector.
80
70
81
-
You can list the available conda environments using a command similar to the following.
71
+
>**Note**: If Docker option is chosen in AI Tools Selector, refer to [Working with Preset Containers](https://github.com/intel/ai-containers/tree/main/preset) to learn how to run the docker and samples.
82
72
83
-
##### Option 1: Clone Conda Environment from AI Toolkit Conda Environment
73
+
###2. (Offline Installer) Activate the AI Tools bundle base environment
84
74
85
-
Please confirm to install Intel AI Toolkit!
75
+
If the default path is used during the installation of AI Tools:
This step is optional if you plan to open the notebook on your local server.
102
+
Install for Jupyter Notebook:
155
103
156
-
## Run the `Intel® Neural Compressor TensorFlow* Getting Started*` Sample
104
+
```
105
+
pip install -r requirements.txt
106
+
```
107
+
108
+
For Jupyter Notebook, refer to [Installing Jupyter](https://jupyter.org/install) for detailed installation instructions.
109
+
110
+
111
+
## Run the Sample
157
112
158
113
> **Note**: Before running the sample, make sure [Environment Setup](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Getting-Started-Samples/INC-Sample-for-TensorFlow#environment-setup) is completed.
159
114
>
160
-
> Linux*:
161
115
Go to the section which corresponds to the installation method chosen in [AI Tools Selector](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-tools-selector.html) to see relevant instructions:
[IPKernelApp] ERROR | No such comm target registered: jupyter.widget.version
241
+
242
+
```
243
+
244
+
#### 3. Follow the instructions to open the URL with the token in your browser
245
+
246
+
#### 4. Select the Notebook
247
+
248
+
```
249
+
inc_sample_tensorflow.ipynb
250
+
```
251
+
252
+
#### 5. Change the kernel to `tensorflow`
253
+
254
+
#### 6. Run every cell in the Notebook in sequence
255
+
256
+
### Docker
257
+
258
+
AI Tools Docker images already have Get Started samples pre-installed. Refer to [Working with Preset Containers](https://github.com/intel/ai-containers/tree/main/preset) to learn how to run the docker and samples.
0 commit comments