Skip to content

Fix Tensorflow -> TensorFlow #2783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/design/alps_submitter.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ LABEL class
```

### Semantic Analyze
Feature Expressions except for Tensorflow Feature Column API should raise an error.
Feature Expressions except for TensorFlow Feature Column API should raise an error.
```sql
/* Not supported */
select * from kaggle_credit_fraud_training_data
Expand Down
6 changes: 3 additions & 3 deletions doc/design/codegen_couler_use_ir.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ In this design, we'll only have `codegen_couler.go` to generate programs to run

For example, to run the training steps, we intend to use couler API call: `couler.{xgboost,tensorflow,elasticdl}.train(model_def, data)` to generate a training step. The `train` function must take lots of parameters to run the training job as the SQLFlow statement describes, see [here](https://github.com/sql-machine-learning/sqlflow/blob/develop/python/runtime/tensorflow/train.py#L52) as an example.

To implement the single `codegen_couler.go` to support generate code that can run either Tensorflow/XGBoost/ElasticDL/ALPS programs, we have below two choices:
To implement the single `codegen_couler.go` to support generate code that can run either TensorFlow/XGBoost/ElasticDL/ALPS programs, we have below two choices:

1. `couler.{tensorflow/xgboost/elasticdl}.train` have different arguments defined, so we can do:

```go
if ir.ModelType == "Tensorflow":
if ir.ModelType == "TensorFlow":
tfFiller := TFFiller{
Estimator: generateTFEstimatorCode(ir),
FeatureColumns: generateFeatureColumnsCode(ir),
Expand Down Expand Up @@ -55,6 +55,6 @@ To implement the single `codegen_couler.go` to support generate code that can ru

We intend to use the solution **No.2** described above for these reasons.

1. If a data scientist needs to add a new type of engine (SVM, SKLearn, PyTorch, etc.) other than Tensorflow/XGBoost/ElasticDL, he/she can use python only to define a `couler.{new_engine}.train` function without modifying the Go code in SQLFlow.
1. If a data scientist needs to add a new type of engine (SVM, SKLearn, PyTorch, etc.) other than TensorFlow/XGBoost/ElasticDL, he/she can use python only to define a `couler.{new_engine}.train` function without modifying the Go code in SQLFlow.
1. `codegen_couler.go` have less code.
1. All submitter unit tests can run in couler.
2 changes: 1 addition & 1 deletion doc/design/couler_sqlflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ Note: You can check more details about the IR definition from [ir.go](/pkg/ir/ir

### SQLFLow Submitter Python Module

An SQLFlow submitter Python module `runtime.{tensorflow,xgboost,elasticdl}.train` accepts an SQLFlow IR with protobuf text format, and then submit a Tensorflow, XGBoost or ElasticDL training job, we can call it like:
An SQLFlow submitter Python module `runtime.{tensorflow,xgboost,elasticdl}.train` accepts an SQLFlow IR with protobuf text format, and then submit a TensorFlow, XGBoost or ElasticDL training job, we can call it like:

``` bash
cat ir.proto_text | python -m runtime.xgboost.train
Expand Down
2 changes: 1 addition & 1 deletion doc/design/diag_attribute_error.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ this can active three advantages at least:

1. Early testing, we can do early testing before running the job; users can wait less time and cluster save resources.
2. More accurate diagnostic message.
3. Model developers do not have to involve dependencies other than Keras or Tensorflow.
3. Model developers do not have to involve dependencies other than Keras or TensorFlow.

## Design

Expand Down
2 changes: 1 addition & 1 deletion doc/design/model_zoo.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ $ sqlflow delete repo model_image:v0.1

For a SELECT program using the SQLFlow syntax extension, the SQLFlow server converts it into a [workflow](workflow.md) and submit the workflow job to a workflow engine like Argo/Tekton on Kubernetes. Each step in the workflow is one SQL statement.

By default, we use a default Docker image to run the training, predicting or explaining job. The default Docker image contains pre-made Tensorflow estimator models, Keras models defined in [sqlflow_models repo](https://github.com/sql-machine-learning/models) and XGBoost. To use a custom model repo Docker image, write SQL statements mentioned above:
By default, we use a default Docker image to run the training, predicting or explaining job. The default Docker image contains pre-made TensorFlow estimator models, Keras models defined in [sqlflow_models repo](https://github.com/sql-machine-learning/models) and XGBoost. To use a custom model repo Docker image, write SQL statements mentioned above:

```sql
SELECT * FROM employee WHERE onboard_year < 2019
Expand Down
2 changes: 1 addition & 1 deletion doc/model_parameter.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ TBD

TBD

## Tensorflow Parameters
## TensorFlow Parameters

### TRAIN

Expand Down
4 changes: 2 additions & 2 deletions doc/sqlflow.org_cn/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
1. [模型解释]()
1. [模型列表](model_list.cn.md)
1. [有监督学习]()
1. [Tensorflow Estimator 模型]()
1. [Tensorflow Keras 模型]()
1. [TensorFlow Estimator 模型]()
1. [TensorFlow Keras 模型]()
1. [XGBoost 模型]()
1. [无监督学习]()
1. [部署]()
Expand Down
2 changes: 1 addition & 1 deletion doc/talk/201906/sqlflow.slide
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ TensorFlow model inputs are dense tensors, which are normally high-dimensional m
- Hash the name string into a 64-bit integer, then bucketizing hashed value into 100 dimension columns.
.image column_fea2.png _ 600

Generated code using Tensorflow "Feature Column" should look like:
Generated code using TensorFlow "Feature Column" should look like:
.code feature_column_code.pysample

* Design: Feature Derivation
Expand Down
2 changes: 1 addition & 1 deletion doc/talk/20190620/sqlflow.slide
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ TensorFlow model inputs are dense tensors, which usually are high-dimensional ma
- Hash the name string into a 64-bit integer, then bucketing hashed value into 100 dimension columns.
.image ../201906/column_fea2.png _ 600

Generated code using Tensorflow "Feature Column" should look like:
Generated code using TensorFlow "Feature Column" should look like:
.code ../201906/feature_column_code.pysample

* Design: Feature Derivation
Expand Down
4 changes: 2 additions & 2 deletions docker/step/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# A Slim Tensorflow Step Image
# A Slim TensorFlow Step Image

This image is used when submitting Argo workflows to run Tensorflow/PAI Tensorflow jobs. To build this image, you should follow the below steps:
This image is used when submitting Argo workflows to run TensorFlow/PAI TensorFlow jobs. To build this image, you should follow the below steps:

1. Go to SQLFlow root directory
```bash
Expand Down
2 changes: 1 addition & 1 deletion go/codegen/alps/template_train.go
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ class SQLFlowEstimatorBuilder(EstimatorBuilder):

if __name__ == "__main__":
if tf_is_version2():
raise ValueError("ALPS must run with Tensorflow == 1.15.x")
raise ValueError("ALPS must run with TensorFlow == 1.15.x")

driver, dsn = "{{.DataSource}}".split("://")
user, passwd, endpoint, odps_project = db.parseMaxComputeDSN(dsn)
Expand Down
6 changes: 3 additions & 3 deletions go/codegen/pai/codegen.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ import (
)

const (
// ModelTypeTF is the mode type that trained by PAI Tensorflow.
// ModelTypeTF is the mode type that trained by PAI TensorFlow.
ModelTypeTF = iota
// ModelTypeXGBoost is the model type that use PAI Tensorflow to train XGBoost models.
// ModelTypeXGBoost is the model type that use PAI TensorFlow to train XGBoost models.
ModelTypeXGBoost
// ModelTypePAIML is the model type that trained by PAI machine learning algorithm toolkit
ModelTypePAIML
Expand Down Expand Up @@ -103,7 +103,7 @@ func genRequirements(isXGBoost bool) (string, error) {
return code.String(), nil
}

// Train generates a Python program a PAI command arguments to train a Tensorflow model.
// Train generates a Python program a PAI command arguments to train a TensorFlow model.
func Train(ir *ir.TrainStmt, session *pb.Session, tarball, paramsFile, modelName, ossModelPathToSave, ossModelPathToLoad, cwd string) (code, paiCmd, requirements string, e error) {
cc, e := GetClusterConfig(ir.Attributes)
if e != nil {
Expand Down
2 changes: 1 addition & 1 deletion go/codegen/tensorflow/docgen.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func DocGenInMarkdown() string {
return doc.String()
}

const docTemplateText = `# Tensorflow Parameters
const docTemplateText = `# TensorFlow Parameters

## TRAIN

Expand Down
2 changes: 1 addition & 1 deletion go/codegen/tensorflow/docgen_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@ import (

func TestDocGenInMarkdown(t *testing.T) {
a := assert.New(t)
a.True(strings.HasPrefix(DocGenInMarkdown(), `# Tensorflow`))
a.True(strings.HasPrefix(DocGenInMarkdown(), `# TensorFlow`))
}
2 changes: 1 addition & 1 deletion go/ir/derivation.go
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@ func deriveLabel(trainStmt *TrainStmt, fmMap FieldDescMap) error {
trainStmt.Label = &NumericColumn{
FieldDesc: fmMap[labelName],
}
// use shape [] if label shape is [1] for Tensorflow scalar label shape should be [].
// use shape [] if label shape is [1] for TensorFlow scalar label shape should be [].
shape := trainStmt.Label.GetFieldDesc()[0].Shape
if len(shape) == 1 && shape[0] == 1 {
trainStmt.Label.GetFieldDesc()[0].Shape = []int{}
Expand Down
4 changes: 2 additions & 2 deletions python/runtime/alps/train_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# limitations under the License.

# NOTE: ALPS supports tensorflow 1.15 currently, should run this example with
# Tensorflow 1.15.x installed.
# TensorFlow 1.15.x installed.

import os
import shutil
Expand Down Expand Up @@ -45,7 +45,7 @@ def _build(self, experiment, run_config):

if __name__ == "__main__":
if tf_is_version2():
raise ValueError("ALPS must run with Tensorflow == 1.15.x")
raise ValueError("ALPS must run with TensorFlow == 1.15.x")
odps_project = os.getenv("SQLFLOW_TEST_DB_MAXCOMPUTE_PROJECT")
odps_conf = OdpsConf(
accessid=os.getenv("SQLFLOW_TEST_DB_MAXCOMPUTE_AK"),
Expand Down
2 changes: 1 addition & 1 deletion python/runtime/feature/derivation.py
Original file line number Diff line number Diff line change
Expand Up @@ -582,7 +582,7 @@ def derive_label(label, fd_map):
assert label_field_desc is not None, \
"deriveLabel: LABEL COLUMN '%s' not found" % label_name

# use shape [] if label shape is [1] for Tensorflow scalar label
# use shape [] if label shape is [1] for TensorFlow scalar label
# shape should be [].
shape = label_field_desc.shape
if shape is None or (len(shape) == 1 and shape[0] == 1):
Expand Down
2 changes: 1 addition & 1 deletion python/runtime/pai/pai_distributed.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
import tensorflow as tf

# This module contain utilities for PAI distributed training.
# Note that currently PAI only support Tensorflow 1.x versions
# Note that currently PAI only support TensorFlow 1.x versions
# importing this module should make sure that currently installed
# tensorflow is version 1.x.

Expand Down
2 changes: 1 addition & 1 deletion python/runtime/pai/submitter.py
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,7 @@ def submit_pai_train(datasource, estimator_string, select, validation_select,
Like: odps://access_id:[email protected]/api?
curr_project=test_ci&scheme=http
estimator_string: string
Tensorflow estimator name, Keras class name, or XGBoost
TensorFlow estimator name, Keras class name, or XGBoost
select: string
The SQL statement for selecting data for train
validation_select: string
Expand Down
2 changes: 1 addition & 1 deletion python/runtime/pai/tensorflow/evaluate.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@

def evaluate(datasource, select, data_table, result_table, oss_model_path,
metrics):
"""PAI Tensorflow evaluate wrapper
"""PAI TensorFlow evaluate wrapper
This function do some preparation for the local evaluation, say,
download the model from OSS, extract metadata and so on.

Expand Down
2 changes: 1 addition & 1 deletion python/runtime/pai/tensorflow/predict.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

def predict(datasource, select, data_table, result_table, label_column,
oss_model_path):
"""PAI Tensorflow prediction wrapper
"""PAI TensorFlow prediction wrapper
This function do some preparation for the local prediction, say,
download the model from OSS, extract metadata and so on.

Expand Down
2 changes: 1 addition & 1 deletion python/runtime/pai/tensorflow/train.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from runtime.tensorflow.input_fn import get_dataset_fn
from runtime.tensorflow.set_log_level import set_log_level

# Disable Tensorflow INFO and WARNING logs
# Disable TensorFlow INFO and WARNING logs
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"


Expand Down
4 changes: 2 additions & 2 deletions python/runtime/tensorflow/explain.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@
init_model_with_feature_column

sns_colors = sns.color_palette('colorblind')
# Disable Tensorflow INFO and WARNING logs
# Disable TensorFlow INFO and WARNING logs
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Use non-interactive background
plt.switch_backend('agg')

# Disable Tensorflow INFO and WARNING logs
# Disable TensorFlow INFO and WARNING logs
if tf_is_version2():
import logging
tf.get_logger().setLevel(logging.ERROR)
Expand Down
4 changes: 2 additions & 2 deletions python/runtime/tensorflow/predict.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@
from runtime.tensorflow.keras_with_feature_column_input import \
init_model_with_feature_column

# Disable Tensorflow INFO and WARNING logs
# Disable TensorFlow INFO and WARNING logs
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Disable Tensorflow INFO and WARNING logs
# Disable TensorFlow INFO and WARNING logs
if tf_is_version2():
import logging
tf.get_logger().setLevel(logging.ERROR)
Expand Down
2 changes: 1 addition & 1 deletion python/runtime/tensorflow/train.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

from ..model_metadata import collect_model_metadata

# Disable Tensorflow INFO and WARNING logs
# Disable TensorFlow INFO and WARNING logs
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"


Expand Down