update

Author: hycis

.gitignore

@@ -1,8 +1,11 @@
 *pyc
 .DS_Store
+<<<<<<< HEAD
 doctrees/
 .buildinfo
 .remote-sync.json
 *tensorboard*
 .coverage.*
 __pycache__/
+=======
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5

.travis.yml

@@ -9,17 +9,29 @@ install:
   - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
       pip install --only-binary=numpy,scipy numpy nose scipy pytest sklearn;
       pip install tensorflow;
+<<<<<<< HEAD
       pip install git+https://github.com/hycis/TensorGraph.git@master;
+=======
+      pip install git+https://github.com/hycis/TensorGraphX.git@master;
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
     fi
 
   - if [[ "$TRAVIS_PYTHON_VERSION" == "3.5" ]]; then
       pip3 install --only-binary=numpy,scipy numpy nose scipy pytest sklearn;
       pip3 install tensorflow;
+<<<<<<< HEAD
       pip3 install git+https://github.com/hycis/TensorGraph.git@master;
     fi
 
 script:
   - echo "TensorGraph Testing.."
+=======
+      pip3 install git+https://github.com/hycis/TensorGraphX.git@master;
+    fi
+
+script:
+  - echo "TensorGraphX Testing.."
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
   - if [[ "$TRAVIS_PYTHON_VERSION" == "2.7" ]]; then
       python -m pytest test;
     fi

LICENCE

@@ -1,4 +1,8 @@
+<<<<<<< HEAD
 Copyright 2015 The TensorGraph Authors.  All rights reserved.
+=======
+Copyright 2015 The TensorGraphX Authors.  All rights reserved.
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
                                  Apache License
                            Version 2.0, January 2004
@@ -188,7 +192,11 @@ Copyright 2015 The TensorGraph Authors.  All rights reserved.
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
+<<<<<<< HEAD
    Copyright 2015, The TensorGraph Authors.
+=======
+   Copyright 2015, The TensorGraphX Authors.
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

MANIFEST.in

@@ -1,2 +1,6 @@
 include README.md LICENCE
+<<<<<<< HEAD
 recursive-include tensorgraph *.py
+=======
+recursive-include tensorgraphx *.py
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5

README.md

@@ -1,3 +1,4 @@
+<<<<<<< HEAD
 `master` [![Build Status](http://54.222.242.222:1010/buildStatus/icon?job=TensorGraph/master)](http://54.222.242.222:1010/job/TensorGraph/master)
 `develop` [![Build Status](http://54.222.242.222:1010/buildStatus/icon?job=TensorGraph/develop)](http://54.222.242.222:1010/job/TensorGraph/develop)
 
@@ -8,18 +9,34 @@ TensorGraph is a simple, lean, and clean framework on TensorFlow for building an
 As deep learning becomes more and more common and the architectures becoming more
 and more complicated, it seems that we need some easy to use framework to quickly
 build these models and that's what TensorGraph is designed for. It's a very simple
+=======
+[![Build Status](https://travis-ci.org/hycis/TensorGraphX.svg?branch=master)](https://travis-ci.org/hycis/TensorGraphX)
+
+# TensorGraphX - Simplicity is Beauty
+TensorGraphX is a simple, lean, and clean framework on TensorFlow for building any imaginable models.
+
+As deep learning becomes more and more common and the architectures becoming more
+and more complicated, it seems that we need some easy to use framework to quickly
+build these models and that's what TensorGraphX is designed for. It's a very simple
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 framework that adds a very thin layer above tensorflow. It is for more advanced
 users who want to have more control and flexibility over his model building and
 who wants efficiency at the same time.
 
 -----
+<<<<<<< HEAD
 ## Target Audience
 TensorGraph is targeted more at intermediate to advance users who feel keras or
+=======
+### Target Audience
+TensorGraphX is targeted more at intermediate to advance users who feel keras or
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 other packages is having too much restrictions and too much black box on model
 building, and someone who don't want to rewrite the standard layers in tensorflow
 constantly. Also for enterprise users who want to share deep learning models
 easily between teams.
 
+<<<<<<< HEAD
 ## Documentation
 
 You can check out the documentation [https://skymed.ai/pages/AI-Platform/TensorGraph/](https://skymed.ai/pages/AI-Platform/TensorGraph/)
@@ -39,16 +56,46 @@ git clone https://skymed.ai/AI-Platform/TensorGraph.git
 export PYTHONPATH=/path/to/TensorGraph:$PYTHONPATH
 ```
 in order for the install to persist via export `PYTHONPATH`. Add `PYTHONPATH=/path/to/TensorGraph:$PYTHONPATH` to your `.bashrc` for linux or
+=======
+-----
+### Install
+
+First you need to install [tensorflow](https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html)
+
+To install tensorgraphx simply do via pip
+```bash
+sudo pip install tensorgraphx
+```
+or for bleeding edge version do
+```bash
+sudo pip install --upgrade git+https://github.com/hycis/TensorGraphX.git@master
+```
+or simply clone and add to `PYTHONPATH`.
+```bash
+git clone https://github.com/hycis/TensorGraphX.git
+export PYTHONPATH=/path/to/TensorGraphX:$PYTHONPATH
+```
+in order for the install to persist via export `PYTHONPATH`. Add `PYTHONPATH=/path/to/TensorGraphX:$PYTHONPATH` to your `.bashrc` for linux or
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 `.bash_profile` for mac. While this method works, you will have to ensure that
 all the dependencies in [setup.py](setup.py) are installed.
 
 -----
+<<<<<<< HEAD
 ## Everything in TensorGraph is about Layers
 Everything in TensorGraph is about layers. A model such as VGG or Resnet can be a layer. An identity block from Resnet or a dense block from Densenet can be a layer as well. Building models in TensorGraph is same as building a toy with lego. For example you can create a new model (layer) by subclass the `BaseModel` layer and use `DenseBlock` layer inside your `ModelA` layer.
 
 ```python
 from tensorgraph.layers import DenseBlock, BaseModel, Flatten, Linear, Softmax
 import tensorgraph as tg
+=======
+### Everything in TensorGraphX is about Layers
+Everything in TensorGraphX is about layers. A model such as VGG or Resnet can be a layer. An identity block from Resnet or a dense block from Densenet can be a layer as well. Building models in TensorGraphX is same as building a toy with lego. For example you can create a new model (layer) by subclass the `BaseModel` layer and use `DenseBlock` layer inside your `ModelA` layer.
+
+```python
+from tensorgraphx.layers import DenseBlock, BaseModel, Flatten, Linear, Softmax
+import tensorgraphx as tg
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
 class ModelA(BaseModel):
     @BaseModel.init_name_scope
@@ -86,6 +133,7 @@ y_train = modelb.train_fprop(X_ph)
 y_test = modelb.test_fprop(X_ph)
 ```
 
+<<<<<<< HEAD
 checkout some well known models in TensorGraph
 1. [VGG16 code](tensorgraph/layers/backbones.py#L37) and [VGG19 code](tensorgraph/layers/backbones.py#L125) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556)
 2. [DenseNet code](tensorgraph/layers/backbones.py#L477) - [Densely Connected Convolutional Networks](https://arxiv.org/abs/1608.06993)
@@ -323,26 +371,96 @@ graph are two separate steps. By splitting them into two separate steps, we ensu
 the flexibility of building our computational graph without the worry of accidental
 reinitialization of the `Variables`.
 We defined three types of nodes
+=======
+checkout some well known models in TensorGraphX
+1. [VGG16 code](tensorgraphx/layers/backbones.py#L37) and [VGG19 code](tensorgraphx/layers/backbones.py#L125) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556)
+2. [DenseNet code](tensorgraphx/layers/backbones.py#L477) - [Densely Connected Convolutional Networks](https://arxiv.org/abs/1608.06993)
+3. [ResNet code](tensorgraphx/layers/backbones.py#L225) - [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)
+4. [Unet code](tensorgraphx/layers/backbones.py#L531) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://arxiv.org/abs/1505.04597)
+
+-----
+### TensorGraphX on Multiple GPUS
+To use tensorgraphx on multiple gpus, you can easily integrate it with [horovod](https://github.com/uber/horovod).
+
+```python
+import horovod.tensorflow as hvd
+from tensorflow.python.framework import ops
+import tensorflow as tf
+hvd.init()
+
+# tensorgraphx model derived previously
+modelb = ModelB()
+X_ph = tf.placeholder()
+y_ph = tf.placeholder()
+y_train = modelb.train_fprop(X_ph)
+y_test = modelb.test_fprop(X_ph)
+
+train_cost = mse(y_train, y_ph)
+test_cost = mse(y_test, y_ph)
+
+opt = tf.train.RMSPropOptimizer(0.001)
+opt = hvd.DistributedOptimizer(opt)
+
+# required for BatchNormalization layer
+update_ops = ops.get_collection(ops.GraphKeys.UPDATE_OPS)
+with ops.control_dependencies(update_ops):
+    train_op = opt.minimize(train_cost)
+
+init_op = tf.group(tf.global_variables_initializer(),
+                   tf.local_variables_initializer())
+bcast = hvd.broadcast_global_variables(0)
+
+# Pin GPU to be used to process local rank (one GPU per process)
+config = tf.ConfigProto()
+config.gpu_options.allow_growth = True
+config.gpu_options.visible_device_list = str(hvd.local_rank())
+
+with tf.Session(graph=graph, config=config) as sess:
+    sess.run(init_op)
+    bcast.run()
+
+    # training model
+    for epoch in range(100):
+        for X,y in train_data:
+            _, loss_train = sess.run([train_op, train_cost], feed_dict={X_ph:X, y_ph:y})
+```
+
+for a full example on [tensorgraphx on horovod](./examples/multi_gpus_horovod.py)
+
+-----
+### How TensorGraphX Works?
+In TensorGraphX, we defined three types of nodes
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
 1. StartNode : for inputs to the graph
 2. HiddenNode : for putting sequential layers inside
 3. EndNode : for getting outputs from the model
 
+<<<<<<< HEAD
 We put all the sequential layers into a `HiddenNode`, `HiddenNode` can be connected
 to another `HiddenNode` or `StartNode`, the nodes are connected together to form
 an architecture. The graph always starts with `StartNode` and ends with `EndNode`.
 Once we have defined an architecture, we can use the `Graph` object to connect the
 path we want in the architecture, there can be multiple StartNodes (s1, s2, etc)
 and multiple EndNodes (e1, e2, etc), we can define which path we want in the
 entire architecture, example to link from `s2` to `e1`. The `StartNode` is where you place
+=======
+We put all the sequential layers into a `HiddenNode`, and connect the hidden nodes
+together to build the architecture that you want. The graph always
+starts with `StartNode` and ends with `EndNode`. The `StartNode` is where you place
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 your starting point, it can be a `placeholder`, a symbolic output from another graph,
 or data output from `tfrecords`. `EndNode` is where you want to get an output from
 the graph, where the output can be used to calculate loss or simply just a peek at the
 outputs at that particular layer. Below shows an
 [example](examples/example.py) of building a tensor graph.
 
 -----
+<<<<<<< HEAD
 ## Graph Example
+=======
+### Graph Example
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
 <img src="draw/graph.png" height="250">
 
@@ -362,19 +480,29 @@ Then define the `HiddenNode` for putting the sequential layers in each `HiddenNo
 ```python
 h1 = HiddenNode(prev=[s1, s2],
                 input_merge_mode=Concat(),
+<<<<<<< HEAD
                 layers=[Linear(y2_dim), RELU()])
 h2 = HiddenNode(prev=[s2],
                 layers=[Linear(y2_dim), RELU()])
 h3 = HiddenNode(prev=[h1, h2],
                 input_merge_mode=Sum(),
                 layers=[Linear(y1_dim), RELU()])
+=======
+                layers=[Linear(y1_dim+y2_dim, y2_dim), RELU()])
+h2 = HiddenNode(prev=[s2],
+                layers=[Linear(y2_dim, y2_dim), RELU()])
+h3 = HiddenNode(prev=[h1, h2],
+                input_merge_mode=Sum(),
+                layers=[Linear(y2_dim, y1_dim), RELU()])
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 ```
 Then define the `EndNode`. `EndNode` is used to back-trace the graph to connect
 the nodes together.
 ```python
 e1 = EndNode(prev=[h3])
 e2 = EndNode(prev=[h2])
 ```
+<<<<<<< HEAD
 Finally build the graph by putting `StartNodes` and `EndNodes` into `Graph`, we
 can choose to use the entire architecture by using all the `StartNodes` and `EndNodes`
 and run the forward propagation to get symbolic output from train mode. The number
@@ -390,13 +518,26 @@ graph = Graph(start=[s2], end=[e1])
 o1, = graph.train_fprop()
 ```
 
+=======
+Finally build the graph by putting `StartNodes` and `EndNodes` into `Graph`
+```python
+graph = Graph(start=[s1, s2], end=[e1, e2])
+```
+Run train forward propagation to get symbolic output from train mode. The number
+of outputs from `graph.train_fprop` is the same as the number of `EndNodes` put
+into `Graph`
+```python
+o1, o2 = graph.train_fprop()
+```
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 Finally build an optimizer to optimize the objective function
 ```python
 o1_mse = tf.reduce_mean((y1 - o1)**2)
 o2_mse = tf.reduce_mean((y2 - o2)**2)
 mse = o1_mse + o2_mse
 optimizer = tf.train.AdamOptimizer(learning_rate).minimize(mse)
 ```
+<<<<<<< HEAD
 
 -----
 ## TensorGraph on Multiple GPUS
@@ -449,6 +590,10 @@ for a full example on [tensorgraph on horovod](./examples/multi_gpus_horovod.py)
 
 -----
 ## Hierachical Softmax Example
+=======
+-----
+### Hierachical Softmax Example
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 Below is another example for building a more powerful [hierachical softmax](examples/hierachical_softmax.py)
 whereby the lower hierachical softmax layer can be conditioned on all the upper
 hierachical softmax layers.
@@ -472,9 +617,15 @@ y3_ph = tf.placeholder('float32', [None, component_dim])
 # define the graph model structure
 start = StartNode(input_vars=[x_ph])
 
+<<<<<<< HEAD
 h1 = HiddenNode(prev=[start], layers=[Linear(component_dim), Softmax()])
 h2 = HiddenNode(prev=[h1], layers=[Linear(component_dim), Softmax()])
 h3 = HiddenNode(prev=[h2], layers=[Linear(component_dim), Softmax()])
+=======
+h1 = HiddenNode(prev=[start], layers=[Linear(x_dim, component_dim), Softmax()])
+h2 = HiddenNode(prev=[h1], layers=[Linear(component_dim, component_dim), Softmax()])
+h3 = HiddenNode(prev=[h2], layers=[Linear(component_dim, component_dim), Softmax()])
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
 
 e1 = EndNode(prev=[h1], input_merge_mode=Sum())
@@ -493,9 +644,15 @@ optimizer = tf.train.AdamOptimizer(learning_rate).minimize(mse)
 ```
 
 -----
+<<<<<<< HEAD
 ## Transfer Learning Example
 Below is an example on transfer learning with bi-modality inputs and merge at
 the middle layer with shared representation, in fact, TensorGraph can be used
+=======
+### Transfer Learning Example
+Below is an example on transfer learning with bi-modality inputs and merge at
+the middle layer with shared representation, in fact, TensorGraphX can be used
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 to build any number of modalities for transfer learning.
 
 <img src="draw/transferlearn.png" height="250">
@@ -518,10 +675,17 @@ y_ph = tf.placeholder('float32', [None, y_dim])
 s1 = StartNode(input_vars=[x1_ph])
 s2 = StartNode(input_vars=[x2_ph])
 
+<<<<<<< HEAD
 h1 = HiddenNode(prev=[s1], layers=[Linear(shared_dim), RELU()])
 h2 = HiddenNode(prev=[s2], layers=[Linear(shared_dim), RELU()])
 h3 = HiddenNode(prev=[h1,h2], input_merge_mode=Sum(),
                 layers=[Linear(y_dim), Softmax()])
+=======
+h1 = HiddenNode(prev=[s1], layers=[Linear(x1_dim, shared_dim), RELU()])
+h2 = HiddenNode(prev=[s2], layers=[Linear(x2_dim, shared_dim), RELU()])
+h3 = HiddenNode(prev=[h1,h2], input_merge_mode=Sum(),
+                layers=[Linear(shared_dim, y_dim), Softmax()])
+>>>>>>> e55a706e1467da7b7c54b6d04055aba847f5a2b5
 
 e1 = EndNode(prev=[h3])
 

docs/index.md

@@ -0,0 +1 @@
+