Skip to content

Commit 9d5b3c4

Browse files
tango4jtango4j
authored
Update INTRO.md
1 parent 2559c5d commit 9d5b3c4

File tree

1 file changed

+116
-10
lines changed

1 file changed

+116
-10
lines changed

‎INTRO.md

+116-10
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,128 @@ This repository aims for comparative analysis of TensorFlow vs PyTorch, for thos
44

55
The whole content was written in Ipython Notebook then converted into MarkDown. Ipython Notebooks in main directory cotains the same content.
66

7-
### TABLE OF CONTENTS
7+
## TABLE OF CONTENTS
88

9-
[**01. Tensor**](https://github.com/tango4j/tensorflow-vs-pytorch#01-tensor)
9+
[**01. Tensor**](https://github.com/tango4j/tensorflow-vs-pytorch#01-tensor)
1010

11-
[1. The Concept of Tensor](https://github.com/tango4j/tensorflow-vs-pytorch#01-tensor)
12-
[2. Tensor Numpy Conversion](https://github.com/tango4j/tensorflow-vs-pytorch#2-tensor-numpy-conversion)
13-
[3. Indentifying The Dimension](https://github.com/tango4j/tensorflow-vs-pytorch#3-indentifying-the-dimension)
14-
[4. Shaping the Tensor Variables](https://github.com/tango4j/tensorflow-vs-pytorch#4-shaping-the-tensor-variables)
11+
> [**1. The Concept of Tensor**](https://github.com/tango4j/tensorflow-vs-pytorch#01-tensor)
12+
>[[TensorFlow] - Tensors and special type of tensors](https://github.com/tango4j/tensorflow-vs-pytorch#tensorflow-tensors-and-special-type-of-tensors)
1513
16-
[**02. Variable**](https://github.com/tango4j/tensorflow-vs-pytorch#02-variables-)
14+
>> [(1) What is TensorFlow "Tensor" ?](https://github.com/tango4j/tensorflow-vs-pytorch#1-what-is-tensorflow-tensor-)
15+
>> [(2) Special type Tensors](https://github.com/tango4j/tensorflow-vs-pytorch#2-special-type-tensors)
16+
>> [(3) Convention for Tensor dimension](https://github.com/tango4j/tensorflow-vs-pytorch#3-convention-for-tensor-dimension)
17+
>> [(4) Numpy to tf.Variable](https://github.com/tango4j/tensorflow-vs-pytorch#4-numpy-to-tfvariable)
18+
>> [(5) Direct declaration](https://github.com/tango4j/tensorflow-vs-pytorch#5-direct-declaration)
19+
>> [(6) Difference Between Special Tensors and tf.Variable (TensorFlow)](https://github.com/tango4j/tensorflow-vs-pytorch#difference-between-special-tensors-and-tfvariable-tensorflow)
1720
18-
[1. Creating a Variable](https://github.com/tango4j/tensorflow-vs-pytorch#1-creating-a-variable)
21+
>[[PyTorch] - Torch tensor and torch.Variable](https://github.com/tango4j/tensorflow-vs-pytorch#pytorch-torch-tensor-and-torchvariable)
22+
>[Basics for PyTorch Tensors.](https://github.com/tango4j/tensorflow-vs-pytorch#basics-for-pytorch-tensors)
1923
20-
[**03. Computation of data**](https://github.com/tango4j/tensorflow-vs-pytorch#03-computaion-of-data)
24+
>>[(1) PyTorch Tensor](https://github.com/tango4j/tensorflow-vs-pytorch#1-pytorch-tensor)
25+
>>[(2) PyTorch's dynamic graph feature](https://github.com/tango4j/tensorflow-vs-pytorch#2-pytorchs-dynamic-graph-feature)
26+
>>[(3) What does torch.autograd.Variable contain?](https://github.com/tango4j/tensorflow-vs-pytorch#3-what-does-torchautogradvariable-contain)
27+
>>[(4) Backpropagation with dynamic graph](https://github.com/tango4j/tensorflow-vs-pytorch#4-backpropagation-with-dynamic-graph)
2128
22-
[1. Dynamic Graph and Static Graph](https://github.com/tango4j/tensorflow-vs-pytorch#1-dynamic-graph-and-static-graph)
29+
>[**2. Tensor Numpy Conversion**](https://github.com/tango4j/tensorflow-vs-pytorch#2-tensor-numpy-conversion)
30+
31+
>[[TensorFlow] tf.convert_to_tensor or .eval()](https://github.com/tango4j/tensorflow-vs-pytorch#tensorflow-tfconvert_to_tensor-or-eval)
32+
>>[Numpy to tf.Tensor](https://github.com/tango4j/tensorflow-vs-pytorch#numpy-to-tftensor)
33+
>>[tf.Tensor to Numpy](https://github.com/tango4j/tensorflow-vs-pytorch#tftensor-to-numpy)
34+
35+
>[[PyTorch] .numpy() or torch.from_numpy()](https://github.com/tango4j/tensorflow-vs-pytorch#pytorch-numpy-or-torchfrom_numpy)
36+
>>[Numpy to torch.Tensor](https://github.com/tango4j/tensorflow-vs-pytorch#numpy-to-torchtensor)
37+
>>[torch.Tensor to Numpy](https://github.com/tango4j/tensorflow-vs-pytorch#torchtensor-to-numpy)
38+
39+
> [**3. Indentifying The Dimension**](https://github.com/tango4j/tensorflow-vs-pytorch#3-indentifying-the-dimension)
40+
41+
> [[TensorFlow] .shape or tf.rank() followed by .eval()](https://github.com/tango4j/tensorflow-vs-pytorch#tensorflow-shape-or-tfrank-followed-by-eval)
42+
>> [.shape variable in TensorFlow](https://github.com/tango4j/tensorflow-vs-pytorch#shape-variable-in-tensorflow)
43+
>> [tf.rank function](https://github.com/tango4j/tensorflow-vs-pytorch#tfrank-function)
44+
45+
>[[PyTorch] .shape or .size()](https://github.com/tango4j/tensorflow-vs-pytorch#pytorch-shape-or-size)
46+
>>[Automatically Displayed PyTorch Tensor Dimension](https://github.com/tango4j/tensorflow-vs-pytorch#automatically-displayed-pytorch-tensor-dimension)
47+
>>[.shape variable in PyTorch](https://github.com/tango4j/tensorflow-vs-pytorch#shape-variable-in-pytorch)
48+
49+
>[**4. Shaping the Tensor Variables**](https://github.com/tango4j/tensorflow-vs-pytorch#4-shaping-the-tensor-variables)
50+
51+
>[[TensorFlow] tf.reshape](https://github.com/tango4j/tensorflow-vs-pytorch#tensorflow-tfreshape)
52+
>>[Reshape tf.Tensor with tf.reshape](https://github.com/tango4j/tensorflow-vs-pytorch#reshape-tftensor-with-tfreshape)
53+
>>[Handling the Rest of Dimension with "-1"](https://github.com/tango4j/tensorflow-vs-pytorch#handling-the-rest-of-dimension-with--1-1)
54+
55+
>[[PyTorch].view() function](https://github.com/tango4j/tensorflow-vs-pytorch#pytorch-view-function)
56+
>> [Reshape PyTorch Tensor with .view()](https://github.com/tango4j/tensorflow-vs-pytorch#reshape-pytorch-tensor-with-view)
57+
>> [Handling the Rest of Dimension with "-1"](https://github.com/tango4j/tensorflow-vs-pytorch#handling-the-rest-of-dimension-with--1-1)
58+
>> [Copy the Dimension of other PyTorch Tensor .view_as()](https://github.com/tango4j/tensorflow-vs-pytorch#copy-the-dimension-of-other-pytorch-tensor-view_as)
59+
60+
> [**5. Shaping the Tensor Variables**](https://github.com/tango4j/tensorflow-vs-pytorch#4-shaping-the-tensor-variables)
61+
62+
> [**6. Datatype Conversion**](https://github.com/tango4j/tensorflow-vs-pytorch#5-datatype-conversion)
63+
64+
> [**7. Printing Variables**](https://github.com/tango4j/tensorflow-vs-pytorch#6-printing-variables)
65+
66+
[**02. Variable**](https://github.com/tango4j/tensorflow-vs-pytorch#02-variables-)
67+
>[**1. Creating a Variable**](https://github.com/tango4j/tensorflow-vs-pytorch#1-creating-a-variable)
68+
>[[TensorFlow]](https://github.com/tango4j/tensorflow-vs-pytorch#tensorflow)
69+
>>[Method 1: tf.get_variable()](https://github.com/tango4j/tensorflow-vs-pytorch#method-1-tfget_variable)
70+
>>[Method 2: tf.Variable](https://github.com/tango4j/tensorflow-vs-pytorch#method-2-tfvariable)
71+
72+
>[[PyTorch] Creating PyTorch Variable - torch.autograd.Variable](https://github.com/tango4j/tensorflow-vs-pytorch#pytorch-creating-pytorch-variable---torchautogradvariable)
73+
>>[The concept of Pytorch Variable](https://github.com/tango4j/tensorflow-vs-pytorch#the-concept-of-pytorch-variable)
74+
75+
[**03. Computation of data**](https://github.com/tango4j/tensorflow-vs-pytorch#03-computaion-of-data)
76+
>[1. Tensorflow VS PyTorch Comparison](https://github.com/tango4j/tensorflow-vs-pytorch#1-tensorflow-vs-pytorch-comparison)
77+
>[2. Dynamic Graph and Static Graph](https://github.com/tango4j/tensorflow-vs-pytorch#1-dynamic-graph-and-static-graph)
78+
79+
- There are a few distinct differences between Tensorflow and Pytorch when it comes to data compuation.
80+
81+
| | TensorFlow | PyTorch |
82+
|---------------|-----------------|----------------|
83+
| Framework | Define-and-run | Define-by-run |
84+
| Graph | Static | Dynamic |
85+
| Debug | Non-native debugger (tfdbg) |pdb(ipdb) Python debugger|
86+
87+
**How "Graph" is defined in each framework?**
88+
89+
#**TensorFlow:**
90+
91+
- Static graph.
92+
93+
- Once define a computational graph and excute the same graph repeatedly.
94+
95+
- Pros:
96+
97+
(1) Optimizes the graph upfront and makes better distributed computation.
98+
99+
(2) Repeated computation does not cause additional computational cost.
100+
101+
102+
- Cons:
103+
104+
(1) Difficult to perform different computation for each data point.
105+
106+
(2) The structure becomes more complicated and harder to debug than dynamic graph.
107+
108+
109+
#**PyTorch:**
110+
111+
- Dynamic graph.
112+
113+
- Does not define a graph in advance. Every forward pass makes a new computational graph.
114+
115+
- Pros:
116+
117+
(1) Debugging is easier than static graph.
118+
119+
(2) Keep the whole structure concise and intuitive.
120+
121+
(3) For each data point and time different computation can be performed.
122+
123+
124+
- Cons:
125+
126+
(1) Repetitive computation can lead to slower computation speed.
127+
128+
(2) Difficult to distribute the work load in the beginning of training.
23129

24130

25131
- There are a few distinct differences between Tensorflow and Pytorch when it comes to data compuation.

0 commit comments

Comments
 (0)