Simplify copy kernel #28428
Closed
Simplify copy kernel #28428
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Oct 22, 2019
zasdfgbnm
added a commit
that referenced
this pull request
Oct 22, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: 8754f6a
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
[ghstack-poisoned]
zasdfgbnm
added a commit
that referenced
this pull request
Oct 22, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: b764aff
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
zasdfgbnm
added a commit
that referenced
this pull request
Oct 23, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: a8356a7
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
zasdfgbnm
added a commit
that referenced
this pull request
Oct 24, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: 9ebae7b
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
zasdfgbnm
added a commit
that referenced
this pull request
Oct 25, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: 54b21a3
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
|
This PR is actually a good test for non-copying type promotion: if someone else breaks the type promotion again by running a |
zasdfgbnm
added a commit
that referenced
this pull request
Oct 26, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: 3fb1c46
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
zasdfgbnm
added a commit
that referenced
this pull request
Oct 26, 2019
Using the new type promotion and dynamic casting added to
`TensorIterator`, the copy kernels could be greatly simplified.
**Script:**
```python
import torch
import timeit
import pandas
import itertools
from tqdm import tqdm
import math
print(torch.__version__)
print()
_10M = 10 * 1024 ** 2
d = {}
for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(),
repeat=2)):
if from_ not in d:
d[from_] = {}
a = torch.zeros(_10M, dtype=from_)
min_ = math.inf
for i in range(100):
start = timeit.default_timer()
a.to(to)
end = timeit.default_timer()
elapsed = end - start
if elapsed < min_:
min_ = elapsed
d[from_][to] = int(elapsed * 1000 * 1000)
pandas.DataFrame(d)
```
**Before:**
**After:**
ghstack-source-id: 1269ecc
Pull Request resolved: #28428
Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. For benchmark, see #28352 (comment) [ghstack-poisoned]
zasdfgbnm
commented
Oct 26, 2019
| // this is done intentionally done after build because copy has a "promotion" | ||
| // rule that always "promote" to target dtype. | ||
| iter.promote_common_dtype(); | ||
| AT_DISPATCH_ALL_TYPES_AND3(kHalf, kBool, kBFloat16, iter.dtype(0), "copy_", [&] { |
There was a problem hiding this comment.
I see a PR trying to enable BFloat16 for CUDA:
https://github.com/pytorch/pytorch/pull/27259/files#diff-6684cb81a1865b7d52d9f2f1789cd0ceR68-R70
So I include it in this PR also, so that I can benchmark
|
There is a small regression on GPU, so I am not sure if this should merge. |
VitalyFedyunin
approved these changes
Oct 27, 2019
zdevito
pushed a commit
to zdevito/ATen
that referenced
this pull request
Oct 28, 2019
Summary: Pull Request resolved: pytorch/pytorch#28428 Using the new type promotion and dynamic casting added to `TensorIterator`, the copy kernels could be greatly simplified. Benchmark on CUDA: ```python import torch import timeit import pandas import itertools from tqdm.notebook import tqdm import math print(torch.__version__) print() _10M = 10 * 1024 ** 2 d = {} for from_, to in tqdm(itertools.product(torch.testing.get_all_dtypes(), repeat=2)): if from_ not in d: d[from_] = {} a = torch.empty(_10M, dtype=from_, device='cuda') min_ = math.inf for i in range(100): torch.cuda.synchronize() start = timeit.default_timer() a.to(to) torch.cuda.synchronize() end = timeit.default_timer() elapsed = end - start if elapsed < min_: min_ = elapsed d[from_][to] = int(min_ * 1000 * 1000) pandas.DataFrame(d) ``` original:  new:  Test Plan: Imported from OSS Differential Revision: D18170995 Pulled By: ezyang fbshipit-source-id: 461b53641813dc6cfa872a094ae917e750c60759
|
Did this break Windows build? |
|
@vishwakftw Seems that we need to add |
|
I'm unlanding the stack |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Using the new type promotion and dynamic casting added to
TensorIterator, the copy kernels could be greatly simplified.Benchmark on CUDA:
original:
new:
Differential Revision: D18170995