Skip to content

load 'AG_NEWS’ failed with torchtext dataset #30557

Closed
@zhixul

Description

@zhixul

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

  1. Fail to run the tutorial of text classification using torchtext.

https://stackoverflow.com/questions/57988897/overflowerror-python-int-too-large-to-convert-to-c-long-torchtext-datasets-text
In line 130:
csv.field_size_limit(sys.maxsize) -- Changed this
csv.field_size_limit(maxInt) -- To this

Expected behavior

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

PyTorch version: 1.2.0
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Microsoft Windows 10 Home
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce RTX 2070
Nvidia driver version: 419.67
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.17.4
[pip] numpydoc==0.9.1
[pip] torch==1.2.0
[pip] torchtext==0.4.0
[conda] blas 1.0 mkl
[conda] mkl 2019.4 245
[conda] mkl-service 2.3.0 py37hb782905_0
[conda] mkl_fft 1.0.15 py37h14836fe_0
[conda] mkl_random 1.1.0 py37h675688f_0
[conda] pytorch 1.2.0 py3.7_cuda100_cudnn7_1 pytorch
[conda] torchtext 0.4.0 pyhb384e40_1 pytorch

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions