Add blood vessel dataset processing script #184

yamengxi · 2020-10-09T13:13:22Z

No description provided.

codecov · 2020-10-09T13:23:07Z

Codecov Report

Merging #184 into master will decrease coverage by 0.11%.
The diff coverage is 77.77%.

@@            Coverage Diff             @@
##           master     #184      +/-   ##
==========================================
- Coverage   83.32%   83.20%   -0.12%     
==========================================
  Files          83       83              
  Lines        3921     3924       +3     
  Branches      617      619       +2     
==========================================
- Hits         3267     3265       -2     
- Misses        520      522       +2     
- Partials      134      137       +3

Flag	Coverage Δ
#unittests	`83.20% <77.77%> (-0.12%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmseg/datasets/custom.py	`85.88% <75.00%> (-0.99%)`	⬇️
mmseg/version.py	`58.33% <100.00%> (ø)`
mmseg/datasets/pipelines/transforms.py	`96.28% <0.00%> (-1.24%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f705071...3f35e25. Read the comment docs.

Junjun2016 · 2020-10-14T17:01:48Z

tools/convert_datasets/drive.py

+            osp.join(out_dir, 'images', 'validation',
+                     osp.splitext(img_name)[0] + '.jpg'), img)
+
+    now_dir = osp.join(tmp_dir, 'test', '1st_manual')


Now, there is no '1st_manual' in the test dataset.

Junjun2016 · 2020-10-14T17:02:15Z

docs/getting_started.md

+
+### DRIVE
+
+The training and validation set of DRIVE could be download from [here](https://drive.grand-challenge.org/). Before that, you should registrate an account.


Now, there is no '1st_manual' in the test dataset.

xvjiarui · 2020-10-15T09:24:20Z

docs/getting_started.md

+To convert STARE dataset to mmsegmentation format, you should run the following command:
+
+```shell
+python tools/convert_datasets/hrf.py ~/stare-images.tar ~/labels-ah.tar ~/labels-vk.tar


Should be stair.py?

Ah......yes, I will fix it.

xvjiarui · 2020-10-15T09:26:13Z

tools/convert_datasets/chase_db1.py

+                         osp.splitext(img_name)[0] + '.jpg'),
+                img[:, :, 0] // 128)


Add comments about //128.

xvjiarui · 2020-10-15T09:30:19Z

tools/convert_datasets/drive.py

+    mmcv.mkdir_or_exist(out_dir)
+    mmcv.mkdir_or_exist(osp.join(out_dir, 'images'))


These are not necessary.
The deepest level mkdir will automatically create them.

I know that, I write this to make the code easy to understand.

xvjiarui · 2020-10-15T09:31:22Z

docs/getting_started.md

+To convert HRF dataset to mmsegmentation format, you should run the following command:
+
+```shell
+python tools/convert_datasets/hrf.py ~/healthy.zip ~/healthy_manualsegm.zip ~/glaucoma.zip ~/glaucoma_manualsegm.zip ~/diabetic_retinopathy.zip ~/diabetic_retinopathy_manualsegm.zip


We may omit ~. We may use /path/to/healthy.zip instead.

xvjiarui · 2020-10-15T09:32:27Z

docs/getting_started.md

+
+### DRIVE
+
+The training and validation set of DRIVE could be download from [here](https://drive.grand-challenge.org/). Before that, you should registrate an account. Now, there is no '1st_manual' in the test dataset, if you need it, please search for it by yourself, we do not provide links to download.


Suggested change

The training and validation set of DRIVE could be download from [here](https://drive.grand-challenge.org/). Before that, you should registrate an account. Now, there is no '1st_manual' in the test dataset, if you need it, please search for it by yourself, we do not provide links to download.

The training and validation set of DRIVE could be download from [here](https://drive.grand-challenge.org/). Before that, you should register an account. Currently '1st_manual' is not provided officially.

xvjiarui · 2020-10-15T09:33:39Z

tools/convert_datasets/hrf.py

+                osp.join(out_dir, 'annotations', 'validation',
+                         osp.splitext(filename)[0] + '.jpg'),
+                img[:, :, 0] // 128)
+        shutil.rmtree(tmp_dir)


Add comments about 128.

xvjiarui · 2020-10-15T09:34:56Z

tools/convert_datasets/stare.py

+
+    for filename in sorted(os.listdir(osp.join(tmp_dir, 'files')))[10:]:
+        img = cv2.imread(osp.join(tmp_dir, 'files', filename))
+        cv2.imwrite(


For all cv2.imwrite, we may replace them with mmcv.imwrite, which could also automatically create a directory.

xvjiarui · 2020-10-15T09:40:09Z

tools/convert_datasets/chase_db1.py

+    mmcv.mkdir_or_exist(osp.join(out_dir, 'annotations', 'training'))
+    mmcv.mkdir_or_exist(osp.join(out_dir, 'annotations', 'validation'))
+
+    tmp_dir = osp.join(out_dir, 'tmp')


I suggest using system tmp instead of hard code 'tmp'. On the other hand, we may add an argument so that user could specify their own tmpdir.

What is the "system tmp"?

You may refer to the doc here.
You may also take a look at this function for reference.

xvjiarui · 2020-10-15T09:41:15Z

tools/convert_datasets/hrf.py

+        mmcv.mkdir_or_exist(tmp_dir)
+        zip_file = zipfile.ZipFile(now_path)
+        zip_file.extractall(tmp_dir)
+        for filename in sorted(os.listdir(tmp_dir))[:5]:


Direct truncate 5 here is not very safe. We may have some check and assert.

xvjiarui · 2020-10-15T09:41:41Z

tools/convert_datasets/hrf.py

+        zip_file = zipfile.ZipFile(now_path)
+        zip_file.extractall(tmp_dir)
+        for filename in sorted(os.listdir(tmp_dir))[:5]:
+            img = cv2.imread(osp.join(tmp_dir, filename))


mmcv.imread

xvjiarui · 2020-10-17T00:34:29Z

docs/getting_started.md

+
+The training and validation set of CHASE DB1 could be download from [here](https://staffnet.kingston.ac.uk/~ku15565/CHASE_DB1/assets/CHASEDB1.zip).
+
+To convert CHASE DB1 dataset to mmsegmentation format, you should run the following command:


Suggested change

To convert CHASE DB1 dataset to mmsegmentation format, you should run the following command:

To convert CHASE DB1 dataset to MMSegmentation format, you should run the following command:

xvjiarui · 2020-10-17T00:40:35Z

tools/convert_datasets/hrf.py

+            assert len(os.listdir(tmp_dir)) == HRF_LEN, \
+                'len(os.listdir(tmp_dir)) != {}'.format(HRF_LEN)
+
+            for filename in sorted(os.listdir(tmp_dir))[:5]:


Hard code in script is not ideal. We may add a variable at top like here

* Add blood vessel dataset processing script * Fix syntax error * Fix syntax error * Fix syntax error * Fix bugs * Fix bugs * Fix bugs * Use safe functions and expand more apis * Use safe functions and expand more apis * Fix hard code and verify dataset integrity

Co-authored-by: lizz <[email protected]>

Update README.md

* val * modify average_clips * support multi-batch * update * update docs * add changelog

yamengxi added 2 commits October 9, 2020 21:08

Add blood vessel dataset processing script

1529354

Fix syntax error

b79e332

yamengxi added 2 commits October 9, 2020 21:26

Fix syntax error

bcef73e

Fix syntax error

18fd2f4

Junjun2016 reviewed Oct 14, 2020

View reviewed changes

yamengxi added 2 commits October 15, 2020 11:21

Fix bugs

feb7e92

Fix bugs

fa564a9

xvjiarui reviewed Oct 15, 2020

View reviewed changes

Fix bugs

9f815ec

xvjiarui reviewed Oct 15, 2020

View reviewed changes

yamengxi added 2 commits October 15, 2020 19:26

Use safe functions and expand more apis

72a1a60

Use safe functions and expand more apis

109467a

xvjiarui reviewed Oct 17, 2020

View reviewed changes

Fix hard code and verify dataset integrity

3f35e25

xvjiarui approved these changes Oct 18, 2020

View reviewed changes

xvjiarui merged commit cb2e0d3 into open-mmlab:master Oct 18, 2020

wjkim81 pushed a commit to wjkim81/mmsegmentation that referenced this pull request Dec 3, 2023

Add ParametricMesh detector. (open-mmlab#184)

1a2b1d8

Co-authored-by: lizz <[email protected]>

michaelzhang-ai pushed a commit to michaelzhang-ai/mmsegmentation that referenced this pull request Mar 22, 2024

Merge pull request open-mmlab#184 from open-mmlab/patch-1

e2495d6

Update README.md

michaelzhang-ai pushed a commit to michaelzhang-ai/mmsegmentation that referenced this pull request Mar 22, 2024

Support multi-batch inference (open-mmlab#184)

ae4b581

* val * modify average_clips * support multi-batch * update * update docs * add changelog


		### DRIVE

		The training and validation set of DRIVE could be download from [here](https://drive.grand-challenge.org/). Before that, you should registrate an account.

		mmcv.mkdir_or_exist(out_dir)
		mmcv.mkdir_or_exist(osp.join(out_dir, 'images'))


		The training and validation set of CHASE DB1 could be download from [here](https://staffnet.kingston.ac.uk/~ku15565/CHASE_DB1/assets/CHASEDB1.zip).

		To convert CHASE DB1 dataset to mmsegmentation format, you should run the following command:

Add blood vessel dataset processing script #184

Add blood vessel dataset processing script #184

Uh oh!

Conversation

yamengxi commented Oct 9, 2020

Uh oh!

codecov bot commented Oct 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xvjiarui Oct 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xvjiarui Oct 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Oct 9, 2020 •

edited

Loading

xvjiarui Oct 15, 2020 •

edited

Loading

xvjiarui Oct 17, 2020 •

edited

Loading