Skip to content

[Bug]: Higher versions of fastdup cannot handle situations within 10 images. #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zczczc11122 opened this issue Dec 5, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@zczczc11122
Copy link

zczczc11122 commented Dec 5, 2024

What happened?

import os 
os.environ['FASTDUP_PRODUCTION'] = '1' 
import fastdup 
fd = fastdup.create(work_dir='work_dir', input_dir='img_cache_folder') 
fd.run(ccthreshold=0.97, d=960, overwrite=True, run_mode=1, high_accuracy=True)

There are only 5 pictures in the img_cache_folder path

I have encountered an error when encountering a lower version of fastdup before, even adding the environment variable 'FASTDUP_PRODUCTION' cannot solve the problem.

Running the above code reports an error
[FATAL] This fastdup version is deprecated, please reinstall from pypi using pip install fastdup

So we upgraded the fastdup version to 2.13, but we encountered new problems with the higher version of fastdup. But when we run the above code to process data within 10 pictures, an error will be reported.
CRITICAL:vl.root.fastdup.pipeline.common.dataset_db_updater:"Error: Insufficient number of valid images in dataset (2). Minimum required images: 10". Please note that only the following image formats are supported: ['.png', '.jpg', '.jpeg', '.gif', '.giff', '.tif', '.tiff', '.heic', '.heif', '.bmp', '.webp', '.jfif', '.jpg2'], info: None, exception: None

Since we are an automated process, there may be situations where there are less than 10 pictures, and this will not cause an error in lower versions (such as version 2.2). Now I want to handle the situation where there are less than 10 pictures. How should I solve it? , because I did not see the documentation related to the 2.XX version here https://visual-layer.readme.io/docs/v02xx-api

What did you expect to see?

Can handle situations within 10 pictures

What version of fastdup were you runnning on?

2.13

What version of Python were you running on?

Python 3.9

Operating System

ubuntu 22.04 LTS

Reproduction steps

No response

Relevant log output

No response

Attach a screenshot [Optional]

No response

Contact Details [Optional]

No response

@zczczc11122 zczczc11122 added the bug Something isn't working label Dec 5, 2024
@dbickson
Copy link
Collaborator

dbickson commented Dec 5, 2024

Hi @zczczc11122 please try to define an environment variable

export MIN_NUM_OF_IMAGES=2

Let us know if this works.

@zczczc11122
Copy link
Author

Hi @zczczc11122 please try to define an environment variable

export MIN_NUM_OF_IMAGES=2

Let us know if this works.

We tried the solution you provided and this solved our problem, thanks for your reply

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants