Skip to content

Add computer-vision slug #2090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 15, 2020
Merged

Add computer-vision slug #2090

merged 2 commits into from
Feb 15, 2020

Conversation

vadixidav
Copy link
Contributor

Right now the closest thing in the list to computer vision is science::robotics. Computer vision algorithms are not always used for robotics. This slug also allows computer vision crates to be found easier instead of being lost among the other robotics crates.

Right now the closest thing in the list to computer vision is `science::robotics`. Computer vision algorithms are not always used for robotics. This slug also allows computer vision crates to be found easier instead of being lost among the other robotics crates.
@rust-highfive
Copy link

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @smarnach (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

Copy link
Contributor

@smarnach smarnach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our current categories mostly seem a lot broader than "computer vision", so I'm not sure this deserves a category of its own. Crates can also be browsed by keyword, e.g.

We don't really seem to have a benchmark to decide what should constitute a category, so I'll consult with the team and get back to you.

@@ -80,6 +80,12 @@ description = """
Algorithms for making data smaller.\
"""

[computer-vision]
name = "Computer Vision"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent with the other category names, the "v" in "vision" should be lower case.

@carols10cents
Copy link
Member

@vadixidav do you know how many crates would fit into this category today?

@vadixidav
Copy link
Contributor Author

vadixidav commented Jan 7, 2020

@carols10cents Some crates:

Ones I know of because I have used them or someone in rust-cv used them

Ones that I found on crates.io just now that look fleshed out

Unfinished or empty crates that probably would have used the computer-vision slug

I am also very actively working on adding as many computer vision algorithms as possible, so this is a growing number. I don't know the amount of crates under other slugs, but computer vision is pretty broad. I have not included some crates that are related to computer vision (or were made for computer vision), but have other uses (such as https://crates.io/crates/hnsw).

@vadixidav
Copy link
Contributor Author

One alternative could be to put the computer-vision slug under the science one. I felt that science wasn't the right place for it, but robotics is also under science, despite that some of the crates there were never intended for or related to science (including most of the computer vision crates, which are also in science::robotics).

@smarnach
Copy link
Contributor

I'm not convinced this deserves a separate category yet. We already have a few rather specialized ones like "Text editors", but the vast majority of categories is more general. I'd be reluctant to add another quite narrow one. I'll leave this open for a while to see who else wants to chime in.

@carols10cents
Copy link
Member

What do you think about a top-level "machine learning" category (and remove the mention of "machine learning" from the description of the science category) with a "computer vision" subcategory?

@vadixidav
Copy link
Contributor Author

@carols10cents So far almost of the crates we have made would not fall under machine learning. Most of them are related to solving or iteratively improving camera or 3d point positions and camera parameters based on points detected and matched with other crates. For instance, most of the functionality in OpenCV is not related to machine learning. Additionally, OpenMVG, which is where many of our ports are sourced from, doesn't utilize any machine learning.

Although machine learning and computer vision have started to overlap greatly in the last several years, I still wouldn't say it falls under machine learning. I feel like computer vision crates that relate to machine learning are more likely just to be considered "machine learning" crates anyways. Perhaps my view is skewed though, as I am not a machine learning specialist, so I am more focused on the other parts of computer vision. We should probably get the opinion of a second person, at least.

@vadixidav
Copy link
Contributor Author

So, it doesn't seem there is much activity here. I think what we should do is agree upon what slug would be a good fit for different computer vision crates.

Some are related to image processing. This might involve, for instance, the Canny edge detector, denoising, gaussian blur/rescale, and other image operations. The closest category for these kinds of crates seems to be the graphics slug, but that doesn't seem appropriate considering the description: "Crates for graphics libraries and applications, including raster and vector graphics primitives such as geometry, curves, and color." It doesn't seem to be related to image processing, so perhaps there should be an image-processing slug. Technically, there is a multimedia tag with the description: "Crates that provide audio, video, and image processing or rendering engines." I don't think that this description intends to include things such as the canny edge detector or other image analysis and processing routines used in computer vision.

Some are related to photogrammetry & multi-view geometry. This will involve crates such as akaze. These currently are best suited to go in science::robotics, because they are sometimes used in SLAM. These descriptors are sometimes used in robot navigation via SLAM, but not always. Faster descriptors (ORB, AKAZE less-so, definitely not SIFT) tend to be used for robotics, while the more robust ones (such as SIFT) are rarely used in robotics, but they are used in SfM (https://en.wikipedia.org/wiki/Structure_from_motion). Crates that are relevant to SfM don't really have a category, but they are used to do things like inspections, geographical surveys, or creating 3d models to preserve historical sites or buildings. SfM itself falls under the concept of computer vision, but it is basically summed up as reconstructing 3d information from 2d images.

There are also crates that do operations on images to prepare them for some of these processes. This would be, for instance, image undistortion. For instance, here is an OpenCV tutorial on this topic in OpenCV with Python. I don't know which category these go into, but the graphics or multimedia slugs seem to be the closest matches.

You also have depth map estimation. This is related to robotics in some cases, so it could go in that slug, but sometimes it is not. Python OpenCV tutorial on this topic is here. I think, at present, this would best go under science::robotics.

Finally, there are the most well-known of all computer vision tasks: convolutional neural networks. They have made a large impact on modern society, but I would argue that since traditional machine learning frameworks (Tensorflow, PyTorch, etc) include these networks as built-in primitives, there is little reason to create a category specifically for them.

I have covered some of the largest components of computer vision here. I believe that while we may not create a computer-vision slug, we might be able to break down the different categories that we see here:

  • Some things seem to be involved in processing imagery, such as the imageproc crate. These crates would probably be best served by having a separate category for image-processing.
  • Some things work with turning 2d information into 3d information. These crates can probably continue to go into both science and science::robotics for the time being.
  • Some things are common to all the above, mostly having to do with camera models, epipolar geometry, focal length, distortion, etc. Perhaps these things can also continue to exist in science and science::robotics, but if there was a category such as science::optics, that would work pretty well for these kinds of things. I guess I am thinking that perhaps the core of "computer vision" might be able to live in a broader category involving cameras or optics, but I don't know what the name of said category would be.

Does anyone have any other thoughts on this? I think we should create a direction for which categories things like this should go, particularly as Rust becomes more relevant in this field.

@jtgeibel
Copy link
Member

@vadixidav I think you make a good case for adding this at the top level, and not under a new machine-learning category. I've added to the agenda for Friday to bring up with the team (since we discussed previously but missed this angle).

@vadixidav
Copy link
Contributor Author

@jtgeibel That is good to hear. It is difficult to find places for many of these crates. I appreciate the time spent discussing this issue in meetings.

@jtgeibel
Copy link
Member

We discussed merging this yesterday, and this is good to go as-is. I should be able to deploy this in the next hour or 2.

@bors r+

@bors
Copy link
Contributor

bors commented Feb 15, 2020

📌 Commit b25daa3 has been approved by jtgeibel

@bors
Copy link
Contributor

bors commented Feb 15, 2020

⌛ Testing commit b25daa3 with merge 4550dc3...

bors added a commit that referenced this pull request Feb 15, 2020
Add computer-vision slug

Right now the closest thing in the list to computer vision is `science::robotics`. Computer vision algorithms are not always used for robotics. This slug also allows computer vision crates to be found easier instead of being lost among the other robotics crates.
@bors
Copy link
Contributor

bors commented Feb 15, 2020

☀️ Test successful - checks-travis
Approved by: jtgeibel
Pushing 4550dc3 to master...

@bors bors merged commit b25daa3 into rust-lang:master Feb 15, 2020
@jtgeibel
Copy link
Member

Deployed. This category should now be available to new publishes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants