Skip to content

Build_disk_index from Iterator #671

@J0SZ

Description

@J0SZ

The documentation of build_disk_index states:

This function will construct a DiskANN disk index. Disk indices are ideal for very large datasets that are too large to fit in memory.

However, it only accepts

  1. Numpy array: Which cannot by definition fit into memory
  2. Path to vector bin file: In order to create this file, I once again need Numpy array: 1)

In other words, this function defeats its own purpose since in order to use it, I need to put all my data that are not supposed to fit into memory... into memory...

I would suggest to make it accept an Iterator[np.Array] or correct the documentation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions