Thank you Rui, my target is: API that can be learned and used under 120 seconds. I don't think more of this is needed, for the basic usage. Then there could be other stuff, but it must be incremental. Easy things must be easy.
And so forth. However there are nice things you can enable, if you want. For instance VADD has the [REDUCE] option that implements random projection in order to reduce the dimensionality of your embeddings. Less memory, and it is faster. And so forth.
Will work on it during xmas, it's quite fun.
It implements on-insertion normalization and int8 quantization. The algorithm to insert nodes inside the HNSW attempts to never leave isolated nodes and to get good quality links and so forth. More hacking needed.
yes loving this! please PLEASE be the hero we need in unfucking the unholy mess of misdirection and useless abstraction that is storing some vectors somewhere for similarity search right now
