What's Changed
Breaking Changes 🛠
- fix!: delta index fragment bitmaps contained previous index coverage by @wjones127 in #3377
New Features 🎉
- feat: enable all datafusion functions by @westonpace in #3381
- feat: support float16/float64 for multivector by @BubbleCal in #3387
- feat: add drop_index by @westonpace in #3382
- feat: add an all null column as a metadata-only operation by @albertlockett in #3391
- feat: finish up variable-length encodings in the full-zip path by @westonpace in #3344
- feat: add testing of string/binary to 2.1 full-zip encoding and fix bugs by @westonpace in #3418
- feat: auto-migrate old index metadata by @wjones127 in #3428
- feat: allow replacement of entire datafile when the schema lines up correctly by @chebbyChefNEQ in #3408
Bug Fixes 🐛
- fix: json schema serializes field metadata by @albertlockett in #3379
- fix: flat FTS would return all unindexed rows by @BubbleCal in #3386
- fix: updating schema/field metadata now retains fragments by @albertlockett in #3384
- fix: full text search index may be corrupted after remapping by @BubbleCal in #3388
- fix: handle the possibility that serialize_expressions returns a memoryview by @westonpace in #3396
- fix: merge_insert with subcols sometimes outputs unexpected nulls by @wjones127 in #3407
- fix: ensure that 'block_size' parameter is properly propagated in the ObjectStore by @vjc578db in #3403
- fix(rust): loosen bytemuck pin by @wjones127 in #3413
- fix: don't compare metadata in merge insert to detect if partial schema by @westonpace in #3412
- fix: support fp16 type in SQ by @chebbyChefNEQ in #3417
- fix: move IO tasks off of CPU runtime in merge_insert by @wjones127 in #3420
- fix: filter out null values when sampling for index training by @wjones127 in #3404
- fix: handle null vectors in flat search by @wjones127 in #3422
- fix: avoid divide-by-zero when training an index with a large dimension by @westonpace in #3426
- fix: bump openssl for CVE by @chebbyChefNEQ in #3431
Performance Improvements 🚀
- perf: avoid re-alloc on assigning PQ by @BubbleCal in #3399
- perf: skip shuffling if there is only 1 partition by @BubbleCal in #3405
New Contributors
Full Changelog: v0.22.0...v0.23.0