Skip to content

Commit 418b593

Browse files
committed
Add vector_concat() and vector_slice() functions
1 parent 7fbef83 commit 418b593

3 files changed

Lines changed: 666 additions & 0 deletions

File tree

README-vector-functions.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Vector Concat and Slice Functions for libSQL
2+
3+
This project adds two new vector functions to libSQL to maintain compatibility with Turso:
4+
5+
1. `vector_concat(X, Y)` - Concatenates two vectors of the same type.
6+
2. `vector_slice(X, start_idx, end_idx)` - Extracts a subvector from start_idx (inclusive) to end_idx (exclusive).
7+
8+
## Implementation
9+
10+
The implementation is based on the existing vector functions in libSQL. The new functions are added to the `vector.c` file in the `libsql-sqlite3/src` directory.
11+
12+
### `vector_concat(X, Y)`
13+
14+
This function concatenates two vectors of the same type. It performs the following steps:
15+
16+
1. Parse the two input vectors
17+
2. Check that both vectors are of the same type
18+
3. Allocate a new vector with dimensions equal to the sum of the dimensions of the input vectors
19+
4. Copy the data from both vectors into the new vector
20+
5. Return the new vector
21+
22+
### `vector_slice(X, start_idx, end_idx)`
23+
24+
This function extracts a slice of a vector from start_idx (inclusive) to end_idx (exclusive). It performs the following steps:
25+
26+
1. Parse the input vector
27+
2. Validate the start and end indices:
28+
- Both must be non-negative
29+
- start_idx must not be greater than end_idx
30+
- Both must be within the bounds of the vector
31+
3. Allocate a new vector with dimensions equal to end_idx - start_idx
32+
4. Copy the appropriate slice of data from the input vector to the new vector
33+
5. Return the new vector
34+
35+
Note: FLOAT1BIT vectors are not yet supported for the slice operation due to the complexity of bit-by-bit extraction.
36+
37+
## Usage
38+
39+
```sql
40+
-- Create a test table with a vector column
41+
CREATE TABLE test_vectors (
42+
id INTEGER PRIMARY KEY,
43+
vec VECTOR
44+
);
45+
46+
-- Insert some test vectors
47+
INSERT INTO test_vectors VALUES (1, vector32(1, 2, 3, 4, 5));
48+
INSERT INTO test_vectors VALUES (2, vector32(6, 7, 8, 9, 10));
49+
50+
-- Concatenate vectors
51+
SELECT vector_extract(vector_concat(vec, vector32(11, 12, 13))) FROM test_vectors WHERE id = 1;
52+
-- Returns: [1.0, 2.0, 3.0, 4.0, 5.0, 11.0, 12.0, 13.0]
53+
54+
-- Slice a vector
55+
SELECT vector_extract(vector_slice(vec, 1, 4)) FROM test_vectors WHERE id = 1;
56+
-- Returns: [2.0, 3.0, 4.0]
57+
```
58+
59+
## Testing
60+
61+
A test file `test_vector_functions.sql` is provided to verify the implementation.
62+
63+
## Building
64+
65+
The implementation is integrated directly into the libSQL SQLite fork. To build it, follow the standard libSQL build process.
66+
67+
## References
68+
69+
- Turso Implementation: https://github.com/tursodatabase/turso/pull/2336
70+
- libSQL Issue: https://github.com/tursodatabase/libsql/issues/2136

0 commit comments

Comments
 (0)