Contributing¶

Guidelines for contributing to Barrel VectorDB.

Development Setup¶

Prerequisites¶

Erlang/OTP 26 or later
rebar3
Python 3.8+ (for embedding providers)

Clone and Build¶

git clone https://gitlab.enki.io/barrel-db/barrel_vectordb.git
cd barrel_vectordb
rebar3 compile

Run Tests¶

# Unit tests (no external dependencies)
rebar3 eunit

# All tests including integration
rebar3 ct

Testing¶

Unit Tests¶

Unit tests use mocking and don't require external dependencies:

rebar3 eunit

Integration Tests¶

Integration tests verify real embedding providers. Setup required backends first:

# Setup for local provider
pip install sentence-transformers

# Setup for Ollama provider
ollama serve &
ollama pull nomic-embed-text

# Setup for OpenAI provider
export OPENAI_API_KEY=sk-...

# Run integration tests
rebar3 eunit --module=barrel_vectordb_integration_tests

Tests automatically skip if their required backend is unavailable.

Benchmarks¶

# Run benchmark suite
rebar3 as bench compile && rebar3 as bench eunit --module=barrel_vectordb_bench

# Backend comparison
./scripts/run_backend_bench.sh --quick

Code Style¶

Follow standard Erlang conventions
Use edoc for documentation
Keep functions focused and small
Add tests for new functionality

Module Structure¶

-module(barrel_vectordb_example).

%% API
-export([public_function/1]).

%% Internal exports
-export_type([my_type/0]).

-type my_type() :: term().

%% @doc Public function documentation.
-spec public_function(Arg) -> Result when
    Arg :: term(),
    Result :: ok | {error, term()}.
public_function(Arg) ->
    internal_function(Arg).

%% Internal functions

internal_function(Arg) ->
    ok.

Pull Requests¶

Fork the repository
Create a feature branch
Make your changes
Add tests
Run the test suite
Submit a pull request

Commit Messages¶

Use clear, descriptive commit messages:

Add support for new embedding provider

- Add barrel_vectordb_embed_newprovider.erl
- Update embed module to support new provider
- Add unit tests

Architecture¶

Core Components¶

barrel_vectordb.erl: Main API module
barrel_vectordb_store.erl: Store process (gen_batch_server)
barrel_vectordb_hnsw.erl: HNSW index implementation
barrel_vectordb_embed.erl: Embedding provider abstraction

Storage¶

RocksDB with column families
Vectors stored with 8-bit quantization
HNSW index checkpoints for fast restart

Clustering¶

barrel_vectordb_mesh.erl: Cluster coordinator
barrel_vectordb_ra_sm.erl: Ra state machine
barrel_vectordb_shard_router.erl: Request routing
barrel_vectordb_scatter.erl: Scatter-gather for search

Reporting Issues¶

Use the GitLab issue tracker:

Report Issues

Include:

Erlang/OTP version
barrel_vectordb version
Steps to reproduce
Expected vs actual behavior
Relevant logs

License¶

Contributions are licensed under Apache-2.0.