The convergence of relational databases and generative AI is reshaping data architecture. By integrating Oracle AI Vector Search, Ollama, and Mistral, developers can build intelligent applications that leverage structured data, semantic search, and local LLMs—all within a unified environment.

My earlier article on Oracle AI Vector explains the basics and shows how to get a docker container.  This container is used in the example below.

What is Ollama? And Why Does It Matter?

Ollama is an open-source tool for running small language models (SLMs) locally, eliminating cloud dependencies. If you have enough computing power and space, you can also run large language models (LLMs) locally. Key features:

  • Local execution: Deploy models like Mistral directly on-premises or in private clouds.

  • Privacy: Sensitive data never leaves your infrastructure.

  • Cost efficiency: Avoid recurring API costs for high-volume workloads.

  • Flexibility: Supports macOS, Linux, and Windows, with GPU acceleration for performance.

Just install Ollama and try it out. How to download and install Ollama can be found here. Once Ollama is installed, adding Mistral (or any supported model) is straightforward. Check available models by running in the shell: ollama pull mistral.

ollama pull mistral

This pulls the Mistral model onto your machine. Alternatively, if a specific variant or quantization is available, you might use: ollama pull mistral-nemo

Why put RDBMS + Vector + LLM in one stack?

The integration of RDBMS, vector technologies and LLMs into one unified architecture makes sense for a variety of reasons, e.g.

Benefit What it means for data architecture
Unified governance Row-level security, auditing, and backups apply to both business facts and the embeddings derived from them.
Lower latency RAG The retrieval step (vector similarity) executes inside the SQL engine; only the final context window is sent to the LLM.
ACID joins on semantics You can SELECT * FROM orders o JOIN faq f ON VECTOR_DISTANCE(f.embedding, o.summary) < 0.12—mixing structured attributes with semantic proximity.
Operational simplicity Fewer moving parts than a polyglot stack with a standalone vector DB plus an external LLM gateway.
Cost & sovereignty Run open-weight models locally; no token fees, no data egress, full GDPR control.

A lot of use cases around RAG (Retrieval augmented generation) and multi-agent applications are feasible.

How to – steps to combine the tools

The demo assumes:

  • Oracle Database 23ai running in a Docker container.

  • Ollama running on the host OS with the Mistral model already pulled.

  • Docker Desktop so the container can reach the host at host.docker.internal

The demo shows how to combine the tools.

First set database permissions.

GRANT db_developer_role, create credential TO userX;
ALTER USER userX QUOTA UNLIMITED ON users;

-- allow network calls and manage ACLs
GRANT EXECUTE ON sys.dbms_network_acl_admin TO userX;
GRANT EXECUTE ON sys.utl_http TO userX;
Add the ACL entry so the user may open outbound sockets.
BEGIN
  sys.DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE(
    host => '*',
    ace => xs$ace_type(privilege_list => xs$name_list('connect'),
                       principal_name => 'userX',
                       principal_type => xs_acl.ptype_db));
END;
/

Connect to the container and verify ollama by calling the command below. The entry host.docker.internal works only on Docker Desktop. On Linux you need the bridge IP 172.17.0.1.
curl -X POST http://host.docker.internal:11434/api/generate -d ‘{“model”:”mistral”,”prompt”:”ping”,”stream”:false}’

Ollama docker test

You should get a JSON payload beginning with “Pong! I’m here to help…” – proving the network path is open.

And finally, call Mistral from PL/SQL via DBMS_VECTOR package. Params define the connection to ollama. The connection goes from Oracle running in a container to ollama running locally on my laptop.

host.docker.internal works only on Docker Desktop. On Linux you need the bridge IP 172.17.0.1.

DECLARE
  params  CLOB := '{
    "provider": "ollama",
    "host"    : "local",
    "url"     : "http://host.docker.internal:11434/api/generate",
    "model"   : "mistral"
  }';
  answer  CLOB;
BEGIN
  answer := DBMS_VECTOR.UTL_TO_GENERATE_TEXT(
              'What is Mistral LLM in one sentence?',
              JSON(params));
  DBMS_OUTPUT.put_line(answer);
END;
/
The output should be similar to the image below.
Oracle AI, Ollama and mistral

The example shows how to combine Oracle AI, Ollama and a SLM. Other steps to RAG (Retrieval Augmented Generation) or multi-agent systems could be

  • Store embeddings in a VECTOR column in a database table.
  • Generate the embedding by using an embedding model that can also be hosted locally and served by Ollama (UTL_TO_EMBEDDINGS in the same PL/SQL package).
  • Semantic search and filter clause to get data.

Summary

Oracle’s AI Vector Search turns the database into a semantic data store; Ollama turns your laptop or edge node into a private LLM gateway with the choice of many LLMs; Mistral provides a fast, unencumbered model that slots right in. Together they let data architects keep governance, performance, and Gen-AI innovation inside a single, converged layer—no extra pipelines, and no sensitive data shipped to the cloud.