Ollama Python Library Tutorial: Build AI Apps Locally in 2025 | Python Developer Guide

gcptutorials.com GenAI

Why Ollama Python?

Ollama has emerged as the go-to solution for running large language models (LLMs) locally, and its Python library (version 0.4.7 as of 2025) simplifies AI integration for developers. This tutorial will guide you through:

Local model deployment without cloud dependencies
Real-time text generation with streaming
Advanced features like context management and temperature control

1. Installation & Setup

System Requirements

Ensure your system meets these minimum specs:

Python 3.8+
8GB RAM (for 7B parameter models)
Ollama service running locally

# Install Python library
pip install ollama

# Download base model
ollama pull llama3.2

2. Basic Chat Workflow

Start with a simple Q&A implementation:

import ollama

response = ollama.chat(
    model='llama3.2',
    messages=[
        {'role': 'system', 'content': 'You are a technical documentation expert'},
        {'role': 'user', 'content': 'Explain gradient descent in simple terms'}
    ]
)
print(response['message']['content'])

Streaming Responses

Handle large outputs efficiently:

stream = ollama.chat(
    model='mistral',
    messages=[{'role': 'user', 'content': 'Describe quantum entanglement'}],
    stream=True
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

3. Advanced Configurations

Context Management

Maintain conversation history:

chat_history = []

def ask(message):
    chat_history.append({'role': 'user', 'content': message})
    response = ollama.chat(model='llama3.2', messages=chat_history)
    chat_history.append(response['message'])
    return response

Parameter Tuning

Control model behavior:

response = ollama.chat(
    model='llama3.2',
    messages=[...],
    options={
        'temperature': 0.7,  # 0-1 scale
        'num_ctx': 4096,     # Context window
        'repeat_penalty': 1.2
    }
)

4. Production-Grade Implementation

Error Handling

Robust implementation pattern:

try:
    response = ollama.chat(model='unknown-model', messages=[...])
except ollama.ResponseError as e:
    if e.status_code == 404:
        print("Model not found - pulling from registry...")
        ollama.pull('unknown-model')
    else:
        raise

Async Operations

For high-performance applications:

import asyncio
from ollama import AsyncClient

async def main():
    client = AsyncClient()
    response = await client.chat(
        model='llama3.2',
        messages=[{'role': 'user', 'content': 'Explain blockchain'}]
    )
    print(response['message']['content'])

asyncio.run(main())

5. Best Practices

Use temperature=0 for code generation tasks
Leverage stream=True for responses >100 tokens
Regularly update models with ollama pull

Conclusion

The Ollama Python library enables developers to harness cutting-edge AI while maintaining full data control. With its simple API and local execution model, it's ideal for: