-

 

Ollama: Local Model Deployment Guide



Ollama

Ollama is an open-source platform  for running and managing large language models (LLMs) locally. It provides a unified interface to download, install, and serve a variety of models without requiring cloud access. Under the hood, Ollama handles model caching, quantization formats, and hardware optimizations so you can focus on building applications rather than wrestling with deployment details.

It’s commonly used to power offline or on-premises AI services where data privacy, latency, or cost constraints make cloud APIs impractical. Developers interact with Ollama through a simple HTTP or gRPC endpoint, sending prompt payloads and receiving streaming or batch completions. By abstracting away the complexities of model hosting, Ollama lets you seamlessly swap between different model families, experiment with compression techniques, and integrate LLM inference into Python scripts, microservices, or containerized pipelines.



Step 1: Install Ollama

Windows

  1. Go to Ollama Downloads

  2. Download the Windows installer .exe

  3. Run the installer and follow prompts

  4. Open PowerShell and verify:
     ollama --version

 Linux

Run this in your terminal:

curl -fsSL https://get.ollama.com | sh

Then verify:

ollama --version

Download Models Using Ollama

 Download qwen2.5:3b (Chat Model)

```

ollama pull qwen2.5:3b

```

This fetches the model weights and configuration into your local Ollama cache.

Download mxbai-embed-large:335m (Embedding Model)

```

ollama pull mxbai-embed-large:335m

```

Start the Ollama Server Locally

Once installed, start the server:

```

ollama serve

```

This launches a local API that handles model loading, chat, and embedding requests.

You can keep this terminal open or run Ollama as a background service.



Model Storage Locations

  • Windows: C:\Users\<YourUser>\.ollama\models\
  • Linux: ~/.ollama/models/

Comments

Post a Comment

Popular posts from this blog

How an AI Agent Works Without a Framework

-
How an AI Agent Works Without a Framework: A Practical Guide AI Agents are complicated. They rely on a combination of models, tools, and messages to understand input, perform tasks, and respond intelligently. Frameworks like LangChain and LangGraph help simplify this process by organizing these parts into reusable workflows. But what happens when you build an agent without the framework? This post walks through a minimal example of an AI agent built from scratch—no orchestration layer, no abstractions. Just raw components working together. By reverse-engineering this agent, you’ll gain a deeper understanding of how frameworks work under the hood. You’re not just learning LangChain or LangGraph—you’re mastering the core mechanics that make agentic systems possible: how tools are registered, how messages are passed, and how models interact with external functions. For the full code Here -> Core Components of an Agent Agent Step-by-Step Breakdown 1. Defining a Model ChatOllama ChatOlla...
Read more

Linear Regression: One Idea, Three Perspectives

-
Image
Linear Regression: One Idea, Three Perspectives Linear regression looks simple on the surface — draw a line through data — but underneath that simplicity is a surprisingly elegant structure. What most people don’t realize is that linear regression is actually one idea expressed in three different mathematical languages : Kaggle Notebook   GitHub Repo Optimization form — learn the line by minimizing a loss function Geometric form — compute the line as a projection Algebraic form — compute the line using means, sums, covariance, and variance Each form tells the same story from a different angle. When you see all three together, the whole picture snaps into place. This post walks through each form using the same dataset and the same goal: find the best‑fit line that predicts y from x. 1. Optimization Form: Learning the Line by Minimizing Loss The optimization view is the one used in machine learning. You start with random parameters, measure how wrong the model is, and...
Read more
-
Image
  Setting Up a Free Virtual Security Lab Building a cybersecurity lab on a low-powered laptop might seem challenging, but with the right setup and optimizations, you can still create an effective environment for security testing and learning. This guide walks you through setting up VMware Workstation Pro , configuring your virtual machines (VMs), and ensuring a secure testing network. For this setup, I decided to start with VMware Workstation Pro on my Windows machine. The GUI-driven interface makes it easier to navigate and understand virtualization concepts. Luckily, Workstation Pro was available for free, making it a great option.. To  download the Windows version from the VMware Workstation Pro website . I had to  click on the blue download Fusion or Workstation button.  At the top of the new page click login   Select option to register.  Once registered, go to My Downloads.  The blue ‘Free Software Downloads available HERE’. I downloaded the Wo...
Read more