Knowledgebase

What Is Ollama?

Ollama is a free, open-source tool that lets you run and manage large language models like LLaMA, Mistral, Gemma, and others on your local machine or server. It simplifies downloading, running, and interacting with models using a single command-line interface.

Step 1: Update All the System Packages

Update all the system packages to make sure your Ubuntu server is up to date.

sudo apt update && sudo apt upgrade -y

Step 2: Install Ollama

Ollama provides an official installation script for Linux.

curl -fsSL https://ollama.com/install.sh | sh

This will download and install the Ollama CLI along with its dependencies.

Once installed, you can verify it by running:

ollama --version

Step 3: Pull a Model (e.g., LLaMA3 or Mistral)

You can browse available models here:

https://ollama.com/library

To run LLaMA3 8B, use:

ollama run llama3

Or, for a lighter model like Mistral:

ollama run mistral

On the first run, Ollama will download the model automatically.

Step 4: Start Chatting

After the model loads, you’ll be dropped into a local chat prompt. Get ChatGPT-style answers by simply entering your questions; there’s no need for API keys, waiting, or internet calls.

Step 5: Use Ollama via API (Optional)

Ollama comes with a local API that is compatible with the OpenAI Chat API. It’s great for integrating with UIs like:

• Open WebUI
• Text Generation WebUI
• LangChain agents
• Custom web apps

Start the Ollama server (it runs automatically), then access it via:

http://localhost:11434

Example cURL request:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt": "Explain black holes like I’m 5"
}'

Optimization Tips

• Use smaller models (like Mistral or Gemma) for lower resource usage.
• Offload to a GPU if your VPS supports it.
• Use --num-ctx to increase the context window when needed.
• Monitor memory usage with htop or glances.

Conclusion

With tools like Ollama, self-hosting your own ChatGPT-like assistant is now a reality, even on mid-range VPSs. With this configuration, you have complete control over creating a chatbot, private assistant, or creative tool.