Motivation

Supposedly, running the llms through api keys is faster and cheaper, then using chatgpt in the normal web interface

Use cases

  • Have multiple projects separated
  • Have a possibility to run it on multiple (single to hundreds) files
  • Have access to web search
  • Be able to back it up
  • Have an ability to fast search through the previous questions

Options

llm

offered from a chatGPT as a first choice based on it being simple terminal like interface not to get overwhelmed

Resource

It looks it have quite extensive docs link Video included in the docs:

Features and modes how it works

  • SQL lite database, where all the interaction is stored, so it can be easily searched

Installation (page 5, section 2.1.1)

Create a separate python environment for installation

for more details on venv see link here.

cd ~/.venvs/
python3 -m venv llm
source llm/bin/activate

Use pip to install llm

pip install llm

Upgrade llm

pip install -U llm

Manage API keys (page 7, section 2.1.6)

llm keys set openai # will open prompt to copy in
llm keys  # to list set keys
llm keys # show path to keys

NOTE: the API key in the json file is not encrypted!!! Would be good to make it encrypted in the future, set the limit for spend on the tokens for some un-harmful level

Configugptration (page 8, section 2.1.7)

Models

List models

llm models

OpenAI Chat: gpt-4o (aliases: 4o) OpenAI Chat: chatgpt-4o-latest (aliases: chatgpt-4o) OpenAI Chat: gpt-4o-mini (aliases: 4o-mini) OpenAI Chat: gpt-4o-audio-preview OpenAI Chat: gpt-4o-audio-preview-2024-12-17 OpenAI Chat: gpt-4o-audio-preview-2024-10-01 OpenAI Chat: gpt-4o-mini-audio-preview OpenAI Chat: gpt-4o-mini-audio-preview-2024-12-17 OpenAI Chat: gpt-4.1 (aliases: 4.1) OpenAI Chat: gpt-4.1-mini (aliases: 4.1-mini) OpenAI Chat: gpt-4.1-nano (aliases: 4.1-nano) OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt) OpenAI Chat: gpt-3.5-turbo-16k (aliases: chatgpt-16k, 3.5-16k) OpenAI Chat: gpt-4 (aliases: 4, gpt4) OpenAI Chat: gpt-4-32k (aliases: 4-32k) OpenAI Chat: gpt-4-1106-preview OpenAI Chat: gpt-4-0125-preview OpenAI Chat: gpt-4-turbo-2024-04-09 OpenAI Chat: gpt-4-turbo (aliases: gpt-4-turbo-preview, 4-turbo, 4t) OpenAI Chat: gpt-4.5-preview-2025-02-27 OpenAI Chat: gpt-4.5-preview (aliases: gpt-4.5) OpenAI Chat: o1 OpenAI Chat: o1-2024-12-17 OpenAI Chat: o1-preview OpenAI Chat: o1-mini OpenAI Chat: o3-mini OpenAI Chat: o3 OpenAI Chat: o4-mini OpenAI Chat: gpt-5 OpenAI Chat: gpt-5-mini OpenAI Chat: gpt-5-nano OpenAI Chat: gpt-5-2025-08-07 OpenAI Chat: gpt-5-mini-2025-08-07 OpenAI Chat: gpt-5-nano-2025-08-07 OpenAI Chat: gpt-5.1 OpenAI Chat: gpt-5.1-chat-latest OpenAI Chat: gpt-5.2 OpenAI Chat: gpt-5.2-chat-latest OpenAI Completion: gpt-3.5-turbo-instruct (aliases: 3.5-instruct, chatgpt-instruct) Default: gpt-4o-mini

See and set the default model (page 9, 2.2.1)

To see the currently used default model

llm models default

To change the default model for a current session

export LLM_MODEL=gpt-4.1-mini
llm 'Ten names for cheesecakes' # Uses gpt-4.1-mini

To use a different model for given query:

llm 'Ten names for cheesecakes' -m gpt-4o

Config directory location

Configuration directory is automatically created in ~/.config/io.datasette.llm/

The variable LLM_USER_PATH can be used to store this path as an environment variable

export LLM_USER_PATH=<absolute-path-to-home>/.config/io.datasette.llm/

Logging

Is not super intuitive overall

llm logs status     #display logging status
llm logs off        #switch off logging
llm logs on         #switch on logging

Navigating logs - most recent

llm logs -n N

gives you the last N logs but it gives you everything, so in case you want to see just the dates you need to use grep/sed afterwards

llm logs -n 5 | grep -A 6 'id:' | sed -e /Model:/d -e /^$/d | less

gives you the date and id of the last N logs, with a first couple lines of a prompt, deletes empty lines and model to make it more concise

Navigating logs - search

llm logs -q 'cheescake' -l -n 3

Searches for the cheescake in the last 3 logs -n 3, putting the latest -l first

Usage (page 9, section 2.2)

To chat

llm chat -m gpt-4.1

Explain given code in a file

cat myscript.py | llm 'explain this code'

System prompts

llm -s 'Limit your output to 5 lines' --save brief5

restricts the output to 5 lines, can be used by:

llm "explain the use of poisson distribution" -t brief5

List the system prompts (templates)

llm templates

template can be used just one (but can contain more information, model etc.), cannot be combined when prompting

Fragments

Fragments are pieces of text which are used to surround the prompt.
These fragments are pieces of text (or other stuff, but not a written text in “”).
YOu can use more fragments at the same time specifying them with an -f tag NOTE: Looks they can be also a bigger documents, so here I do not understand the attachment vs fragment.

Save fragments:

llm fragments set <fragment-alias> <fragment-file>

List fragments:

llm fragments

Use fragments

llm "prompt" -f fragment1 -f fragment2

Use attachments

llm "extract text" -a image1.jpg -a image2.jpg

looks the attachments follow the symbolic links on ubuntu 24.04

cat image.jpg | llm "describe this image" -a -

System prompts

llm 'SQL to calculate total sales by month' \
--system 'You are an exaggerated sentient cheesecake that knows SQL and talks about cheesecake a lot'

Setting up the project folders

1) Put the LLM data store inside your project

Use the environment variable that tells llm where its “user dir” is.

Common pattern:

# from your project root
export LLM_USER_DIR="$PWD/.llm"

Now when you run llm ..., it will create/use:

  • a SQLite database in that directory (conversations, logs, etc.)
  • your project-local fragments/templates/etc. (depending on what you use)

To make it automatic when you cd into the project, put it in a .envrc (direnv) or your shell config.

Example with direnv:

# in project root
echo 'export LLM_USER_DIR="$PWD/.llm"' > .envrc
direnv allow

Now cd into the folder ⇒ llm uses .llm/ for that project.

2) Keep different conversations in different folders

You have two main approaches:

Give each folder its own LLM_USER_DIR.

Example:

# Folder A
cd ~/work/projectA
export LLM_USER_DIR="$PWD/.llm"

# Folder B
cd ~/work/projectB
export LLM_USER_DIR="$PWD/.llm"

Each folder gets its own .llm/ and therefore its own conversation database.

How does it “know” to append to the conversation in that folder?
Because “the conversation history” is stored in the SQLite DB inside that LLM_USER_DIR. When you’re in that folder (and LLM_USER_DIR points there), you’re using that folder’s DB, so you’re naturally continuing the history that’s already in it.

3) Make it automatic per folder

If you want “whatever folder I’m in becomes the conversation namespace”, you can do:

  • direnv per folder (best UX), or
  • a shell function that sets LLM_USER_DIR to something like "$PWD/.llm" before invoking llm.

Quick sanity check / questions

To give you the most accurate commands, tell me:

  1. What OS/shell are you using? (bash/zsh/fish, macOS/Linux/Windows)
  2. Which llm workflow are you using—llm chat or llm prompt?
  3. Do you want one conversation thread per folder, or many threads per folder (named/ID-based) while still keeping them project-local?