n8n Ollama Chat Model nodeConfigure Ollama Chat Model in n8n.

This dropdown names the model that generates the completion. It is the one setting you always fill, and it has to match a model you have already pulled onto the Ollama machine; the Ollama Models Library lists what is available to download.

• Model: required. The official node documentation lists Llama2, Llama2 13B, Llama2 70B and Llama2 Uncensored among the choices.

Some models reason step by step before answering. With thinking mode on, which is the default, that reasoning is kept separate from the output. Turn it off and the model writes content directly. It only changes anything on models that support the feature.

• Enable Thinking: a toggle, on by default, that separates the model thinking process from what the workflow receives.

Randomness in the generated text is what this number controls. Lower values make output more focused and deterministic. Higher values make it more diverse and random, which also raises the risk of the model inventing things.

• Sampling Temperature: a number; low for repeatable answers, high for varied ones.

At every step the model ranks candidate tokens, the word fragments it builds text from. Top K caps how many of the highest probability ones stay in the running. A higher value increases diversity but may reduce coherence. Set it to -1 to disable the cap entirely.

• Top K: a number limiting the highest probability vocabulary tokens considered at each step.

Rather than counting tokens, this one works on cumulative probability: the model picks from the smallest set of tokens whose combined probability passes the threshold. The documented effect is more human-like text with fewer repetitions.

• Top P: a number; a lower value ignores the less probable options.

Tokens that already appeared in the generated text get penalized, and this value sets how hard. Higher values discourage repetition. It scales with how often something has been said, so the more a phrase recurs, the stronger the push away from it.

• Frequency Penalty: a number adjusting the penalty for tokens already present in the output.

After a generation finishes, Ollama can hold the model in memory instead of unloading it. This field says for how long, written as a duration such as 1h30m for one hour and thirty minutes. Useful for models a workflow hits often.

• Keep Alive: a string duration in the 1h30m format, controlling how long the loaded model stays in memory after use.

Tight on graphics memory? This switch reduces memory usage at the cost of slower generation. It is meant for GPUs with limited memory, and it is a real trade rather than a free optimization.

• Low VRAM Mode: a toggle that trades generation speed for lower memory usage.

Machines with several graphics cards need to know which one carries the main computation. This field takes that GPU identifier. On a single-GPU box there is nothing to decide here, and the documentation says to change it only when you have multiple GPUs.

• Main GPU ID: a number naming the GPU used for the main computation.

Prompt processing happens in batches, and this sets their size. Larger batches may improve generation speed, but they increase memory usage. It is the throughput lever that has nothing to do with output quality.

• Context Batch Size: a number setting the batch size used for prompt processing.

The context window is how much text the model keeps in view while generating the next token. This number caps it. Smaller values reduce memory usage; larger values give the model more to work with.

• Context Length: the maximum number of tokens used as context for generating the next token.

Parallel processing across cards is set here: the value is how many GPUs Ollama should use. Leave it at -1 and it auto-detects what the machine has, which is the sane default on a box you control.

• Number of GPUs: a number; -1 means auto-detection.

Output length gets its ceiling from this field. Set it to -1 for no limit, though the documentation warns against large values, because they can lead to very long outputs and the run time that comes with them.

• Max Tokens to Generate: the maximum number of tokens produced; -1 removes the limit.

Processing on the CPU uses as many threads as this value allows, and 0 hands the count back to Ollama, which detects it. Worth touching when the machine does more than serve models.

• Number of CPU Threads: a number; 0 triggers auto-detection.

Line breaks are tokens too. Switch this on and the model becomes less likely to generate them, which encourages longer continuous sequences of text instead of short chopped-up lines.

• Penalize Newlines: a toggle making newline characters less likely in the output.

Presence, not frequency, is the criterion here: tokens are penalized simply for having shown up in the text so far. Positive values push the model toward new material, encouraging diversity.

• Presence Penalty: a number; positive values penalize tokens that have already appeared.

A model stuck in a loop, repeating the same clause over and over, is what this factor is for. Higher values discourage repetition more strongly. Set it to 1.0 and the repetition penalty is off.

• Repetition Penalty: a number adjusting the penalty factor for repeated tokens; 1.0 disables it.

Swapping, when the system moves memory pages to disk, slows a model down badly. Locking it in memory prevents that and can improve performance, but it requires enough free memory to hold the whole model.

• Use Memory Locking: a toggle that pins the model in memory to prevent swapping.

Loading the model file through memory mapping can reduce memory usage, though it may impact performance. The documentation recommends keeping it enabled, so this is a setting you disable deliberately rather than by default.

• Use Memory Mapping: a toggle for memory-mapped model loading, recommended on.

Only the model vocabulary gets loaded, without the weights. There is no real generation behind it, which is exactly the point: it makes tokenization quick to test.

• Load Vocabulary Only: a toggle that loads the vocabulary and skips the weights.

Downstream nodes rarely want prose. This selector sets the format of the API response, so the answer arrives in a shape the rest of the workflow can handle.

• Output Format: two choices, default (Default) and json (JSON).