n8n Ollama Model nodeConfigure Ollama Model in n8n.

Nothing runs until this dropdown has a value. It names the model that will generate the completion, and the choice decides everything downstream: quality, speed, and how much memory the machine gives up while the model is loaded.

• Model : the model which will generate the completion. The official node documentation lists Llama2, Llama2 13B, Llama2 70B and Llama2 Uncensored; to download models, visit the Ollama Models Library.

Reasoning models keep a scratchpad before they answer. This switch decides whether that scratchpad stays separate from the result, and it only means anything on models that support the mode.

• Enable Thinking : whether to enable (default) thinking mode for supported models. Enabled, the model's thinking process is separated from the output; disabled, the model outputs content directly.

Randomness has a dial, and this is it. The same prompt run twice can come back worded differently, and how far apart those two answers land is exactly what this number controls.

• Sampling Temperature : controls the randomness of the generated text. Lower values make the output more focused and deterministic, while higher values make it more diverse and random.

At each step the model ranks its whole vocabulary, then picks from the top of that ranking. A token is a fragment of text, roughly a word or part of one. This option sets how deep into the ranking the sampler is allowed to look.

• Top K : limits the number of highest probability vocabulary tokens to consider at each step. A higher value increases diversity but may reduce coherence. Set to -1 to disable.

Instead of counting candidates, this one counts probability. It keeps adding tokens to the pool until their combined probability crosses the threshold you set, then stops.

• Top P : chooses from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Helps generate more human-like text by reducing repetitions.

Repetition creeps in when the same token keeps winning the ranking. This penalty leans against tokens in proportion to how often they have already been used in the answer being written.

• Frequency Penalty : adjusts the penalty for tokens that have already appeared in the generated text. Higher values discourage repetition.

Loading a local model into memory costs real seconds. This field says how long Ollama should hold it there once the call is done, so the next run starts warm instead of paying that cost again.

• Keep Alive : specifies the duration to keep the loaded model in memory after use. Useful for frequently used models. Format: 1h30m for one hour and thirty minutes.

Small graphics cards choke on large models. Switching this on trades speed for room, which is often the difference between a chain that runs and one that fails on the machine you actually have.

• Low VRAM Mode : whether to activate low VRAM mode, which reduces memory usage at the cost of slower generation speed. Useful for GPUs with limited memory.

A machine with several graphics cards has to be told which one leads the computation. Left alone, this field is a non-event; on a multi-card server it decides where the load lands.

• Main GPU ID : specifies the ID of the GPU to use for the main computation. Only change this if you have multiple GPUs.

Before generating anything, the model has to read the prompt, and it reads it in batches. Widening the batch moves that reading phase along faster, at a cost you pay in memory.

• Context Batch Size : sets the batch size for prompt processing. Larger batch sizes may improve generation speed but increase memory usage.

Context is a budget. Everything the model is allowed to look at while writing the next token, the prompt and what it has produced so far, has to fit inside this number.

• Context Length : the maximum number of tokens to use as context for generating the next token. Smaller values reduce memory usage, while larger values provide more context to the model.

Spreading the work over several cards is something you ask for explicitly here. The field counts the cards, where Main GPU ID names the one in charge.

• Number of GPUs : specifies the number of GPUs to use for parallel processing. Set to -1 for auto-detection.

Answers can run away. This ceiling stops the model mid-flight, which protects both the execution time of the workflow and whatever node is waiting downstream for a field of reasonable size.

• Max Tokens to Generate : the maximum number of tokens to generate. Set to -1 for no limit. Be cautious with a large value, as it can lead to very long outputs.

Part of the work always lands on the processor, and on a machine without a usable graphics card all of it does. This field decides how many threads Ollama is allowed to claim.

• Number of CPU Threads : specifies the number of CPU threads to use for processing. Set to 0 for auto-detection.

Line breaks are tokens too, and a model that reaches for them often produces answers chopped into fragments. This switch pushes it the other way.

• Penalize Newlines : whether the model will be less likely to generate newline characters, encouraging longer continuous sequences of text.

Presence and frequency sound alike but count differently. This one cares only that a token has shown up at all, not how many times, and it nudges the model toward vocabulary it has not touched yet.

• Presence Penalty : adjusts the penalty for tokens based on their presence in the generated text so far. Positive values penalize tokens that have already appeared, encouraging diversity.

A third repetition control sits beside the other two, and this is the blunt one: a single factor applied to tokens that come back around.

• Repetition Penalty : adjusts the penalty factor for repeated tokens. Higher values more strongly discourage repetition. Set to 1.0 to disable repetition penalty.

An operating system under pressure will swap a loaded model out to disk, and the next call crawls. Locking pins it in place so that cannot happen.

• Use Memory Locking : whether to lock the model in memory to prevent swapping. This can improve performance but requires sufficient available memory.

How the weights get read off disk is itself a choice. Memory mapping lets the system page them in as needed instead of pulling everything into RAM first.

• Use Memory Mapping : whether to use memory mapping for loading the model. This can reduce memory usage but may impact performance. Recommended to keep enabled.

Sometimes the weights are beside the point. Loading only the vocabulary skips them entirely, which is fast and produces no real completion.

• Load Vocabulary Only : whether to only load the model vocabulary without the weights. Useful for quickly testing tokenization.

The shape of the API response is selectable, which matters when the next node expects structured data rather than prose it has to parse.

• Output Format : specifies the format of the API response, either default (Default) or json (JSON).