AI & ML Glossary

AI systems that can autonomously plan, use tools, and take actions to accomplish complex multi-step goals with minimal human intervention.

A unified entry point that routes requests to multiple AI providers, handling authentication, rate limiting, and load balancing. Vincony is an API gateway.

A component in neural networks that allows the model to focus on relevant parts of the input when generating each part of the output.

Standardized tests used to evaluate and compare AI model performance. Common benchmarks include MMLU, HumanEval, and MT-Bench.

A prompting technique that encourages the model to break down complex reasoning into intermediate steps, improving accuracy on logical tasks.

Splitting documents into smaller segments for processing by LLMs or storage in vector databases. Chunk size affects retrieval quality.

Automated systems that detect and block harmful, toxic, or policy-violating content in AI inputs and outputs.

The maximum number of tokens an LLM can process in a single request. GPT-4 Turbo supports 128K tokens; Claude 3 supports 200K.

Training a smaller, faster 'student' model to replicate the behavior of a larger 'teacher' model, achieving similar quality at lower cost.

A simpler alternative to RLHF that directly optimizes language models from human preference data without needing a separate reward model.

A dense vector representation of text in a continuous vector space, where semantically similar texts are positioned closer together.

Systematic testing of AI model outputs against quality metrics including accuracy, helpfulness, safety, and task-specific criteria.

Automatically switching to an alternative AI model or provider when the primary one is unavailable or returns errors.

Providing a small number of examples in the prompt to guide the model's output format and behavior without any fine-tuning.

The process of further training a pre-trained model on a specific dataset to adapt it for a particular task or domain.

Techniques to anchor AI outputs in verifiable facts, typically through RAG or tool use, reducing hallucinations.

Safety mechanisms that constrain AI outputs to prevent harmful, biased, or off-topic responses. Can be rule-based or model-based.

When an AI model generates plausible-sounding but factually incorrect or fabricated information not grounded in its training data or provided context.

The process of using a trained model to generate predictions or outputs from new input data. Inference speed and cost are key production concerns.

Techniques used to bypass an AI model's safety filters and alignment training to produce restricted or harmful outputs.

A neural network trained on massive text datasets to understand and generate human-like language. Examples include GPT-4, Claude, and Llama.

The time delay between sending a request to an AI model and receiving the first token of the response. Critical for real-time applications.

A parameter-efficient fine-tuning technique that trains small adapter layers instead of modifying all model weights, dramatically reducing compute requirements.

A model architecture that uses multiple specialized sub-networks (experts) and a gating mechanism to route inputs to the most relevant experts, improving efficiency.

Automatically selecting the optimal AI model for a given task based on complexity, cost, speed, or quality requirements.

An architecture where multiple specialized AI agents collaborate to solve complex tasks, each handling a specific role or subtask.

An extension of attention that runs multiple attention operations in parallel, allowing the model to attend to information from different representation subspaces.

AI models that can process and generate multiple types of data — text, images, audio, video — within a single model. Examples: GPT-4V, Gemini.

Managing and coordinating multiple AI models, agents, or pipeline steps to work together in a defined workflow.

Storing and reusing processed prompt prefixes to reduce latency and cost for repeated or similar requests.

The practice of designing and optimizing input prompts to elicit desired outputs from LLMs, including techniques like few-shot learning and chain-of-thought.

A security attack where malicious instructions are hidden in user input to manipulate the AI model into ignoring its system prompt or performing unintended actions.

A technique combining quantization with LoRA to enable fine-tuning of large models on consumer-grade GPUs with minimal memory.

Reducing model precision (e.g., from 32-bit to 4-bit) to decrease memory usage and increase inference speed with minimal quality loss.

A technique that enhances LLM responses by first retrieving relevant documents from an external knowledge base and injecting them into the prompt context.

Controlling the number of API requests a user or application can make within a time period to prevent abuse and ensure fair usage.

A framework where AI models alternate between reasoning about a problem and taking actions (tool calls) to solve it iteratively.

Systematically testing AI systems by attempting to elicit harmful, biased, or incorrect outputs to identify vulnerabilities before deployment.

A training technique where human preferences are used to create a reward model that guides the LLM toward more helpful and harmless outputs.

Search that understands the meaning and intent behind queries rather than relying on keyword matching, powered by embeddings.

Delivering AI model output token-by-token as it's generated, rather than waiting for the complete response. Improves perceived latency.

An initial instruction given to an LLM that defines its behavior, personality, and constraints for the entire conversation.

A parameter controlling the randomness of model outputs. Lower values (0.0-0.3) produce deterministic outputs; higher values (0.7-1.0) increase creativity.

The number of tokens or requests a model can process per unit of time. Higher throughput reduces cost per query at scale.

The basic unit of text processed by an LLM. A token can be a word, subword, or character depending on the tokenizer used.

An algorithm that converts text into tokens (numbers) that a model can process. Common tokenizers include BPE (Byte-Pair Encoding) and SentencePiece.

The ability of an LLM to invoke external functions, APIs, or tools to perform actions beyond text generation (e.g., web search, code execution).

A sampling method that considers only tokens whose cumulative probability exceeds a threshold P, balancing diversity and coherence.

Using knowledge gained from training on one task to improve performance on a different but related task, the foundation of modern LLM usage.

A deep learning architecture using self-attention mechanisms that processes input data in parallel, forming the backbone of modern LLMs.

An advanced reasoning technique where the model explores multiple reasoning paths simultaneously and evaluates which leads to the best answer.

The latency measurement from when a request is sent to when the first token of the response is generated. Key metric for streaming UX.

A specialized database optimized for storing and querying high-dimensional vector embeddings. Examples include Pinecone, Weaviate, and Qdrant.

A model that combines visual understanding with language capabilities, able to analyze images and answer questions about them.

Asking a model to perform a task without any examples, relying entirely on its pre-trained knowledge and the task description.