Skip to content

README.md

Latest commit

 

History

History
80 lines (53 loc) · 3.52 KB

README.md

File metadata and controls

80 lines (53 loc) · 3.52 KB

Granite Switch Tutorials

Quick Start (5 minutes)

Tutorial Format Description
Hello Adapter Script Minimal adapter invocation (HuggingFace)
Hello Mellea Notebook Mellea intrinsics intro (vLLM)

How-To Guides

Guide Description
Using Mellea with Granite Switch Connect Mellea to a Granite Switch model
Bring Your Own Adapter Train, compose, and use custom adapters

Learning Paths

Path 1: Low-Level Understanding (HuggingFace)

Best for: Understanding how Granite Switch works at the control-token level

The HuggingFace examples show how adapters are activated via control tokens. This is useful for understanding the underlying mechanics, but for actual inference, use Mellea (Path 2), which provides constrained decoding, prompt formatting, and proper input/output processing.

  1. Prerequisites
  2. Hello Adapter — see control tokens in action
  3. Granite Switch with HuggingFace — detailed walkthrough

Path 2: Inference with Mellea (Recommended)

Best for: All inference use cases — development through production

Mellea is the correct way to invoke Granite Switch capabilities. It handles constrained decoding, prompt rewriting, and input/output processing automatically. Currently supports vLLM; HuggingFace support coming soon.

  1. Prerequisites
  2. Hello Mellea
  3. RAG Pipeline — full RAG with ChromaDB

Composing Models

Before running inference, you need a composed Granite Switch model. Options:

  1. Use pre-composed models from HuggingFace (recommended for getting started)
  2. Compose your own — see Compose Your Checkpoint

Path 3: Bring Your Own Adapter

Best for: Custom adapter development

  1. Bring Your Own Adapter Guide

Notebooks

Interactive Jupyter tutorials in notebooks/:

Notebook Topics Duration
01_granite_switch_with_hf.ipynb Compose + HuggingFace backend, adapter_name= invocation, Core + Guardian adapters in a multi-turn conversation 20 min
02_govt_rag_pipeline.ipynb Full RAG pipeline, ChromaDB, Guardian 30 min
03_compose_granite_switch.ipynb Compose a checkpoint from adapter libraries 15 min

External Resources

Resource Description
Mellea IBM's library for writing Generative Programs
Granite aLoRA Adapters Official adapter libraries on HuggingFace
vLLM Documentation High-performance inference
Granite Models Base Granite models

Reference Documentation

For technical details, see docs/: