docs: dspy integration blog

[akihikokuroda]

dspy integration blog

/schedule 2026-05-06

[ajbozarth]

Great start, here's a few items to address:

There's a typo in the file name: dspy-inegration.md -> dspy-integration.md

[ajbozarth]

I ran the code snippets as well and got the following additional notes:

[ajbozarth]

model="mellea-ollama" appears in 12 of the 15 code examples. Per the source, this parameter is metadata only — it doesn't select the Ollama backend; the actual model comes from start_session() configuration. Suggest dropping it from all examples (the default "mellea" is fine) to avoid implying it matters.

[ajbozarth]

This LGTM, if #35 is merged we can add /schedule 2026-05-06 to the description to let this auto-merge on Wednesday.

[planetf1]

Tested with mellea_dspy==0.1.0 and dspy on Python 3.12. Five findings below — two code-correctness items (A, B) and three reader-experience gaps (C–E). None repeats prior comments.

[planetf1]

Narrative nit — a short 'which to pick' paragraph at the end of the feature tour.

[ajbozarth]

removing my approval for now, at least until @planetf1 's blocker is addressed

[planetf1]

Just the one blocker with the sample code that should be addressed - otherwise just nits/style comments

[ajbozarth]

@akihikokuroda a couple follow ups, but otherwise all of @planetf1 review items have been addressed

[planetf1]

Thanks @ajbozarth for the response above -- I think there's still a remaining issue.

MelleaBestOfN / MelleaRefine use verification.py in mellea_dspy which is rule based. So only requirements that match explicit patterns are enforced - for example:

  - "Must be under X words" / "at least X" / "between X and Y" / "under X characters"
  - "Must mention X" / "Must include X" / "Must not mention X"
  - "Must be in bullet points" / "numbered list" / "valid JSON"
  - "Must be concise" / "detailed" / "professional"

there's a default case of _create_default_reward which does a substring match

In our BestOfN example, two phrases are used : Must be one word and Must be a proper noun. These aren't rule-matched by the list above - so fall through the default case - so the threshold check is pointless. I think this is only working because the model happens to be answering tersely - it is not a reproducible pattern which we need in an example?

There's also the Refine example which uses Must be under 50 words -- that works great as there's a _create_max_words_reward, but then we have Must mention AI and validation which extracts as ai and validation with a substring match - ie doesn't.

So this is about a concern there are silient errors and/or not showing fully how the requirement works -- even though running it, it looks as if it might work

[planetf1]

This could work for BestOfN

Complete fixed BestOfN snippet — combining the session-state restore with handled requirement patterns:

import dspy
from mellea import start_session
from mellea_dspy import MelleaLM, MelleaBestOfN

m = start_session()
dspy.configure(lm=MelleaLM(mellea_session=m))  # neutral global LM, no inherited requirements

qa = dspy.ChainOfThought("question -> answer")

best_of_5 = MelleaBestOfN(
    module=qa,
    N=5,
    requirements=[
        "Must be under 3 words",      # routes to _create_max_words_reward
        "Must be professional",       # routes to _create_professional_reward
    ],
    threshold=0.8,
)

result = best_of_5(question="What is the capital of Belgium?")
print(result.answer)
# Brussels

Maybe also add an explanation of BestForN about the fact we're just pattern checking (in verification.py)

For the Refine example, we just need to correct the phrase

requirements=[
    "Must be under 50 words",
    "Must mention AI",              # single term — extracts "ai", substring match
],

[planetf1]

LGTM - thanks for the last batch of fixes - tested code execution and it's working as described!

[github-actions]

Scheduled merge for 2026-05-06 — added to the merge queue.