feat(stage-ui): process for reasoning models

[TECozens]

Description

Before

After

New Features

This PR will provide a collapsible reasoning box to the assistant chat, it renders only the speech in the chat
TTS ignores reasoning

Reasoning is defined by any response which is wrapped in tags <think> , <thoughts> etc, this excludes and does not have an affect on SPECIAL_TOKENS

Provided is use-chat-session/ summary.ts containing helper functions to grab the reasoning from a chat session, I believe we can use this to create summaries and store those are memories or use the reasoning directly with little processing.

Linked Issues

#835

Additional Context

There were a lot of challenges in retrospect so I have kept a couple of the ideas in mind for future feature requests and PRs

I think the biggest challenge is determining if the input comes from a reasoning model and what parts of text were reasoning.

I think that if a toggle is needed I could add one to enable/disable reasoning filter, but at present it moves reasoning content if found to a collapsible box and TTS ignores it during stream.

Still i found the streaming and tts to be quite difficult to understand at first like if the response was completed and we performed TTS on the final response that might be easier than doing it per chunk we parse as we do.

Open to suggestions but I found limitations in trying to work around the current approach, if all if fine then we can push this!

I also think that this burns tokens like crazy (Current Release) Significantly less now with only speech parts

Testing

Unit Test Response Categoriser

Example Test Input

    // Special tokens like <|EMOTE_HAPPY|> and <|DELAY:1|> should be included in reasoning
    const text = 'Hello <reasoning>thinking <|EMOTE_HAPPY|> about this <|DELAY:1|></reasoning> world!'

# Run only the response-categoriser tests
pnpm exec vitest run packages/stage-ui/src/composables/response-categoriser.test.ts

# Check the console logs for more verbose output
pnpm exec vitest watch packages/stage-ui/src/composables/response-categoriser.test.ts --reporter=verbos   

Sample Output:

 Test: should handle <think> with special tokens like <|EMOTE_HAPPY|>
  Input text: Hello <think>thinking <|EMOTE_HAPPY|> about this <|DELAY:1|> and that</think> world!
  Segments found: 1
  Tag name: think
  Segment content: thinking <|EMOTE_HAPPY|> about this <|DELAY:1|> and that
  Has <|EMOTE_HAPPY|>: true
  Has <|DELAY:1|>: true
  Reasoning: thinking <|EMOTE_HAPPY|> about this <|DELAY:1|> and that
  Speech: Hello world!

[gemini-code-assist]

Summary of Changes

Hello @TECozens, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the chat assistant's user experience by intelligently separating AI-generated reasoning from direct speech. It introduces a dedicated, collapsible UI element to present the AI's thought process, while simultaneously ensuring that Text-to-Speech (TTS) output remains clear and concise by excluding this reasoning. The changes involve robust streaming categorization logic and new utility functions to manage and leverage this categorized content, improving both visual clarity and audio relevance in AI interactions.

Highlights

• Reasoning Box UI: Introduces a new collapsible UI component, ChatResponseCategorization.vue, to display AI reasoning content within the assistant chat. This box will only render if reasoning is present and is designed to be collapsible to manage screen space.
• Text-to-Speech (TTS) Filtering: Implements a mechanism to filter out AI reasoning content from Text-to-Speech (TTS) output. Only the 'speech' portions of the AI's response will be vocalized, ensuring a more natural and focused audio experience.
• Dynamic Response Categorization: Adds a new composable, responseCategorizer.ts, which dynamically extracts and categorizes parts of an AI's response into 'speech' and 'reasoning' based on XML-like tags (e.g., <think>, <thoughts>). This categorization works both for static responses and streaming data.
• Chat Memory Utilities: Provides new helper functions in use-chat-memory.ts to easily extract, combine, and summarize reasoning and speech content from chat sessions, laying groundwork for potential future features like memory or summarization.
• LLM Marker Parser Enhancement: Extends the useLlmmarkerParser with an onEnd callback, allowing for final processing of the complete streamed text, which is crucial for accurate categorization of speech and reasoning.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This PR introduces a great feature for separating reasoning from speech in assistant responses. The implementation is comprehensive, covering streaming, UI display, and data storage. I've found a few areas for improvement:

• A critical bug in the llmmarkerParser that could duplicate text.
• A potential data inconsistency in the chat.ts store when reconciling streamed content.
• Some opportunities for code simplification and performance improvement in the Vue component and composables.
• The streaming response categorizer has some fragility in its tag parsing that could be improved for robustness.

Overall, this is a solid contribution. Addressing these points will make it even better.

[TECozens]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a well-architected feature to separate "reasoning" from "speech" in assistant responses using tagged content. The implementation is clean, adding new components and composables to handle the UI and logic, and integrating them effectively into the chat store's streaming process. My review focuses on a few areas for improvement, including enhancing code clarity by removing redundancy, improving robustness by avoiding non-null assertions, addressing a potential performance bottleneck in the streaming parser, and improving maintainability by replacing a magic number with a constant. Overall, this is a solid contribution that adds valuable capabilities.

[TECozens]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a robust mechanism for separating 'reasoning' from 'speech' in assistant responses by parsing XML-like tags. The implementation is well-structured, adding a new responseCategorizer composable that handles both streaming and full-text categorization, and integrates it into the chat store to filter content for TTS and display. The UI is updated with a collapsible section for reasoning. I've identified a few areas for improvement, mainly around error handling and robustness in the new parsing logic.

[TECozens]

Updated 👍🏼 ready for review

[nekomeowww]

Generally LGTM.

[TECozens]

Generally LGTM.

I think now it should (Hooray, shall be my first open source contribution)

[github-actions]

✅ Deploy to Cloudflare Workers (Preview) for stage-web ready!

Name	Link
🔍 Latest deploy log	https://github.com/moeru-ai/airi/actions/runs/20726062697
😎 Deploy Preview	https://71330a1e-moeru-ai-airi.kwaa.workers.dev
🚀 Pull Request Preview	https://pr-859-moeru-ai-airi.kwaa.workers.dev