docs(web-search): add loader engine guidance#1170
docs(web-search): add loader engine guidance#1170kayomarz97 wants to merge 1 commit intoopen-webui:mainfrom
Conversation
ℹ️ Documentation PR Guidelines👋 Welcome! This is an automated message posted on all new documentation PRs to help guide our contributors. Just because this comment appeared doesn't mean you have done anything wrong! Please ensure you're using the correct branches: Target branch (where you're merging TO):
Source branch (where you're merging FROM):
If your docs PR depends on a pending PR in open-webui/open-webui:
Please adjust your PR target branch, source branch, and/or draft status accordingly if needed. |
9cc0835 to
3a3d9c8
Compare
|
|
||
| # Web Loader Engines | ||
|
|
||
| After your search engine returns URLs, Open WebUI still needs to fetch the page content. The **Web Loader Engine** controls how that content is retrieved for traditional web search, URL fetching, and features like [Save Search Results to Knowledge](./save-to-knowledge). |
There was a problem hiding this comment.
only true for default tool calling mode needs rewording generally
| | --- | --- | --- | --- | --- | | ||
| | `safe_web` | Static docs, blogs, and simple HTML pages | No browser rendering | None | Fastest and lightest | | ||
| | `playwright` | Single-page apps and JavaScript-heavy sites | Yes | Browser install or remote Playwright endpoint | Slower and heavier than `safe_web` | | ||
| | `firecrawl` | Cleaner extracted content from difficult or noisy pages | Usually yes, via Firecrawl service | Firecrawl API key and service access | External service, may add cost and network dependency | |
There was a problem hiding this comment.
pretty sure custom web loader is missing (external/custom)
|
|
||
| Tradeoffs: | ||
|
|
||
| - It is slower and uses more CPU and memory than `safe_web`. |
| - Because extraction happens outside Open WebUI, it adds an external network dependency. | ||
|
|
||
| Useful settings: | ||
|
|
There was a problem hiding this comment.
missing external/custom
|
thanks please address the above |
Summary
Closes #447.
Verification
Notes