Skip to content

docs: improve worker docs, and add internals docs#2334

Merged
dunglas merged 16 commits intomainfrom
docs/improve
May 3, 2026
Merged

docs: improve worker docs, and add internals docs#2334
dunglas merged 16 commits intomainfrom
docs/improve

Conversation

@dunglas
Copy link
Copy Markdown
Member

@dunglas dunglas commented Apr 8, 2026

No description provided.

Copilot AI review requested due to automatic review settings April 8, 2026 15:12
@dunglas dunglas changed the title docs: improver worker docs, and add internals docs docs: improve worker docs, and add internals docs Apr 8, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the FrankenPHP worker documentation to better describe what state does (and doesn’t) get reset between requests in worker mode, helping users avoid cross-request data leakage and unexpected behavior.

Changes:

  • Documented that most PHP superglobals are reset between requests, while $_ENV is not.
  • Added a “State Persistence” section explaining which PHP state persists across requests, with an example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/worker.md Outdated
Comment thread docs/worker.md Outdated
Comment thread docs/extensions.md Outdated
Copy link
Copy Markdown
Member

@alexandre-daubois alexandre-daubois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me once Superlinter is happy 🙂

Comment thread docs/worker.md Outdated
dunglas added 5 commits May 3, 2026 19:57
Updated notes regarding the `GEN_STUB_SCRIPT` environment variable and clarified that the source file is never modified. Added emphasis on naming conventions for PHP constants.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
- Soften the $_ENV note: state the current behavior without claiming a performance rationale.
- Simplify the state-persistence example loop.
- Clarify that Symfony and Laravel Octane reset most state, but user services may need to implement Symfony's ResetInterface.
- Add a `text` language to fenced state-machine and protocol diagrams in internals.md to satisfy markdownlint MD040.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Igor PHP is a static linter that catches state-leak bugs (missing ResetInterface, stateful properties, mutable statics, exit/die, superglobal writes) before they hit production.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/internals.md Outdated
Comment on lines +216 to +218
2. Each request copies `main_thread_env` into `$_SERVER`
3. `frankenphp_putenv()` / `frankenphp_getenv()` use a thread-local `sandboxed_env` copy, preventing race conditions on the global environment
4. The sandboxed environment is reset between requests via `reset_sandboxed_environment()`
Comment thread docs/internals.md
Comment on lines +57 to +79
```text
Lifecycle: Reserved → Booting → Inactive → Ready ⇄ (processing)
Shutdown: ShuttingDown → Done → Reserved
Restart: Restarting → Yielding → Ready
Handler transition: TransitionRequested → TransitionInProgress → TransitionComplete
```

| State | Description |
| ---------------------- | ------------------------------------------------------------------------------------ |
| `Reserved` | Thread slot allocated but not booted. Can be booted on demand. |
| `Booting` | Underlying POSIX thread is starting up. |
| `Inactive` | Thread is alive but has no handler assigned. Minimal memory footprint. |
| `Ready` | Thread has a handler and is ready to accept work. |
| `ShuttingDown` | Thread is shutting down. |
| `Done` | Thread has completely shut down. Transitions back to `Reserved` for potential reuse. |
| `Restarting` | Worker thread is being restarted (e.g., via admin API or file watcher). |
| `Yielding` | Worker thread has yielded control and is waiting to be re-activated. |
| `TransitionRequested` | A handler change has been requested from the Go side. |
| `TransitionInProgress` | The C thread has acknowledged the transition request. |
| `TransitionComplete` | The Go side has installed the new handler. |
@dunglas
Copy link
Copy Markdown
Member Author

dunglas commented May 3, 2026

Added @KevinMartinsDev's Igor-Php! https://github.com/igor-php/igor-php

- Add the missing BootRequested, Rebooting and RebootReady states to the lifecycle diagram and table, and reference internal/state/state.go for the full set.
- Correct the environment-sandboxing description: $_SERVER is built from a copy of main_thread_env plus request-specific variables, $_ENV is populated from the same snapshot via php_import_environment_variables, and sandboxed_env is reset after each script execution (lazy re-init on next getenv/putenv).

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/internals.md
Comment on lines +1 to +14
# Internals

This document explains FrankenPHP's internal architecture, focusing on thread management, the state machine, and the CGO boundary between Go and C/PHP.

## Overview

FrankenPHP embeds the PHP interpreter directly into Go via CGO. Each PHP execution runs on a real POSIX thread (not a goroutine) because PHP's ZTS (Zend Thread Safety) model requires it. Go orchestrates these threads through a state machine, while C handles the PHP SAPI lifecycle.

The main layers are:

1. **Go layer** (`frankenphp.go`, `phpthread.go`, `threadworker.go`, `threadregular.go`, `scaling.go`): Thread pool management, request routing, auto-scaling
2. **C layer** (`frankenphp.c`): PHP SAPI implementation, script execution loop, superglobal management
3. **State machine** (`internal/state/`): Synchronization between Go goroutines and C threads

Make the architecture overview discoverable from the main contributor entry point.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/internals.md Outdated

### Downscaling

A separate goroutine (`startDownScalingThreads`) periodically checks (every 5s) for idle auto-scaled threads. Threads idle longer than `maxIdleTime` (default 5s) are shut down, up to 10 per cycle.
Comment thread docs/internals.md Outdated
6. `go_frankenphp_finish_worker_request()` cleans up the request context
7. The PHP script loops back to step 3

Worker threads are restarted when the script exits (exit code 0), with exponential backoff on failure.
- Worker restart: clarify that exits after frankenphp_handle_request() restart immediately whether clean or due to a fatal error; exponential backoff applies only to consecutive startup failures.
- Downscaling: idle Ready threads are converted to Inactive (not shut down); the full-stop path is currently disabled because of memory-leaking PECL extensions.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
@dunglas dunglas requested a review from Copilot May 3, 2026 18:29
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

docs/worker.md:1

  • This claim conflicts with the new docs/internals.md “Environment Sandboxing” section, which says $_ENV is populated from a snapshot per request. If worker-mode behaves differently (e.g., $_ENV isn’t reset between frankenphp_handle_request() iterations), please reconcile the two docs by clarifying the scope (regular mode vs worker mode; per-script-execution vs per-worker-request) and explicitly stating when $_ENV is (and isn’t) rebuilt.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/internals.md Outdated
Comment on lines +223 to +225
2. For each request, `$_SERVER` is built from a copy of `main_thread_env` plus request-specific variables (in `frankenphp_register_server_vars`); `$_ENV` is populated from the same snapshot through PHP's `php_import_environment_variables` hook
3. `frankenphp_putenv()` / `frankenphp_getenv()` operate on a thread-local `sandboxed_env` initialized lazily from `main_thread_env`, preventing race conditions on the global C environment
4. After each script execution, `reset_sandboxed_environment()` releases `sandboxed_env`; the next call re-initializes it from `main_thread_env`
Comment thread docs/symfony.md
Comment on lines +52 to +54
### Auditing Worker Compatibility

[Igor PHP](https://github.com/igor-php/igor-php) is a static linter that scans Symfony projects for state leaks before they bite in production: services missing `ResetInterface`, stateful properties that aren't reset, mutable local statics, `exit()`/`die()` calls, and superglobal writes. It audits your application code as well as services declared in `vendor/`.
Comment thread docs/es/worker.md Outdated

Si un script de worker falla con un código de salida distinto de cero, FrankenPHP lo reiniciará con una estrategia de retroceso exponencial.
Si el script de worker permanece activo más tiempo que el último retroceso * 2,
Si el script de worker permanece activo más tiempo que el último retroceso \* 2,
- Internals: distinguish regular vs worker mode in the Environment Sandboxing section. \$_SERVER is rebuilt on every request (including each worker iteration), \$_ENV is only populated at script startup, sandboxed_env is only released when the script exits.
- Worker docs (en/es/fr/ru/pt-br): replace the awkward backslash-escaped "\* 2" with inline code "last backoff * 2" for readability.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
@dunglas dunglas requested a review from Copilot May 3, 2026 18:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

docs/worker.md:1

  • Using inline-code to avoid escaping * makes this read like a literal identifier (last backoff) rather than prose. Consider either escaping the asterisk (e.g., last backoff \\* 2) or using a multiplication symbol (e.g., “last backoff × 2”) so this remains readable as documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/ru/worker.md Outdated

Если worker-скрипт завершится с ненулевым кодом выхода, FrankenPHP перезапустит его со стратегией экспоненциальной задержки.
Если скрипт воркера остается активным дольше, чем время последней задержки \* 2, FrankenPHP не будет применять штраф к worker-скрипту и перезапустит его снова.
Если скрипт воркера остается активным дольше, чем `время последней задержки * 2`, FrankenPHP не будет применять штраф к worker-скрипту и перезапустит его снова.
Comment thread docs/ja/hot-reload.md
Comment on lines +148 to +149
- **Idiomorph**が検出された場合、更新されたコンテンツをフェッチし、現在のHTMLを新しい状態に合わせてモーフィングし、状態を失うことなく即座に変更を適用します。
- それ以外の場合、`window.location.reload()`が呼び出されてページがリフレッシュされます。
Comment thread docs/cn/hot-reload.md
Comment on lines +148 to +149
- 如果检测到 **Idiomorph**,它会获取更新的内容并修改当前的 HTML 以匹配新状态,即时应用更改而不会丢失状态。
- 否则,将调用 `window.location.reload()` 来刷新页面。
Comment thread docs/internals.md Outdated
Comment on lines +11 to +13
1. **Go layer** (`frankenphp.go`, `phpthread.go`, `threadworker.go`, `threadregular.go`, `scaling.go`): Thread pool management, request routing, auto-scaling
2. **C layer** (`frankenphp.c`): PHP SAPI implementation, script execution loop, superglobal management
3. **State machine** (`internal/state/`): Synchronization between Go goroutines and C threads
Comment thread docs/es/classic.md Outdated

Al igual que Caddy, FrankenPHP acepta un número ilimitado de conexiones y utiliza un [número fijo de hilos](config.md#caddyfile-config) para atenderlas. La cantidad de conexiones aceptadas y en cola está limitada únicamente por los recursos disponibles del sistema.
El *pool* de hilos de PHP opera con un número fijo de hilos inicializados al inicio, comparable al modo estático de PHP-FPM. También es posible permitir que los hilos [escale automáticamente en tiempo de ejecución](performance.md#max_threads), similar al modo dinámico de PHP-FPM.
El _pool_ de hilos de PHP opera con un número fijo de hilos inicializados al inicio, comparable al modo estático de PHP-FPM. También es posible permitir que los hilos [escale automáticamente en tiempo de ejecución](performance.md#max_threads), similar al modo dinámico de PHP-FPM.
- Internals: refer to file globs and the C header alongside frankenphp.c rather than enumerating individual Go files, so the layer description doesn't drift when files are renamed or split.
- Worker docs (en/es/fr/ru/pt-br): replace the inline-code "last backoff * 2" with the math symbol "× 2" so it reads as prose, not as a code identifier.
- es/classic.md: fix subject-verb agreement ("hilos escale" -> "hilos escalen").

The two remaining nits (3-space vs 4-space sub-list indent in cn/ja hot-reload.md) are skipped: 3 spaces is CommonMark-correct for ordered-list continuation and the lint check passes.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
@dunglas dunglas requested a review from Copilot May 3, 2026 18:47
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

docs/worker.md:1

  • This section calls out $_ENV persistence, but it doesn’t mention related environment mutation APIs (e.g., putenv()), which are commonly used as an alternative to writing $_ENV. Since this PR adds detailed internals coverage, consider adding a short note here (or an explicit link to the relevant internals section) so users don’t miss that other environment writes can persist too in worker mode.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/symfony.md

```console
composer require --dev igor-php/igor-php
vendor/bin/igor-php .
Comment thread docs/internals.md Outdated
### Memory Management

- **Go → C strings**: `C.CString()` allocates with `malloc()`. The C side is responsible for freeing (e.g., `frankenphp_free_request_context()` frees cookie data).
- **Go string pinning**: `thread.Pin()` / `thread.Unpin()` pins Go memory so C can safely reference it during script execution without copying. Unpinned after each script execution.
Pin/Unpin clarification: name the underlying type (`runtime.Pinner` embedded in `phpThread` in `phpthread.go`) and link to the stdlib doc, so readers can locate the API.

Skipping the recurring `vendor/bin/igor-php` nit: the igor-php README documents `igor-php` as the binary name, so the path is correct.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

docs/worker.md:1

  • The wording “Most superglobals … are automatically reset” plus an explicit list can be misleading because it reads as an exhaustive/authoritative set of what is reset; it also omits commonly referenced superglobals like $_SESSION and $_GLOBALS. Consider rephrasing to make it explicit that this is about request-related superglobals and that the list is non-exhaustive (or expand the list/clarify exceptions) so readers don’t assume anything not mentioned is safe/reset.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/extensions.md Outdated
```

> [!NOTE]
>
Comment thread docs/internals.md
Comment on lines +11 to +13
1. **Go layer** (top-level `*.go` files such as `frankenphp.go`, `phpthread.go`, `thread*.go`, `scaling.go`): Thread pool management, request routing, auto-scaling
2. **C layer** (`frankenphp.c`, `frankenphp.h`): PHP SAPI implementation, script execution loop, superglobal management
3. **State machine** (`internal/state/`): Synchronization between Go goroutines and C threads
Drop the empty quoted lines I added inside the [!NOTE] and [!IMPORTANT] admonitions in extensions.md. They produced an empty paragraph in some renderers and weren't present in the original blocks on main.

Skipping the recurring internals.md filenames nit: the current "such as" framing already signals examples rather than canonical entry points, and concrete file references are more useful for readers locating code than directory-only links.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

docs/worker.md:1

  • The warning is clear, but it may be incomplete for readers who use environment APIs rather than $_ENV. Consider adding a short follow-up sentence (or link to internals.md#environment-sandboxing) clarifying whether putenv() / getenv() changes also persist for the lifetime of the worker script on that thread, since that’s a common source of cross-request leakage in long-lived workers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/es/extension-workers.md Outdated
Comment on lines +140 to +143
| **Server** | `WithWorkerOnServerStartup` | `func()` | Configuración global. Se ejecuta **Una vez**. Ejemplo: Conectar a NATS/Redis. |
| **Server** | `WithWorkerOnServerShutdown` | `func()` | Limpieza global. Se ejecuta **Una vez**. Ejemplo: Cerrar conexiones compartidas. |
| **Thread** | `WithWorkerOnReady` | `func(threadID int)` | Configuración por hilo. Llamado cuando un hilo inicia. Recibe el ID del hilo. |
| **Thread** | `WithWorkerOnShutdown` | `func(threadID int)` | Limpieza por hilo. Recibe el ID del hilo. |
| **Thread** | `WithWorkerOnReady` | `func(threadID int)` | Configuración por hilo. Llamado cuando un hilo inicia. Recibe el ID del hilo. |
| **Thread** | `WithWorkerOnShutdown` | `func(threadID int)` | Limpieza por hilo. Recibe el ID del hilo. |
Comment thread docs/internals.md Outdated
Comment on lines +204 to +216
A single goroutine (`startUpscalingThreads`) reads from an unbuffered `scaleChan`:

1. A request handler can't find an available thread
2. It sends the request context to `scaleChan`
3. The scaling goroutine checks:
- Has the request been stalled long enough? (minimum 5ms)
- Is CPU usage below the threshold? (80%)
- Is the thread limit reached?
4. If all checks pass, a new thread is booted and assigned

### Downscaling

A separate goroutine (`startDownScalingThreads`) periodically checks (every 5s) for idle auto-scaled threads. Threads in `Ready` state idle longer than `maxIdleTime` (default 5s) are converted to `Inactive` via `convertToInactiveThread()` (up to 10 per cycle). They are not fully stopped: a code path exists for that, but it is currently disabled because some PECL extensions leak memory and prevent threads from cleanly shutting down.
- es/extension-workers.md: translate the "Server"/"Thread" column labels in the hooks table to "Servidor"/"Hilo" so the Spanish doc is consistent.
- internals.md: drop the private function names from the Upscaling/Downscaling sections; describe the goroutines by behavior instead, so the doc doesn't drift if those identifiers are renamed.

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
@dunglas dunglas requested a review from Copilot May 3, 2026 21:00
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/extensions.md Outdated
- **Automatic list detection** - When converting to PHP, automatically detects if array should be a packed list or hashmap
- **Nested Arrays** - Arrays can be nested and will convert all support types automatically (`int64`,`float64`,`string`,`bool`,`nil`,`AssociativeArray`,`map[string]any`,`[]any`)
- **Automatic list detection** - When converting to PHP, automatically detects if the array should be a packed list or a hashmap
- **Nested Arrays** - Arrays can be nested and will convert all support types automatically (`int64`, `float64`, `string`, `bool`, `nil`, `AssociativeArray`, `map[string]any`, `[]any`)
Comment thread docs/fr/hot-reload.md Outdated
@@ -84,7 +84,7 @@ Le serveur détecte les modifications et publie les modifications automatiquemen
FrankenPHP expose l'URL du Hub Mercure à utiliser pour s'abonner aux modifications de fichiers via la variable d'environnement `$_SERVER['FRANKENPHP_HOT_RELOAD']`.

La bibliothèque JavaScript [frankenphp-hot-reload](https://www.npmjs.com/package/frankenphp-hot-reload) gére la logique côté client.
- extensions.md: "support types" -> "supported types".
- fr/hot-reload.md: typo "gére" -> "gère".

Signed-off-by: Kévin Dunglas <kevin@dunglas.fr>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dunglas dunglas merged commit cc8dfa8 into main May 3, 2026
31 of 35 checks passed
@dunglas dunglas deleted the docs/improve branch May 3, 2026 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants