Merge pull request #19787 from blakeblackshear/dev

0.17 Release
2026-10-22 03:02:09 +10:00 · 2026-02-26 21:03:59 -06:00
parent dd8282ff3c 7df3622243
commit 0310a9654d
1435 changed files with 114866 additions and 18149 deletions
@@ -25,7 +25,7 @@ Examples of available modules are:

 - `frigate.app`
 - `frigate.mqtt`
- `frigate.object_detection`
+- `frigate.object_detection.base`
 - `detector.<detector_name>`
 - `watchdog.<camera_name>`
 - `ffmpeg.<camera_name>.<sorted_roles>` NOTE: All FFmpeg logs are sent as `error` level.
@@ -53,6 +53,17 @@ environment_vars:
  VARIABLE_NAME: variable_value
 ```

+#### TensorFlow Thread Configuration
+
+If you encounter thread creation errors during classification model training, you can limit TensorFlow's thread usage:
+
+```yaml
+environment_vars:
+  TF_INTRA_OP_PARALLELISM_THREADS: "2" # Threads within operations (0 = use default)
+  TF_INTER_OP_PARALLELISM_THREADS: "2" # Threads between operations (0 = use default)
+  TF_DATASET_THREAD_POOL_SIZE: "2" # Data pipeline threads (0 = use default)
+```
+
 ### `database`

 Tracked object and recording information is managed in a sqlite database at `/config/frigate.db`. If that database is deleted, recordings will be orphaned and will need to be cleaned up manually. They also won't show up in the Media Browser within Home Assistant.
@@ -177,9 +188,11 @@ listen [::]:5000 ipv6only=off;
 By default, Frigate runs at the root path (`/`). However some setups require to run Frigate under a custom path prefix (e.g. `/frigate`), especially when Frigate is located behind a reverse proxy that requires path-based routing.

 ### Set Base Path via HTTP Header
+
 The preferred way to configure the base path is through the `X-Ingress-Path` HTTP header, which needs to be set to the desired base path in an upstream reverse proxy.

 For example, in Nginx:
+
 ```
 location /frigate {
    proxy_set_header X-Ingress-Path /frigate;
@@ -188,9 +201,11 @@ location /frigate {
 ```

 ### Set Base Path via Environment Variable
+
 When it is not feasible to set the base path via a HTTP header, it can also be set via the `FRIGATE_BASE_PATH` environment variable in the Docker Compose file.

 For example:
+
 ```
 services:
  frigate:
@@ -200,6 +215,7 @@ services:
 ```

 This can be used for example to access Frigate via a Tailscale agent (https), by simply forwarding all requests to the base path (http):
+
 ```
 tailscale serve --https=443 --bg --set-path /frigate http://localhost:5000/frigate
 ```
@@ -218,7 +234,7 @@ To do this:

 ### Custom go2rtc version

-Frigate currently includes go2rtc v1.9.9, there may be certain cases where you want to run a different version of go2rtc.
+Frigate currently includes go2rtc v1.9.10, there may be certain cases where you want to run a different version of go2rtc.

 To do this:

@@ -242,7 +258,7 @@ curl -X POST http://frigate_host:5000/api/config/save -d @config.json
 if you'd like you can use your yaml config directly by using [`yq`](https://github.com/mikefarah/yq) to convert it to json:

 ```bash
-yq r -j config.yml | curl -X POST http://frigate_host:5000/api/config/save -d @-
+yq -o=json '.' config.yaml | curl -X POST 'http://frigate_host:5000/api/config/save?save_option=saveonly' --data-binary @-
 ```

 ### Via Command Line
@@ -50,7 +50,7 @@ cameras:

 ### Configuring Minimum Volume

-The audio detector uses volume levels in the same way that motion in a camera feed is used for object detection. This means that frigate will not run audio detection unless the audio volume is above the configured level in order to reduce resource usage. Audio levels can vary widely between camera models so it is important to run tests to see what volume levels are. MQTT explorer can be used on the audio topic to see what volume level is being detected.
+The audio detector uses volume levels in the same way that motion in a camera feed is used for object detection. This means that Frigate will not run audio detection unless the audio volume is above the configured level in order to reduce resource usage. Audio levels can vary widely between camera models so it is important to run tests to see what volume levels are. The Debug view in the Frigate UI has an Audio tab for cameras that have the `audio` role assigned where a graph and the current levels are is displayed. The `min_volume` parameter should be set to the minimum the `RMS` level required to run audio detection.

 :::tip

@@ -72,3 +72,106 @@ audio:
    - speech
    - yell
 ```
+
+### Audio Transcription
+
+Frigate supports fully local audio transcription using either `sherpa-onnx` or OpenAI’s open-source Whisper models via `faster-whisper`. The goal of this feature is to support Semantic Search for `speech` audio events. Frigate is not intended to act as a continuous, fully-automatic speech transcription service — automatically transcribing all speech (or queuing many audio events for transcription) requires substantial CPU (or GPU) resources and is impractical on most systems. For this reason, transcriptions for events are initiated manually from the UI or the API rather than being run continuously in the background.
+
+Transcription accuracy also depends heavily on the quality of your camera's microphone and recording conditions. Many cameras use inexpensive microphones, and distance to the speaker, low audio bitrate, or background noise can significantly reduce transcription quality. If you need higher accuracy, more robust long-running queues, or large-scale automatic transcription, consider using the HTTP API in combination with an automation platform and a cloud transcription service.
+
+#### Configuration
+
+To enable transcription, enable it in your config. Note that audio detection must also be enabled as described above in order to use audio transcription features.
+
+```yaml
+audio_transcription:
+  enabled: True
+  device: ...
+  model_size: ...
+```
+
+Disable audio transcription for select cameras at the camera level:
+
+```yaml
+cameras:
+  back_yard:
+    ...
+    audio_transcription:
+      enabled: False
+```
+
+:::note
+
+Audio detection must be enabled and configured as described above in order to use audio transcription features.
+
+:::
+
+The optional config parameters that can be set at the global level include:
+
+- **`enabled`**: Enable or disable the audio transcription feature.
+  - Default: `False`
+  - It is recommended to only configure the features at the global level, and enable it at the individual camera level.
+- **`device`**: Device to use to run transcription and translation models.
+  - Default: `CPU`
+  - This can be `CPU` or `GPU`. The `sherpa-onnx` models are lightweight and run on the CPU only. The `whisper` models can run on GPU but are only supported on CUDA hardware.
+- **`model_size`**: The size of the model used for live transcription.
+  - Default: `small`
+  - This can be `small` or `large`. The `small` setting uses `sherpa-onnx` models that are fast, lightweight, and always run on the CPU but are not as accurate as the `whisper` model.
+  - This config option applies to **live transcription only**. Recorded `speech` events will always use a different `whisper` model (and can be accelerated for CUDA hardware if available with `device: GPU`).
+- **`language`**: Defines the language used by `whisper` to translate `speech` audio events (and live audio only if using the `large` model).
+  - Default: `en`
+  - You must use a valid [language code](https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10).
+  - Transcriptions for `speech` events are translated.
+  - Live audio is translated only if you are using the `large` model. The `small` `sherpa-onnx` model is English-only.
+
+The only field that is valid at the camera level is `enabled`.
+
+#### Live transcription
+
+The single camera Live view in the Frigate UI supports live transcription of audio for streams defined with the `audio` role. Use the Enable/Disable Live Audio Transcription button/switch to toggle transcription processing. When speech is heard, the UI will display a black box over the top of the camera stream with text. The MQTT topic `frigate/<camera_name>/audio/transcription` will also be updated in real-time with transcribed text.
+
+Results can be error-prone due to a number of factors, including:
+
+- Poor quality camera microphone
+- Distance of the audio source to the camera microphone
+- Low audio bitrate setting in the camera
+- Background noise
+- Using the `small` model - it's fast, but not accurate for poor quality audio
+
+For speech sources close to the camera with minimal background noise, use the `small` model.
+
+If you have CUDA hardware, you can experiment with the `large` `whisper` model on GPU. Performance is not quite as fast as the `sherpa-onnx` `small` model, but live transcription is far more accurate. Using the `large` model with CPU will likely be too slow for real-time transcription.
+
+#### Transcription and translation of `speech` audio events
+
+Any `speech` events in Explore can be transcribed and/or translated through the Transcribe button in the Tracked Object Details pane.
+
+In order to use transcription and translation for past events, you must enable audio detection and define `speech` as an audio type to listen for in your config. To have `speech` events translated into the language of your choice, set the `language` config parameter with the correct [language code](https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10).
+
+The transcribed/translated speech will appear in the description box in the Tracked Object Details pane. If Semantic Search is enabled, embeddings are generated for the transcription text and are fully searchable using the description search type.
+
+:::note
+
+Only one `speech` event may be transcribed at a time. Frigate does not automatically transcribe `speech` events or implement a queue for long-running transcription model inference.
+
+:::
+
+Recorded `speech` events will always use a `whisper` model, regardless of the `model_size` config setting. Without a supported Nvidia GPU, generating transcriptions for longer `speech` events may take a fair amount of time, so be patient.
+
+#### FAQ
+
+1. Why doesn't Frigate automatically transcribe all `speech` events?
+
+   Frigate does not implement a queue mechanism for speech transcription, and adding one is not trivial. A proper queue would need backpressure, prioritization, memory/disk buffering, retry logic, crash recovery, and safeguards to prevent unbounded growth when events outpace processing. That’s a significant amount of complexity for a feature that, in most real-world environments, would mostly just churn through low-value noise.
+
+   Because transcription is **serialized (one event at a time)** and speech events can be generated far faster than they can be processed, an auto-transcribe toggle would very quickly create an ever-growing backlog and degrade core functionality. For the amount of engineering and risk involved, it adds **very little practical value** for the majority of deployments, which are often on low-powered, edge hardware.
+
+   If you hear speech that’s actually important and worth saving/indexing for the future, **just press the transcribe button in Explore** on that specific `speech` event - that keeps things explicit, reliable, and under your control.
+
+   Other options are being considered for future versions of Frigate to add transcription options that support external `whisper` Docker containers. A single transcription service could then be shared by Frigate and other applications (for example, Home Assistant Voice), and run on more powerful machines when available.
+
+2. Why don't you save live transcription text and use that for `speech` events?
+
+   There’s no guarantee that a `speech` event is even created from the exact audio that went through the transcription model. Live transcription and `speech` event creation are **separate, asynchronous processes**. Even when both are correctly configured, trying to align the **precise start and end time of a speech event** with whatever audio the model happened to be processing at that moment is unreliable.
+
+   Automatically persisting that data would often result in **misaligned, partial, or irrelevant transcripts**, while still incurring all of the CPU, storage, and privacy costs of transcription. That’s why Frigate treats transcription as an **explicit, user-initiated action** rather than an automatic side-effect of every `speech` event.
@@ -29,6 +29,10 @@ auth:
  reset_admin_password: true
 ```

+## Password guidance
+
+Constructing secure passwords and managing them properly is important. Frigate requires a minimum length of 12 characters. For guidance on password standards see [NIST SP 800-63B](https://pages.nist.gov/800-63-3/sp800-63b.html). To learn what makes a password truly secure, read this [article](https://medium.com/peerio/how-to-build-a-billion-dollar-password-3d92568d9277).
+
 ## Login failure rate limiting

 In order to limit the risk of brute force attacks, rate limiting is available for login failures. This is implemented with SlowApi, and the string notation for valid values is available in [the documentation](https://limits.readthedocs.io/en/stable/quickstart.html#examples).
@@ -59,6 +63,7 @@ The default session length for user authentication in Frigate is 24 hours. This
 While the default provides a balance of security and convenience, you can customize this duration to suit your specific security requirements and user experience preferences. The session length is configured in seconds.

 The default value of `86400` will expire the authentication session after 24 hours. Some other examples:
+
 - `0`: Setting the session length to 0 will require a user to log in every time they access the application or after a very short, immediate timeout.
 - `604800`: Setting the session length to 604800 will require a user to log in if the token is not refreshed for 7 days.

@@ -80,7 +85,7 @@ python3 -c 'import secrets; print(secrets.token_hex(64))'
 Frigate looks for a JWT token secret in the following order:

 1. An environment variable named `FRIGATE_JWT_SECRET`
-2. A docker secret named `FRIGATE_JWT_SECRET` in `/run/secrets/`
+2. A file named `FRIGATE_JWT_SECRET` in the directory specified by the `CREDENTIALS_DIRECTORY` environment variable (defaults to the Docker Secrets directory: `/run/secrets/`)
 3. A `jwt_secret` option from the Home Assistant Add-on options
 4. A `.jwt_secret` file in the config directory

@@ -123,7 +128,7 @@ proxy:
    role: x-forwarded-groups
 ```

-Frigate supports both `admin` and `viewer` roles (see below). When using port `8971`, Frigate validates these headers and subsequent requests use the headers `remote-user` and `remote-role` for authorization.
+Frigate supports `admin`, `viewer`, and custom roles (see below). When using port `8971`, Frigate validates these headers and subsequent requests use the headers `remote-user` and `remote-role` for authorization.

 A default role can be provided. Any value in the mapped `role` header will override the default.

@@ -133,6 +138,38 @@ proxy:
  default_role: viewer
 ```

+## Role mapping
+
+In some environments, upstream identity providers (OIDC, SAML, LDAP, etc.) do not pass a Frigate-compatible role directly, but instead pass one or more group claims. To handle this, Frigate supports a `role_map` that translates upstream group names into Frigate’s internal roles (`admin`, `viewer`, or custom).
+
+```yaml
+proxy:
+  ...
+  header_map:
+    user: x-forwarded-user
+    role: x-forwarded-groups
+    role_map:
+      admin:
+        - sysadmins
+        - access-level-security
+      viewer:
+        - camera-viewer
+      operator:  # Custom role mapping
+        - operators
+```
+
+In this example:
+
+- If the proxy passes a role header containing `sysadmins` or `access-level-security`, the user is assigned the `admin` role.
+- If the proxy passes a role header containing `camera-viewer`, the user is assigned the `viewer` role.
+- If the proxy passes a role header containing `operators`, the user is assigned the `operator` custom role.
+- If no mapping matches, Frigate falls back to `default_role` if configured.
+- If `role_map` is not defined, Frigate assumes the role header directly contains `admin`, `viewer`, or a custom role name.
+
+**Note on matching semantics:**
+
+- Admin precedence: if the `admin` mapping matches, Frigate resolves the session to `admin` to avoid accidental downgrade when a user belongs to multiple groups (for example both `admin` and `viewer` groups).
+
 #### Port Considerations

 **Authenticated Port (8971)**
@@ -141,6 +178,7 @@ proxy:
 - The `remote-role` header determines the user’s privileges:
  - **admin** → Full access (user management, configuration changes).
  - **viewer** → Read-only access.
+  - **Custom roles** → Read-only access limited to the cameras defined in `auth.roles[role]`.
 - Ensure your **proxy sends both user and role headers** for proper role enforcement.

 **Unauthenticated Port (5000)**
@@ -186,6 +224,41 @@ Frigate supports user roles to control access to certain features in the UI and

 - **admin**: Full access to all features, including user management and configuration.
 - **viewer**: Read-only access to the UI and API, including viewing cameras, review items, and historical footage. Configuration editor and settings in the UI are inaccessible.
+- **Custom Roles**: Arbitrary role names (alphanumeric, dots/underscores) with specific camera permissions. These extend the system for granular access (e.g., "operator" for select cameras).
+
+### Custom Roles and Camera Access
+
+The viewer role provides read-only access to all cameras in the UI and API. Custom roles allow admins to limit read-only access to specific cameras. Each role specifies an array of allowed camera names. If a user is assigned a custom role, their account is like the **viewer** role - they can only view Live, Review/History, Explore, and Export for the designated cameras. Backend API endpoints enforce this server-side (e.g., returning 403 for unauthorized cameras), and the frontend UI filters content accordingly (e.g., camera dropdowns show only permitted options).
+
+### Role Configuration Example
+
+```yaml
+cameras:
+  front_door:
+    # ... camera config
+  side_yard:
+    # ... camera config
+  garage:
+    # ... camera config
+
+auth:
+  enabled: true
+  roles:
+    operator: # Custom role
+      - front_door
+      - garage # Operator can access front and garage
+    neighbor:
+      - side_yard
+```
+
+If you want to provide access to all cameras to a specific user, just use the **viewer** role.
+
+### Managing User Roles
+
+1. Log in as an **admin** user via port `8971` (preferred), or unauthenticated via port `5000`.
+2. Navigate to **Settings**.
+3. In the **Users** section, edit a user’s role by selecting from available roles (admin, viewer, or custom).
+4. In the **Roles** section, add/edit/delete custom roles (select cameras via switches). Deleting a role auto-reassigns users to "viewer".

 ### Role Enforcement

@@ -205,3 +278,42 @@ To use role-based access control, you must connect to Frigate via the **authenti
 1. Log in as an **admin** user via port `8971`.
 2. Navigate to **Settings > Users**.
 3. Edit a user’s role by selecting **admin** or **viewer**.
+
+## API Authentication Guide
+
+### Getting a Bearer Token
+
+To use the Frigate API, you need to authenticate first. Follow these steps to obtain a Bearer token:
+
+#### 1. Login
+
+Make a POST request to `/login` with your credentials:
+
+```bash
+curl -i -X POST https://frigate_ip:8971/api/login \
+  -H "Content-Type: application/json" \
+  -d '{"user": "admin", "password": "your_password"}'
+```
+
+:::note
+
+You may need to include `-k` in the argument list in these steps (eg: `curl -k -i -X POST ...`) if your Frigate instance is using a self-signed certificate.
+
+:::
+
+The response will contain a cookie with the JWT token.
+
+#### 2. Using the Bearer Token
+
+Once you have the token, include it in the Authorization header for subsequent requests:
+
+```bash
+curl -H "Authorization: Bearer <your_token>" https://frigate_ip:8971/api/profile
+```
+
+#### 3. Token Lifecycle
+
+- Tokens are valid for the configured session length
+- Tokens are automatically refreshed when you visit the `/auth` endpoint
+- Tokens are invalidated when the user's password is changed
+- Use `/logout` to clear your session cookie
@@ -21,7 +21,7 @@ Frigate autotracking functions with PTZ cameras capable of relative movement wit

 Many cheaper or older PTZs may not support this standard. Frigate will report an error message in the log and disable autotracking if your PTZ is unsupported.

-Alternatively, you can download and run [this simple Python script](https://gist.github.com/hawkeye217/152a1d4ba80760dac95d46e143d37112), replacing the details on line 4 with your camera's IP address, ONVIF port, username, and password to check your camera.
+The FeatureList on the [ONVIF Conformant Products Database](https://www.onvif.org/conformant-products/) can provide a starting point to determine a camera's compatibility with Frigate's autotracking. Look to see if a camera lists `PTZRelative`, `PTZRelativePanTilt` and/or `PTZRelativeZoom`. These features are required for autotracking, but some cameras still fail to respond even if they claim support.

 A growing list of cameras and brands that have been reported by users to work with Frigate's autotracking can be found [here](cameras.md).

@@ -147,7 +147,7 @@ WEB Digest Algorithm  - MD5
 Reolink has many different camera models with inconsistently supported features and behavior. The below table shows a summary of various features and recommendations.

 | Camera Resolution | Camera Generation         | Recommended Stream Type           | Additional Notes                                                        |
-| ----------------  | ------------------------- | --------------------------------  | ----------------------------------------------------------------------- |
+| ----------------- | ------------------------- | --------------------------------- | ----------------------------------------------------------------------- |
 | 5MP or lower      | All                       | http-flv                          | Stream is h264                                                          |
 | 6MP or higher     | Latest (ex: Duo3, CX-8##) | http-flv with ffmpeg 8.0, or rtsp | This uses the new http-flv-enhanced over H265 which requires ffmpeg 8.0 |
 | 6MP or higher     | Older (ex: RLC-8##)       | rtsp                              |                                                                         |
@@ -244,7 +244,7 @@ go2rtc:
      - rtspx://192.168.1.1:7441/abcdefghijk
 ```

-[See the go2rtc docs for more information](https://github.com/AlexxIT/go2rtc/tree/v1.9.9#source-rtsp)
+[See the go2rtc docs for more information](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#source-rtsp)

 In the Unifi 2.0 update Unifi Protect Cameras had a change in audio sample rate which causes issues for ffmpeg. The input rate needs to be set for record if used directly with unifi protect.

@@ -267,6 +267,7 @@ Some community members have found better performance on Wyze cameras by using an
 To use a USB camera (webcam) with Frigate, the recommendation is to use go2rtc's [FFmpeg Device](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-ffmpeg-device) support:

 - Preparation outside of Frigate:
+
  - Get USB camera path. Run `v4l2-ctl --list-devices` to get a listing of locally-connected cameras available. (You may need to install `v4l-utils` in a way appropriate for your Linux distribution). In the sample configuration below, we use `video=0` to correlate with a detected device path of `/dev/video0`
  - Get USB camera formats & resolutions. Run `ffmpeg -f v4l2 -list_formats all -i /dev/video0` to get an idea of what formats and resolutions the USB Camera supports. In the sample configuration below, we use a width of 1024 and height of 576 in the stream and detection settings based on what was reported back.
  - If using Frigate in a container (e.g. Docker on TrueNAS), ensure you have USB Passthrough support enabled, along with a specific Host Device (`/dev/video0`) + Container Device (`/dev/video0`) listed.
@@ -79,6 +79,12 @@ cameras:

 If the ONVIF connection is successful, PTZ controls will be available in the camera's WebUI.

+:::note
+
+Some cameras use a separate ONVIF/service account that is distinct from the device administrator credentials. If ONVIF authentication fails with the admin account, try creating or using an ONVIF/service user in the camera's firmware. Refer to your camera manufacturer's documentation for more.
+
+:::
+
 :::tip

 If your ONVIF camera does not require authentication credentials, you may still need to specify an empty string for `user` and `password`, eg: `user: ""` and `password: ""`.
@@ -89,32 +95,36 @@ An ONVIF-capable camera that supports relative movement within the field of view

 ## ONVIF PTZ camera recommendations

-This list of working and non-working PTZ cameras is based on user feedback.
+This list of working and non-working PTZ cameras is based on user feedback. If you'd like to report specific quirks or issues with a manufacturer or camera that would be helpful for other users, open a pull request to add to this list.

-| Brand or specific camera     | PTZ Controls | Autotracking | Notes                                                                                                                                           |
-| ---------------------------- | :----------: | :----------: | ----------------------------------------------------------------------------------------------------------------------------------------------- |
-| Amcrest                      |      ✅      |      ✅      | ⛔️ Generally, Amcrest should work, but some older models (like the common IP2M-841) don't support autotracking                                 |
-| Amcrest ASH21                |      ✅      |      ❌      | ONVIF service port: 80                                                                                                                          |
-| Amcrest IP4M-S2112EW-AI      |      ✅      |      ❌      | FOV relative movement not supported.                                                                                                            |
-| Amcrest IP5M-1190EW          |      ✅      |      ❌      | ONVIF Port: 80. FOV relative movement not supported.                                                                                            |
-| Annke CZ504                  |      ✅      |      ✅      | Annke support provide specific firmware ([V5.7.1 build 250227](https://github.com/pierrepinon/annke_cz504/raw/refs/heads/main/digicap_V5-7-1_build_250227.dav)) to fix issue with ONVIF "TranslationSpaceFov" |
-| Axis Q-6155E                 |      ✅      |      ❌      | ONVIF service port: 80; Camera does not support MoveStatus.
-| Ctronics PTZ                 |      ✅      |      ❌      |                                                                                                                                                 |
-| Dahua                        |      ✅      |      ✅      | Some low-end Dahuas (lite series, among others) have been reported to not support autotracking                                                  |
-| Dahua DH-SD2A500HB           |      ✅      |      ❌      |                                                                                                                                                 |
-| Dahua DH-SD49825GB-HNR       |      ✅      |      ✅      |                                                                                                                                                 |
-| Dahua DH-P5AE-PV             |      ❌      |      ❌      |                                                                                                                                                 |
-| Foscam R5                    |      ✅      |      ❌      |                                                                                                                                                 |
-| Hanwha XNP-6550RH            |      ✅      |      ❌      |                                                                                                                                                 |
-| Hikvision                    |      ✅      |      ❌      | Incomplete ONVIF support (MoveStatus won't update even on latest firmware) - reported with HWP-N4215IH-DE and DS-2DE3304W-DE, but likely others |
-| Hikvision DS-2DE3A404IWG-E/W |      ✅      |      ✅      |                                                                                                                                                 |
-| Reolink                      |      ✅      |      ❌      |                                                                                                                                                 |
-| Speco O8P32X                 |      ✅      |      ❌      |                                                                                                                                                 |
-| Sunba 405-D20X               |      ✅      |      ❌      | Incomplete ONVIF support reported on original, and 4k models. All models are suspected incompatable.                                            |
-| Tapo                         |      ✅      |      ❌      | Many models supported, ONVIF Service Port: 2020                                                                                                 |
-| Uniview IPC672LR-AX4DUPK     |      ✅      |      ❌      | Firmware says FOV relative movement is supported, but camera doesn't actually move when sending ONVIF commands                                  |
-| Uniview IPC6612SR-X33-VG     |      ✅      |      ✅      | Leave `calibrate_on_startup` as `False`. A user has reported that zooming with `absolute` is working.                                           |
-| Vikylin PTZ-2804X-I2         |      ❌      |      ❌      | Incomplete ONVIF support                                                                                                                        |
+The FeatureList on the [ONVIF Conformant Products Database](https://www.onvif.org/conformant-products/) can provide a starting point to determine a camera's compatibility with Frigate's autotracking. Look to see if a camera lists `PTZRelative`, `PTZRelativePanTilt` and/or `PTZRelativeZoom`. These features are required for autotracking, but some cameras still fail to respond even if they claim support. If they are missing, autotracking will not work (though basic PTZ in the WebUI might). Avoid cameras with no database entry unless they are confirmed as working below.
+
+| Brand or specific camera     | PTZ Controls | Autotracking | Notes                                                                                                                                                                                                                                                             |
+| ---------------------------- | :----------: | :----------: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Amcrest                      |      ✅      |      ✅      | ⛔️ Generally, Amcrest should work, but some older models (like the common IP2M-841) don't support autotracking                                                                                                                                                    |
+| Amcrest ASH21                |      ✅      |      ❌      | ONVIF service port: 80                                                                                                                                                                                                                                            |
+| Amcrest IP4M-S2112EW-AI      |      ✅      |      ❌      | FOV relative movement not supported.                                                                                                                                                                                                                              |
+| Amcrest IP5M-1190EW          |      ✅      |      ❌      | ONVIF Port: 80. FOV relative movement not supported.                                                                                                                                                                                                              |
+| Annke CZ504                  |      ✅      |      ✅      | Annke support provide specific firmware ([V5.7.1 build 250227](https://github.com/pierrepinon/annke_cz504/raw/refs/heads/main/digicap_V5-7-1_build_250227.dav)) to fix issue with ONVIF "TranslationSpaceFov"                                                     |
+| Axis Q-6155E                 |      ✅      |      ❌      | ONVIF service port: 80; Camera does not support MoveStatus.                                                                                                                                                                                                       |
+| Ctronics PTZ                 |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Dahua                        |      ✅      |      ✅      | Some low-end Dahuas (lite series, picoo series (commonly), among others) have been reported to not support autotracking. These models usually don't have a four digit model number with chassis prefix and options postfix (e.g. DH-P5AE-PV vs DH-SD49825GB-HNR). |
+| Dahua DH-SD2A500HB           |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Dahua DH-SD49825GB-HNR       |      ✅      |      ✅      |                                                                                                                                                                                                                                                                   |
+| Dahua DH-P5AE-PV             |      ❌      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Foscam                       |      ✅      |      ❌      | In general support PTZ, but not relative move. There are no official ONVIF certifications and tests available on the ONVIF Conformant Products Database                                                                                                           |
+| Foscam R5                    |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Foscam SD4                   |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Hanwha XNP-6550RH            |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Hikvision                    |      ✅      |      ❌      | Incomplete ONVIF support (MoveStatus won't update even on latest firmware) - reported with HWP-N4215IH-DE and DS-2DE3304W-DE, but likely others                                                                                                                   |
+| Hikvision DS-2DE3A404IWG-E/W |      ✅      |      ✅      |                                                                                                                                                                                                                                                                   |
+| Reolink                      |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Speco O8P32X                 |      ✅      |      ❌      |                                                                                                                                                                                                                                                                   |
+| Sunba 405-D20X               |      ✅      |      ❌      | Incomplete ONVIF support reported on original, and 4k models. All models are suspected incompatable.                                                                                                                                                              |
+| Tapo                         |      ✅      |      ❌      | Many models supported, ONVIF Service Port: 2020                                                                                                                                                                                                                   |
+| Uniview IPC672LR-AX4DUPK     |      ✅      |      ❌      | Firmware says FOV relative movement is supported, but camera doesn't actually move when sending ONVIF commands                                                                                                                                                    |
+| Uniview IPC6612SR-X33-VG     |      ✅      |      ✅      | Leave `calibrate_on_startup` as `False`. A user has reported that zooming with `absolute` is working.                                                                                                                                                             |
+| Vikylin PTZ-2804X-I2         |      ❌      |      ❌      | Incomplete ONVIF support                                                                                                                                                                                                                                          |

 ## Setting up camera groups

@@ -135,3 +145,7 @@ camera_groups:
    icon: LuCar
    order: 0
 ```
+
+## Two-Way Audio
+
+See the guide [here](/configuration/live/#two-way-talk)
@@ -0,0 +1,130 @@
+---
+id: object_classification
+title: Object Classification
+---
+
+Object classification allows you to train a custom MobileNetV2 classification model to run on tracked objects (persons, cars, animals, etc.) to identify a finer category or attribute for that object. Classification results are visible in the Tracked Object Details pane in Explore, through the `frigate/tracked_object_details` MQTT topic, in Home Assistant sensors via the official Frigate integration, or through the event endpoints in the HTTP API.
+
+## Minimum System Requirements
+
+Object classification models are lightweight and run very fast on CPU. Inference should be usable on virtually any machine that can run Frigate.
+
+Training the model does briefly use a high amount of system resources for about 1–3 minutes per training run. On lower-power devices, training may take longer.
+
+A CPU with AVX instructions is required for training and inference.
+
+## Classes
+
+Classes are the categories your model will learn to distinguish between. Each class represents a distinct visual category that the model will predict.
+
+For object classification:
+
+- Define classes that represent different types or attributes of the detected object
+- Examples: For `person` objects, classes might be `delivery_person`, `resident`, `stranger`
+- Include a `none` class for objects that don't fit any specific category
+- Keep classes visually distinct to improve accuracy
+
+### Classification Type
+
+- **Sub label**:
+
+  - Applied to the object’s `sub_label` field.
+  - Ideal for a single, more specific identity or type.
+  - Example: `cat` → `Leo`, `Charlie`, `None`.
+
+- **Attribute**:
+  - Added as metadata to the object, visible in the Tracked Object Details pane in Explore, `frigate/events` MQTT messages, and the HTTP API response as `<model_name>: <predicted_value>`.
+  - Ideal when multiple attributes can coexist independently.
+  - Example: Detecting if a `person` in a construction yard is wearing a helmet or not, and if they are wearing a yellow vest or not.
+
+:::note
+
+A tracked object can only have a single sub label. If you are using Triggers or Face Recognition and you configure an object classification model for `person` using the sub label type, your sub label may not be assigned correctly as it depends on which enrichment completes its analysis first. This could also occur with `car` objects that are assigned a sub label for a delivery carrier. Consider using the `attribute` type instead.
+
+:::
+
+## Assignment Requirements
+
+Sub labels and attributes are only assigned when both conditions are met:
+
+1. **Threshold**: Each classification attempt must have a confidence score that meets or exceeds the configured `threshold` (default: `0.8`).
+2. **Class Consensus**: After at least 3 classification attempts, 60% of attempts must agree on the same class label. If the consensus class is `none`, no assignment is made.
+
+This two-step verification prevents false positives by requiring consistent predictions across multiple frames before assigning a sub label or attribute.
+
+## Example use cases
+
+### Sub label
+
+- **Known pet vs unknown**: For `dog` objects, set sub label to your pet’s name (e.g., `buddy`) or `none` for others.
+- **Mail truck vs normal car**: For `car`, classify as `mail_truck` vs `car` to filter important arrivals.
+- **Delivery vs non-delivery person**: For `person`, classify `delivery` vs `visitor` based on uniform/props.
+
+### Attributes
+
+- **Backpack**: For `person`, add attribute `backpack: yes/no`.
+- **Helmet**: For `person` (worksite), add `helmet: yes/no`.
+- **Leash**: For `dog`, add `leash: yes/no` (useful for park or yard rules).
+- **Ladder rack**: For `truck`, add `ladder_rack: yes/no` to flag service vehicles.
+
+## Configuration
+
+Object classification is configured as a custom classification model. Each model has its own name and settings. You must list which object labels should be classified.
+
+```yaml
+classification:
+  custom:
+    dog:
+      threshold: 0.8
+      object_config:
+        objects: [dog] # object labels to classify
+        classification_type: sub_label # or: attribute
+```
+
+An optional config, `save_attempts`, can be set as a key under the model name. This defines the number of classification attempts to save in the Recent Classifications tab. For object classification models, the default is 200.
+
+## Training the model
+
+Creating and training the model is done within the Frigate UI using the `Classification` page. The process consists of two steps:
+
+### Step 1: Name and Define
+
+Enter a name for your model, select the object label to classify (e.g., `person`, `dog`, `car`), choose the classification type (sub label or attribute), and define your classes. Frigate will automatically include a `none` class for objects that don't fit any specific category.
+
+For example: To classify your two cats, create a model named "Our Cats" and create two classes, "Charlie" and "Leo". A third class, "none", will be created automatically for other neighborhood cats that are not your own.
+
+### Step 2: Assign Training Examples
+
+The system will automatically generate example images from detected objects matching your selected label. You'll be guided through each class one at a time to select which images represent that class. Any images not assigned to a specific class will automatically be assigned to `none` when you complete the last class. Once all images are processed, training will begin automatically.
+
+When choosing which objects to classify, start with a small number of visually distinct classes and ensure your training samples match camera viewpoints and distances typical for those objects.
+
+If examples for some of your classes do not appear in the grid, you can continue configuring the model without them. New images will begin to appear in the Recent Classifications view. When your missing classes are seen, classify them from this view and retrain your model.
+
+### Improving the Model
+
+- **Problem framing**: Keep classes visually distinct and relevant to the chosen object types.
+- **Data collection**: Use the model’s Recent Classification tab to gather balanced examples across times of day, weather, and distances.
+- **Preprocessing**: Ensure examples reflect object crops similar to Frigate’s boxes; keep the subject centered.
+- **Labels**: Keep label names short and consistent; include a `none` class if you plan to ignore uncertain predictions for sub labels.
+- **Threshold**: Tune `threshold` per model to reduce false assignments. Start at `0.8` and adjust based on validation.
+
+## Debugging Classification Models
+
+To troubleshoot issues with object classification models, enable debug logging to see detailed information about classification attempts, scores, and consensus calculations.
+
+Enable debug logs for classification models by adding `frigate.data_processing.real_time.custom_classification: debug` to your `logger` configuration. These logs are verbose, so only keep this enabled when necessary. Restart Frigate after this change.
+
+```yaml
+logger:
+  default: info
+  logs:
+    frigate.data_processing.real_time.custom_classification: debug
+```
+
+The debug logs will show:
+
+- Classification probabilities for each attempt
+- Whether scores meet the threshold requirement
+- Consensus calculations and when assignments are made
+- Object classification history and weighted scores
@@ -0,0 +1,107 @@
+---
+id: state_classification
+title: State Classification
+---
+
+State classification allows you to train a custom MobileNetV2 classification model on a fixed region of your camera frame(s) to determine a current state. The model can be configured to run on a schedule and/or when motion is detected in that region. Classification results are available through the `frigate/<camera_name>/classification/<model_name>` MQTT topic and in Home Assistant sensors via the official Frigate integration.
+
+## Minimum System Requirements
+
+State classification models are lightweight and run very fast on CPU. Inference should be usable on virtually any machine that can run Frigate.
+
+Training the model does briefly use a high amount of system resources for about 1–3 minutes per training run. On lower-power devices, training may take longer.
+
+A CPU with AVX instructions is required for training and inference.
+
+## Classes
+
+Classes are the different states an area on your camera can be in. Each class represents a distinct visual state that the model will learn to recognize.
+
+For state classification:
+
+- Define classes that represent mutually exclusive states
+- Examples: `open` and `closed` for a garage door, `on` and `off` for lights
+- Use at least 2 classes (typically binary states work best)
+- Keep class names clear and descriptive
+
+## Example use cases
+
+- **Door state**: Detect if a garage or front door is open vs closed.
+- **Gate state**: Track if a driveway gate is open or closed.
+- **Trash day**: Bins at curb vs no bins present.
+- **Pool cover**: Cover on vs off.
+
+## Configuration
+
+State classification is configured as a custom classification model. Each model has its own name and settings. You must provide at least one camera crop under `state_config.cameras`.
+
+```yaml
+classification:
+  custom:
+    front_door:
+      threshold: 0.8
+      state_config:
+        motion: true # run when motion overlaps the crop
+        interval: 10 # also run every N seconds (optional)
+        cameras:
+          front:
+            crop: [0, 180, 220, 400]
+```
+
+An optional config, `save_attempts`, can be set as a key under the model name. This defines the number of classification attempts to save in the Recent Classifications tab. For state classification models, the default is 100.
+
+## Training the model
+
+Creating and training the model is done within the Frigate UI using the `Classification` page. The process consists of three steps:
+
+### Step 1: Name and Define
+
+Enter a name for your model and define at least 2 classes (states) that represent mutually exclusive states. For example, `open` and `closed` for a door, or `on` and `off` for lights.
+
+### Step 2: Select the Crop Area
+
+Choose one or more cameras and draw a rectangle over the area of interest for each camera. The crop should be tight around the region you want to classify to avoid extra signals unrelated to what is being classified. You can drag and resize the rectangle to adjust the crop area.
+
+### Step 3: Assign Training Examples
+
+The system will automatically generate example images from your camera feeds. You'll be guided through each class one at a time to select which images represent that state. It's not strictly required to select all images you see. If a state is missing from the samples, you can train it from the Recent tab later.
+
+Once some images are assigned, training will begin automatically.
+
+### Improving the Model
+
+- **Problem framing**: Keep classes visually distinct and state-focused (e.g., `open`, `closed`, `unknown`). Avoid combining object identity with state in a single model unless necessary.
+- **Data collection**: Use the model's Recent Classifications tab to gather balanced examples across times of day and weather.
+- **When to train**: Focus on cases where the model is entirely incorrect or flips between states when it should not. There's no need to train additional images when the model is already working consistently.
+- **Selecting training images**: Images scoring below 100% due to new conditions (e.g., first snow of the year, seasonal changes) or variations (e.g., objects temporarily in view, insects at night) are good candidates for training, as they represent scenarios different from the default state. Training these lower-scoring images that differ from existing training data helps prevent overfitting. Avoid training large quantities of images that look very similar, especially if they already score 100% as this can lead to overfitting.
+
+## Debugging Classification Models
+
+To troubleshoot issues with state classification models, enable debug logging to see detailed information about classification attempts, scores, and state verification.
+
+Enable debug logs for classification models by adding `frigate.data_processing.real_time.custom_classification: debug` to your `logger` configuration. These logs are verbose, so only keep this enabled when necessary. Restart Frigate after this change.
+
+```yaml
+logger:
+  default: info
+  logs:
+    frigate.data_processing.real_time.custom_classification: debug
+```
+
+The debug logs will show:
+
+- Classification probabilities for each attempt
+- Whether scores meet the threshold requirement
+- State verification progress (consecutive detections needed)
+- When state changes are published
+
+### Recent Classifications
+
+For state classification, images are only added to recent classifications under specific circumstances:
+
+- **First detection**: The first classification attempt for a camera is always saved
+- **State changes**: Images are saved when the detected state differs from the current verified state
+- **Pending verification**: Images are saved when there's a pending state change being verified (requires 3 consecutive identical states)
+- **Low confidence**: Images with scores below 100% are saved even if the state matches the current state (useful for training)
+
+Images are **not** saved when the state is stable (detected state matches current state) **and** the score is 100%. This prevents unnecessary storage of redundant high-confidence classifications.
@@ -24,7 +24,7 @@ Frigate needs to first detect a `person` before it can detect and recognize a fa
 Frigate has support for two face recognition model types:

 - **small**: Frigate will run a FaceNet embedding model to recognize faces, which runs locally on the CPU. This model is optimized for efficiency and is not as accurate.
- **large**: Frigate will run a large ArcFace embedding model that is optimized for accuracy. It is only recommended to be run when an integrated or dedicated GPU is available.
+- **large**: Frigate will run a large ArcFace embedding model that is optimized for accuracy. It is only recommended to be run when an integrated or dedicated GPU / NPU is available.

 In both cases, a lightweight face landmark detection model is also used to align faces before running recognition.

@@ -34,7 +34,7 @@ All of these features run locally on your system.

 The `small` model is optimized for efficiency and runs on the CPU, most CPUs should run the model efficiently.

-The `large` model is optimized for accuracy, an integrated or discrete GPU is required. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.
+The `large` model is optimized for accuracy, an integrated or discrete GPU / NPU is required. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.

 ## Configuration

@@ -70,9 +70,12 @@ Fine-tune face recognition with these optional parameters at the global level of
 - `min_faces`: Min face recognitions for the sub label to be applied to the person object.
  - Default: `1`
 - `save_attempts`: Number of images of recognized faces to save for training.
-  - Default: `100`.
+  - Default: `200`.
 - `blur_confidence_filter`: Enables a filter that calculates how blurry the face is and adjusts the confidence based on this.
  - Default: `True`.
+- `device`: Target a specific device to run the face recognition model on (multi-GPU installation).
+  - Default: `None`.
+  - Note: This setting is only applicable when using the `large` model. See [onnxruntime's provider options](https://onnxruntime.ai/docs/execution-providers/)

 ## Usage

@@ -111,9 +114,9 @@ When choosing images to include in the face training set it is recommended to al

 :::

-### Understanding the Train Tab
+### Understanding the Recent Recognitions Tab

-The Train tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching.
+The Recent Recognitions tab in the face library displays recent face recognition attempts. Detected face images are grouped according to the person they were identified as potentially matching.

 Each face image is labeled with a name (or `Unknown`) along with the confidence score of the recognition attempt. While each image can be used to train the system for a specific person, not all images are suitable for training.

@@ -137,7 +140,7 @@ Once front-facing images are performing well, start choosing slightly off-angle

 Start with the [Usage](#usage) section and re-read the [Model Requirements](#model-requirements) above.

-1. Ensure `person` is being _detected_. A `person` will automatically be scanned by Frigate for a face. Any detected faces will appear in the Train tab in the Frigate UI's Face Library.
+1. Ensure `person` is being _detected_. A `person` will automatically be scanned by Frigate for a face. Any detected faces will appear in the Recent Recognitions tab in the Frigate UI's Face Library.

   If you are using a Frigate+ or `face` detecting model:

@@ -185,7 +188,7 @@ Avoid training on images that already score highly, as this can lead to over-fit
 No, face recognition does not support negative training (i.e., explicitly telling it who someone is _not_). Instead, the best approach is to improve the training data by using a more diverse and representative set of images for each person.
 For more guidance, refer to the section above on improving recognition accuracy.

-### I see scores above the threshold in the train tab, but a sub label wasn't assigned?
+### I see scores above the threshold in the Recent Recognitions tab, but a sub label wasn't assigned?

 The Frigate considers the recognition scores across all recognition attempts for each person object. The scores are continually weighted based on the area of the face, and a sub label will only be assigned to person if a person is confidently recognized consistently. This avoids cases where a single high confidence recognition would throw off the results.

@@ -1,230 +0,0 @@
---
-id: genai
-title: Generative AI
---
-
-Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail.
-
-Requests for a description are sent off automatically to your AI provider at the end of the tracked object's lifecycle, or can optionally be sent earlier after a number of significantly changed frames, for example in use in more real-time notifications. Descriptions can also be regenerated manually via the Frigate UI. Note that if you are manually entering a description for tracked objects prior to its end, this will be overwritten by the generated response.
-
-## Configuration
-
-Generative AI can be enabled for all cameras or only for specific cameras. There are currently 3 native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI section below.
-
-To use Generative AI, you must define a single provider at the global level of your Frigate configuration. If the provider you choose requires an API key, you may either directly paste it in your configuration, or store it in an environment variable prefixed with `FRIGATE_`.
-
-```yaml
-genai:
-  enabled: True
-  provider: gemini
-  api_key: "{FRIGATE_GEMINI_API_KEY}"
-  model: gemini-2.0-flash
-
-cameras:
-  front_camera:
-    genai:
-      enabled: True # <- enable GenAI for your front camera
-      use_snapshot: True
-      objects:
-        - person
-      required_zones:
-        - steps
-  indoor_camera:
-    genai:
-      enabled: False # <- disable GenAI for your indoor camera
-```
-
-By default, descriptions will be generated for all tracked objects and all zones. But you can also optionally specify `objects` and `required_zones` to only generate descriptions for certain tracked objects or zones.
-
-Optionally, you can generate the description using a snapshot (if enabled) by setting `use_snapshot` to `True`. By default, this is set to `False`, which sends the uncompressed images from the `detect` stream collected over the object's lifetime to the model. Once the object lifecycle ends, only a single compressed and cropped thumbnail is saved with the tracked object. Using a snapshot might be useful when you want to _regenerate_ a tracked object's description as it will provide the AI with a higher-quality image (typically downscaled by the AI itself) than the cropped/compressed thumbnail. Using a snapshot otherwise has a trade-off in that only a single image is sent to your provider, which will limit the model's ability to determine object movement or direction.
-
-## Ollama
-
-:::warning
-
-Using Ollama on CPU is not recommended, high inference times make using Generative AI impractical.
-
-:::
-
-[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It provides a nice API over [llama.cpp](https://github.com/ggerganov/llama.cpp). It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
-
-Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available.
-
-Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-does-ollama-handle-concurrent-requests).
-
-### Supported Models
-
-You must use a vision capable model with Frigate. Current model variants can be found [in their model library](https://ollama.com/library). At the time of writing, this includes `llava`, `llava-llama3`, `llava-phi3`, and `moondream`. Note that Frigate will not automatically download the model you specify in your config, you must download the model to your local instance of Ollama first i.e. by running `ollama pull llava:7b` on your Ollama server/Docker container. Note that the model specified in Frigate's config must match the downloaded model tag.
-
-:::note
-
-You should have at least 8 GB of RAM available (or VRAM if running on GPU) to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
-
-:::
-
-### Configuration
-
-```yaml
-genai:
-  enabled: True
-  provider: ollama
-  base_url: http://localhost:11434
-  model: llava:7b
-```
-
-## Google Gemini
-
-Google Gemini has a free tier allowing [15 queries per minute](https://ai.google.dev/pricing) to the API, which is more than sufficient for standard Frigate usage.
-
-### Supported Models
-
-You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://ai.google.dev/gemini-api/docs/models/gemini).
-
-### Get API Key
-
-To start using Gemini, you must first get an API key from [Google AI Studio](https://aistudio.google.com).
-
-1. Accept the Terms of Service
-2. Click "Get API Key" from the right hand navigation
-3. Click "Create API key in new project"
-4. Copy the API key for use in your config
-
-### Configuration
-
-```yaml
-genai:
-  enabled: True
-  provider: gemini
-  api_key: "{FRIGATE_GEMINI_API_KEY}"
-  model: gemini-2.0-flash
-```
-
-:::note
-
-To use a different Gemini-compatible API endpoint, set the `GEMINI_BASE_URL` environment variable to your provider's API URL.
-
-:::
-
-## OpenAI
-
-OpenAI does not have a free tier for their API. With the release of gpt-4o, pricing has been reduced and each generation should cost fractions of a cent if you choose to go this route.
-
-### Supported Models
-
-You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://platform.openai.com/docs/models).
-
-### Get API Key
-
-To start using OpenAI, you must first [create an API key](https://platform.openai.com/api-keys) and [configure billing](https://platform.openai.com/settings/organization/billing/overview).
-
-### Configuration
-
-```yaml
-genai:
-  enabled: True
-  provider: openai
-  api_key: "{FRIGATE_OPENAI_API_KEY}"
-  model: gpt-4o
-```
-
-:::note
-
-To use a different OpenAI-compatible API endpoint, set the `OPENAI_BASE_URL` environment variable to your provider's API URL.
-
-:::
-
-## Azure OpenAI
-
-Microsoft offers several vision models through Azure OpenAI. A subscription is required.
-
-### Supported Models
-
-You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).
-
-### Create Resource and Get API Key
-
-To start using Azure OpenAI, you must first [create a resource](https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource). You'll need your API key, model name, and resource URL, which must include the `api-version` parameter (see the example below).
-
-### Configuration
-
-```yaml
-genai:
-  enabled: True
-  provider: azure_openai
-  base_url: https://instance.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview
-  model: gpt-5-mini
-  api_key: "{FRIGATE_OPENAI_API_KEY}"
-```
-
-## Usage and Best Practices
-
-Frigate's thumbnail search excels at identifying specific details about tracked objects – for example, using an "image caption" approach to find a "person wearing a yellow vest," "a white dog running across the lawn," or "a red car on a residential street." To enhance this further, Frigate’s default prompts are designed to ask your AI provider about the intent behind the object's actions, rather than just describing its appearance.
-
-While generating simple descriptions of detected objects is useful, understanding intent provides a deeper layer of insight. Instead of just recognizing "what" is in a scene, Frigate’s default prompts aim to infer "why" it might be there or "what" it could do next. Descriptions tell you what’s happening, but intent gives context. For instance, a person walking toward a door might seem like a visitor, but if they’re moving quickly after hours, you can infer a potential break-in attempt. Detecting a person loitering near a door at night can trigger an alert sooner than simply noting "a person standing by the door," helping you respond based on the situation’s context.
-
-### Using GenAI for notifications
-
-Frigate provides an [MQTT topic](/integrations/mqtt), `frigate/tracked_object_update`, that is updated with a JSON payload containing `event_id` and `description` when your AI provider returns a description for a tracked object. This description could be used directly in notifications, such as sending alerts to your phone or making audio announcements. If additional details from the tracked object are needed, you can query the [HTTP API](/integrations/api/event-events-event-id-get) using the `event_id`, eg: `http://frigate_ip:5000/api/events/<event_id>`.
-
-If looking to get notifications earlier than when an object ceases to be tracked, an additional send trigger can be configured of `after_significant_updates`.
-
-```yaml
-genai:
-  send_triggers:
-    tracked_object_end: true # default
-    after_significant_updates: 3 # how many updates to a tracked object before we should send an image
-```
-
-## Custom Prompts
-
-Frigate sends multiple frames from the tracked object along with a prompt to your Generative AI provider asking it to generate a description. The default prompt is as follows:
-
-```
-Analyze the sequence of images containing the {label}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next.
-```
-
-:::tip
-
-Prompts can use variable replacements `{label}`, `{sub_label}`, and `{camera}` to substitute information from the tracked object as part of the prompt.
-
-:::
-
-You are also able to define custom prompts in your configuration.
-
-```yaml
-genai:
-  enabled: True
-  provider: ollama
-  base_url: http://localhost:11434
-  model: llava
-  prompt: "Analyze the {label} in these images from the {camera} security camera. Focus on the actions, behavior, and potential intent of the {label}, rather than just describing its appearance."
-  object_prompts:
-    person: "Examine the main person in these images. What are they doing and what might their actions suggest about their intent (e.g., approaching a door, leaving an area, standing still)? Do not describe the surroundings or static details."
-    car: "Observe the primary vehicle in these images. Focus on its movement, direction, or purpose (e.g., parking, approaching, circling). If it's a delivery vehicle, mention the company."
-```
-
-Prompts can also be overriden at the camera level to provide a more detailed prompt to the model about your specific camera, if you desire.
-
-```yaml
-cameras:
-  front_door:
-    genai:
-      use_snapshot: True
-      prompt: "Analyze the {label} in these images from the {camera} security camera at the front door. Focus on the actions and potential intent of the {label}."
-      object_prompts:
-        person: "Examine the person in these images. What are they doing, and how might their actions suggest their purpose (e.g., delivering something, approaching, leaving)? If they are carrying or interacting with a package, include details about its source or destination."
-        cat: "Observe the cat in these images. Focus on its movement and intent (e.g., wandering, hunting, interacting with objects). If the cat is near the flower pots or engaging in any specific actions, mention it."
-      objects:
-        - person
-        - cat
-      required_zones:
-        - steps
-```
-
-### Experiment with prompts
-
-Many providers also have a public facing chat interface for their models. Download a couple of different thumbnails or snapshots from Frigate and try new things in the playground to get descriptions to your liking before updating the prompt in Frigate.
-
- OpenAI - [ChatGPT](https://chatgpt.com)
- Gemini - [Google AI Studio](https://aistudio.google.com)
- Ollama - [Open WebUI](https://docs.openwebui.com/)
@@ -0,0 +1,188 @@
+---
+id: genai_config
+title: Configuring Generative AI
+---
+
+## Configuration
+
+A Generative AI provider can be configured in the global config, which will make the Generative AI features available for use. There are currently 3 native providers available to integrate with Frigate. Other providers that support the OpenAI standard API can also be used. See the OpenAI section below.
+
+To use Generative AI, you must define a single provider at the global level of your Frigate configuration. If the provider you choose requires an API key, you may either directly paste it in your configuration, or store it in an environment variable prefixed with `FRIGATE_`.
+
+## Ollama
+
+:::warning
+
+Using Ollama on CPU is not recommended, high inference times make using Generative AI impractical.
+
+:::
+
+[Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.
+
+Most of the 7b parameter 4-bit vision models will fit inside 8GB of VRAM. There is also a [Docker container](https://hub.docker.com/r/ollama/ollama) available.
+
+Parallel requests also come with some caveats. You will need to set `OLLAMA_NUM_PARALLEL=1` and choose a `OLLAMA_MAX_QUEUE` and `OLLAMA_MAX_LOADED_MODELS` values that are appropriate for your hardware and preferences. See the [Ollama documentation](https://docs.ollama.com/faq#how-does-ollama-handle-concurrent-requests).
+
+### Model Types: Instruct vs Thinking
+
+Most vision-language models are available as **instruct** models, which are fine-tuned to follow instructions and respond concisely to prompts. However, some models (such as certain Qwen-VL or minigpt variants) offer both **instruct** and **thinking** versions.
+
+- **Instruct models** are always recommended for use with Frigate. These models generate direct, relevant, actionable descriptions that best fit Frigate's object and event summary use case.
+- **Thinking models** are fine-tuned for more free-form, open-ended, and speculative outputs, which are typically not concise and may not provide the practical summaries Frigate expects. For this reason, Frigate does **not** recommend or support using thinking models.
+
+Some models are labeled as **hybrid** (capable of both thinking and instruct tasks). In these cases, Frigate will always use instruct-style prompts and specifically disables thinking-mode behaviors to ensure concise, useful responses.
+
+**Recommendation:**
+Always select the `-instruct` or documented instruct/tagged variant of any model you use in your Frigate configuration. If in doubt, refer to your model provider’s documentation or model library for guidance on the correct model variant to use.
+
+### Supported Models
+
+You must use a vision capable model with Frigate. Current model variants can be found [in their model library](https://ollama.com/library). Note that Frigate will not automatically download the model you specify in your config, Ollama will try to download the model but it may take longer than the timeout, it is recommended to pull the model beforehand by running `ollama pull your_model` on your Ollama server/Docker container. Note that the model specified in Frigate's config must match the downloaded model tag.
+
+:::info
+
+Each model is available in multiple parameter sizes (3b, 4b, 8b, etc.). Larger sizes are more capable of complex tasks and understanding of situations, but requires more memory and computational resources. It is recommended to try multiple models and experiment to see which performs best.
+
+:::
+
+:::tip
+
+If you are trying to use a single model for Frigate and HomeAssistant, it will need to support vision and tools calling. qwen3-VL supports vision and tools simultaneously in Ollama.
+
+:::
+
+The following models are recommended:
+
+| Model         | Notes                                                                |
+| ------------- | -------------------------------------------------------------------- |
+| `qwen3-vl`    | Strong visual and situational understanding, higher vram requirement |
+| `Intern3.5VL` | Relatively fast with good vision comprehension                       |
+| `gemma3`      | Strong frame-to-frame understanding, slower inference times          |
+| `qwen2.5-vl`  | Fast but capable model with good vision comprehension                |
+
+:::note
+
+You should have at least 8 GB of RAM available (or VRAM if running on GPU) to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
+
+:::
+
+#### Ollama Cloud models
+
+Ollama also supports [cloud models](https://ollama.com/cloud), where your local Ollama instance handles requests from Frigate, but model inference is performed in the cloud. Set up Ollama locally, sign in with your Ollama account, and specify the cloud model name in your Frigate config. For more details, see the Ollama cloud model [docs](https://docs.ollama.com/cloud).
+
+### Configuration
+
+```yaml
+genai:
+  provider: ollama
+  base_url: http://localhost:11434
+  model: qwen3-vl:4b
+```
+
+## Google Gemini
+
+Google Gemini has a [free tier](https://ai.google.dev/pricing) for the API, however the limits may not be sufficient for standard Frigate usage. Choose a plan appropriate for your installation.
+
+### Supported Models
+
+You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://ai.google.dev/gemini-api/docs/models/gemini).
+
+### Get API Key
+
+To start using Gemini, you must first get an API key from [Google AI Studio](https://aistudio.google.com).
+
+1. Accept the Terms of Service
+2. Click "Get API Key" from the right hand navigation
+3. Click "Create API key in new project"
+4. Copy the API key for use in your config
+
+### Configuration
+
+```yaml
+genai:
+  provider: gemini
+  api_key: "{FRIGATE_GEMINI_API_KEY}"
+  model: gemini-2.5-flash
+```
+
+:::note
+
+To use a different Gemini-compatible API endpoint, set the `provider_options` with the `base_url` key to your provider's API URL. For example:
+
+```
+genai:
+  provider: gemini
+  ...
+  provider_options:
+    base_url: https://...
+```
+
+Other HTTP options are available, see the [python-genai documentation](https://github.com/googleapis/python-genai).
+
+:::
+
+## OpenAI
+
+OpenAI does not have a free tier for their API. With the release of gpt-4o, pricing has been reduced and each generation should cost fractions of a cent if you choose to go this route.
+
+### Supported Models
+
+You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://platform.openai.com/docs/models).
+
+### Get API Key
+
+To start using OpenAI, you must first [create an API key](https://platform.openai.com/api-keys) and [configure billing](https://platform.openai.com/settings/organization/billing/overview).
+
+### Configuration
+
+```yaml
+genai:
+  provider: openai
+  api_key: "{FRIGATE_OPENAI_API_KEY}"
+  model: gpt-4o
+```
+
+:::note
+
+To use a different OpenAI-compatible API endpoint, set the `OPENAI_BASE_URL` environment variable to your provider's API URL.
+
+:::
+
+:::tip
+
+For OpenAI-compatible servers (such as llama.cpp) that don't expose the configured context size in the API response, you can manually specify the context size in `provider_options`:
+
+```yaml
+genai:
+  provider: openai
+  base_url: http://your-llama-server
+  model: your-model-name
+  provider_options:
+    context_size: 8192 # Specify the configured context size
+```
+
+This ensures Frigate uses the correct context window size when generating prompts.
+
+:::
+
+## Azure OpenAI
+
+Microsoft offers several vision models through Azure OpenAI. A subscription is required.
+
+### Supported Models
+
+You must use a vision capable model with Frigate. Current model variants can be found [in their documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).
+
+### Create Resource and Get API Key
+
+To start using Azure OpenAI, you must first [create a resource](https://learn.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource). You'll need your API key, model name, and resource URL, which must include the `api-version` parameter (see the example below).
+
+### Configuration
+
+```yaml
+genai:
+  provider: azure_openai
+  base_url: https://instance.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview
+  model: gpt-5-mini
+  api_key: "{FRIGATE_OPENAI_API_KEY}"
+```
@@ -0,0 +1,78 @@
+---
+id: genai_objects
+title: Object Descriptions
+---
+
+Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail.
+
+Requests for a description are sent off automatically to your AI provider at the end of the tracked object's lifecycle, or can optionally be sent earlier after a number of significantly changed frames, for example in use in more real-time notifications. Descriptions can also be regenerated manually via the Frigate UI. Note that if you are manually entering a description for tracked objects prior to its end, this will be overwritten by the generated response.
+
+By default, descriptions will be generated for all tracked objects and all zones. But you can also optionally specify `objects` and `required_zones` to only generate descriptions for certain tracked objects or zones.
+
+Optionally, you can generate the description using a snapshot (if enabled) by setting `use_snapshot` to `True`. By default, this is set to `False`, which sends the uncompressed images from the `detect` stream collected over the object's lifetime to the model. Once the object lifecycle ends, only a single compressed and cropped thumbnail is saved with the tracked object. Using a snapshot might be useful when you want to _regenerate_ a tracked object's description as it will provide the AI with a higher-quality image (typically downscaled by the AI itself) than the cropped/compressed thumbnail. Using a snapshot otherwise has a trade-off in that only a single image is sent to your provider, which will limit the model's ability to determine object movement or direction.
+
+Generative AI object descriptions can also be toggled dynamically for a camera via MQTT with the topic `frigate/<camera_name>/object_descriptions/set`. See the [MQTT documentation](/integrations/mqtt/#frigatecamera_nameobjectdescriptionsset).
+
+## Usage and Best Practices
+
+Frigate's thumbnail search excels at identifying specific details about tracked objects – for example, using an "image caption" approach to find a "person wearing a yellow vest," "a white dog running across the lawn," or "a red car on a residential street." To enhance this further, Frigate’s default prompts are designed to ask your AI provider about the intent behind the object's actions, rather than just describing its appearance.
+
+While generating simple descriptions of detected objects is useful, understanding intent provides a deeper layer of insight. Instead of just recognizing "what" is in a scene, Frigate’s default prompts aim to infer "why" it might be there or "what" it could do next. Descriptions tell you what’s happening, but intent gives context. For instance, a person walking toward a door might seem like a visitor, but if they’re moving quickly after hours, you can infer a potential break-in attempt. Detecting a person loitering near a door at night can trigger an alert sooner than simply noting "a person standing by the door," helping you respond based on the situation’s context.
+
+## Custom Prompts
+
+Frigate sends multiple frames from the tracked object along with a prompt to your Generative AI provider asking it to generate a description. The default prompt is as follows:
+
+```
+Analyze the sequence of images containing the {label}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next.
+```
+
+:::tip
+
+Prompts can use variable replacements `{label}`, `{sub_label}`, and `{camera}` to substitute information from the tracked object as part of the prompt.
+
+:::
+
+You are also able to define custom prompts in your configuration.
+
+```yaml
+genai:
+  provider: ollama
+  base_url: http://localhost:11434
+  model: qwen3-vl:8b-instruct
+
+objects:
+  genai:
+    prompt: "Analyze the {label} in these images from the {camera} security camera. Focus on the actions, behavior, and potential intent of the {label}, rather than just describing its appearance."
+    object_prompts:
+      person: "Examine the main person in these images. What are they doing and what might their actions suggest about their intent (e.g., approaching a door, leaving an area, standing still)? Do not describe the surroundings or static details."
+      car: "Observe the primary vehicle in these images. Focus on its movement, direction, or purpose (e.g., parking, approaching, circling). If it's a delivery vehicle, mention the company."
+```
+
+Prompts can also be overridden at the camera level to provide a more detailed prompt to the model about your specific camera, if you desire.
+
+```yaml
+cameras:
+  front_door:
+    objects:
+      genai:
+        enabled: True
+        use_snapshot: True
+        prompt: "Analyze the {label} in these images from the {camera} security camera at the front door. Focus on the actions and potential intent of the {label}."
+        object_prompts:
+          person: "Examine the person in these images. What are they doing, and how might their actions suggest their purpose (e.g., delivering something, approaching, leaving)? If they are carrying or interacting with a package, include details about its source or destination."
+          cat: "Observe the cat in these images. Focus on its movement and intent (e.g., wandering, hunting, interacting with objects). If the cat is near the flower pots or engaging in any specific actions, mention it."
+        objects:
+          - person
+          - cat
+        required_zones:
+          - steps
+```
+
+### Experiment with prompts
+
+Many providers also have a public facing chat interface for their models. Download a couple of different thumbnails or snapshots from Frigate and try new things in the playground to get descriptions to your liking before updating the prompt in Frigate.
+
+- OpenAI - [ChatGPT](https://chatgpt.com)
+- Gemini - [Google AI Studio](https://aistudio.google.com)
+- Ollama - [Open WebUI](https://docs.openwebui.com/)
@@ -0,0 +1,134 @@
+---
+id: genai_review
+title: Review Summaries
+---
+
+Generative AI can be used to automatically generate structured summaries of review items. These summaries will show up in Frigate's native notifications as well as in the UI. Generative AI can also be used to take a collection of summaries over a period of time and provide a report, which may be useful to get a quick report of everything that happened while out for some amount of time.
+
+Requests for a summary are requested automatically to your AI provider for alert review items when the activity has ended, they can also be optionally enabled for detections as well.
+
+Generative AI review summaries can also be toggled dynamically for a [camera via MQTT](/integrations/mqtt/#frigatecamera_namereviewdescriptionsset).
+
+## Review Summary Usage and Best Practices
+
+Review summaries provide structured JSON responses that are saved for each review item:
+
+```
+- `title` (string): A concise, direct title that describes the purpose or overall action (e.g., "Person taking out trash", "Joe walking dog").
+- `scene` (string): A narrative description of what happens across the sequence from start to finish, including setting, detected objects, and their observable actions.
+- `shortSummary` (string): A brief 2-sentence summary of the scene, suitable for notifications. This is a condensed version of the scene description.
+- `confidence` (float): 0-1 confidence in the analysis. Higher confidence when objects/actions are clearly visible and context is unambiguous.
+- `other_concerns` (list): List of user-defined concerns that may need additional investigation.
+- `potential_threat_level` (integer): 0, 1, or 2 as defined below.
+```
+
+This will show in multiple places in the UI to give additional context about each activity, and allow viewing more details when extra attention is required. Frigate's built in notifications will automatically show the title and `shortSummary` when the data is available, while the full `scene` description is available in the UI for detailed review.
+
+### Defining Typical Activity
+
+Each installation and even camera can have different parameters for what is considered suspicious activity. Frigate allows the `activity_context_prompt` to be defined globally and at the camera level, which allows you to define more specifically what should be considered normal activity. It is important that this is not overly specific as it can sway the output of the response.
+
+<details>
+  <summary>Default Activity Context Prompt</summary>
+
+```yaml
+review:
+  genai:
+    activity_context_prompt: |
+      ### Normal Activity Indicators (Level 0)
+      - Known/verified people in any zone at any time
+      - People with pets in residential areas
+      - Deliveries or services during daytime/evening (6 AM - 10 PM): carrying packages to doors/porches, placing items, leaving
+      - Services/maintenance workers with visible tools, uniforms, or service vehicles during daytime
+      - Activity confined to public areas only (sidewalks, streets) without entering property at any time
+
+      ### Suspicious Activity Indicators (Level 1)
+      - **Testing or attempting to open doors/windows/handles on vehicles or buildings** — ALWAYS Level 1 regardless of time or duration
+      - **Unidentified person in private areas (driveways, near vehicles/buildings) during late night/early morning (11 PM - 5 AM)** — ALWAYS Level 1 regardless of activity or duration
+      - Taking items that don't belong to them (packages, objects from porches/driveways)
+      - Climbing or jumping fences/barriers to access property
+      - Attempting to conceal actions or items from view
+      - Prolonged loitering: remaining in same area without visible purpose throughout most of the sequence
+
+      ### Critical Threat Indicators (Level 2)
+      - Holding break-in tools (crowbars, pry bars, bolt cutters)
+      - Weapons visible (guns, knives, bats used aggressively)
+      - Forced entry in progress
+      - Physical aggression or violence
+      - Active property damage or theft in progress
+
+      ### Assessment Guidance
+      Evaluate in this order:
+
+      1. **If person is verified/known** → Level 0 regardless of time or activity
+      2. **If person is unidentified:**
+        - Check time: If late night/early morning (11 PM - 5 AM) AND in private areas (driveways, near vehicles/buildings) → Level 1
+        - Check actions: If testing doors/handles, taking items, climbing → Level 1
+        - Otherwise, if daytime/evening (6 AM - 10 PM) with clear legitimate purpose (delivery, service worker) → Level 0
+      3. **Escalate to Level 2 if:** Weapons, break-in tools, forced entry in progress, violence, or active property damage visible (escalates from Level 0 or 1)
+
+      The mere presence of an unidentified person in private areas during late night hours is inherently suspicious and warrants human review, regardless of what activity they appear to be doing or how brief the sequence is.
+```
+
+</details>
+
+### Image Source
+
+By default, review summaries use preview images (cached preview frames) which have a lower resolution but use fewer tokens per image. For better image quality and more detailed analysis, you can configure Frigate to extract frames directly from recordings at a higher resolution:
+
+```yaml
+review:
+  genai:
+    enabled: true
+    image_source: recordings # Options: "preview" (default) or "recordings"
+```
+
+When using `recordings`, frames are extracted at 480px height while maintaining the camera's original aspect ratio, providing better detail for the LLM while being mindful of context window size. This is particularly useful for scenarios where fine details matter, such as identifying license plates, reading text, or analyzing distant objects.
+
+The number of frames sent to the LLM is dynamically calculated based on:
+
+- Your LLM provider's context window size
+- The camera's resolution and aspect ratio (ultrawide cameras like 32:9 use more tokens per image)
+- The image source (recordings use more tokens than preview images)
+
+Frame counts are automatically optimized to use ~98% of the available context window while capping at 20 frames maximum to ensure reasonable inference times. Note that using recordings will:
+
+- Provide higher quality images to the LLM (480p vs 180p preview images)
+- Use more tokens per image due to higher resolution
+- Result in fewer frames being sent for ultrawide cameras due to larger image size
+- Require that recordings are enabled for the camera
+
+If recordings are not available for a given time period, the system will automatically fall back to using preview frames.
+
+### Additional Concerns
+
+Along with the concern of suspicious activity or immediate threat, you may have concerns such as animals in your garden or a gate being left open. These concerns can be configured so that the review summaries will make note of them if the activity requires additional review. For example:
+
+```yaml
+review:
+  genai:
+    enabled: true
+    additional_concerns:
+      - animals in the garden
+```
+
+### Preferred Language
+
+By default, review summaries are generated in English. You can configure Frigate to generate summaries in your preferred language by setting the `preferred_language` option:
+
+```yaml
+review:
+  genai:
+    enabled: true
+    preferred_language: Spanish
+```
+
+## Review Reports
+
+Along with individual review item summaries, Generative AI can also produce a single report of review items from all cameras marked "suspicious" over a specified time period (for example, a daily summary of suspicious activity while you're on vacation).
+
+### Requesting Reports Programmatically
+
+Review reports can be requested via the [API](/integrations/api/generate-review-summary-review-summarize-start-start-ts-end-end-ts-post) by sending a POST request to `/api/review/summarize/start/{start_ts}/end/{end_ts}` with Unix timestamps.
+
+For Home Assistant users, there is a built-in service (`frigate.review_summarize`) that makes it easy to request review reports as part of automations or scripts. This allows you to automatically generate daily summaries, vacation reports, or custom time period reports based on your specific needs.
@@ -5,25 +5,27 @@ title: Enrichments

 # Enrichments

-Some of Frigate's enrichments can use a discrete GPU for accelerated processing.
+Some of Frigate's enrichments can use a discrete GPU or integrated GPU for accelerated processing.

 ## Requirements

-Object detection and enrichments (like Semantic Search, Face Recognition, and License Plate Recognition) are independent features. To use a GPU for object detection, see the [Object Detectors](/configuration/object_detectors.md) documentation. If you want to use your GPU for any supported enrichments, you must choose the appropriate Frigate Docker image for your GPU and configure the enrichment according to its specific documentation.
+Object detection and enrichments (like Semantic Search, Face Recognition, and License Plate Recognition) are independent features. To use a GPU / NPU for object detection, see the [Object Detectors](/configuration/object_detectors.md) documentation. If you want to use your GPU for any supported enrichments, you must choose the appropriate Frigate Docker image for your GPU / NPU and configure the enrichment according to its specific documentation.

 - **AMD**
-
-  - ROCm will automatically be detected and used for enrichments in the `-rocm` Frigate image.
+  - ROCm support in the `-rocm` Frigate image is automatically detected for enrichments, but only some enrichment models are available due to ROCm's focus on LLMs and limited stability with certain neural network models. Frigate disables models that perform poorly or are unstable to ensure reliable operation, so only compatible enrichments may be active.

 - **Intel**
-
  - OpenVINO will automatically be detected and used for enrichments in the default Frigate image.
+  - **Note:** Intel NPUs have limited model support for enrichments. GPU is recommended for enrichments when available.

 - **Nvidia**
  - Nvidia GPUs will automatically be detected and used for enrichments in the `-tensorrt` Frigate image.
  - Jetson devices will automatically be detected and used for enrichments in the `-tensorrt-jp6` Frigate image.

-Utilizing a GPU for enrichments does not require you to use the same GPU for object detection. For example, you can run the `tensorrt` Docker image for enrichments and still use other dedicated hardware like a Coral or Hailo for object detection. However, one combination that is not supported is TensorRT for object detection and OpenVINO for enrichments.
+- **RockChip**
+  - RockChip NPU will automatically be detected and used for semantic search v1 and face recognition in the `-rk` Frigate image.
+
+Utilizing a GPU for enrichments does not require you to use the same GPU for object detection. For example, you can run the `tensorrt` Docker image to run enrichments on an Nvidia GPU and still use other dedicated hardware like a Coral or Hailo for object detection. However, one combination that is not supported is the `tensorrt` image for object detection on an Nvidia GPU and Intel iGPU for enrichments.

 :::note

@@ -3,78 +3,65 @@ id: hardware_acceleration_video
 title: Video Decoding
 ---

+import CommunityBadge from '@site/src/components/CommunityBadge';
+
 # Video Decoding

-It is highly recommended to use a GPU for hardware acceleration video decoding in Frigate. Some types of hardware acceleration are detected and used automatically, but you may need to update your configuration to enable hardware accelerated decoding in ffmpeg.
+It is highly recommended to use an integrated or discrete GPU for hardware acceleration video decoding in Frigate.

-Depending on your system, these parameters may not be compatible. More information on hardware accelerated decoding for ffmpeg can be found here: https://trac.ffmpeg.org/wiki/HWAccelIntro
+Some types of hardware acceleration are detected and used automatically, but you may need to update your configuration to enable hardware accelerated decoding in ffmpeg. To verify that hardware acceleration is working:
+- Check the logs: A message will either say that hardware acceleration was automatically detected, or there will be a warning that no hardware acceleration was automatically detected
+- If hardware acceleration is specified in the config, verification can be done by ensuring the logs are free from errors. There is no CPU fallback for hardware acceleration.

+:::info

-## Raspberry Pi 3/4
+Frigate supports presets for optimal hardware accelerated video decoding:

-Ensure you increase the allocated RAM for your GPU to at least 128 (`raspi-config` > Performance Options > GPU Memory).
-If you are using the HA Add-on, you may need to use the full access variant and turn off _Protection mode_ for hardware acceleration.
+**AMD**

-```yaml
-# if you want to decode a h264 stream
-ffmpeg:
-  hwaccel_args: preset-rpi-64-h264
+- [AMD](#amd-based-cpus): Frigate can utilize modern AMD integrated GPUs and AMD discrete GPUs to accelerate video decoding.

-# if you want to decode a h265 (hevc) stream
-ffmpeg:
-  hwaccel_args: preset-rpi-64-h265
-```
+**Intel**

-:::note
+- [Intel](#intel-based-cpus): Frigate can utilize most Intel integrated GPUs and Arc GPUs to accelerate video decoding.

-If running Frigate through Docker, you either need to run in privileged mode or
-map the `/dev/video*` devices to Frigate. With Docker Compose add:
+**Nvidia GPU**

-```yaml
-services:
-  frigate:
-    ...
-    devices:
-      - /dev/video11:/dev/video11
-```
+- [Nvidia GPU](#nvidia-gpus): Frigate can utilize most modern Nvidia GPUs to accelerate video decoding.

-Or with `docker run`:
+**Raspberry Pi 3/4**

-```bash
-docker run -d \
-  --name frigate \
-  ...
-  --device /dev/video11 \
-  ghcr.io/blakeblackshear/frigate:stable
-```
+- [Raspberry Pi](#raspberry-pi-34): Frigate can utilize the media engine in the Raspberry Pi 3 and 4 to slightly accelerate video decoding.

-`/dev/video11` is the correct device (on Raspberry Pi 4B). You can check
-by running the following and looking for `H264`:
+**Nvidia Jetson** <CommunityBadge />

-```bash
-for d in /dev/video*; do
-  echo -e "---\n$d"
-  v4l2-ctl --list-formats-ext -d $d
-done
-```
+- [Jetson](#nvidia-jetson): Frigate can utilize the media engine in Jetson hardware to accelerate video decoding.

-Or map in all the `/dev/video*` devices.
+**Rockchip** <CommunityBadge />
+
+- [RKNN](#rockchip-platform): Frigate can utilize the media engine in RockChip SOCs to accelerate video decoding.
+
+**Other Hardware**
+
+Depending on your system, these presets may not be compatible, and you may need to use manual hwaccel args to take advantage of your hardware. More information on hardware accelerated decoding for ffmpeg can be found here: https://trac.ffmpeg.org/wiki/HWAccelIntro

 :::

 ## Intel-based CPUs

+Frigate can utilize most Intel integrated GPUs and Arc GPUs to accelerate video decoding.
+
 :::info

 **Recommended hwaccel Preset**

-| CPU Generation | Intel Driver | Recommended Preset  | Notes                                |
-| -------------- | ------------ | ------------------- | ------------------------------------ |
-| gen1 - gen5    | i965         | preset-vaapi        | qsv is not supported                 |
-| gen6 - gen7    | iHD          | preset-vaapi        | qsv is not supported                 |
-| gen8 - gen12   | iHD          | preset-vaapi        | preset-intel-qsv-\* can also be used |
-| gen13+         | iHD / Xe     | preset-intel-qsv-\* |                                      |
-| Intel Arc GPU  | iHD / Xe     | preset-intel-qsv-\* |                                      |
+| CPU Generation | Intel Driver | Recommended Preset  | Notes                                       |
+| -------------- | ------------ | ------------------- | ------------------------------------------- |
+| gen1 - gen5    | i965         | preset-vaapi        | qsv is not supported, may not support H.265 |
+| gen6 - gen7    | iHD          | preset-vaapi        | qsv is not supported                        |
+| gen8 - gen12   | iHD          | preset-vaapi        | preset-intel-qsv-\* can also be used        |
+| gen13+         | iHD / Xe     | preset-intel-qsv-\* |                                             |
+| Intel Arc GPU  | iHD / Xe     | preset-intel-qsv-\* |                                             |

 :::

@@ -195,15 +182,17 @@ telemetry:

 If you are passing in a device path, make sure you've passed the device through to the container.

-## AMD/ATI GPUs (Radeon HD 2000 and newer GPUs) via libva-mesa-driver
+## AMD-based CPUs

-VAAPI supports automatic profile selection so it will work automatically with both H.264 and H.265 streams.
+Frigate can utilize modern AMD integrated GPUs and AMD GPUs to accelerate video decoding using VAAPI.

-:::note
+### Configuring Radeon Driver

 You need to change the driver to `radeonsi` by adding the following environment variable `LIBVA_DRIVER_NAME=radeonsi` to your docker-compose file or [in the `config.yml` for HA Add-on users](advanced.md#environment_vars).

-:::
+### Via VAAPI
+
+VAAPI supports automatic profile selection so it will work automatically with both H.264 and H.265 streams.

 ```yaml
 ffmpeg:
@@ -264,7 +253,7 @@ processes:

 :::note

-`nvidia-smi` may not show `ffmpeg` processes when run inside the container [due to docker limitations](https://github.com/NVIDIA/nvidia-docker/issues/179#issuecomment-645579458).
+`nvidia-smi` will not show `ffmpeg` processes when run inside the container [due to docker limitations](https://github.com/NVIDIA/nvidia-docker/issues/179#issuecomment-645579458).

 :::

@@ -300,12 +289,63 @@ If you do not see these processes, check the `docker logs` for the container and

 These instructions were originally based on the [Jellyfin documentation](https://jellyfin.org/docs/general/administration/hardware-acceleration.html#nvidia-hardware-acceleration-on-docker-linux).

+## Raspberry Pi 3/4
+
+Ensure you increase the allocated RAM for your GPU to at least 128 (`raspi-config` > Performance Options > GPU Memory).
+If you are using the HA Add-on, you may need to use the full access variant and turn off _Protection mode_ for hardware acceleration.
+
+```yaml
+# if you want to decode a h264 stream
+ffmpeg:
+  hwaccel_args: preset-rpi-64-h264
+
+# if you want to decode a h265 (hevc) stream
+ffmpeg:
+  hwaccel_args: preset-rpi-64-h265
+```
+
+:::note
+
+If running Frigate through Docker, you either need to run in privileged mode or
+map the `/dev/video*` devices to Frigate. With Docker Compose add:
+
+```yaml
+services:
+  frigate:
+    ...
+    devices:
+      - /dev/video11:/dev/video11
+```
+
+Or with `docker run`:
+
+```bash
+docker run -d \
+  --name frigate \
+  ...
+  --device /dev/video11 \
+  ghcr.io/blakeblackshear/frigate:stable
+```
+
+`/dev/video11` is the correct device (on Raspberry Pi 4B). You can check
+by running the following and looking for `H264`:
+
+```bash
+for d in /dev/video*; do
+  echo -e "---\n$d"
+  v4l2-ctl --list-formats-ext -d $d
+done
+```
+
+Or map in all the `/dev/video*` devices.
+
+:::
+
 # Community Supported

-## NVIDIA Jetson (Orin AGX, Orin NX, Orin Nano\*, Xavier AGX, Xavier NX, TX2, TX1, Nano)
+## NVIDIA Jetson

-A separate set of docker images is available that is based on Jetpack/L4T. They come with an `ffmpeg` build
-with codecs that use the Jetson's dedicated media engine. If your Jetson host is running Jetpack 6.0+ use the `stable-tensorrt-jp6` tagged image. Note that the Orin Nano has no video encoder, so frigate will use software encoding on this platform, but the image will still allow hardware decoding and tensorrt object detection.
+A separate set of docker images is available for Jetson devices. They come with an `ffmpeg` build with codecs that use the Jetson's dedicated media engine. If your Jetson host is running Jetpack 6.0+ use the `stable-tensorrt-jp6` tagged image. Note that the Orin Nano has no video encoder, so frigate will use software encoding on this platform, but the image will still allow hardware decoding and tensorrt object detection.

 You will need to use the image with the nvidia container runtime:

@@ -427,3 +467,29 @@ cameras:
 ```

 :::
+
+## Synaptics
+
+Hardware accelerated video de-/encoding is supported on Synpatics SL-series SoC.
+
+### Prerequisites
+
+Make sure to follow the [Synaptics specific installation instructions](/frigate/installation#synaptics).
+
+### Configuration
+
+Add one of the following FFmpeg presets to your `config.yml` to enable hardware video processing:
+
+```yaml
+ffmpeg:
+  hwaccel_args: -c:v h264_v4l2m2m
+  input_args: preset-rtsp-restream
+output_args:
+  record: preset-record-generic-audio-aac
+```
+
+:::warning
+
+Make sure that your SoC supports hardware acceleration for your input stream and your input stream is h264 encoding. For example, if your camera streams with h264 encoding, your SoC must be able to de- and encode with it. If you are unsure whether your SoC meets the requirements, take a look at the datasheet.
+
+:::
@@ -29,12 +29,12 @@ cameras:

 When running Frigate through the HA Add-on, the Frigate `/config` directory is mapped to `/addon_configs/<addon_directory>` in the host, where `<addon_directory>` is specific to the variant of the Frigate Add-on you are running.

-| Add-on Variant             | Configuration directory                      |
-| -------------------------- | -------------------------------------------- |
-| Frigate                    | `/addon_configs/ccab4aaf_frigate`            |
-| Frigate (Full Access)      | `/addon_configs/ccab4aaf_frigate-fa`         |
-| Frigate Beta               | `/addon_configs/ccab4aaf_frigate-beta`       |
-| Frigate Beta (Full Access) | `/addon_configs/ccab4aaf_frigate-fa-beta`    |
+| Add-on Variant             | Configuration directory                   |
+| -------------------------- | ----------------------------------------- |
+| Frigate                    | `/addon_configs/ccab4aaf_frigate`         |
+| Frigate (Full Access)      | `/addon_configs/ccab4aaf_frigate-fa`      |
+| Frigate Beta               | `/addon_configs/ccab4aaf_frigate-beta`    |
+| Frigate Beta (Full Access) | `/addon_configs/ccab4aaf_frigate-fa-beta` |

 **Whenever you see `/config` in the documentation, it refers to this directory.**

@@ -109,15 +109,16 @@ detectors:

 record:
  enabled: True
-  retain:
+  motion:
    days: 7
-    mode: motion
  alerts:
    retain:
      days: 30
+      mode: motion
  detections:
    retain:
      days: 30
+      mode: motion

 snapshots:
  enabled: True
@@ -165,15 +166,16 @@ detectors:

 record:
  enabled: True
-  retain:
+  motion:
    days: 7
-    mode: motion
  alerts:
    retain:
      days: 30
+      mode: motion
  detections:
    retain:
      days: 30
+      mode: motion

 snapshots:
  enabled: True
@@ -231,15 +233,16 @@ model:

 record:
  enabled: True
-  retain:
+  motion:
    days: 7
-    mode: motion
  alerts:
    retain:
      days: 30
+      mode: motion
  detections:
    retain:
      days: 30
+      mode: motion

 snapshots:
  enabled: True
@@ -3,18 +3,18 @@ id: license_plate_recognition
 title: License Plate Recognition (LPR)
 ---

-Frigate can recognize license plates on vehicles and automatically add the detected characters to the `recognized_license_plate` field or a known name as a `sub_label` to tracked objects of type `car` or `motorcycle`. A common use case may be to read the license plates of cars pulling into a driveway or cars passing by on a street.
+Frigate can recognize license plates on vehicles and automatically add the detected characters to the `recognized_license_plate` field or a [known](#matching) name as a `sub_label` to tracked objects of type `car` or `motorcycle`. A common use case may be to read the license plates of cars pulling into a driveway or cars passing by on a street.

-LPR works best when the license plate is clearly visible to the camera. For moving vehicles, Frigate continuously refines the recognition process, keeping the most confident result. However, LPR does not run on stationary vehicles.
+LPR works best when the license plate is clearly visible to the camera. For moving vehicles, Frigate continuously refines the recognition process, keeping the most confident result. When a vehicle becomes stationary, LPR continues to run for a short time after to attempt recognition.

 When a plate is recognized, the details are:

- Added as a `sub_label` (if known) or the `recognized_license_plate` field (if unknown) to a tracked object.
- Viewable in the Review Item Details pane in Review (sub labels).
+- Added as a `sub_label` (if [known](#matching)) or the `recognized_license_plate` field (if unknown) to a tracked object.
+- Viewable in the Details pane in Review/History.
 - Viewable in the Tracked Object Details pane in Explore (sub labels and recognized license plates).
 - Filterable through the More Filters menu in Explore.
- Published via the `frigate/events` MQTT topic as a `sub_label` (known) or `recognized_license_plate` (unknown) for the `car` or `motorcycle` tracked object.
- Published via the `frigate/tracked_object_update` MQTT topic with `name` (if known) and `plate`.
+- Published via the `frigate/events` MQTT topic as a `sub_label` ([known](#matching)) or `recognized_license_plate` (unknown) for the `car` or `motorcycle` tracked object.
+- Published via the `frigate/tracked_object_update` MQTT topic with `name` (if [known](#matching)) and `plate`.

 ## Model Requirements

@@ -31,6 +31,7 @@ In the default mode, Frigate's LPR needs to first detect a `car` or `motorcycle`
 ## Minimum System Requirements

 License plate recognition works by running AI models locally on your system. The YOLOv9 plate detector model and the OCR models ([PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)) are relatively lightweight and can run on your CPU or GPU, depending on your configuration. At least 4GB of RAM is required.
+
 ## Configuration

 License plate recognition is disabled by default. Enable it in your config file:
@@ -66,12 +67,15 @@ Fine-tune the LPR feature using these optional parameters at the global level of
 - **`min_area`**: Defines the minimum area (in pixels) a license plate must be before recognition runs.
  - Default: `1000` pixels. Note: this is intentionally set very low as it is an _area_ measurement (length x width). For reference, 1000 pixels represents a ~32x32 pixel square in your camera image.
  - Depending on the resolution of your camera's `detect` stream, you can increase this value to ignore small or distant plates.
- **`device`**: Device to use to run license plate recognition models.
-  - Default: `CPU`
-  - This can be `CPU` or `GPU`. For users without a model that detects license plates natively, using a GPU may increase performance of the models, especially the YOLOv9 license plate detector model. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.
- **`model_size`**: The size of the model used to detect text on plates.
+- **`device`**: Device to use to run license plate detection _and_ recognition models.
+  - Default: `None`
+  - This is auto-selected by Frigate and can be `CPU`, `GPU`, or the GPU's device number. For users without a model that detects license plates natively, using a GPU may increase performance of the YOLOv9 license plate detector model. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation. However, for users who run a model that detects `license_plate` natively, there is little to no performance gain reported with running LPR on GPU compared to the CPU.
+- **`model_size`**: The size of the model used to identify regions of text on plates.
  - Default: `small`
-  - This can be `small` or `large`. The `large` model uses an enhanced text detector and is more accurate at finding text on plates but slower than the `small` model. For most users, the small model is recommended. For users in countries with multiple lines of text on plates, the large model is recommended. Note that using the large model does not improve _text recognition_, but it may improve _text detection_.
+  - This can be `small` or `large`.
+  - The `small` model is fast and identifies groups of Latin and Chinese characters.
+  - The `large` model identifies Latin characters only, and uses an enhanced text detector to find characters on multi-line plates. It is significantly slower than the `small` model.
+  - If your country or region does not use multi-line plates, you should use the `small` model as performance is much better for single-line plates.

 ### Recognition

@@ -101,6 +105,32 @@ Fine-tune the LPR feature using these optional parameters at the global level of
  - This setting is best adjusted at the camera level if running LPR on multiple cameras.
  - If Frigate is already recognizing plates correctly, leave this setting at the default of `0`. However, if you're experiencing frequent character issues or incomplete plates and you can already easily read the plates yourself, try increasing the value gradually, starting at 5 and adjusting as needed. You should see how different enhancement levels affect your plates. Use the `debug_save_plates` configuration option (see below).

+### Normalization Rules
+
+- **`replace_rules`**: List of regex replacement rules to normalize detected plates. These rules are applied sequentially and are applied _before_ the `format` regex, if specified. Each rule must have a `pattern` (which can be a string or a regex) and `replacement` (a string, which also supports [backrefs](https://docs.python.org/3/library/re.html#re.sub) like `\1`). These rules are useful for dealing with common OCR issues like noise characters, separators, or confusions (e.g., 'O'→'0').
+
+These rules must be defined at the global level of your `lpr` config.
+
+```yaml
+lpr:
+  replace_rules:
+    - pattern: "[%#*?]" # Remove noise symbols
+      replacement: ""
+    - pattern: "[= ]" # Normalize = or space to dash
+      replacement: "-"
+    - pattern: "O" # Swap 'O' to '0' (common OCR error)
+      replacement: "0"
+    - pattern: "I" # Swap 'I' to '1'
+      replacement: "1"
+    - pattern: '(\w{3})(\w{3})' # Split 6 chars into groups (e.g., ABC123 → ABC-123) - use single quotes to preserve backslashes
+      replacement: '\1-\2'
+```
+
+- Rules fire in order: In the example above: clean noise first, then separators, then swaps, then splits.
+- Backrefs (`\1`, `\2`) allow dynamic replacements (e.g., capture groups).
+- Any changes made by the rules are printed to the LPR debug log.
+- Tip: You can test patterns with tools like regex101.com.
+
 ### Debugging

 - **`debug_save_plates`**: Set to `True` to save captured text on plates for debugging. These images are stored in `/media/frigate/clips/lpr`, organized into subdirectories by `<camera>/<event_id>`, and named based on the capture timestamp.
@@ -135,6 +165,9 @@ lpr:
  recognition_threshold: 0.85
  format: "^[A-Z]{2} [A-Z][0-9]{4}$" # Only recognize plates that are two letters, followed by a space, followed by a single letter and 4 numbers
  match_distance: 1 # Allow one character variation in plate matching
+  replace_rules:
+    - pattern: "O"
+      replacement: "0" # Replace the letter O with the number 0 in every plate
  known_plates:
    Delivery Van:
      - "RJ K5678"
@@ -145,7 +178,7 @@ lpr:

 :::note

-If you want to detect cars on cameras but don't want to use resources to run LPR on those cars, you should disable LPR for those specific cameras.
+If a camera is configured to detect `car` or `motorcycle` but you don't want Frigate to run LPR for that camera, disable LPR at the camera level:

 ```yaml
 cameras:
@@ -273,7 +306,7 @@ With this setup:
 - Review items will always be classified as a `detection`.
 - Snapshots will always be saved.
 - Zones and object masks are **not** used.
- The `frigate/events` MQTT topic will **not** publish tracked object updates with the license plate bounding box and score, though `frigate/reviews` will publish if recordings are enabled. If a plate is recognized as a known plate, publishing will occur with an updated `sub_label` field. If characters are recognized, publishing will occur with an updated `recognized_license_plate` field.
+- The `frigate/events` MQTT topic will **not** publish tracked object updates with the license plate bounding box and score, though `frigate/reviews` will publish if recordings are enabled. If a plate is recognized as a [known](#matching) plate, publishing will occur with an updated `sub_label` field. If characters are recognized, publishing will occur with an updated `recognized_license_plate` field.
 - License plate snapshots are saved at the highest-scoring moment and appear in Explore.
 - Debug view will not show `license_plate` bounding boxes.

@@ -341,9 +374,20 @@ Use `match_distance` to allow small character mismatches. Alternatively, define

 Start with ["Why isn't my license plate being detected and recognized?"](#why-isnt-my-license-plate-being-detected-and-recognized). If you are still having issues, work through these steps.

-1. Enable debug logs to see exactly what Frigate is doing.
+1. Start with a simplified LPR config.

-   - Enable debug logs for LPR by adding `frigate.data_processing.common.license_plate: debug` to your `logger` configuration. These logs are _very_ verbose, so only keep this enabled when necessary.
+   - Remove or comment out everything in your LPR config, including `min_area`, `min_plate_length`, `format`, `known_plates`, or `enhancement` values so that the only values left are `enabled` and `debug_save_plates`. This will run LPR with Frigate's default values.
+
+     ```yaml
+     lpr:
+       enabled: true
+       device: CPU
+       debug_save_plates: true
+     ```
+
+2. Enable debug logs to see exactly what Frigate is doing.
+
+   - Enable debug logs for LPR by adding `frigate.data_processing.common.license_plate: debug` to your `logger` configuration. These logs are _very_ verbose, so only keep this enabled when necessary. Restart Frigate after this change.

     ```yaml
     logger:
@@ -352,7 +396,7 @@ Start with ["Why isn't my license plate being detected and recognized?"](#why-is
         frigate.data_processing.common.license_plate: debug
     ```

-2. Ensure your plates are being _detected_.
+3. Ensure your plates are being _detected_.

   If you are using a Frigate+ or `license_plate` detecting model:

@@ -365,7 +409,7 @@ Start with ["Why isn't my license plate being detected and recognized?"](#why-is
   - Watch the debug logs for messages from the YOLOv9 plate detector.
   - You may need to adjust your `detection_threshold` if your plates are not being detected.

-3. Ensure the characters on detected plates are being _recognized_.
+4. Ensure the characters on detected plates are being _recognized_.

   - Enable `debug_save_plates` to save images of detected text on plates to the clips directory (`/media/frigate/clips/lpr`). Ensure these images are readable and the text is clear.
   - Watch the debug view to see plates recognized in real-time. For non-dedicated LPR cameras, the `car` or `motorcycle` label will change to the recognized plate when LPR is enabled and working.
@@ -389,6 +433,6 @@ If you are using a model that natively detects `license_plate`, add an _object m

 If you are not using a model that natively detects `license_plate` or you are using dedicated LPR camera mode, only a _motion mask_ over your text is required.

-### I see "Error running ... model" in my logs. How can I fix this?
+### I see "Error running ... model" in my logs, or my inference time is very high. How can I fix this?

 This usually happens when your GPU is unable to compile or use one of the LPR models. Set your `device` to `CPU` and try again. GPU acceleration only provides a slight performance increase, and the models are lightweight enough to run without issue on most CPUs.
@@ -177,6 +177,10 @@ For devices that support two way talk, Frigate can be configured to use the feat

 To use the Reolink Doorbell with two way talk, you should use the [recommended Reolink configuration](/configuration/camera_specific#reolink-cameras)

+As a starting point to check compatibility for your camera, view the list of cameras supported for two-way talk on the [go2rtc repository](https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#two-way-audio). For cameras in the category `ONVIF Profile T`, you can use the [ONVIF Conformant Products Database](https://www.onvif.org/conformant-products/)'s FeatureList to check for the presence of `AudioOutput`. A camera that supports `ONVIF Profile T` _usually_ supports this, but due to inconsistent support, a camera that explicitly lists this feature may still not work. If no entry for your camera exists on the database, it is recommended not to buy it or to consult with the manufacturer's support on the feature availability.
+
+To prevent go2rtc from blocking other applications from accessing your camera's two-way audio, you must configure your stream with `#backchannel=0`. See [preventing go2rtc from blocking two-way audio](/configuration/restream#two-way-talk-restream) in the restream documentation.
+
 ### Streaming options on camera group dashboards

 Frigate provides a dialog in the Camera Group Edit pane with several options for streaming on a camera group's dashboard. These settings are _per device_ and are saved in your device's local storage.
@@ -213,6 +217,42 @@ For restreamed cameras, go2rtc remains active but does not use system resources

 Note that disabling a camera through the config file (`enabled: False`) removes all related UI elements, including historical footage access. To retain access while disabling the camera, keep it enabled in the config and use the UI or MQTT to disable it temporarily.

+### Live player error messages
+
+When your browser runs into problems playing back your camera streams, it will log short error messages to the browser console. They indicate playback, codec, or network issues on the client/browser side, not something server side with Frigate itself. Below are the common messages you may see and simple actions you can take to try to resolve them.
+
+- **startup**
+
+  - What it means: The player failed to initialize or connect to the live stream (network or startup error).
+  - What to try: Reload the Live view or click _Reset_. Verify `go2rtc` is running and the camera stream is reachable. Try switching to a different stream from the Live UI dropdown (if available) or use a different browser.
+
+  - Possible console messages from the player code:
+
+    - `Error opening MediaSource.`
+    - `Browser reported a network error.`
+    - `Max error count ${errorCount} exceeded.` (the numeric value will vary)
+
+- **mse-decode**
+
+  - What it means: The browser reported a decoding error while trying to play the stream, which usually is a result of a codec incompatibility or corrupted frames.
+  - What to try: Check the browser console for the supported and negotiated codecs. Ensure your camera/restream is using H.264 video and AAC audio (these are the most compatible). If your camera uses a non-standard audio codec, configure `go2rtc` to transcode the stream to AAC. Try another browser (some browsers have stricter MSE/codec support) and, for iPhone, ensure you're on iOS 17.1 or newer.
+
+  - Possible console messages from the player code:
+
+    - `Safari cannot open MediaSource.`
+    - `Safari reported InvalidStateError.`
+    - `Safari reported decoding errors.`
+
+- **stalled**
+
+  - What it means: Playback has stalled because the player has fallen too far behind live (extended buffering or no data arriving).
+  - What to try: This is usually indicative of the browser struggling to decode too many high-resolution streams at once. Try selecting a lower-bandwidth stream (substream), reduce the number of live streams open, improve the network connection, or lower the camera resolution. Also check your camera's keyframe (I-frame) interval — shorter intervals make playback start and recover faster. You can also try increasing the timeout value in the UI pane of Frigate's settings.
+
+  - Possible console messages from the player code:
+
+    - `Buffer time (10 seconds) exceeded, browser may not be playing media correctly.`
+    - `Media playback has stalled after <n> seconds due to insufficient buffering or a network interruption.` (the seconds value will vary)
+
 ## Live view FAQ

 1. **Why don't I have audio in my Live view?**
@@ -229,7 +269,27 @@ Note that disabling a camera through the config file (`enabled: False`) removes

   If you are using continuous streaming or you are loading more than a few high resolution streams at once on the dashboard, your browser may struggle to begin playback of your streams before the timeout. Frigate always prioritizes showing a live stream as quickly as possible, even if it is a lower quality jsmpeg stream. You can use the "Reset" link/button to try loading your high resolution stream again.

-   If you are still experiencing Frigate falling back to low bandwidth mode, you may need to adjust your camera's settings per the [recommendations above](#camera_settings_recommendations).
+   Errors in stream playback (e.g., connection failures, codec issues, or buffering timeouts) that cause the fallback to low bandwidth mode (jsmpeg) are logged to the browser console for easier debugging. These errors may include:
+
+   - Network issues (e.g., MSE or WebRTC network connection problems).
+   - Unsupported codecs or stream formats (e.g., H.265 in WebRTC, which is not supported in some browsers).
+   - Buffering timeouts or low bandwidth conditions causing fallback to jsmpeg.
+   - Browser compatibility problems (e.g., iOS Safari limitations with MSE).
+
+   To view browser console logs:
+
+   1. Open the Frigate Live View in your browser.
+   2. Open the browser's Developer Tools (F12 or right-click > Inspect > Console tab).
+   3. Reproduce the error (e.g., load a problematic stream or simulate network issues).
+   4. Look for messages prefixed with the camera name.
+
+   These logs help identify if the issue is player-specific (MSE vs. WebRTC) or related to camera configuration (e.g., go2rtc streams, codecs). If you see frequent errors:
+
+   - Verify your camera's H.264/AAC settings (see [Frigate's camera settings recommendations](#camera_settings_recommendations)).
+   - Check go2rtc configuration for transcoding (e.g., audio to AAC/OPUS).
+   - Test with a different stream via the UI dropdown (if `live -> streams` is configured).
+   - For WebRTC-specific issues, ensure port 8555 is forwarded and candidates are set (see (WebRTC Extra Configuration)(#webrtc-extra-configuration)).
+   - If your cameras are streaming at a high resolution, your browser may be struggling to load all of the streams before the buffering timeout occurs. Frigate prioritizes showing a true live view as quickly as possible. If the fallback occurs often, change your live view settings to use a lower bandwidth substream.

 3. **It doesn't seem like my cameras are streaming on the Live dashboard. Why?**

@@ -256,3 +316,38 @@ Note that disabling a camera through the config file (`enabled: False`) removes
 7. **My camera streams have lots of visual artifacts / distortion.**

   Some cameras don't include the hardware to support multiple connections to the high resolution stream, and this can cause unexpected behavior. In this case it is recommended to [restream](./restream.md) the high resolution stream so that it can be used for live view and recordings.
+
+8. **Why does my camera stream switch aspect ratios on the Live dashboard?**
+
+   Your camera may change aspect ratios on the dashboard because Frigate uses different streams for different purposes. With go2rtc and Smart Streaming, Frigate shows a static image from the `detect` stream when no activity is present, and switches to the live stream when motion is detected. The camera image will change size if your streams use different aspect ratios.
+
+   To prevent this, make the `detect` stream match the go2rtc live stream's aspect ratio (resolution does not need to match, just the aspect ratio). You can either adjust the camera's output resolution or set the `width` and `height` values in your config's `detect` section to a resolution with an aspect ratio that matches.
+
+   Example: Resolutions from two streams
+
+   - Mismatched (may cause aspect ratio switching on the dashboard):
+
+     - Live/go2rtc stream: 1920x1080 (16:9)
+     - Detect stream: 640x352 (~1.82:1, not 16:9)
+
+   - Matched (prevents switching):
+     - Live/go2rtc stream: 1920x1080 (16:9)
+     - Detect stream: 640x360 (16:9)
+
+   You can update the detect settings in your camera config to match the aspect ratio of your go2rtc live stream. For example:
+
+   ```yaml
+   cameras:
+     front_door:
+       detect:
+         width: 640
+         height: 360 # set this to 360 instead of 352
+       ffmpeg:
+         inputs:
+           - path: rtsp://127.0.0.1:8554/front_door # main stream 1920x1080
+             roles:
+               - record
+           - path: rtsp://127.0.0.1:8554/front_door_sub # sub stream 640x352
+             roles:
+               - detect
+   ```
@@ -28,7 +28,6 @@ To create a poly mask:
 5. Click the plus icon under the type of mask or zone you would like to create
 6. Click on the camera's latest image to create the points for a masked area. Click the first point again to close the polygon.
 7. When you've finished creating your mask, press Save.
-8. Restart Frigate to apply your changes.

 Your config file will be updated with the relative coordinates of the mask/zone:

@@ -3,6 +3,8 @@ id: object_detectors
 title: Object Detectors
 ---

+import CommunityBadge from '@site/src/components/CommunityBadge';
+
 # Supported Hardware

 :::info
@@ -11,14 +13,20 @@ Frigate supports multiple different detectors that work on different types of ha

 **Most Hardware**

- [Coral EdgeTPU](#edge-tpu-detector): The Google Coral EdgeTPU is available in USB and m.2 format allowing for a wide range of compatibility with devices.
+- [Coral EdgeTPU](#edge-tpu-detector): The Google Coral EdgeTPU is available in USB, Mini PCIe, and m.2 formats allowing for a wide range of compatibility with devices.
 - [Hailo](#hailo-8): The Hailo8 and Hailo8L AI Acceleration module is available in m.2 format with a HAT for RPi devices, offering a wide range of compatibility with devices.
+- <CommunityBadge /> [MemryX](#memryx-mx3): The MX3 Acceleration module is available in m.2 format, offering broad compatibility across various platforms.
+- <CommunityBadge /> [DeGirum](#degirum): Service for using hardware devices in the cloud or locally. Hardware and models provided on the cloud on [their website](https://hub.degirum.com).

 **AMD**

 - [ROCm](#amdrocm-gpu-detector): ROCm can run on AMD Discrete GPUs to provide efficient object detection.
 - [ONNX](#onnx): ROCm will automatically be detected and used as a detector in the `-rocm` Frigate image when a supported ONNX model is configured.

+**Apple Silicon**
+
+- [Apple Silicon](#apple-silicon-detector): Apple Silicon can run on M1 and newer Apple Silicon devices.
+
 **Intel**

 - [OpenVino](#openvino-detector): OpenVino can run on Intel Arc GPUs, Intel integrated GPUs, and Intel CPUs to provide efficient object detection.
@@ -26,17 +34,21 @@ Frigate supports multiple different detectors that work on different types of ha

 **Nvidia GPU**

- [ONNX](#onnx): TensorRT will automatically be detected and used as a detector in the `-tensorrt` Frigate image when a supported ONNX model is configured.
+- [ONNX](#onnx): Nvidia GPUs will automatically be detected and used as a detector in the `-tensorrt` Frigate image when a supported ONNX model is configured.

-**Nvidia Jetson**
+**Nvidia Jetson** <CommunityBadge />

 - [TensortRT](#nvidia-tensorrt-detector): TensorRT can run on Jetson devices, using one of many default models.
 - [ONNX](#onnx): TensorRT will automatically be detected and used as a detector in the `-tensorrt-jp6` Frigate image when a supported ONNX model is configured.

-**Rockchip**
+**Rockchip** <CommunityBadge />

 - [RKNN](#rockchip-platform): RKNN models can run on Rockchip devices with included NPUs.

+**Synaptics** <CommunityBadge />
+
+- [Synaptics](#synaptics): synap models can run on Synaptics devices(e.g astra machina) with included NPUs.
+
 **For Testing**

 - [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results.
@@ -53,16 +65,14 @@ This does not affect using hardware for accelerating other tasks such as [semant

 # Officially Supported Detectors

-Frigate provides the following builtin detector types: `cpu`, `edgetpu`, `hailo8l`, `onnx`, `openvino`, `rknn`, and `tensorrt`. By default, Frigate will use a single CPU detector. Other detectors may require additional configuration as described below. When using multiple detectors they will run in dedicated processes, but pull from a common queue of detection requests from across all cameras.
+Frigate provides a number of builtin detector types. By default, Frigate will use a single CPU detector. Other detectors may require additional configuration as described below. When using multiple detectors they will run in dedicated processes, but pull from a common queue of detection requests from across all cameras.

 ## Edge TPU Detector

-The Edge TPU detector type runs a TensorFlow Lite model utilizing the Google Coral delegate for hardware acceleration. To configure an Edge TPU detector, set the `"type"` attribute to `"edgetpu"`.
+The Edge TPU detector type runs TensorFlow Lite models utilizing the Google Coral delegate for hardware acceleration. To configure an Edge TPU detector, set the `"type"` attribute to `"edgetpu"`.

 The Edge TPU device can be specified using the `"device"` attribute according to the [Documentation for the TensorFlow Lite Python API](https://coral.ai/docs/edgetpu/multiple-edgetpu/#using-the-tensorflow-lite-python-api). If not set, the delegate will use the first device it finds.

-A TensorFlow Lite model is provided in the container at `/edgetpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`.
-
 :::tip

 See [common Edge TPU troubleshooting steps](/troubleshooting/edgetpu) if the Edge TPU is not detected.
@@ -134,6 +144,50 @@ detectors:
    device: pci
 ```

+### EdgeTPU Supported Models
+
+| Model                   | Notes                                       |
+| ----------------------- | ------------------------------------------- |
+| [Mobiledet](#mobiledet) | Default model                               |
+| [YOLOv9](#yolov9)       | More accurate but slower than default model |
+
+#### Mobiledet
+
+A TensorFlow Lite model is provided in the container at `/edgetpu_model.tflite` and is used by this detector type by default. To provide your own model, bind mount the file into the container and provide the path with `model.path`.
+
+#### YOLOv9
+
+YOLOv9 models that are compiled for TensorFlow Lite and properly quantized are supported, but not included by default. [Instructions](#yolov9-for-google-coral-support) for downloading a model with support for the Google Coral.
+
+:::tip
+
+**Frigate+ Users:** Follow the [instructions](../integrations/plus#use-models) to set a model ID in your config file.
+
+:::
+
+<details>
+  <summary>YOLOv9 Setup & Config</summary>
+
+After placing the downloaded files for the tflite model and labels in your config folder, you can use the following configuration:
+
+```yaml
+detectors:
+  coral:
+    type: edgetpu
+    device: usb
+
+model:
+  model_type: yolo-generic
+  width: 320 # <--- should match the imgsize of the model, typically 320
+  height: 320 # <--- should match the imgsize of the model, typically 320
+  path: /config/model_cache/yolov9-s-relu6-best_320_int8_edgetpu.tflite
+  labelmap_path: /config/labels-coco17.txt
+```
+
+Note that due to hardware limitations of the Coral, the labelmap is a subset of the COCO labels and includes only 17 object classes.
+
+</details>
+
 ---

 ## Hailo-8
@@ -243,41 +297,55 @@ Hailo8 supports all models in the Hailo Model Zoo that include HailoRT post-proc

 ## OpenVINO Detector

-The OpenVINO detector type runs an OpenVINO IR model on AMD and Intel CPUs, Intel GPUs and Intel VPU hardware. To configure an OpenVINO detector, set the `"type"` attribute to `"openvino"`.
+The OpenVINO detector type runs an OpenVINO IR model on AMD and Intel CPUs, Intel GPUs and Intel NPUs. To configure an OpenVINO detector, set the `"type"` attribute to `"openvino"`.

-The OpenVINO device to be used is specified using the `"device"` attribute according to the naming conventions in the [Device Documentation](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes.html). The most common devices are `CPU` and `GPU`. Currently, there is a known issue with using `AUTO`. For backwards compatibility, Frigate will attempt to use `GPU` if `AUTO` is set in your configuration.
+The OpenVINO device to be used is specified using the `"device"` attribute according to the naming conventions in the [Device Documentation](https://docs.openvino.ai/2025/openvino-workflow/running-inference/inference-devices-and-modes.html). The most common devices are `CPU`, `GPU`, or `NPU`.

-OpenVINO is supported on 6th Gen Intel platforms (Skylake) and newer. It will also run on AMD CPUs despite having no official support for it. A supported Intel platform is required to use the `GPU` device with OpenVINO. For detailed system requirements, see [OpenVINO System Requirements](https://docs.openvino.ai/2024/about-openvino/release-notes-openvino/system-requirements.html)
+OpenVINO is supported on 6th Gen Intel platforms (Skylake) and newer. It will also run on AMD CPUs despite having no official support for it. A supported Intel platform is required to use the `GPU` or `NPU` device with OpenVINO. For detailed system requirements, see [OpenVINO System Requirements](https://docs.openvino.ai/2025/about-openvino/release-notes-openvino/system-requirements.html)

 :::tip

+**NPU + GPU Systems:** If you have both NPU and GPU available (Intel Core Ultra processors), use NPU for object detection and GPU for enrichments (semantic search, face recognition, etc.) for best performance and compatibility.
+
 When using many cameras one detector may not be enough to keep up. Multiple detectors can be defined assuming GPU resources are available. An example configuration would be:

 ```yaml
 detectors:
  ov_0:
    type: openvino
-    device: GPU
+    device: GPU # or NPU
  ov_1:
    type: openvino
-    device: GPU
+    device: GPU # or NPU
 ```

 :::

-### Supported Models
+### OpenVINO Supported Models
+
+| Model                                 | GPU | NPU | Notes                                                        |
+| ------------------------------------- | --- | --- | ------------------------------------------------------------ |
+| [YOLOv9](#yolo-v3-v4-v7-v9)           | ✅  | ✅  | Recommended for GPU & NPU                                    |
+| [RF-DETR](#rf-detr)                   | ✅  | ✅  | Requires XE iGPU or Arc                                      |
+| [YOLO-NAS](#yolo-nas)                 | ✅  | ✅  |                                                              |
+| [MobileNet v2](#ssdlite-mobilenet-v2) | ✅  | ✅  | Fast and lightweight model, less accurate than larger models |
+| [YOLOX](#yolox)                       | ✅  | ?   |                                                              |
+| [D-FINE](#d-fine)                     | ❌  | ❌  |                                                              |

 #### SSDLite MobileNet v2

 An OpenVINO model is provided in the container at `/openvino-model/ssdlite_mobilenet_v2.xml` and is used by this detector type by default. The model comes from Intel's Open Model Zoo [SSDLite MobileNet V2](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssdlite_mobilenet_v2) and is converted to an FP16 precision IR model.

+<details>
+  <summary>MobileNet v2 Config</summary>
+
 Use the model configuration shown below when using the OpenVINO detector with the default OpenVINO model:

 ```yaml
 detectors:
  ov:
    type: openvino
-    device: GPU
+    device: GPU # Or NPU

 model:
  width: 300
@@ -288,6 +356,8 @@ model:
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
 ```

+</details>
+
 #### YOLOX

 This detector also supports YOLOX. Frigate does not come with any YOLOX models preloaded, so you will need to supply your own models.
@@ -296,6 +366,9 @@ This detector also supports YOLOX. Frigate does not come with any YOLOX models p

 [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) models are supported, but not included by default. See [the models section](#downloading-yolo-nas-model) for more information on downloading the YOLO-NAS model for use in Frigate.

+<details>
+  <summary>YOLO-NAS Setup & Config</summary>
+
 After placing the downloaded onnx model in your config folder, you can use the following configuration:

 ```yaml
@@ -316,6 +389,8 @@ model:

 Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.

+</details>
+
 #### YOLO (v3, v4, v7, v9)

 YOLOv3, YOLOv4, YOLOv7, and [YOLOv9](https://github.com/WongKinYiu/yolov9) models are supported, but not included by default.
@@ -326,9 +401,12 @@ The YOLO detector has been designed to support YOLOv3, YOLOv4, YOLOv7, and YOLOv

 :::

+<details>
+  <summary>YOLOv Setup & Config</summary>
+
 :::warning

-If you are using a Frigate+ YOLOv9 model, you should not define any of the below `model` parameters in your config except for `path`. See [the Frigate+ model docs](/plus/first_model#step-3-set-your-model-id-in-the-config) for more information on setting up your model.
+If you are using a Frigate+ model, you should not define any of the below `model` parameters in your config except for `path`. See [the Frigate+ model docs](/plus/first_model#step-3-set-your-model-id-in-the-config) for more information on setting up your model.

 :::

@@ -338,7 +416,7 @@ After placing the downloaded onnx model in your config folder, you can use the f
 detectors:
  ov:
    type: openvino
-    device: GPU
+    device: GPU # or NPU

 model:
  model_type: yolo-generic
@@ -352,6 +430,8 @@ model:

 Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.

+</details>
+
 #### RF-DETR

 [RF-DETR](https://github.com/roboflow/rf-detr) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-rf-detr-model) for more informatoin on downloading the RF-DETR model for use in Frigate.
@@ -362,6 +442,9 @@ Due to the size and complexity of the RF-DETR model, it is only recommended to b

 :::

+<details>
+  <summary>RF-DETR Setup & Config</summary>
+
 After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:

 ```yaml
@@ -379,6 +462,8 @@ model:
  path: /config/model_cache/rfdetr.onnx
 ```

+</details>
+
 #### D-FINE

 [D-FINE](https://github.com/Peterande/D-FINE) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-d-fine-model) for more information on downloading the D-FINE model for use in Frigate.
@@ -389,6 +474,9 @@ Currently D-FINE models only run on OpenVINO in CPU mode, GPUs currently fail to

 :::

+<details>
+  <summary>D-FINE Setup & Config</summary>
+
 After placing the downloaded onnx model in your config/model_cache folder, you can use the following configuration:

 ```yaml
@@ -403,7 +491,63 @@ model:
  height: 640
  input_tensor: nchw
  input_dtype: float
-  path: /config/model_cache/dfine_s_obj2coco.onnx
+  path: /config/model_cache/dfine-s.onnx
+  labelmap_path: /labelmap/coco-80.txt
+```
+
+Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.
+
+</details>
+
+## Apple Silicon detector
+
+The NPU in Apple Silicon can't be accessed from within a container, so the [Apple Silicon detector client](https://github.com/frigate-nvr/apple-silicon-detector) must first be setup. It is recommended to use the Frigate docker image with `-standard-arm64` suffix, for example `ghcr.io/blakeblackshear/frigate:stable-standard-arm64`.
+
+### Setup
+
+1. Setup the [Apple Silicon detector client](https://github.com/frigate-nvr/apple-silicon-detector) and run the client
+2. Configure the detector in Frigate and startup Frigate
+
+### Configuration
+
+Using the detector config below will connect to the client:
+
+```yaml
+detectors:
+  apple-silicon:
+    type: zmq
+    endpoint: tcp://host.docker.internal:5555
+```
+
+### Apple Silicon Supported Models
+
+There is no default model provided, the following formats are supported:
+
+#### YOLO (v3, v4, v7, v9)
+
+YOLOv3, YOLOv4, YOLOv7, and [YOLOv9](https://github.com/WongKinYiu/yolov9) models are supported, but not included by default.
+
+:::tip
+
+The YOLO detector has been designed to support YOLOv3, YOLOv4, YOLOv7, and YOLOv9 models, but may support other YOLO model architectures as well. See [the models section](#downloading-yolo-models) for more information on downloading YOLO models for use in Frigate.
+
+:::
+
+When Frigate is started with the following config it will connect to the detector client and transfer the model automatically:
+
+```yaml
+detectors:
+  apple-silicon:
+    type: zmq
+    endpoint: tcp://host.docker.internal:5555
+
+model:
+  model_type: yolo-generic
+  width: 320 # <--- should match the imgsize set during model export
+  height: 320 # <--- should match the imgsize set during model export
+  input_tensor: nchw
+  input_dtype: float
+  path: /config/model_cache/yolo.onnx
  labelmap_path: /labelmap/coco-80.txt
 ```

@@ -489,7 +633,18 @@ We unset the `HSA_OVERRIDE_GFX_VERSION` to prevent an existing override from mes
 $ docker exec -it frigate /bin/bash -c '(unset HSA_OVERRIDE_GFX_VERSION && /opt/rocm/bin/rocminfo |grep gfx)'
 ```

-### Supported Models
+### ROCm Supported Models
+
+:::tip
+
+The AMD GPU kernel is known problematic especially when converting models to mxr format. The recommended approach is:
+
+1. Disable object detection in the config.
+2. Startup Frigate with the onnx detector configured, the main object detection model will be converted to mxr format and cached in the config directory.
+3. Once this is finished as indicated by the logs, enable object detection in the UI and confirm that it is working correctly.
+4. Re-enable object detection in the config.
+
+:::

 See [ONNX supported models](#supported-models) for supported models, there are some caveats:

@@ -505,11 +660,9 @@ ONNX is an open format for building machine learning models, Frigate supports ru
 If the correct build is used for your GPU then the GPU will be detected and used automatically.

 - **AMD**
-
  - ROCm will automatically be detected and used with the ONNX detector in the `-rocm` Frigate image.

 - **Intel**
-
  - OpenVINO will automatically be detected and used with the ONNX detector in the default Frigate image.

 - **Nvidia**
@@ -532,7 +685,15 @@ detectors:

 :::

-### Supported Models
+### ONNX Supported Models
+
+| Model                         | Nvidia GPU | AMD GPU | Notes                                               |
+| ----------------------------- | ---------- | ------- | --------------------------------------------------- |
+| [YOLOv9](#yolo-v3-v4-v7-v9-2) | ✅         | ✅      | Supports CUDA Graphs for optimal Nvidia performance |
+| [RF-DETR](#rf-detr)           | ✅         | ❌      | Supports CUDA Graphs for optimal Nvidia performance |
+| [YOLO-NAS](#yolo-nas-1)       | ⚠️         | ⚠️      | Not supported by CUDA Graphs                        |
+| [YOLOX](#yolox-1)             | ✅         | ✅      | Supports CUDA Graphs for optimal Nvidia performance |
+| [D-FINE](#d-fine)             | ⚠️         | ❌      | Not supported by CUDA Graphs                        |

 There is no default model provided, the following formats are supported:

@@ -540,6 +701,9 @@ There is no default model provided, the following formats are supported:

 [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) models are supported, but not included by default. See [the models section](#downloading-yolo-nas-model) for more information on downloading the YOLO-NAS model for use in Frigate.

+<details>
+  <summary>YOLO-NAS Setup & Config</summary>
+
 :::warning

 If you are using a Frigate+ YOLO-NAS model, you should not define any of the below `model` parameters in your config except for `path`. See [the Frigate+ model docs](/plus/first_model#step-3-set-your-model-id-in-the-config) for more information on setting up your model.
@@ -563,6 +727,8 @@ model:
  labelmap_path: /labelmap/coco-80.txt
 ```

+</details>
+
 #### YOLO (v3, v4, v7, v9)

 YOLOv3, YOLOv4, YOLOv7, and [YOLOv9](https://github.com/WongKinYiu/yolov9) models are supported, but not included by default.
@@ -573,9 +739,12 @@ The YOLO detector has been designed to support YOLOv3, YOLOv4, YOLOv7, and YOLOv

 :::

+<details>
+  <summary>YOLOv Setup & Config</summary>
+
 :::warning

-If you are using a Frigate+ YOLOv9 model, you should not define any of the below `model` parameters in your config except for `path`. See [the Frigate+ model docs](/plus/first_model#step-3-set-your-model-id-in-the-config) for more information on setting up your model.
+If you are using a Frigate+ model, you should not define any of the below `model` parameters in your config except for `path`. See [the Frigate+ model docs](/plus/first_model#step-3-set-your-model-id-in-the-config) for more information on setting up your model.

 :::

@@ -596,12 +765,17 @@ model:
  labelmap_path: /labelmap/coco-80.txt
 ```

+</details>
+
 Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.

 #### YOLOx

 [YOLOx](https://github.com/Megvii-BaseDetection/YOLOX) models are supported, but not included by default. See [the models section](#downloading-yolo-models) for more information on downloading the YOLOx model for use in Frigate.

+<details>
+  <summary>YOLOx Setup & Config</summary>
+
 After placing the downloaded onnx model in your config folder, you can use the following configuration:

 ```yaml
@@ -621,10 +795,15 @@ model:

 Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.

+</details>
+
 #### RF-DETR

 [RF-DETR](https://github.com/roboflow/rf-detr) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-rf-detr-model) for more information on downloading the RF-DETR model for use in Frigate.

+<details>
+  <summary>RF-DETR Setup & Config</summary>
+
 After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:

 ```yaml
@@ -641,10 +820,15 @@ model:
  path: /config/model_cache/rfdetr.onnx
 ```

+</details>
+
 #### D-FINE

 [D-FINE](https://github.com/Peterande/D-FINE) is a DETR based model. The ONNX exported models are supported, but not included by default. See [the models section](#downloading-d-fine-model) for more information on downloading the D-FINE model for use in Frigate.

+<details>
+  <summary>D-FINE Setup & Config</summary>
+
 After placing the downloaded onnx model in your `config/model_cache` folder, you can use the following configuration:

 ```yaml
@@ -662,6 +846,8 @@ model:
  labelmap_path: /labelmap/coco-80.txt
 ```

+</details>
+
 Note that the labelmap uses a subset of the complete COCO label set that has only 80 objects.

 ## CPU Detector (not recommended)
@@ -717,6 +903,196 @@ To verify that the integration is working correctly, start Frigate and observe t

 # Community Supported Detectors

+## MemryX MX3
+
+This detector is available for use with the MemryX MX3 accelerator M.2 module. Frigate supports the MX3 on compatible hardware platforms, providing efficient and high-performance object detection.
+
+See the [installation docs](../frigate/installation.md#memryx-mx3) for information on configuring the MemryX hardware.
+
+To configure a MemryX detector, simply set the `type` attribute to `memryx` and follow the configuration guide below.
+
+### Configuration
+
+To configure the MemryX detector, use the following example configuration:
+
+#### Single PCIe MemryX MX3
+
+```yaml
+detectors:
+  memx0:
+    type: memryx
+    device: PCIe:0
+```
+
+#### Multiple PCIe MemryX MX3 Modules
+
+```yaml
+detectors:
+  memx0:
+    type: memryx
+    device: PCIe:0
+
+  memx1:
+    type: memryx
+    device: PCIe:1
+
+  memx2:
+    type: memryx
+    device: PCIe:2
+```
+
+### Supported Models
+
+MemryX `.dfp` models are automatically downloaded at runtime, if enabled, to the container at `/memryx_models/model_folder/`.
+
+#### YOLO-NAS
+
+The [YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) model included in this detector is downloaded from the [Models Section](#downloading-yolo-nas-model) and compiled to DFP with [mx_nc](https://developer.memryx.com/tools/neural_compiler.html#usage).
+
+**Note:** The default model for the MemryX detector is YOLO-NAS 320x320.
+
+The input size for **YOLO-NAS** can be set to either **320x320** (default) or **640x640**.
+
+- The default size of **320x320** is optimized for lower CPU usage and faster inference times.
+
+##### Configuration
+
+Below is the recommended configuration for using the **YOLO-NAS** (small) model with the MemryX detector:
+
+```yaml
+detectors:
+  memx0:
+    type: memryx
+    device: PCIe:0
+
+model:
+  model_type: yolonas
+  width: 320 # (Can be set to 640 for higher resolution)
+  height: 320 # (Can be set to 640 for higher resolution)
+  input_tensor: nchw
+  input_dtype: float
+  labelmap_path: /labelmap/coco-80.txt
+  # Optional: The model is normally fetched through the runtime, so 'path' can be omitted unless you want to use a custom or local model.
+  # path: /config/yolonas.zip
+  # The .zip file must contain:
+  # ├── yolonas.dfp          (a file ending with .dfp)
+  # └── yolonas_post.onnx    (optional; only if the model includes a cropped post-processing network)
+```
+
+#### YOLOv9
+
+The YOLOv9s model included in this detector is downloaded from [the original GitHub](https://github.com/WongKinYiu/yolov9) like in the [Models Section](#yolov9-1) and compiled to DFP with [mx_nc](https://developer.memryx.com/tools/neural_compiler.html#usage).
+
+##### Configuration
+
+Below is the recommended configuration for using the **YOLOv9** (small) model with the MemryX detector:
+
+```yaml
+detectors:
+  memx0:
+    type: memryx
+    device: PCIe:0
+
+model:
+  model_type: yolo-generic
+  width: 320 # (Can be set to 640 for higher resolution)
+  height: 320 # (Can be set to 640 for higher resolution)
+  input_tensor: nchw
+  input_dtype: float
+  labelmap_path: /labelmap/coco-80.txt
+  # Optional: The model is normally fetched through the runtime, so 'path' can be omitted unless you want to use a custom or local model.
+  # path: /config/yolov9.zip
+  # The .zip file must contain:
+  # ├── yolov9.dfp          (a file ending with .dfp)
+```
+
+#### YOLOX
+
+The model is sourced from the [OpenCV Model Zoo](https://github.com/opencv/opencv_zoo) and precompiled to DFP.
+
+##### Configuration
+
+Below is the recommended configuration for using the **YOLOX** (small) model with the MemryX detector:
+
+```yaml
+detectors:
+  memx0:
+    type: memryx
+    device: PCIe:0
+
+model:
+  model_type: yolox
+  width: 640
+  height: 640
+  input_tensor: nchw
+  input_dtype: float_denorm
+  labelmap_path: /labelmap/coco-80.txt
+  # Optional: The model is normally fetched through the runtime, so 'path' can be omitted unless you want to use a custom or local model.
+  # path: /config/yolox.zip
+  # The .zip file must contain:
+  # ├── yolox.dfp          (a file ending with .dfp)
+```
+
+#### SSDLite MobileNet v2
+
+The model is sourced from the [OpenMMLab Model Zoo](https://mmdeploy-oss.openmmlab.com/model/mmdet-det/ssdlite-e8679f.onnx) and has been converted to DFP.
+
+##### Configuration
+
+Below is the recommended configuration for using the **SSDLite MobileNet v2** model with the MemryX detector:
+
+```yaml
+detectors:
+  memx0:
+    type: memryx
+    device: PCIe:0
+
+model:
+  model_type: ssd
+  width: 320
+  height: 320
+  input_tensor: nchw
+  input_dtype: float
+  labelmap_path: /labelmap/coco-80.txt
+  # Optional: The model is normally fetched through the runtime, so 'path' can be omitted unless you want to use a custom or local model.
+  # path: /config/ssdlite_mobilenet.zip
+  # The .zip file must contain:
+  # ├── ssdlite_mobilenet.dfp          (a file ending with .dfp)
+  # └── ssdlite_mobilenet_post.onnx    (optional; only if the model includes a cropped post-processing network)
+```
+
+#### Using a Custom Model
+
+To use your own model:
+
+1.  Package your compiled model into a `.zip` file.
+
+2.  The `.zip` must contain the compiled `.dfp` file.
+
+3.  Depending on the model, the compiler may also generate a cropped post-processing network. If present, it will be named with the suffix `_post.onnx`.
+
+4.  Bind-mount the `.zip` file into the container and specify its path using `model.path` in your config.
+
+5.  Update the `labelmap_path` to match your custom model's labels.
+
+For detailed instructions on compiling models, refer to the [MemryX Compiler](https://developer.memryx.com/tools/neural_compiler.html#usage) docs and [Tutorials](https://developer.memryx.com/tutorials/tutorials.html).
+
+```yaml
+# The detector automatically selects the default model if nothing is provided in the config.
+#
+# Optionally, you can specify a local model path as a .zip file to override the default.
+# If a local path is provided and the file exists, it will be used instead of downloading.
+#
+# Example:
+# path: /config/yolonas.zip
+#
+# The .zip file must contain:
+# ├── yolonas.dfp          (a file ending with .dfp)
+# └── yolonas_post.onnx    (optional; only if the model includes a cropped post-processing network)
+```
+
+---
+
 ## NVidia TensorRT Detector

 Nvidia Jetson devices may be used for object detection using the TensorRT libraries. Due to the size of the additional libraries, this detector is only provided in images with the `-tensorrt-jp6` tag suffix, e.g. `ghcr.io/blakeblackshear/frigate:stable-tensorrt-jp6`. This detector is designed to work with Yolo models for object detection.
@@ -799,6 +1175,41 @@ model:
  height: 320 # MUST match the chosen model i.e yolov7-320 -> 320 yolov4-416 -> 416
 ```

+## Synaptics
+
+Hardware accelerated object detection is supported on the following SoCs:
+
+- SL1680
+
+This implementation uses the [Synaptics model conversion](https://synaptics-synap.github.io/doc/v/latest/docs/manual/introduction.html#offline-model-conversion), version v3.1.0.
+
+This implementation is based on sdk `v1.5.0`.
+
+See the [installation docs](../frigate/installation.md#synaptics) for information on configuring the SL-series NPU hardware.
+
+### Configuration
+
+When configuring the Synap detector, you have to specify the model: a local **path**.
+
+#### SSD Mobilenet
+
+A synap model is provided in the container at /mobilenet.synap and is used by this detector type by default. The model comes from [Synap-release Github](https://github.com/synaptics-astra/synap-release/tree/v1.5.0/models/dolphin/object_detection/coco/model/mobilenet224_full80).
+
+Use the model configuration shown below when using the synaptics detector with the default synap model:
+
+```yaml
+detectors: # required
+  synap_npu: # required
+    type: synaptics # required
+
+model: # required
+  path: /synaptics/mobilenet.synap # required
+  width: 224 # required
+  height: 224 # required
+  tensor_format: nhwc # default value (optional. If you change the model, it is required)
+  labelmap_path: /labelmap/coco-80.txt # required
+```
+
 ## Rockchip platform

 Hardware accelerated object detection is supported on the following SoCs:
@@ -842,7 +1253,7 @@ $ cat /sys/kernel/debug/rknpu/load

 :::

-### Supported Models
+### RockChip Supported Models

 This `config.yml` shows all relevant options to configure the detector and explains them. All values shown are the default values (except for two). Lines that are required at least to use the detector are labeled as required, all other lines are optional.

@@ -968,6 +1379,105 @@ Explanation of the paramters:
  - **example**: Specifying `output_name = "frigate-{quant}-{input_basename}-{soc}-v{tk_version}"` could result in a model called `frigate-i8-my_model-rk3588-v2.3.0.rknn`.
 - `config`: Configuration passed to `rknn-toolkit2` for model conversion. For an explanation of all available parameters have a look at section "2.2. Model configuration" of [this manual](https://github.com/MarcA711/rknn-toolkit2/releases/download/v2.3.2/03_Rockchip_RKNPU_API_Reference_RKNN_Toolkit2_V2.3.2_EN.pdf).

+## DeGirum
+
+DeGirum is a detector that can use any type of hardware listed on [their website](https://hub.degirum.com). DeGirum can be used with local hardware through a DeGirum AI Server, or through the use of `@local`. You can also connect directly to DeGirum's AI Hub to run inferences. **Please Note:** This detector _cannot_ be used for commercial purposes.
+
+### Configuration
+
+#### AI Server Inference
+
+Before starting with the config file for this section, you must first launch an AI server. DeGirum has an AI server ready to use as a docker container. Add this to your `docker-compose.yml` to get started:
+
+```yaml
+degirum_detector:
+  container_name: degirum
+  image: degirum/aiserver:latest
+  privileged: true
+  ports:
+    - "8778:8778"
+```
+
+All supported hardware will automatically be found on your AI server host as long as relevant runtimes and drivers are properly installed on your machine. Refer to [DeGirum's docs site](https://docs.degirum.com/pysdk/runtimes-and-drivers) if you have any trouble.
+
+Once completed, changing the `config.yml` file is simple.
+
+```yaml
+degirum_detector:
+  type: degirum
+  location: degirum # Set to service name (degirum_detector), container_name (degirum), or a host:port (192.168.29.4:8778)
+  zoo: degirum/public # DeGirum's public model zoo. Zoo name should be in format "workspace/zoo_name". degirum/public is available to everyone, so feel free to use it if you don't know where to start. If you aren't pulling a model from the AI Hub, leave this and 'token' blank.
+  token: dg_example_token # For authentication with the AI Hub. Get this token through the "tokens" section on the main page of the [AI Hub](https://hub.degirum.com). This can be left blank if you're pulling a model from the public zoo and running inferences on your local hardware using @local or a local DeGirum AI Server
+```
+
+Setting up a model in the `config.yml` is similar to setting up an AI server.
+You can set it to:
+
+- A model listed on the [AI Hub](https://hub.degirum.com), given that the correct zoo name is listed in your detector
+  - If this is what you choose to do, the correct model will be downloaded onto your machine before running.
+- A local directory acting as a zoo. See DeGirum's docs site [for more information](https://docs.degirum.com/pysdk/user-guide-pysdk/organizing-models#model-zoo-directory-structure).
+- A path to some model.json.
+
+```yaml
+model:
+  path: ./mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1 # directory to model .json and file
+  width: 300 # width is in the model name as the first number in the "int"x"int" section
+  height: 300 # height is in the model name as the second number in the "int"x"int" section
+  input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
+```
+
+#### Local Inference
+
+It is also possible to eliminate the need for an AI server and run the hardware directly. The benefit of this approach is that you eliminate any bottlenecks that occur when transferring prediction results from the AI server docker container to the frigate one. However, the method of implementing local inference is different for every device and hardware combination, so it's usually more trouble than it's worth. A general guideline to achieve this would be:
+
+1. Ensuring that the frigate docker container has the runtime you want to use. So for instance, running `@local` for Hailo means making sure the container you're using has the Hailo runtime installed.
+2. To double check the runtime is detected by the DeGirum detector, make sure the `degirum sys-info` command properly shows whatever runtimes you mean to install.
+3. Create a DeGirum detector in your `config.yml` file.
+
+```yaml
+degirum_detector:
+  type: degirum
+  location: "@local" # For accessing AI Hub devices and models
+  zoo: degirum/public # DeGirum's public model zoo. Zoo name should be in format "workspace/zoo_name". degirum/public is available to everyone, so feel free to use it if you don't know where to start.
+  token: dg_example_token # For authentication with the AI Hub. Get this token through the "tokens" section on the main page of the [AI Hub](https://hub.degirum.com). This can be left blank if you're pulling a model from the public zoo and running inferences on your local hardware using @local or a local DeGirum AI Server
+```
+
+Once `degirum_detector` is setup, you can choose a model through 'model' section in the `config.yml` file.
+
+```yaml
+model:
+  path: mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
+  width: 300 # width is in the model name as the first number in the "int"x"int" section
+  height: 300 # height is in the model name as the second number in the "int"x"int" section
+  input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
+```
+
+#### AI Hub Cloud Inference
+
+If you do not possess whatever hardware you want to run, there's also the option to run cloud inferences. Do note that your detection fps might need to be lowered as network latency does significantly slow down this method of detection. For use with Frigate, we highly recommend using a local AI server as described above. To set up cloud inferences,
+
+1. Sign up at [DeGirum's AI Hub](https://hub.degirum.com).
+2. Get an access token.
+3. Create a DeGirum detector in your `config.yml` file.
+
+```yaml
+degirum_detector:
+  type: degirum
+  location: "@cloud" # For accessing AI Hub devices and models
+  zoo: degirum/public # DeGirum's public model zoo. Zoo name should be in format "workspace/zoo_name". degirum/public is available to everyone, so feel free to use it if you don't know where to start.
+  token: dg_example_token # For authentication with the AI Hub. Get this token through the "tokens" section on the main page of the (AI Hub)[https://hub.degirum.com).
+```
+
+Once `degirum_detector` is setup, you can choose a model through 'model' section in the `config.yml` file.
+
+```yaml
+model:
+  path: mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1
+  width: 300 # width is in the model name as the first number in the "int"x"int" section
+  height: 300 # height is in the model name as the second number in the "int"x"int" section
+  input_pixel_format: rgb/bgr # look at the model.json to figure out which to put here
+```
+
 # Models

 Some model types are not included in Frigate by default.
@@ -1008,11 +1518,11 @@ RF-DETR can be exported as ONNX by running the command below. You can copy and p

 ```sh
 docker build . --build-arg MODEL_SIZE=Nano --rm --output . -f- <<'EOF'
-FROM python:3.11 AS build
+FROM python:3.12 AS build
 RUN apt-get update && apt-get install --no-install-recommends -y libgl1 && rm -rf /var/lib/apt/lists/*
-COPY --from=ghcr.io/astral-sh/uv:0.8.0 /uv /bin/
+COPY --from=ghcr.io/astral-sh/uv:0.10.4 /uv /bin/
 WORKDIR /rfdetr
-RUN uv pip install --system rfdetr[onnxexport] torch==2.8.0 onnx==1.19.1 onnxscript
+RUN uv pip install --system rfdetr[onnxexport] torch==2.8.0 onnx==1.19.1 transformers==4.57.6 onnxscript
 ARG MODEL_SIZE
 RUN python3 -c "from rfdetr import RFDETR${MODEL_SIZE}; x = RFDETR${MODEL_SIZE}(resolution=320); x.export(simplify=True)"
 FROM scratch
@@ -1050,7 +1560,11 @@ cd tensorrt_demos/yolo
 python3 yolo_to_onnx.py -m yolov7-320
 ```

-#### YOLOv9
+#### YOLOv9 for Google Coral Support
+
+[Download the model](https://github.com/dbro/frigate-detector-edgetpu-yolo9/releases/download/v1.0/yolov9-s-relu6-best_320_int8_edgetpu.tflite), bind mount the file into the container, and provide the path with `model.path`. Note that the linked model requires a 17-label [labelmap file](https://raw.githubusercontent.com/dbro/frigate-detector-edgetpu-yolo9/refs/heads/main/labels-coco17.txt) that includes only 17 COCO classes.
+
+#### YOLOv9 for other detectors

 YOLOv9 model can be exported as ONNX using the command below. You can copy and paste the whole thing to your terminal and execute, altering `MODEL_SIZE=t` and `IMG_SIZE=320` in the first line to the [model size](https://github.com/WongKinYiu/yolov9#performance) you would like to convert (available model sizes are `t`, `s`, `m`, `c`, and `e`, common image sizes are `320` and `640`).

@@ -13,34 +13,34 @@ H265 recordings can be viewed in Chrome 108+, Edge and Safari only. All other br

 ### Most conservative: Ensure all video is saved

-For users deploying Frigate in environments where it is important to have contiguous video stored even if there was no detectable motion, the following config will store all video for 3 days. After 3 days, only video containing motion and overlapping with alerts or detections will be retained until 30 days have passed.
+For users deploying Frigate in environments where it is important to have contiguous video stored even if there was no detectable motion, the following config will store all video for 3 days. After 3 days, only video containing motion will be saved for 7 days. After 7 days, only video containing motion and overlapping with alerts or detections will be retained until 30 days have passed.

 ```yaml
 record:
  enabled: True
-  retain:
+  continuous:
    days: 3
-    mode: all
+  motion:
+    days: 7
  alerts:
    retain:
      days: 30
-      mode: motion
+      mode: all
  detections:
    retain:
      days: 30
-      mode: motion
+      mode: all
 ```

 ### Reduced storage: Only saving video when motion is detected

-In order to reduce storage requirements, you can adjust your config to only retain video where motion was detected.
+In order to reduce storage requirements, you can adjust your config to only retain video where motion / activity was detected.

 ```yaml
 record:
  enabled: True
-  retain:
+  motion:
    days: 3
-    mode: motion
  alerts:
    retain:
      days: 30
@@ -53,12 +53,12 @@ record:

 ### Minimum: Alerts only

-If you only want to retain video that occurs during a tracked object, this config will discard video unless an alert is ongoing.
+If you only want to retain video that occurs during activity caused by tracked object(s), this config will discard video unless an alert is ongoing.

 ```yaml
 record:
  enabled: True
-  retain:
+  continuous:
    days: 0
  alerts:
    retain:
@@ -80,15 +80,17 @@ Retention configs support decimals meaning they can be configured to retain `0.5

 :::

-### Continuous Recording
+### Continuous and Motion Recording

-The number of days to retain continuous recordings can be set via the following config where X is a number, by default continuous recording is disabled.
+The number of days to retain continuous and motion recordings can be set via the following config where X is a number, by default continuous recording is disabled.

 ```yaml
 record:
  enabled: True
-  retain:
+  continuous:
    days: 1 # <- number of days to keep continuous recordings
+  motion:
+    days: 2 # <- number of days to keep motion recordings
 ```

 Continuous recording supports different retention modes [which are described below](#what-do-the-different-retain-modes-mean)
@@ -112,38 +114,6 @@ This configuration will retain recording segments that overlap with alerts and d

 **WARNING**: Recordings still must be enabled in the config. If a camera has recordings disabled in the config, enabling via the methods listed above will have no effect.

-## What do the different retain modes mean?
-
-Frigate saves from the stream with the `record` role in 10 second segments. These options determine which recording segments are kept for continuous recording (but can also affect tracked objects).
-
-Let's say you have Frigate configured so that your doorbell camera would retain the last **2** days of continuous recording.
-
- With the `all` option all 48 hours of those two days would be kept and viewable.
- With the `motion` option the only parts of those 48 hours would be segments that Frigate detected motion. This is the middle ground option that won't keep all 48 hours, but will likely keep all segments of interest along with the potential for some extra segments.
- With the `active_objects` option the only segments that would be kept are those where there was a true positive object that was not considered stationary.
-
-The same options are available with alerts and detections, except it will only save the recordings when it overlaps with a review item of that type.
-
-A configuration example of the above retain modes where all `motion` segments are stored for 7 days and `active objects` are stored for 14 days would be as follows:
-
-```yaml
-record:
-  enabled: True
-  retain:
-    days: 7
-    mode: motion
-  alerts:
-    retain:
-      days: 14
-      mode: active_objects
-  detections:
-    retain:
-      days: 14
-      mode: active_objects
-```
-
-The above configuration example can be added globally or on a per camera basis.
-
 ## Can I have "continuous" recordings, but only at certain times?

 Using Frigate UI, Home Assistant, or MQTT, cameras can be automated to only record in certain situations or at certain times.
@@ -73,6 +73,12 @@ tls:
  # Optional: Enable TLS for port 8971 (default: shown below)
  enabled: True

+# Optional: IPv6 configuration
+networking:
+  # Optional: Enable IPv6 on 5000, and 8971 if tls is configured (default: shown below)
+  ipv6:
+    enabled: False
+
 # Optional: Proxy configuration
 proxy:
  # Optional: Mapping for headers from upstream proxies. Only used if Frigate's auth
@@ -82,7 +88,13 @@ proxy:
  #       See the docs for more info.
  header_map:
    user: x-forwarded-user
-    role: x-forwarded-role
+    role: x-forwarded-groups
+    role_map:
+      admin:
+        - sysadmins
+        - access-level-security
+      viewer:
+        - camera-viewer
  # Optional: Url for logging out a user. This sets the location of the logout url in
  # the UI.
  logout_url: /api/logout
@@ -111,7 +123,7 @@ auth:
  # Optional: Refresh time in seconds (default: shown below)
  # When the session is going to expire in less time than this setting,
  # it will be refreshed back to the session_length.
-  refresh_time: 43200 # 12 hours
+  refresh_time: 1800 # 30 minutes
  # Optional: Rate limiting for login failures to help prevent brute force
  # login attacks (default: shown below)
  # See the docs for more information on valid values
@@ -228,11 +240,13 @@ birdseye:
    scaling_factor: 2.0
    # Optional: Maximum number of cameras to show at one time, showing the most recent (default: show all cameras)
    max_cameras: 1
+  # Optional: Frames-per-second to re-send the last composed Birdseye frame when idle (no motion or active updates). (default: shown below)
+  idle_heartbeat_fps: 0.0

 # Optional: ffmpeg configuration
 # More information about presets at https://docs.frigate.video/configuration/ffmpeg_presets
 ffmpeg:
-  # Optional: ffmpeg binry path (default: shown below)
+  # Optional: ffmpeg binary path (default: shown below)
  # can also be set to `7.0` or `5.0` to specify one of the included versions
  # or can be set to any path that holds `bin/ffmpeg` & `bin/ffprobe`
  path: "default"
@@ -256,6 +270,8 @@ ffmpeg:
  retry_interval: 10
  # Optional: Set tag on HEVC (H.265) recording stream to improve compatibility with Apple players. (default: shown below)
  apple_compatibility: false
+  # Optional: Set the index of the GPU to use for hardware acceleration. (default: shown below)
+  gpu: 0

 # Optional: Detect configuration
 # NOTE: Can be overridden at the camera level
@@ -275,6 +291,9 @@ detect:
  max_disappeared: 25
  # Optional: Configuration for stationary object tracking
  stationary:
+    # Optional: Stationary classifier that uses visual characteristics to determine if an object
+    # is stationary even if the box changes enough to be considered motion (default: shown below).
+    classifier: True
    # Optional: Frequency for confirming stationary objects (default: same as threshold)
    # When set to 1, object detection will run to confirm the object still exists on every frame.
    # If set to 10, object detection will run to confirm the object still exists on every 10th frame.
@@ -339,6 +358,33 @@ objects:
      # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
      # Checks based on the bottom center of the bounding box of the object
      mask: 0.000,0.000,0.781,0.000,0.781,0.278,0.000,0.278
+  # Optional: Configuration for AI generated tracked object descriptions
+  genai:
+    # Optional: Enable AI object description generation (default: shown below)
+    enabled: False
+    # Optional: Use the object snapshot instead of thumbnails for description generation (default: shown below)
+    use_snapshot: False
+    # Optional: The default prompt for generating descriptions. Can use replacement
+    # variables like "label", "sub_label", "camera" to make more dynamic. (default: shown below)
+    prompt: "Describe the {label} in the sequence of images with as much detail as possible. Do not describe the background."
+    # Optional: Object specific prompts to customize description results
+    # Format: {label}: {prompt}
+    object_prompts:
+      person: "My special person prompt."
+    # Optional: objects to generate descriptions for (default: all objects that are tracked)
+    objects:
+      - person
+      - cat
+    # Optional: Restrict generation to objects that entered any of the listed zones (default: none, all zones qualify)
+    required_zones: []
+    # Optional: What triggers to use to send frames for a tracked object to generative AI (default: shown below)
+    send_triggers:
+      # Once the object is no longer tracked
+      tracked_object_end: True
+      # Optional: After X many significant updates are received (default: shown below)
+      after_significant_updates: None
+    # Optional: Save thumbnails sent to generative AI for review/debugging purposes (default: shown below)
+    debug_save_thumbnails: False

 # Optional: Review configuration
 # NOTE: Can be overridden at the camera level
@@ -351,6 +397,8 @@ review:
    labels:
      - car
      - person
+    # Time to cutoff alerts after no alert-causing activity has occurred (default: shown below)
+    cutoff_time: 40
    # Optional: required zones for an object to be marked as an alert (default: none)
    # NOTE: when settings required zones globally, this zone must exist on all cameras
    #       or the config will be considered invalid. In that case the required_zones
@@ -365,12 +413,36 @@ review:
    labels:
      - car
      - person
+    # Time to cutoff detections after no detection-causing activity has occurred (default: shown below)
+    cutoff_time: 30
    # Optional: required zones for an object to be marked as a detection (default: none)
    # NOTE: when settings required zones globally, this zone must exist on all cameras
    #       or the config will be considered invalid. In that case the required_zones
    #       should be configured at the camera level.
    required_zones:
      - driveway
+  # Optional: GenAI Review Summary Configuration
+  genai:
+    # Optional: Enable the GenAI review summary feature (default: shown below)
+    enabled: False
+    # Optional: Enable GenAI review summaries for alerts (default: shown below)
+    alerts: True
+    # Optional: Enable GenAI review summaries for detections (default: shown below)
+    detections: False
+    # Optional: Activity Context Prompt to give context to the GenAI what activity is and is not suspicious.
+    # It is important to be direct and detailed. See documentation for the default prompt structure.
+    activity_context_prompt: """Define what is and is not suspicious
+"""
+    # Optional: Image source for GenAI (default: preview)
+    # Options: "preview" (uses cached preview frames at ~180p) or "recordings" (extracts frames from recordings at 480p)
+    # Using "recordings" provides better image quality but uses more tokens per image.
+    # Frame count is automatically calculated based on context window size, aspect ratio, and image source (capped at 20 frames).
+    image_source: preview
+    # Optional: Additional concerns that the GenAI should make note of (default: None)
+    additional_concerns:
+      - Animals in the garden
+    # Optional: Preferred response language (default: English)
+    preferred_language: English

 # Optional: Motion configuration
 # NOTE: Can be overridden at the camera level
@@ -440,18 +512,18 @@ record:
  expire_interval: 60
  # Optional: Two-way sync recordings database with disk on startup and once a day (default: shown below).
  sync_recordings: False
-  # Optional: Retention settings for recording
-  retain:
+  # Optional: Continuous retention settings
+  continuous:
+    # Optional: Number of days to retain recordings regardless of tracked objects or motion (default: shown below)
+    # NOTE: This should be set to 0 and retention should be defined in alerts and detections section below
+    #       if you only want to retain recordings of alerts and detections.
+    days: 0
+  # Optional: Motion retention settings
+  motion:
    # Optional: Number of days to retain recordings regardless of tracked objects (default: shown below)
    # NOTE: This should be set to 0 and retention should be defined in alerts and detections section below
    #       if you only want to retain recordings of alerts and detections.
    days: 0
-    # Optional: Mode for retention. Available options are: all, motion, and active_objects
-    #   all - save all recording segments regardless of activity
-    #   motion - save all recordings segments with any detected motion
-    #   active_objects - save all recording segments with active/moving objects
-    # NOTE: this mode only applies when the days setting above is greater than 0
-    mode: all
  # Optional: Recording Export Settings
  export:
    # Optional: Timelapse Output Args (default: shown below).
@@ -476,7 +548,7 @@ record:
    # Optional: Retention settings for recordings of alerts
    retain:
      # Required: Retention days (default: shown below)
-      days: 14
+      days: 10
      # Optional: Mode for retention. (default: shown below)
      #   all - save all recording segments for alerts regardless of activity
      #   motion - save all recordings segments for alerts with any detected motion
@@ -496,7 +568,7 @@ record:
    # Optional: Retention settings for recordings of detections
    retain:
      # Required: Retention days (default: shown below)
-      days: 14
+      days: 10
      # Optional: Mode for retention. (default: shown below)
      #   all - save all recording segments for detections regardless of activity
      #   motion - save all recordings segments for detections with any detected motion
@@ -513,7 +585,7 @@ record:
 snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  enabled: False
-  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
+  # Optional: save a clean copy of the snapshot image (default: shown below)
  clean_copy: True
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: False
@@ -546,6 +618,9 @@ semantic_search:
  # Optional: Set the model size used for embeddings. (default: shown below)
  # NOTE: small model runs on CPU and large model runs on GPU
  model_size: "small"
+  # Optional: Target a specific device to run the model (default: shown below)
+  # NOTE: See https://onnxruntime.ai/docs/execution-providers/ for more information
+  device: None

 # Optional: Configuration for face recognition capability
 # NOTE: enabled, min_area can be overridden at the camera level
@@ -564,11 +639,14 @@ face_recognition:
  # Optional: Min face recognitions for the sub label to be applied to the person object (default: shown below)
  min_faces: 1
  # Optional: Number of images of recognized faces to save for training (default: shown below)
-  save_attempts: 100
+  save_attempts: 200
  # Optional: Apply a blur quality filter to adjust confidence based on the blur level of the image (default: shown below)
  blur_confidence_filter: True
  # Optional: Set the model size used face recognition. (default: shown below)
  model_size: small
+  # Optional: Target a specific device to run the model (default: shown below)
+  # NOTE: See https://onnxruntime.ai/docs/execution-providers/ for more information
+  device: None

 # Optional: Configuration for license plate recognition capability
 # NOTE: enabled, min_area, and enhancement can be overridden at the camera level
@@ -576,6 +654,7 @@ lpr:
  # Optional: Enable license plate recognition (default: shown below)
  enabled: False
  # Optional: The device to run the models on (default: shown below)
+  # NOTE: See https://onnxruntime.ai/docs/execution-providers/ for more information
  device: CPU
  # Optional: Set the model size used for text detection. (default: shown below)
  model_size: small
@@ -598,30 +677,82 @@ lpr:
  enhancement: 0
  # Optional: Save plate images to /media/frigate/clips/lpr for debugging purposes (default: shown below)
  debug_save_plates: False
+  # Optional: List of regex replacement rules to normalize detected plates (default: shown below)
+  replace_rules: {}

-# Optional: Configuration for AI generated tracked object descriptions
+# Optional: Configuration for AI / LLM provider
 # WARNING: Depending on the provider, this will send thumbnails over the internet
-# to Google or OpenAI's LLMs to generate descriptions. It can be overridden at
-# the camera level (enabled: False) to enhance privacy for indoor cameras.
+# to Google or OpenAI's LLMs to generate descriptions. GenAI features can be configured at
+# the camera level to enhance privacy for indoor cameras.
 genai:
-  # Optional: Enable AI description generation (default: shown below)
-  enabled: False
-  # Required if enabled: Provider must be one of ollama, gemini, or openai
+  # Required: Provider must be one of ollama, gemini, or openai
  provider: ollama
  # Required if provider is ollama. May also be used for an OpenAI API compatible backend with the openai provider.
  base_url: http://localhost::11434
  # Required if gemini or openai
  api_key: "{FRIGATE_GENAI_API_KEY}"
-  # Optional: The default prompt for generating descriptions. Can use replacement
-  # variables like "label", "sub_label", "camera" to make more dynamic. (default: shown below)
-  prompt: "Describe the {label} in the sequence of images with as much detail as possible. Do not describe the background."
-  # Optional: Object specific prompts to customize description results
-  # Format: {label}: {prompt}
-  object_prompts:
-    person: "My special person prompt."
+  # Required: The model to use with the provider.
+  model: gemini-1.5-flash
+  # Optional additional args to pass to the GenAI Provider (default: None)
+  provider_options:
+    keep_alive: -1
+  # Optional: Options to pass during inference calls (default: {})
+  runtime_options:
+    temperature: 0.7
+
+# Optional: Configuration for audio transcription
+# NOTE: only the enabled option can be overridden at the camera level
+audio_transcription:
+  # Optional: Enable live and speech event audio transcription (default: shown below)
+  enabled: False
+  # Optional: The device to run the models on for live transcription. (default: shown below)
+  device: CPU
+  # Optional: Set the model size used for live transcription. (default: shown below)
+  model_size: small
+  # Optional: Set the language used for transcription translation. (default: shown below)
+  # List of language codes: https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10
+  language: en
+
+# Optional: Configuration for classification models
+classification:
+  # Optional: Configuration for bird classification
+  bird:
+    # Optional: Enable bird classification (default: shown below)
+    enabled: False
+    # Optional: Minimum classification score required to be considered a match (default: shown below)
+    threshold: 0.9
+  custom:
+    # Required: name of the classification model
+    model_name:
+      # Optional: Enable running the model (default: shown below)
+      enabled: True
+      # Optional: Name of classification model (default: shown below)
+      name: None
+      # Optional: Classification score threshold to change the state (default: shown below)
+      threshold: 0.8
+      # Optional: Number of classification attempts to save in the recent classifications tab (default: shown below)
+      # NOTE: Defaults to 200 for object classification and 100 for state classification if not specified
+      save_attempts: None
+      # Optional: Object classification configuration
+      object_config:
+        # Required: Object types to classify
+        objects: [dog]
+        # Optional: Type of classification that is applied (default: shown below)
+        classification_type: sub_label
+      # Optional: State classification configuration
+      state_config:
+        # Required: Cameras to run classification on
+        cameras:
+          camera_name:
+            # Required: Crop of image frame on this camera to run classification on
+            crop: [0, 180, 220, 400]
+        # Optional: If classification should be run when motion is detected in the crop (default: shown below)
+        motion: False
+        # Optional: Interval to run classification on in seconds (default: shown below)
+        interval: None

 # Optional: Restream configuration
-# Uses https://github.com/AlexxIT/go2rtc (v1.9.9)
+# Uses https://github.com/AlexxIT/go2rtc (v1.9.10)
 # NOTE: The default go2rtc API port (1984) must be used,
 #       changing this port for the integrated go2rtc instance is not supported.
 go2rtc:
@@ -720,6 +851,8 @@ cameras:
      # NOTE: This must be different than any camera names, but can match with another zone on another
      #       camera.
      front_steps:
+        # Optional: A friendly name or descriptive text for the zones
+        friendly_name: ""
        # Required: List of x,y coordinates to define the polygon of the zone.
        # NOTE: Presence in a zone is evaluated only based on the bottom center of the objects bounding box.
        coordinates: 0.033,0.306,0.324,0.138,0.439,0.185,0.042,0.428
@@ -781,7 +914,7 @@ cameras:
      user: admin
      # Optional: password for login.
      password: admin
-      # Optional: Skip TLS verification from the ONVIF server (default: shown below)
+      # Optional: Skip TLS verification and disable digest authentication for the ONVIF server (default: shown below)
      tls_insecure: False
      # Optional: Ignores time synchronization mismatches between the camera and the server during authentication.
      # Using NTP on both ends is recommended and this should only be set to True in a "safe" environment due to the security risk it represents.
@@ -827,33 +960,27 @@ cameras:
      # By default the cameras are sorted alphabetically.
      order: 0

-    # Optional: Configuration for AI generated tracked object descriptions
-    genai:
-      # Optional: Enable AI description generation (default: shown below)
-      enabled: False
-      # Optional: Use the object snapshot instead of thumbnails for description generation (default: shown below)
-      use_snapshot: False
-      # Optional: The default prompt for generating descriptions. Can use replacement
-      # variables like "label", "sub_label", "camera" to make more dynamic. (default: shown below)
-      prompt: "Describe the {label} in the sequence of images with as much detail as possible. Do not describe the background."
-      # Optional: Object specific prompts to customize description results
-      # Format: {label}: {prompt}
-      object_prompts:
-        person: "My special person prompt."
-      # Optional: objects to generate descriptions for (default: all objects that are tracked)
-      objects:
-        - person
-        - cat
-      # Optional: Restrict generation to objects that entered any of the listed zones (default: none, all zones qualify)
-      required_zones: []
-      # Optional: What triggers to use to send frames for a tracked object to generative AI (default: shown below)
-      send_triggers:
-        # Once the object is no longer tracked
-        tracked_object_end: True
-        # Optional: After X many significant updates are received (default: shown below)
-        after_significant_updates: None
-      # Optional: Save thumbnails sent to generative AI for review/debugging purposes (default: shown below)
-      debug_save_thumbnails: False
+    # Optional: Configuration for triggers to automate actions based on semantic search results.
+    triggers:
+      # Required: Unique identifier for the trigger (generated automatically from friendly_name if not specified).
+      trigger_name:
+        # Required: Enable or disable the trigger. (default: shown below)
+        enabled: true
+        # Optional: A friendly name or descriptive text for the trigger
+        friendly_name: Unique name or descriptive text
+        # Type of trigger, either `thumbnail` for image-based matching or `description` for text-based matching. (default: none)
+        type: thumbnail
+        # Reference data for matching, either an event ID for `thumbnail` or a text string for `description`. (default: none)
+        data: 1751565549.853251-b69j73
+        # Similarity threshold for triggering. (default: shown below)
+        threshold: 0.8
+        # List of actions to perform when the trigger fires. (default: none)
+        # Available options:
+        # - `notification` (send a webpush notification)
+        # - `sub_label` (add trigger friendly name as a sub label to the triggering tracked object)
+        # - `attribute` (add trigger's name and similarity score as a data attribute to the triggering tracked object)
+        actions:
+          - notification

 # Optional
 ui:
@@ -878,10 +1005,6 @@ ui:
  #    full: 8:15:22 PM Mountain Standard Time
  # (default: shown below).
  time_style: medium
-  # Optional: Ability to manually override the date / time styling to use strftime format
-  # https://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html
-  # possible values are shown above (default: not set)
-  strftime_fmt: "%Y/%m/%d %H:%M"
  # Optional: Set the unit system to either "imperial" or "metric" (default: metric)
  # Used in the UI and in MQTT topics
  unit_system: metric
@@ -7,7 +7,7 @@ title: Restream

 Frigate can restream your video feed as an RTSP feed for other applications such as Home Assistant to utilize it at `rtsp://<frigate_host>:8554/<camera_name>`. Port 8554 must be open. [This allows you to use a video feed for detection in Frigate and Home Assistant live view at the same time without having to make two separate connections to the camera](#reduce-connections-to-camera). The video feed is copied from the original video feed directly to avoid re-encoding. This feed does not include any annotation by Frigate.

-Frigate uses [go2rtc](https://github.com/AlexxIT/go2rtc/tree/v1.9.9) to provide its restream and MSE/WebRTC capabilities. The go2rtc config is hosted at the `go2rtc` in the config, see [go2rtc docs](https://github.com/AlexxIT/go2rtc/tree/v1.9.9#configuration) for more advanced configurations and features.
+Frigate uses [go2rtc](https://github.com/AlexxIT/go2rtc/tree/v1.9.10) to provide its restream and MSE/WebRTC capabilities. The go2rtc config is hosted at the `go2rtc` in the config, see [go2rtc docs](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#configuration) for more advanced configurations and features.

 :::note

@@ -24,6 +24,12 @@ birdseye:
  restream: True
 ```

+:::tip
+
+To improve connection speed when using Birdseye via restream you can enable a small idle heartbeat by setting `birdseye.idle_heartbeat_fps` to a low value (e.g. `1–2`). This makes Frigate periodically push the last frame even when no motion is detected, reducing initial connection latency.
+
+:::
+
 ### Securing Restream With Authentication

 The go2rtc restream can be secured with RTSP based username / password authentication. Ex:
@@ -154,6 +160,31 @@ go2rtc:

 See [this comment](https://github.com/AlexxIT/go2rtc/issues/1217#issuecomment-2242296489) for more information.

+## Preventing go2rtc from blocking two-way audio {#two-way-talk-restream}
+
+For cameras that support two-way talk, go2rtc will automatically establish an audio output backchannel when connecting to an RTSP stream. This backchannel blocks access to the camera's audio output for two-way talk functionality, preventing both Frigate and other applications from using it.
+
+To prevent this, you must configure two separate stream instances:
+
+1. One stream instance with `#backchannel=0` for Frigate's viewing, recording, and detection (prevents go2rtc from establishing the blocking backchannel)
+2. A second stream instance without `#backchannel=0` for two-way talk functionality (can be used by Frigate's WebRTC viewer or other applications)
+
+Configuration example:
+
+```yaml
+go2rtc:
+  streams:
+    front_door:
+      - rtsp://user:password@10.0.10.10:554/cam/realmonitor?channel=1&subtype=2#backchannel=0
+    front_door_twoway:
+      - rtsp://user:password@10.0.10.10:554/cam/realmonitor?channel=1&subtype=2
+```
+
+In this configuration:
+
+- `front_door` stream is used by Frigate for viewing, recording, and detection. The `#backchannel=0` parameter prevents go2rtc from establishing the audio output backchannel, so it won't block two-way talk access.
+- `front_door_twoway` stream is used for two-way talk functionality. This stream can be used by Frigate's WebRTC viewer when two-way talk is enabled, or by other applications (like Home Assistant Advanced Camera Card) that need access to the camera's audio output channel.
+
 ## Security: Restricted Stream Sources

 For security reasons, the `echo:`, `expr:`, and `exec:` stream sources are disabled by default in go2rtc. These sources allow arbitrary command execution and can pose security risks if misconfigured.
@@ -175,7 +206,13 @@ Enabling arbitrary exec sources allows execution of arbitrary commands through g

 ## Advanced Restream Configurations

-The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.9#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below:
+The [exec](https://github.com/AlexxIT/go2rtc/tree/v1.9.10#source-exec) source in go2rtc can be used for custom ffmpeg commands. An example is below:
+
+:::warning
+
+The `exec:`, `echo:`, and `expr:` sources are disabled by default for security. You must set `GO2RTC_ALLOW_ARBITRARY_EXEC=true` to use them. See [Security: Restricted Stream Sources](#security-restricted-stream-sources) for more information.
+
+:::

 :::warning

@@ -39,7 +39,7 @@ If you are enabling Semantic Search for the first time, be advised that Frigate

 The [V1 model from Jina](https://huggingface.co/jinaai/jina-clip-v1) has a vision model which is able to embed both images and text into the same vector space, which allows `image -> image` and `text -> image` similarity searches. Frigate uses this model on tracked objects to encode the thumbnail image and store it in the database. When searching for tracked objects via text in the search box, Frigate will perform a `text -> image` similarity search against this embedding. When clicking "Find Similar" in the tracked object detail pane, Frigate will perform an `image -> image` similarity search to retrieve the closest matching thumbnails.

-The V1 text model is used to embed tracked object descriptions and perform searches against them. Descriptions can be created, viewed, and modified on the Explore page when clicking on thumbnail of a tracked object. See [the Generative AI docs](/configuration/genai.md) for more information on how to automatically generate tracked object descriptions.
+The V1 text model is used to embed tracked object descriptions and perform searches against them. Descriptions can be created, viewed, and modified on the Explore page when clicking on thumbnail of a tracked object. See [the object description docs](/configuration/genai/objects.md) for more information on how to automatically generate tracked object descriptions.

 Differently weighted versions of the Jina models are available and can be selected by setting the `model_size` config option as `small` or `large`:

@@ -78,17 +78,21 @@ Switching between V1 and V2 requires reindexing your embeddings. The embeddings

 ### GPU Acceleration

-The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU hardware, when available. This depends on the Docker build that is used.
+The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU hardware, when available. This depends on the Docker build that is used. You can also target a specific device in a multi-GPU installation.

 ```yaml
 semantic_search:
  enabled: True
  model_size: large
+  # Optional, if using the 'large' model in a multi-GPU installation
+  device: 0
 ```

 :::info

-If the correct build is used for your GPU and the `large` model is configured, then the GPU will be detected and used automatically.
+If the correct build is used for your GPU / NPU and the `large` model is configured, then the GPU will be detected and used automatically.
+Specify the `device` option to target a specific GPU in a multi-GPU system (see [onnxruntime's provider options](https://onnxruntime.ai/docs/execution-providers/)).
+If you do not specify a device, the first available GPU will be used.

 See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.

@@ -102,3 +106,61 @@ See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_
 4. Make your search language and tone closely match exactly what you're looking for. If you are using thumbnail search, **phrase your query as an image caption**. Searching for "red car" may not work as well as "red sedan driving down a residential street on a sunny day".
 5. Semantic search on thumbnails tends to return better results when matching large subjects that take up most of the frame. Small things like "cat" tend to not work well.
 6. Experiment! Find a tracked object you want to test and start typing keywords and phrases to see what works for you.
+
+## Triggers
+
+Triggers utilize Semantic Search to automate actions when a tracked object matches a specified image or description. Triggers can be configured so that Frigate executes a specific actions when a tracked object's image or description matches a predefined image or text, based on a similarity threshold. Triggers are managed per camera and can be configured via the Frigate UI in the Settings page under the Triggers tab.
+
+:::note
+
+Semantic Search must be enabled to use Triggers.
+
+:::
+
+### Configuration
+
+Triggers are defined within the `semantic_search` configuration for each camera in your Frigate configuration file or through the UI. Each trigger consists of a `friendly_name`, a `type` (either `thumbnail` or `description`), a `data` field (the reference image event ID or text), a `threshold` for similarity matching, and a list of `actions` to perform when the trigger fires - `notification`, `sub_label`, and `attribute`.
+
+Triggers are best configured through the Frigate UI.
+
+#### Managing Triggers in the UI
+
+1. Navigate to the **Settings** page and select the **Triggers** tab.
+2. Choose a camera from the dropdown menu to view or manage its triggers.
+3. Click **Add Trigger** to create a new trigger or use the pencil icon to edit an existing one.
+4. In the **Create Trigger** wizard:
+   - Enter a **Name** for the trigger (e.g., "Red Car Alert").
+   - Enter a descriptive **Friendly Name** for the trigger (e.g., "Red car on the driveway camera").
+   - Select the **Type** (`Thumbnail` or `Description`).
+   - For `Thumbnail`, select an image to trigger this action when a similar thumbnail image is detected, based on the threshold.
+   - For `Description`, enter text to trigger this action when a similar tracked object description is detected.
+   - Set the **Threshold** for similarity matching.
+   - Select **Actions** to perform when the trigger fires.
+     If native webpush notifications are enabled, check the `Send Notification` box to send a notification.
+     Check the `Add Sub Label` box to add the trigger's friendly name as a sub label to any triggering tracked objects.
+     Check the `Add Attribute` box to add the trigger's internal ID (e.g., "red_car_alert") to a data attribute on the tracked object that can be processed via the API or MQTT.
+5. Save the trigger to update the configuration and store the embedding in the database.
+
+When a trigger fires, the UI highlights the trigger with a blue dot for 3 seconds for easy identification. Additionally, the UI will show the last date/time and tracked object ID that activated your trigger. The last triggered timestamp is not saved to the database or persisted through restarts of Frigate.
+
+### Usage and Best Practices
+
+1. **Thumbnail Triggers**: Select a representative image (event ID) from the Explore page that closely matches the object you want to detect. For best results, choose images where the object is prominent and fills most of the frame.
+2. **Description Triggers**: Write concise, specific text descriptions (e.g., "Person in a red jacket") that align with the tracked object’s description. Avoid vague terms to improve matching accuracy.
+3. **Threshold Tuning**: Adjust the threshold to balance sensitivity and specificity. A higher threshold (e.g., 0.8) requires closer matches, reducing false positives but potentially missing similar objects. A lower threshold (e.g., 0.6) is more inclusive but may trigger more often.
+4. **Using Explore**: Use the context menu or right-click / long-press on a tracked object in the Grid View in Explore to quickly add a trigger based on the tracked object's thumbnail.
+5. **Editing triggers**: For the best experience, triggers should be edited via the UI. However, Frigate will ensure triggers edited in the config will be synced with triggers created and edited in the UI.
+
+### Notes
+
+- Triggers rely on the same Jina AI CLIP models (V1 or V2) used for semantic search. Ensure `semantic_search` is enabled and properly configured.
+- Reindexing embeddings (via the UI or `reindex: True`) does not affect trigger configurations but may update the embeddings used for matching.
+- For optimal performance, use a system with sufficient RAM (8GB minimum, 16GB recommended) and a GPU for `large` model configurations, as described in the Semantic Search requirements.
+
+### FAQ
+
+#### Why can't I create a trigger on thumbnails for some text, like "person with a blue shirt" and have it trigger when a person with a blue shirt is detected?
+
+TL;DR: Text-to-image triggers aren’t supported because CLIP can confuse similar images and give inconsistent scores, making automation unreliable. The same word–image pair can give different scores and the score ranges can be too close together to set a clear cutoff.
+
+Text-to-image triggers are not supported due to fundamental limitations of CLIP-based similarity search. While CLIP works well for exploratory, manual queries, it is unreliable for automated triggers based on a threshold. Issues include embedding drift (the same text–image pair can yield different cosine distances over time), lack of true semantic grounding (visually similar but incorrect matches), and unstable thresholding (distance distributions are dataset-dependent and often too tightly clustered to separate relevant from irrelevant results). Instead, it is recommended to set up a workflow with thumbnail triggers: first use text search to manually select 3–5 representative reference tracked objects, then configure thumbnail triggers based on that visual similarity. This provides robust automation without the semantic ambiguity of text to image matching.
@@ -9,4 +9,25 @@ Snapshots are accessible in the UI in the Explore pane. This allows for quick su

 To only save snapshots for objects that enter a specific zone, [see the zone docs](./zones.md#restricting-snapshots-to-specific-zones)

-Snapshots sent via MQTT are configured in the [config file](https://docs.frigate.video/configuration/) under `cameras -> your_camera -> mqtt`
+Snapshots sent via MQTT are configured in the [config file](/configuration) under `cameras -> your_camera -> mqtt`
+
+## Frame Selection
+
+Frigate does not save every frame — it picks a single "best" frame for each tracked object and uses it for both the snapshot and clean copy. As the object is tracked across frames, Frigate continuously evaluates whether the current frame is better than the previous best based on detection confidence, object size, and the presence of key attributes like faces or license plates. Frames where the object touches the edge of the frame are deprioritized. The snapshot is written to disk once tracking ends using whichever frame was determined to be the best.
+
+MQTT snapshots are published more frequently — each time a better thumbnail frame is found during tracking, or when the current best image is older than `best_image_timeout` (default: 60s). These use their own annotation settings configured under `cameras -> your_camera -> mqtt`.
+
+## Clean Copy
+
+Frigate can produce up to two snapshot files per event, each used in different places:
+
+| Version | File | Annotations | Used by |
+| --- | --- | --- | --- |
+| **Regular snapshot** | `<camera>-<id>.jpg` | Respects your `timestamp`, `bounding_box`, `crop`, and `height` settings | API (`/api/events/<id>/snapshot.jpg`), MQTT (`<camera>/<label>/snapshot`), Explore pane in the UI |
+| **Clean copy** | `<camera>-<id>-clean.webp` | Always unannotated — no bounding box, no timestamp, no crop, full resolution | API (`/api/events/<id>/snapshot-clean.webp`), [Frigate+](/plus/first_model) submissions, "Download Clean Snapshot" in the UI |
+
+MQTT snapshots are configured separately under `cameras -> your_camera -> mqtt` and are unrelated to the clean copy.
+
+The clean copy is required for submitting events to [Frigate+](/plus/first_model) — if you plan to use Frigate+, keep `clean_copy` enabled regardless of your other snapshot settings.
+
+If you are not using Frigate+ and `timestamp`, `bounding_box`, and `crop` are all disabled, the regular snapshot is already effectively clean, so `clean_copy` provides no benefit and only uses additional disk space. You can safely set `clean_copy: False` in this case.
@@ -27,6 +27,7 @@ cameras:
          - entire_yard
    zones:
      entire_yard:
+        friendly_name: Entire yard # You can use characters from any language text
        coordinates: ...
 ```

@@ -44,8 +45,10 @@ cameras:
          - edge_yard
    zones:
      edge_yard:
+        friendly_name: Edge yard # You can use characters from any language text
        coordinates: ...
      inner_yard:
+        friendly_name: Inner yard # You can use characters from any language text
        coordinates: ...
 ```

@@ -59,6 +62,7 @@ cameras:
        - entire_yard
    zones:
      entire_yard:
+        friendly_name: Entire yard
        coordinates: ...
 ```

@@ -82,13 +86,16 @@ cameras:

 Only car objects can trigger the `front_yard_street` zone and only person can trigger the `entire_yard`. Objects will be tracked for any `person` that enter anywhere in the yard, and for cars only if they enter the street.

+
 ### Zone Loitering

 Sometimes objects are expected to be passing through a zone, but an object loitering in an area is unexpected. Zones can be configured to have a minimum loitering time after which the object will be considered in the zone.

 :::note

-When using loitering zones, a review item will remain active until the object leaves. Loitering zones are only meant to be used in areas where loitering is not expected behavior.
+When using loitering zones, a review item will behave in the following way:
+- When a person is in a loitering zone, the review item will remain active until the person leaves the loitering zone, regardless of if they are stationary.
+- When any other object is in a loitering zone, the review item will remain active until the loitering time is met. Then if the object is stationary the review item will end.

 :::