AI recognition

Rerinku View’s AI recognition runs entirely on the machine that runs Rerinku View. Neither video nor results are uploaded. It has two parts:

• Object detection: recognizes people, vehicles (cars, trucks, buses, motorcycles, bicycles) and pets (dogs, cats, birds). The model supports the 80 COCO classes, and the 9 most common ones can be checked directly.
• Face recognition: building on person detection, it recognizes people registered in the face library and marks everyone else as Unknown or To confirm.

Each recognition creates an event. The event has a snapshot with the objects marked, and a video clip that starts 5 seconds before the object was first seen. See Events and notifications.

Turning it on

1. Open Settings → Smart detection. The first time, follow the prompt to download a detection model. Models come from Hugging Face; in mainland China, you can switch Model download source to China mirror.
2. In the camera list, turn on Object detection for the cameras you want analyzed. For cameras that should recognize people, also turn on Face recognition.
3. Fine-tune each camera under Device details → Settings → Smart detection: which stream to analyze, which objects to detect, and which zones to detect in. You can also turn on View detection frame to watch the detection boxes live.

Face recognition builds on object detection. Turning on face recognition also turns on object detection for that camera and checks Person. While face recognition is on, turning off object detection or unchecking Person asks you to confirm first; once you confirm, both are turned off.

Models

Model	Size	Use
YOLOX-Nano	3.5 MB	Object detection, lightest on resources
YOLOX-Tiny	19 MB	Object detection, recommended
YOLOX-S	34 MB	Object detection, more accurate, uses more resources
YuNet + ArcFace	About 250 MB	Face detection and recognition, downloaded when you turn on face recognition

Models are stored under models/ in the data directory (change this with MODELS_PATH). A model that’s in use can’t be deleted; turn off the feature that uses it, then delete it. You can also upload a YOLOX ONNX model you exported yourself.

When events fire

• Only moving things count. Rerinku View runs motion detection first and runs inference only on the areas of the picture that changed. It also scans the whole picture once every 10 seconds.
• Stationary objects aren’t reported. An object that hasn’t moved for about 10 seconds counts as stationary. An object that hasn’t moved since detection started (such as a car that’s always parked there) creates no event.
• One continuous appearance is one event. Several objects that appear at the same time are merged into one event. The event ends when the objects leave.
• Default thresholds: 5 frames analyzed per second (adjustable from 1 to 15), a per-frame confidence of 0.5, and 0.7 to confirm an object. Person and Car are detected by default. All of these can be changed under Settings → Smart detection → Advanced, and each camera can override them.
• Motion masks: areas that are always moving but don’t matter (trees, a road) can be excluded in the camera’s detection zones.

Controlling how often events fire

Settings → Smart detection → Advanced has two throttling options. Both are off by default, and each camera can have its own settings:

• Event interval: how long this camera pauses detection after an event ends (up to 24 hours). During the pause, video keeps recording as usual but no inference runs, and the status shows “Event interval, detecting again at HH:MM”. Restarting Rerinku View resumes detection right away.
• Longest event: the longest an event can last (10 seconds to 24 hours). When that time is reached, the event ends.

Recognition page

The Recognition page in the sidebar brings the results together:

• Objects: filter detected objects by category (people, vehicles, pets), camera and time.
• People:
  • Detections: every face captured, sorted into Recognized, To confirm and Unknown. You can assign a face that’s waiting for confirmation to a person.
  • Face library: register the people to recognize, with up to 5 photos each. You can put them in groups (Family, Friends, Regulars), merge duplicate people, and test how well recognition works.

Recognized names are written into the event, and also show up in phone notifications and MQTT.

Supported platforms

The recognition component ships with the program. Rerinku View unpacks it automatically at startup, so there’s nothing to install separately.

Platform	AI recognition
Linux (x86-64)	Supported; needs glibc 2.34 or later
Linux (ARM64)	Supported; needs glibc 2.34 or later
Windows 10 / 11 (x64)	Supported, no runtime to install; tries GPU acceleration (DirectML) when there’s a graphics card
Windows (ARM64)	Not supported yet
macOS (Apple silicon)	Supported; needs macOS 14 Sonoma or later
macOS (Intel)	Not supported
Docker image (x86-64)	Supported; the recognition component and Intel / AMD GPU drivers are built in

On platforms without AI recognition, live view, recording, playback and everything else work as usual. Recognition features aren’t hidden: the top of the Recognition page and of Settings → Smart detection explains why, and the related switches are grayed out.

Linux

You need glibc 2.34 or later. Mainstream distributions from the last few years all have it, for example Debian 12+, Ubuntu 22.04+, Fedora 35+, the RHEL 9 family, and Raspberry Pi OS (Bookworm), which is based on Debian 12. musl-based distributions such as Alpine don’t. If you’re not sure, check:

ldd --version

The number at the end of the first line is the glibc version. On Linux, inference currently runs on the CPU.

The Docker image (x86-64) also has the recognition component built in, with nothing extra to install. See Quick Start.

Windows

For Windows 10 / 11 (x64). The recognition component is built in, and you don’t need to install any runtime.

The Windows version tries to run object detection and face recognition on the GPU. It uses DirectML, so any graphics card that supports Direct3D 12 works, including NVIDIA, AMD and Intel (integrated graphics too). When the GPU can’t be used (no graphics card, a virtual machine, a driver problem), it falls back to the CPU automatically and recognition keeps working.

• Under Settings → Smart detection → Advanced → Detection, set Inference device to Auto (GPU first) (the default) or CPU only. It takes effect as soon as you save.
• To check whether the GPU is in use, look at Performance at the bottom of Settings → Smart detection. The Inference device item shows the graphics card’s name; if the GPU isn’t in use, it explains why.
• GPU acceleration needs a fairly recent graphics driver. If the GPU isn’t in use, update the driver first and try again.

macOS

You need a Mac with Apple silicon (M1 or later) running macOS 14 Sonoma or later. The recognition component is built in, with nothing extra to install. Inference currently runs on the CPU.

• Intel Macs don’t support AI recognition: ONNX Runtime, the inference library, stopped shipping Intel builds as of 1.24. Everything else works normally.
• Running the Intel version on an Apple silicon Mac (translated by Rosetta): switch to the Apple silicon (arm64) version to use AI recognition.
• macOS 13 or earlier: upgrade to macOS 14 or later to use AI recognition.

To check your chip and macOS version, click the Apple menu in the top-left corner of the screen and choose About This Mac. An Apple silicon Mac shows a Chip item (such as Apple M1); an Intel Mac shows Processor instead. The macOS version is in the same window.

Video decoding

To see what’s in the picture, recognition first has to decode the camera’s video into images. Live view and recording forward the camera’s original stream and need no decoding; only cameras with recognition are decoded.

Rerinku View uses the GPU’s video decoder first (hardware decoding, which uses almost no CPU). Only formats the GPU can’t decode, or a GPU that fails, are handed to the CPU (software decoding):

Platform	Hardware decoding
Linux (x86-64)	VA-API (Intel, AMD), NVDEC (NVIDIA)
Linux (ARM64)	Not supported yet; uses software decoding
Windows	D3D11VA (NVIDIA, AMD, Intel)
macOS	VideoToolbox

At startup, Rerinku View uses a short built-in video to test-decode H.264 and H.265 on each GPU, and hands a GPU only the formats it actually decoded. Settings → Video decoding shows whether each camera is using hardware or software decoding right now, why hardware isn’t used, and which GPUs were found and which formats each one can decode.

Hardware decoding on a native Linux install needs:

• The GPU driver: intel-media-va-driver for Intel (i965-va-driver for older GPUs), mesa-va-drivers for AMD, and NVIDIA’s official driver 470 or newer for NVIDIA (the open-source nouveau driver won’t work).
• The user running Rerinku View in the render and video groups: sudo usermod -aG render,video USER. After logging in again or restarting the service, click Detect again under Settings → Video decoding.

The Docker image already includes the Intel and AMD GPU drivers, so the host needs no drivers and no group changes. Start the container with --device /dev/dri (under devices: in Compose) and it decodes on the GPU; the container gets access to the GPU automatically. NVIDIA GPUs need the NVIDIA driver and the NVIDIA Container Toolkit installed on the host, and --gpus all when starting the container.

If a GPU has problems as soon as it starts decoding, set the environment variable VIDEO_HW_DECODE=false to force software decoding only.

How much computing power you need

There’s no single hardware requirement. It depends on the number of cameras, their resolution and the model. For reference: on the CPU of a Ryzen 7 5825U laptop, one YOLOX-Tiny inference takes about 22 ms, so at 5 frames per second it can handle about 8 cameras. Decoding one 1080p video takes about 0.35–0.6 of a CPU core.

The most accurate way to find out is Performance at the bottom of Settings → Smart detection. It measures the actual inference speed and estimates how many more cameras this machine can run recognition on.