Files
carplay/README.md
T

95 lines
8.4 KiB
Markdown
Raw Normal View History

2026-05-01 21:51:25 -07:00
# Frame Stream Player
A small web app that plays a remote video stream without using browser video decoding. The server uses `ffmpeg` to decode the input URL into:
- an MP3 audio stream served to a normal `<audio>` element
- timed JPEG image frames sent over a WebSocket and painted onto a `<canvas>`
2026-05-01 21:51:25 -07:00
This is meant for machines where image and audio decoding work but browser video decoding is unavailable or unreliable.
## Run
```sh
npm install
npm start
```
Open `http://localhost:3000`, paste a direct HTTP(S) stream URL, YouTube link, or X/Twitter video post URL, and click `Play`. Use `Queue` to add a URL to Recents without starting playback.
2026-05-01 21:51:25 -07:00
2026-06-14 18:10:10 -07:00
Set `LOCAL_VIDEOS=/path/to/videos` before starting the server to enable `Play Local`. The picker lets you navigate directories under that root, search the library, and play selected files through the server-side ffmpeg pipeline as MP3 audio plus JPEG canvas frames.
You need an `ffmpeg` binary with decoders for the stream's video and audio codecs. YouTube and X/Twitter page links also need `yt-dlp`; Docker installs it automatically from the upstream master branch at image build time. If your stream is H.264 or HEVC, make sure your installed `ffmpeg` actually includes those decoders. You can point the app at different binaries with:
2026-05-01 21:51:25 -07:00
```sh
FFMPEG_PATH=/path/to/ffmpeg npm start
2026-06-11 21:13:48 -07:00
YT_DLP_PATH=/path/to/yt-dlp npm start
2026-05-01 21:51:25 -07:00
```
## Docker
```sh
docker build -t frame-stream-player .
docker run --rm -p 3000:3000 frame-stream-player
```
Then open `http://localhost:3000`.
For Docker Compose:
```sh
docker compose -f docker-compose-example.yml up --build
```
2026-05-01 23:21:16 -07:00
The app uses CPU decoding by default, so no video device is required. The compose example includes commented VAAPI/NVIDIA passthrough options for future hardware-accelerated `ffmpeg` setups, but hardware acceleration is usually only useful when server CPU is saturated.
2026-05-01 21:51:25 -07:00
2026-05-04 20:33:14 -07:00
Recently played URLs and favorites are stored globally by the backend. In Docker Compose, they are persisted in the `frame-stream-data` named volume.
2026-05-01 22:08:50 -07:00
2026-06-14 18:10:10 -07:00
To use local playback in Docker, mount a host video directory into the container and set `LOCAL_VIDEOS` to the container path, for example `/app/local-videos`.
2026-05-01 22:41:51 -07:00
`ffmpeg` worker lifecycle, stderr warnings/errors, and source proxy open/close events are written to stdout/stderr, so they appear in `docker logs`. For more detail while debugging a stream, set `FFMPEG_LOG_LEVEL=info` in Docker Compose and run:
```sh
docker logs -f frame-stream-player
```
The app sets `FFMPEG_INPUT_SEEKABLE=0` by default so `ffmpeg` reads unknown/live stream inputs sequentially and avoids extra HTTP range connections. Recognized recorded files and sessions with a known duration automatically allow HTTP range requests from the start, so ffmpeg can jump to MP4 metadata instead of downloading the entire file first. For ffmpeg-owned HTTP inputs, reconnect handling is enabled by default with `FFMPEG_HTTP_RECONNECT=1`. `FFMPEG_HTTP_RECONNECT_MAX_RETRIES` is applied only when the installed `ffmpeg` supports that HTTP protocol option.
2026-05-01 22:41:51 -07:00
YouTube and X/Twitter URLs are resolved server-side with `yt-dlp`, with the app's Node runtime explicitly enabled for JavaScript challenges. Both `x.com` and `twitter.com` links work, including their `www`, `m`, and `mobile` variants. For posts with multiple videos, the first video plays unless the URL selects one with `/video/N`. The YouTube default selector is `bestvideo[protocol=https]+bestaudio[protocol=https]/bestvideo+bestaudio/best`. X/Twitter defaults to `best[protocol=https]/bestvideo[protocol=https]+bestaudio[protocol=https]`, preferring progressive MP4s because the source proxy does not rewrite relative HLS segment URLs. Audio and video can have separate URLs and request headers: each is proxied to its corresponding ffmpeg worker in `split` mode. Combined audio/video URLs also work. Finite, non-live duration metadata from yt-dlp enables seeking without another source probe. Recents and favorites keep the original page URL. Tune the selected format with `YT_DLP_FORMAT` and the resolver timeout with `YT_DLP_TIMEOUT_MS`.
Set `SINGLE_CONNECTION_HOSTS` to a comma-separated list of IPTV subscription hosts, for example `oceanstream.me`. Matching hosts and their subdomains always use `relay`, skip metadata probes, and disable seeking and recovery probes, even if the URL looks like a recorded file or the global mode is `split`. This protects subscriptions that permit only one upstream stream. Include any alternate provider hostnames in the list.
2026-06-11 21:13:48 -07:00
JPEG frames are dropped when the browser WebSocket falls behind instead of letting stale frames queue indefinitely. Tune the server-side backlog cap with `MAX_WS_BUFFER_BYTES`; the default is `2097152`.
2026-05-04 00:00:34 -07:00
2026-06-07 00:44:45 -07:00
In single mode, audio output from `ffmpeg` is buffered before it is written to the browser so short HTTP backpressure pauses are less likely to stall frame generation. Tune the cap with `MAX_AUDIO_QUEUE_BYTES`; the default is `4194304`.
2026-05-01 23:21:16 -07:00
Playback uses `PLAYBACK_CONNECTION_MODE=split` by default. The Docker Compose example sets `PLAYBACK_CONNECTION_MODE=relay` so IPTV-style streams can be tested with one upstream connection.
Available playback modes:
- `split`: Separate source connections and separate `ffmpeg` workers for audio and JPEG frames. This is usually the smoothest mode.
- `relay`: One source connection from the backend, then the compressed input bytes are teed into separate audio and frame `ffmpeg` workers. This is intended for IPTV hosts that stop early or reject multiple active connections.
- `single`: One source connection and one `ffmpeg` worker with both audio and frame outputs. This is the simplest one-connection fallback, but audio and frame delivery can affect each other.
Finite-duration streams are treated as recorded video and become seekable once metadata is available. Metadata probing is enabled by default except in `relay`, where it is disabled for unknown/live URLs to avoid extra upstream connections. Local files, YouTube and X/Twitter URLs without resolved duration metadata, recognized media file extensions, and Jellyfin download/stream URLs are probed automatically. Set `METADATA_PROBE_ENABLED=1` to probe other URLs as well.
Recognized recorded files use `split` immediately when the configured mode is `relay`, even while duration detection is pending. Other relay sessions switch to `split` once a finite duration is known. Failed probes for likely recorded sources retry up to three times in the background, with increasing timeouts (by default 4, 12, then 30 seconds) and short delays. `METADATA_PROBE_TIMEOUT_MS` sets the initial timeout; each attempt is capped at 30 seconds. Seeking becomes available when the probe succeeds, without restarting playback.
2026-05-26 19:11:19 -07:00
2026-06-07 00:44:45 -07:00
Relay mode uses bounded per-worker input queues so one branch can briefly lag without immediately stalling the other. Tune the cap with `MAX_RELAY_BRANCH_QUEUE_BYTES`; the default is `8388608`.
2026-05-01 22:50:34 -07:00
2026-05-01 21:51:25 -07:00
## Tuning
The UI intentionally hides these settings, but the backend still supports them through `POST /api/session`.
2026-06-07 00:52:37 -07:00
- Frame rate defaults to `24fps`. Lower it if the client cannot keep up.
2026-06-11 10:29:07 -07:00
- Max width defaults to `960px`, and the client now caps each session to its viewport width. Lower `DEFAULT_FRAME_WIDTH` first if bandwidth or image decode is the bottleneck.
- JPEG quality uses ffmpeg's `-q:v` scale, where lower is better/larger. Set the default with `JPEG_QUALITY`; `7` is the fallback, `2` is high quality, and `18` is rough but much lighter.
2026-06-07 00:52:37 -07:00
- Audio defaults to stereo MP3 at `160k`. Tune with `DEFAULT_AUDIO_BITRATE`, `DEFAULT_AUDIO_CHANNELS`, and `DEFAULT_AUDIO_SAMPLE_RATE`.
2026-05-01 21:51:25 -07:00
## Tradeoffs
JPEG frames are used instead of PNG or GIF. PNG is usually too large for 24fps video, and GIF has poor quality and weak timing control. JPEG is simple, browser-native, streamable per frame, and lets the audio element act as the playback clock.
2026-05-01 23:21:16 -07:00
The default split mode starts separate `ffmpeg` workers for audio and frames. That is simple and usually smoother for direct files and many HTTP streams, but live streams can have small startup offset differences and some hosts only allow one active connection. Relay mode avoids that host-side issue while keeping separate audio/frame workers, but it works best with sequential stream containers such as MPEG-TS. Single mode is kept as a fallback. The input URL is proxied or relayed by the backend before it is handed to `ffmpeg`, so query-string tokens are not exposed in `ffmpeg` process arguments.
2026-05-01 21:51:25 -07:00
Arbitrary URLs are still fetched by your server, so do not expose this app publicly without adding authentication and URL allowlisting.