Compare commits

..
Author SHA1 Message Date
buzzert cedd0c6aa2 ios: simplify share extension to saved-player tap-to-play
Show only configured players, enqueue and play immediately on row tap,
and keep the URL on one truncated line. Remove extension discovery,
selection state, and the bottom action while retaining progress and
existing duplicate/partial-failure guards. Preserve main-app discovery.

Update share-extension guidance to match the simplified flow.
2026-10-21 01:06:45 -07:00
buzzert 174a8f8255 Merge share extension for iOS release
TestFlight / testflight (push) Successful in 1m45s
* commit '87239557059e0bf8618647d066fe733188f89984':
  ios: add share extension to queue and play video links
2026-10-20 22:46:46 -07:00
buzzert a846d9b4ec Optimize screen sharing latency and frame rate 2026-08-29 20:10:55 -07:00
11 changed files with 368 additions and 226 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ It allows you to remotely manage mpv's playqueue as well as the ability to searc
## Sharing video links ## Sharing video links
The iOS and Mac Catalyst app includes a QueueCube share extension. Share one The iOS and Mac Catalyst app includes a QueueCube share extension. Share one
video link, choose a player, and tap **Queue & Play** to append and play it while video link and tap a saved player to append and play it while
keeping the existing queue. See [share extension setup, API behavior, and validation](ios/SHARE_EXTENSION.md). keeping the existing queue. See [share extension setup, API behavior, and validation](ios/SHARE_EXTENSION.md).
## iOS TestFlight releases ## iOS TestFlight releases
+3 -2
View File
@@ -13,7 +13,7 @@ QueueCube is a SwiftUI-based jukebox client application for iOS and macOS (via M
- **API.swift**: Central networking layer that handles all communication with the jukebox server. Includes REST API methods for playback control, playlist management, and WebSocket events for real-time updates. - **API.swift**: Central networking layer that handles all communication with the jukebox server. Includes REST API methods for playback control, playlist management, and WebSocket events for real-time updates.
- **ContentView.swift**: Main view controller containing the `MainViewModel` that coordinates between UI components and API calls. Handles WebSocket event processing and data flow. - **ContentView.swift**: Main view controller containing the `MainViewModel` that coordinates between UI components and API calls. Handles WebSocket event processing and data flow.
- **Server.swift**: Represents individual jukebox servers with support for both manual configuration and Bonjour service discovery. - **Server.swift**: Represents individual jukebox servers with support for both manual configuration and Bonjour service discovery.
- **ServerDiscovery.swift**: Shared Bonjour browsing and bounded, cancellation-safe endpoint resolution for the app and share extension. - **ServerDiscovery.swift**: Main-app Bonjour browsing and bounded, cancellation-safe endpoint resolution for Add Server.
- **RequestBuilder.swift**: Shared extension-safe HTTP builder and error handling. - **RequestBuilder.swift**: Shared extension-safe HTTP builder and error handling.
- **SharePlayback.swift**: Append once, select the new queue item, unpause, and verify playback using existing endpoints. - **SharePlayback.swift**: Append once, select the new queue item, unpause, and verify playback using existing endpoints.
- **Settings.swift**: Manages multiple server configurations stored in UserDefaults with validation through `SettingsViewModel` that tests connectivity on URL changes. - **Settings.swift**: Manages multiple server configurations stored in UserDefaults with validation through `SettingsViewModel` that tests connectivity on URL changes.
@@ -99,5 +99,6 @@ QueueCubeApp
## Share extension ## Share extension
`QueueCubeShare` is embedded in iOS and Mac Catalyst builds. It reads an App Group `QueueCubeShare` is embedded in iOS and Mac Catalyst builds. It reads an App Group
settings snapshot and reuses the API/discovery code. See [SHARE_EXTENSION.md](SHARE_EXTENSION.md) settings snapshot and reuses the API code. Tapping a saved player queues and plays
the shared link; discovery remains in the main app. See [SHARE_EXTENSION.md](SHARE_EXTENSION.md)
for authentication limitations, queue/play failure semantics, signing setup, and focused tests. for authentication limitations, queue/play failure semantics, signing setup, and focused tests.
-4
View File
@@ -11,7 +11,6 @@
CD2026102000000000000201 /* RequestBuilder.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000101 /* RequestBuilder.swift */; }; CD2026102000000000000201 /* RequestBuilder.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000101 /* RequestBuilder.swift */; };
CD2026102000000000000202 /* Server.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000102 /* Server.swift */; }; CD2026102000000000000202 /* Server.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000102 /* Server.swift */; };
CD2026102000000000000203 /* Settings.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000103 /* Settings.swift */; }; CD2026102000000000000203 /* Settings.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000103 /* Settings.swift */; };
CD2026102000000000000204 /* ServerDiscovery.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000104 /* ServerDiscovery.swift */; };
CD2026102000000000000205 /* SharedVideo.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000105 /* SharedVideo.swift */; }; CD2026102000000000000205 /* SharedVideo.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000105 /* SharedVideo.swift */; };
CD2026102000000000000206 /* SharePlayback.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000106 /* SharePlayback.swift */; }; CD2026102000000000000206 /* SharePlayback.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000106 /* SharePlayback.swift */; };
CD2026102000000000000207 /* SharedInput.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000107 /* SharedInput.swift */; }; CD2026102000000000000207 /* SharedInput.swift in Sources */ = {isa = PBXBuildFile; fileRef = CD2026102000000000000107 /* SharedInput.swift */; };
@@ -49,7 +48,6 @@
CD2026102000000000000101 /* RequestBuilder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RequestBuilder.swift; sourceTree = "<group>"; }; CD2026102000000000000101 /* RequestBuilder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RequestBuilder.swift; sourceTree = "<group>"; };
CD2026102000000000000102 /* Server.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Server.swift; sourceTree = "<group>"; }; CD2026102000000000000102 /* Server.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Server.swift; sourceTree = "<group>"; };
CD2026102000000000000103 /* Settings.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Settings.swift; sourceTree = "<group>"; }; CD2026102000000000000103 /* Settings.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Settings.swift; sourceTree = "<group>"; };
CD2026102000000000000104 /* ServerDiscovery.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ServerDiscovery.swift; sourceTree = "<group>"; };
CD2026102000000000000105 /* SharedVideo.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SharedVideo.swift; sourceTree = "<group>"; }; CD2026102000000000000105 /* SharedVideo.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SharedVideo.swift; sourceTree = "<group>"; };
CD2026102000000000000106 /* SharePlayback.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SharePlayback.swift; sourceTree = "<group>"; }; CD2026102000000000000106 /* SharePlayback.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SharePlayback.swift; sourceTree = "<group>"; };
CD2026102000000000000107 /* SharedInput.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SharedInput.swift; sourceTree = "<group>"; }; CD2026102000000000000107 /* SharedInput.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SharedInput.swift; sourceTree = "<group>"; };
@@ -137,7 +135,6 @@
CD2026102000000000000101 /* RequestBuilder.swift */, CD2026102000000000000101 /* RequestBuilder.swift */,
CD2026102000000000000102 /* Server.swift */, CD2026102000000000000102 /* Server.swift */,
CD2026102000000000000103 /* Settings.swift */, CD2026102000000000000103 /* Settings.swift */,
CD2026102000000000000104 /* ServerDiscovery.swift */,
CD2026102000000000000105 /* SharedVideo.swift */, CD2026102000000000000105 /* SharedVideo.swift */,
CD2026102000000000000106 /* SharePlayback.swift */, CD2026102000000000000106 /* SharePlayback.swift */,
CD2026102000000000000107 /* SharedInput.swift */, CD2026102000000000000107 /* SharedInput.swift */,
@@ -272,7 +269,6 @@
CD2026102000000000000201 /* RequestBuilder.swift in Sources */, CD2026102000000000000201 /* RequestBuilder.swift in Sources */,
CD2026102000000000000202 /* Server.swift in Sources */, CD2026102000000000000202 /* Server.swift in Sources */,
CD2026102000000000000203 /* Settings.swift in Sources */, CD2026102000000000000203 /* Settings.swift in Sources */,
CD2026102000000000000204 /* ServerDiscovery.swift in Sources */,
CD2026102000000000000205 /* SharedVideo.swift in Sources */, CD2026102000000000000205 /* SharedVideo.swift in Sources */,
CD2026102000000000000206 /* SharePlayback.swift in Sources */, CD2026102000000000000206 /* SharePlayback.swift in Sources */,
CD2026102000000000000207 /* SharedInput.swift in Sources */, CD2026102000000000000207 /* SharedInput.swift in Sources */,
+1 -1
View File
@@ -1,4 +1,4 @@
// Shared Bonjour discovery for the app and share extension. // Bonjour discovery for the main app's Add Server view.
import Foundation import Foundation
import Network import Network
+1 -5
View File
@@ -4,12 +4,8 @@
<dict> <dict>
<key>ITSAppUsesNonExemptEncryption</key> <key>ITSAppUsesNonExemptEncryption</key>
<false/> <false/>
<key>NSBonjourServices</key>
<array>
<string>_queuecube._tcp.</string>
</array>
<key>NSLocalNetworkUsageDescription</key> <key>NSLocalNetworkUsageDescription</key>
<string>QueueCube discovers nearby players so you can choose where to play this video.</string> <string>QueueCube connects to your saved players to queue and play this video.</string>
<key>NSAppTransportSecurity</key> <key>NSAppTransportSecurity</key>
<dict> <dict>
<key>NSAllowsArbitraryLoads</key> <key>NSAllowsArbitraryLoads</key>
+20 -90
View File
@@ -9,8 +9,8 @@ final class ShareViewController: UIViewController {
override func viewDidLoad() { override func viewDidLoad() {
super.viewDidLoad() super.viewDidLoad()
preferredContentSize = CGSize(width: 480, height: 600) preferredContentSize = CGSize(width: 480, height: 600)
let host = UIHostingController(rootView: ShareView(model: model, send: { [weak self] in let host = UIHostingController(rootView: ShareView(model: model, send: { [weak self] server in
self?.send() self?.send(to: server)
}, close: { [weak self] in }, close: { [weak self] in
self?.close() self?.close()
})) }))
@@ -24,7 +24,6 @@ final class ShareViewController: UIViewController {
host.view.bottomAnchor.constraint(equalTo: view.bottomAnchor) host.view.bottomAnchor.constraint(equalTo: view.bottomAnchor)
]) ])
host.didMove(toParent: self) host.didMove(toParent: self)
model.discovery.start()
inputTask = Task { [weak self] in inputTask = Task { [weak self] in
guard let self else { return } guard let self else { return }
do { do {
@@ -37,12 +36,10 @@ final class ShareViewController: UIViewController {
} }
} }
private func send() { private func send(to server: Server) {
guard !model.sending, !model.finished, let url = model.url, guard !model.sending, !model.finished, let url = model.url else { return }
let server = model.selectedServer else { return }
model.sending = true model.sending = true
model.message = nil model.message = nil
model.discovery.stop()
playbackTask = Task { [weak self] in playbackTask = Task { [weak self] in
guard let self else { return } guard let self else { return }
do { do {
@@ -55,7 +52,6 @@ final class ShareViewController: UIViewController {
guard !Task.isCancelled else { return } guard !Task.isCancelled else { return }
model.message = error.localizedDescription model.message = error.localizedDescription
model.finished = !error.canRetry model.finished = !error.canRetry
if error.canRetry { model.discovery.start() }
} catch { } catch {
guard !Task.isCancelled else { return } guard !Task.isCancelled else { return }
model.message = "The request could not be confirmed. Check the player in QueueCube before sharing again." model.message = "The request could not be confirmed. Check the player in QueueCube before sharing again."
@@ -68,7 +64,6 @@ final class ShareViewController: UIViewController {
private func close() { private func close() {
inputTask?.cancel() inputTask?.cancel()
playbackTask?.cancel() playbackTask?.cancel()
model.stop()
if model.finished { if model.finished {
extensionContext?.completeRequest(returningItems: nil) extensionContext?.completeRequest(returningItems: nil)
} else { } else {
@@ -80,7 +75,6 @@ final class ShareViewController: UIViewController {
super.viewDidDisappear(animated) super.viewDidDisappear(animated)
inputTask?.cancel() inputTask?.cancel()
playbackTask?.cancel() playbackTask?.cancel()
model.stop()
} }
deinit { deinit {
@@ -91,53 +85,18 @@ final class ShareViewController: UIViewController {
@MainActor @Observable @MainActor @Observable
private final class ShareViewModel { private final class ShareViewModel {
let discovery = ServerDiscovery() let configuredServers = Settings.fromSharedDefaults().configuredServers
let configuredServers: [Server]
var selectedServer: Server?
var url: URL? var url: URL?
var loading = true var loading = true
var sending = false var sending = false
var finished = false var finished = false
var succeeded = false var succeeded = false
var resolving = false
var message: String? var message: String?
private var resolutionTask: Task<Void, Never>?
init() {
let settings = Settings.fromSharedDefaults()
configuredServers = settings.configuredServers
selectedServer = settings.selectedServer.flatMap { selected in
settings.configuredServers.first { $0.id == selected.id }
} ?? settings.configuredServers.first
}
func select(_ endpoint: DiscoveredEndpoint) {
resolutionTask?.cancel()
resolving = true
message = nil
resolutionTask = Task {
do {
let server = try await endpoint.resolve()
try Task.checkCancellation()
selectedServer = server
} catch {
if !Task.isCancelled {
message = "Could not connect to that player. Check its connection and try again."
}
}
if !Task.isCancelled { resolving = false }
}
}
func stop() {
resolutionTask?.cancel()
discovery.stop()
}
} }
private struct ShareView: View { private struct ShareView: View {
@Bindable var model: ShareViewModel @Bindable var model: ShareViewModel
let send: () -> Void let send: (Server) -> Void
let close: () -> Void let close: () -> Void
var body: some View { var body: some View {
@@ -149,40 +108,35 @@ private struct ShareView: View {
} else if let url = model.url { } else if let url = model.url {
Text(url.absoluteString) Text(url.absoluteString)
.font(.callout) .font(.callout)
.lineLimit(3) .lineLimit(1)
.truncationMode(.middle)
} }
} }
if !model.finished { if !model.finished {
Section("Choose player") { Section("Choose player") {
if model.configuredServers.isEmpty { if model.configuredServers.isEmpty {
Text("Open QueueCube once to share your saved players, or choose a nearby player below. Allow local network access in QueueCube to reach nearby players.") Text("Open QueueCube to set up your players.")
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
ForEach(model.configuredServers) { server in ForEach(model.configuredServers) { server in
Button { Button {
model.selectedServer = server send(server)
} label: { } label: {
playerLabel(server.displayName, selected: model.selectedServer?.id == server.id) HStack {
Label(server.displayName, systemImage: "hifispeaker.fill")
Spacer()
} }
.disabled(model.sending || model.resolving) .foregroundStyle(Color.primary)
.contentShape(Rectangle())
} }
if let selected = model.selectedServer, .tint(.primary)
!model.configuredServers.contains(where: { $0.id == selected.id }) { .disabled(model.url == nil || model.sending)
playerLabel(selected.displayName, selected: true)
} }
} }
if !model.sending {
Section("Nearby players") {
if model.discovery.servers.isEmpty {
Text(model.discovery.errorMessage ?? "Looking for QueueCube players on your local network…")
.foregroundStyle(.secondary)
}
ForEach(model.discovery.servers) { endpoint in
Button(endpoint.displayName) { model.select(endpoint) }
.disabled(model.resolving)
}
if model.resolving { ProgressView("Connecting…") }
} }
if model.sending {
Section {
ProgressView("Queuing and starting playback…")
} }
} }
if let message = model.message { if let message = model.message {
@@ -191,22 +145,6 @@ private struct ShareView: View {
.foregroundStyle(model.succeeded ? Color.green : Color.primary) .foregroundStyle(model.succeeded ? Color.green : Color.primary)
} }
} }
if !model.finished {
Section {
Button(action: send) {
HStack {
Spacer()
if model.sending { ProgressView() }
Text(model.sending ? "Queuing and starting playback…" : "Queue & Play")
Spacer()
}
}
.disabled(model.url == nil || model.selectedServer == nil || model.sending || model.resolving)
Text("Adds this link to the queue and plays it now. Existing queued items are kept.")
.font(.caption)
.foregroundStyle(.secondary)
}
}
} }
.navigationTitle("QueueCube") .navigationTitle("QueueCube")
.navigationBarTitleDisplayMode(.inline) .navigationBarTitleDisplayMode(.inline)
@@ -218,12 +156,4 @@ private struct ShareView: View {
} }
} }
} }
private func playerLabel(_ name: String, selected: Bool) -> some View {
HStack {
Label(name, systemImage: "hifispeaker.fill")
Spacer()
if selected { Image(systemName: "checkmark").accessibilityLabel("Selected") }
}
}
} }
+8 -8
View File
@@ -2,15 +2,15 @@
Open QueueCube once after updating to make saved players available to the share Open QueueCube once after updating to make saved players available to the share
extension. In Safari or a video app, share one HTTP(S) link, choose QueueCube, extension. In Safari or a video app, share one HTTP(S) link, choose QueueCube,
select a saved or nearby player, and tap **Queue & Play**. The link is appended and tap a saved player. The link is appended
and played immediately; existing queue entries remain. A successful request and played immediately; existing queue entries remain. A successful request
shows the selected player and a Done button. Video support depends on the shows the selected player and a Done button. Video support depends on the
player's existing mpv/yt-dlp support. player's existing mpv/yt-dlp support.
Nearby players use the same `_queuecube._tcp.` Bonjour discovery as Add Server. The extension shows only saved players and does not browse for nearby players.
Resolved IPv4 interface suffixes are removed; scoped IPv6 addresses remain valid. Add players manually or with Bonjour discovery in QueueCube Settings first.
Allow local network access in the containing app first. Players can also be Allow local network access in the containing app to reach local players.
configured manually in QueueCube Settings. The extension handles URL attachments The extension handles URL attachments
(including UTF-8 data and the absolute-URL property-list form observed from (including UTF-8 data and the absolute-URL property-list form observed from
UIKit) and plain text containing one link. URL data keeps the same single-link UIKit) and plain text containing one link. URL data keeps the same single-link
validation; property lists with a nonempty base or properties dictionary and validation; property lists with a nonempty base or properties dictionary and
@@ -25,7 +25,7 @@ The app and `net.buzzert.QueueCube.Share` extension share
publishes a single versioned settings snapshot on read/save. The extension reads publishes a single versioned settings snapshot on read/save. The extension reads
that snapshot without changing the app's selected player or server list. No that snapshot without changing the app's selected player or server list. No
queued link, credentials store, background transfer, or keychain group is added. queued link, credentials store, background transfer, or keychain group is added.
The extension has Bonjour/local network declarations and the same HTTP transport The extension has a local network usage declaration and the same HTTP transport
compatibility as the existing app; Catalyst adds only app sandbox and outgoing compatibility as the existing app; Catalyst adds only app sandbox and outgoing
network access. network access.
@@ -94,8 +94,8 @@ app/extension versions synchronized. Do not apply a single global bundle ID or
provisioning profile override to both targets. provisioning profile override to both targets.
For manual UI validation, use a dedicated simulator or test installation: For manual UI validation, use a dedicated simulator or test installation:
configure a test player in QueueCube, share a URL from Safari, verify saved and configure a test player in QueueCube, share a URL from Safari, verify saved-player
nearby player selection, Queue & Play success, cancellation, invalid text, and tap-to-play, the empty saved-player state, cancellation, invalid text, and
an offline/partially failing player. Confirm the old queue is retained and the an offline/partially failing player. Confirm the old queue is retained and the
new item is current and unpaused. On macOS also confirm QueueCube appears in new item is current and unpaused. On macOS also confirm QueueCube appears in
Share extensions and can be invoked from a host app. Record signed-device and Share extensions and can be invoked from a host app. Record signed-device and
+24 -21
View File
@@ -375,29 +375,27 @@ export class MediaPlayer {
siteName: "Screen Sharing", siteName: "Screen Sharing",
}); });
// Special options for mpv to better handle screen sharing (AI recommended...) // Start from mpv's maintained low-latency profile, then keep only a
// couple of frames of demuxed data ready for smooth 60 fps playback.
await this.loadFile(url, "replace", false, [ await this.loadFile(url, "replace", false, [
"demuxer-lavf-o=fflags=+nobuffer+discardcorrupt", // Reduce buffering and discard corrupt frames "profile=low-latency",
"demuxer-lavf-o=analyzeduration=100000", // Reduce analyze duration "cache=no",
"demuxer-lavf-o=probesize=1000000", // Reduce probe size "force-seekable=no",
"untimed=yes", // Ignore timing info "demuxer-readahead-secs=0.05",
"cache=no", // Disable cache "demuxer-max-bytes=256K",
"force-seekable=yes", // Force seekable "demuxer-mkv-probe-start-time=no",
"no-cache=yes", // Disable cache "vd-lavc-threads=2",
"demuxer-max-bytes=500K", // Limit demuxer buffer "vd-lavc-fast=yes",
"demuxer-readahead-secs=0.1", // Reduce readahead "audio-buffer=0",
"hr-seek=no", // Disable high-res seeking "pulse-buffer=20",
"video-sync=display-resample", // Better sync mode "pipewire-buffer=20",
"video-latency-hacks=yes", // Enable latency hacks "alsa-buffer-time=20000",
"audio-sync=yes", // Enable audio sync "initial-audio-sync=no",
"audio-buffer=0.1", // Reduce audio buffer
"audio-channels=stereo", // Force stereo audio
"audio-samplerate=44100", // Match sample rate
"audio-format=s16", // Use 16-bit audio
]); ]);
// Make sure it's playing // loadfile normally unpauses, but make that explicit without adding the
setTimeout(() => this.play(), 100); // previous fixed 100 ms startup delay.
await this.play();
} }
public async play() { public async play() {
@@ -475,13 +473,18 @@ export class MediaPlayer {
private async loadFile(url: string, mode: string, fetchMetadata: boolean = true, options: string[] = []) { private async loadFile(url: string, mode: string, fetchMetadata: boolean = true, options: string[] = []) {
this.lastLoadCandidate = url; this.lastLoadCandidate = url;
this.playbackErrors.delete(url); this.playbackErrors.delete(url);
this.modify(UserEvent.PlaylistUpdate, () => this.writeCommand("loadfile", [url, mode, "-1", options.join(',')])); const loadPromise = this.modify(
UserEvent.PlaylistUpdate,
() => this.writeCommand("loadfile", [url, mode, "-1", options.join(',')]),
);
if (fetchMetadata && this.shouldFetchLinkPreview(url)) { if (fetchMetadata && this.shouldFetchLinkPreview(url)) {
this.fetchMetadataAndNotify(url).catch(error => { this.fetchMetadataAndNotify(url).catch(error => {
console.warn(`Failed to fetch metadata for ${this.loggableUrl(url)}:`, error); console.warn(`Failed to fetch metadata for ${this.loggableUrl(url)}:`, error);
}); });
} }
await loadPromise;
} }
private async modify<T>(event: UserEvent, func: () => Promise<T>): Promise<T> { private async modify<T>(event: UserEvent, func: () => Promise<T>): Promise<T> {
+86 -41
View File
@@ -26,6 +26,7 @@ import { searchInvidious, fetchThumbnail } from "./InvidiousAPI";
import { PlaylistItem } from './types'; import { PlaylistItem } from './types';
import { PassThrough } from "stream"; import { PassThrough } from "stream";
import { AddressInfo } from "net"; import { AddressInfo } from "net";
import { WebSocket } from "ws";
const app = express(); const app = express();
app.use(express.json()); app.use(express.json());
@@ -43,6 +44,21 @@ const mediaPlayer = new MediaPlayer();
// Create a shared stream that both endpoints can access // Create a shared stream that both endpoints can access
let activeScreenshareStream: PassThrough | null = null; let activeScreenshareStream: PassThrough | null = null;
let activeScreenshareMimeType: string | null = null; let activeScreenshareMimeType: string | null = null;
let activeScreenshareSocket: WebSocket | null = null;
const screenshareStreamsClosedByViewer = new WeakSet<PassThrough>();
const SCREENSHARE_STREAM_HIGH_WATER_MARK = 32 * 1024;
const getScreenshareContentType = (mimeType: unknown): string => {
if (typeof mimeType !== "string") {
return "application/octet-stream";
}
const contentType = mimeType.split(";", 1)[0].trim().toLowerCase();
return contentType === "video/webm" || contentType === "video/mp4"
? contentType
: "application/octet-stream";
};
const withErrorHandling = (func: (req: any, res: any) => Promise<any>) => { const withErrorHandling = (func: (req: any, res: any) => Promise<any>) => {
return async (req: any, res: any) => { return async (req: any, res: any) => {
@@ -158,75 +174,104 @@ apiRouter.ws("/events", (ws, req) => {
// using the fact that QueueCube/MPV is based all around streaming URLs, so the active // using the fact that QueueCube/MPV is based all around streaming URLs, so the active
// screenshare stream manifests as just another URL to play. // screenshare stream manifests as just another URL to play.
apiRouter.get("/screenshareStream", withErrorHandling(async (req, res) => { apiRouter.get("/screenshareStream", withErrorHandling(async (req, res) => {
res.setHeader("Content-Type", activeScreenshareMimeType || "video/mp4"); const screenshareStream = activeScreenshareStream;
if (!screenshareStream) {
res.status(503).send("No active screen sharing session");
return;
}
req.socket.setNoDelay(true);
res.setHeader("Content-Type", activeScreenshareMimeType || "application/octet-stream");
res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate"); res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
res.setHeader("Pragma", "no-cache"); res.setHeader("Pragma", "no-cache");
res.setHeader("Expires", "0"); res.setHeader("Expires", "0");
res.setHeader("Connection", "keep-alive"); res.setHeader("Connection", "keep-alive");
res.setHeader("Transfer-Encoding", "chunked"); res.setHeader("Transfer-Encoding", "chunked");
res.setHeader("Accept-Ranges", "none");
res.flushHeaders();
if (!activeScreenshareStream) { res.on('close', () => {
res.status(503).send("No active screen sharing session"); screenshareStream.unpipe(res);
return;
}
// Handle client disconnection
req.on('close', () => {
console.log("Screenshare viewer disconnected"); console.log("Screenshare viewer disconnected");
// If mpv goes away unexpectedly, stop capture instead of leaving an
// unconsumed stream to build backpressure forever.
if (activeScreenshareStream === screenshareStream && !screenshareStream.writableEnded) {
screenshareStreamsClosedByViewer.add(screenshareStream);
activeScreenshareSocket?.close(1011, "Screen sharing viewer disconnected");
}
}); });
// Configure stream for low latency screenshareStream.pipe(res);
activeScreenshareStream.setMaxListeners(0);
// Pipe with immediate flush
activeScreenshareStream.pipe(res, { end: false });
})); }));
apiRouter.ws("/screenshare", (ws, req) => { apiRouter.ws("/screenshare", (ws, req) => {
const mimeType = req.query.mimeType as string; const mimeType = getScreenshareContentType(req.query.mimeType);
console.log("Screen sharing client connected with mimeType: " + mimeType); console.log("Screen sharing client connected with mimeType: " + mimeType);
ws.binaryType = "arraybuffer"; req.socket.setNoDelay(true);
let firstChunk = false; // Supersede an old sender without allowing its close handler to tear down
// the new session.
const previousSocket = activeScreenshareSocket;
const previousStream = activeScreenshareStream;
activeScreenshareSocket = ws;
activeScreenshareMimeType = mimeType;
activeScreenshareStream = new PassThrough({
highWaterMark: SCREENSHARE_STREAM_HIGH_WATER_MARK,
allowHalfOpen: false,
});
const screenshareStream = activeScreenshareStream;
let waitingForDrain = false;
// Configure WebSocket for low latency previousStream?.end();
ws.setMaxListeners(0); if (previousSocket && previousSocket.readyState === WebSocket.OPEN) {
ws.binaryType = "arraybuffer"; previousSocket.close(1000, "A newer screen sharing session started");
}
// Start mpv while the browser starts its recorder, instead of waiting for a
// whole encoded chunk before beginning the HTTP connection and demux setup.
const port = (server.address() as AddressInfo).port;
const url = `http://127.0.0.1:${port}/api/screenshareStream`;
console.log(`Starting screen share stream at ${url}`);
mediaPlayer.initiateScreenSharing(url).catch((error: unknown) => {
console.error("Failed to start screen sharing:", error);
if (activeScreenshareSocket === ws) {
ws.close(1011, "Unable to start the media player");
}
});
ws.on('message', (data: any) => { ws.on('message', (data: any) => {
const buffer = data instanceof Buffer ? data : Buffer.from(data); const buffer = data instanceof Buffer ? data : Buffer.from(data);
if (!firstChunk) { if (activeScreenshareStream !== screenshareStream) {
firstChunk = true; return;
}
const port = (server.address() as AddressInfo).port; if (!screenshareStream.write(buffer) && !waitingForDrain) {
const url = `http://localhost:${port}/api/screenshareStream`; waitingForDrain = true;
console.log(`Starting screen share stream at ${url}`); ws.pause();
screenshareStream.once('drain', () => {
// Create new shared stream with immediate flush waitingForDrain = false;
activeScreenshareStream = new PassThrough({ if (activeScreenshareStream === screenshareStream && ws.readyState === WebSocket.OPEN) {
highWaterMark: 1024 * 1024, // 1MB buffer ws.resume();
allowHalfOpen: false }
});
activeScreenshareStream.write(buffer);
mediaPlayer.initiateScreenSharing(url);
} else if (activeScreenshareStream) {
// Write with immediate flush
activeScreenshareStream.write(buffer, () => {
activeScreenshareStream?.cork();
activeScreenshareStream?.uncork();
}); });
} }
}); });
ws.on('close', () => { ws.on('close', () => {
console.log("Screen sharing client disconnected"); console.log("Screen sharing client disconnected");
if (activeScreenshareStream) { if (activeScreenshareStream === screenshareStream) {
activeScreenshareStream.end(); screenshareStream.end();
activeScreenshareStream = null; activeScreenshareStream = null;
activeScreenshareMimeType = null;
activeScreenshareSocket = null;
if (!screenshareStreamsClosedByViewer.has(screenshareStream)) {
mediaPlayer.stop().catch((error: unknown) => {
console.error("Failed to stop screen sharing:", error);
});
}
} }
mediaPlayer.stop();
}); });
}); });
+3 -1
View File
@@ -203,7 +203,9 @@ export const API = {
startScreenShare(mimeType: string): WebSocket { startScreenShare(mimeType: string): WebSocket {
const protocol = window.location.protocol === 'https:' ? 'wss' : 'ws'; const protocol = window.location.protocol === 'https:' ? 'wss' : 'ws';
const ws = new WebSocket(`${protocol}://${window.location.host}/api/screenshare?mimeType=${mimeType}`); const ws = new WebSocket(
`${protocol}://${window.location.host}/api/screenshare?mimeType=${encodeURIComponent(mimeType)}`,
);
return ws; return ws;
} }
}; };
+215 -46
View File
@@ -1,6 +1,17 @@
import { useState, useEffect, useRef, useCallback } from 'react'; import { useState, useEffect, useRef, useCallback } from 'react';
import { API } from '../api/player'; import { API } from '../api/player';
const TARGET_FRAME_RATE = 60;
const MAX_CAPTURE_WIDTH = 1920;
const MAX_CAPTURE_HEIGHT = 1080;
const RECORDER_TIMESLICE_MS = 16;
const MIN_VIDEO_BIT_RATE = 2_500_000;
const MAX_VIDEO_BIT_RATE = 8_000_000;
const VIDEO_BITS_PER_PIXEL = 0.065;
const SOCKET_BUFFER_TARGET_MS = 75;
const MIN_SOCKET_BUFFER_BYTES = 32 * 1024;
const MIN_RESUME_BUFFER_BYTES = 8 * 1024;
interface UseScreenShareResult { interface UseScreenShareResult {
isScreenSharing: boolean; isScreenSharing: boolean;
isScreenSharingSupported: boolean; isScreenSharingSupported: boolean;
@@ -8,15 +19,21 @@ interface UseScreenShareResult {
stopScreenShare: () => void; stopScreenShare: () => void;
} }
function getBestSupportedMimeType() { function getBestSupportedMimeType(hasAudio: boolean): string | undefined {
// Ordered by preference (best first) - all of these include audio+video // VP8 is substantially cheaper to encode than VP9 on most systems. Prefer it
const mimeTypes = [ // for real-time capture, then use hardware-friendly H.264 where available.
'video/webm;codecs=vp9,opus', // Best quality, good compression const mimeTypes = hasAudio ? [
'video/webm;codecs=vp8,opus', // Good fallback, well supported 'video/webm;codecs=vp8,opus',
'video/webm;codecs=h264,opus', // Better compatibility with some systems 'video/mp4;codecs=avc1.42E01E,mp4a.40.2',
'video/mp4;codecs=h264,aac', // Good for Safari but may not be supported for MediaRecorder 'video/webm;codecs=vp9,opus',
'video/webm', // Generic fallback (browser will choose codecs) 'video/webm',
'video/mp4' // Last resort 'video/mp4',
] : [
'video/webm;codecs=vp8',
'video/mp4;codecs=avc1.42E01E',
'video/webm;codecs=vp9',
'video/webm',
'video/mp4',
]; ];
// Find the first supported mimetype // Find the first supported mimetype
@@ -27,15 +44,42 @@ function getBestSupportedMimeType() {
} }
} }
// If none are supported, return null or a basic fallback
console.warn('No preferred mime types supported by this browser'); console.warn('No preferred mime types supported by this browser');
return 'video/webm'; // Most basic fallback return undefined;
}
function getTargetVideoBitRate(track: MediaStreamTrack): number {
const settings = track.getSettings();
const width = settings.width ?? MAX_CAPTURE_WIDTH;
const height = settings.height ?? MAX_CAPTURE_HEIGHT;
const frameRate = Math.min(settings.frameRate ?? TARGET_FRAME_RATE, TARGET_FRAME_RATE);
const target = width * height * frameRate * VIDEO_BITS_PER_PIXEL;
return Math.round(Math.min(MAX_VIDEO_BIT_RATE, Math.max(MIN_VIDEO_BIT_RATE, target)));
}
function getSocketBufferLimits(recorder: MediaRecorder) {
const bytesPerSecond = (recorder.videoBitsPerSecond + recorder.audioBitsPerSecond) / 8;
const maximum = Math.max(
MIN_SOCKET_BUFFER_BYTES,
Math.ceil(bytesPerSecond * SOCKET_BUFFER_TARGET_MS / 1000),
);
return {
maximum,
resumeAt: Math.max(MIN_RESUME_BUFFER_BYTES, Math.floor(maximum / 4)),
};
} }
export const useScreenShare = (): UseScreenShareResult => { export const useScreenShare = (): UseScreenShareResult => {
const [isScreenSharing, setIsScreenSharing] = useState(false); const [isScreenSharing, setIsScreenSharing] = useState(false);
const [isScreenSharingSupported, setIsScreenSharingSupported] = useState(false); const [isScreenSharingSupported, setIsScreenSharingSupported] = useState(false);
const screenShareSocketRef = useRef<WebSocket | null>(null); const screenShareSocketRef = useRef<WebSocket | null>(null);
const mediaRecorderRef = useRef<MediaRecorder | null>(null);
const mediaStreamRef = useRef<MediaStream | null>(null);
const backpressureTimerRef = useRef<number | null>(null);
const isStartingRef = useRef(false);
const sessionGenerationRef = useRef(0);
// Check if screen sharing is supported // Check if screen sharing is supported
useEffect(() => { useEffect(() => {
@@ -47,80 +91,205 @@ export const useScreenShare = (): UseScreenShareResult => {
}, []); }, []);
const stopScreenShare = useCallback(() => { const stopScreenShare = useCallback(() => {
if (screenShareSocketRef.current) { sessionGenerationRef.current += 1;
screenShareSocketRef.current.close(); isStartingRef.current = false;
screenShareSocketRef.current = null;
if (backpressureTimerRef.current !== null) {
window.clearInterval(backpressureTimerRef.current);
backpressureTimerRef.current = null;
} }
const recorder = mediaRecorderRef.current;
mediaRecorderRef.current = null;
if (recorder) {
recorder.ondataavailable = null;
recorder.onerror = null;
if (recorder.state !== 'inactive') {
recorder.stop();
}
}
const mediaStream = mediaStreamRef.current;
mediaStreamRef.current = null;
mediaStream?.getTracks().forEach((track) => {
track.onended = null;
track.stop();
});
const socket = screenShareSocketRef.current;
screenShareSocketRef.current = null;
if (socket) {
socket.onopen = null;
socket.onclose = null;
socket.onerror = null;
if (socket.readyState === WebSocket.CONNECTING || socket.readyState === WebSocket.OPEN) {
socket.close();
}
}
setIsScreenSharing(false); setIsScreenSharing(false);
}, []); }, []);
const startScreenShare = useCallback(async () => { const startScreenShare = useCallback(async () => {
if (isStartingRef.current || screenShareSocketRef.current) {
return;
}
isStartingRef.current = true;
const sessionGeneration = ++sessionGenerationRef.current;
try { try {
const mediaStream = await navigator.mediaDevices.getDisplayMedia({ const mediaStream = await navigator.mediaDevices.getDisplayMedia({
video: true, video: {
frameRate: { ideal: TARGET_FRAME_RATE, max: TARGET_FRAME_RATE },
width: { max: MAX_CAPTURE_WIDTH },
height: { max: MAX_CAPTURE_HEIGHT },
},
audio: true, audio: true,
}); });
let mimeType = getBestSupportedMimeType(); // getDisplayMedia cannot be programmatically dismissed. If the user
console.log('Using MIME type:', mimeType); // stopped or restarted while its picker was open, discard this result.
if (sessionGenerationRef.current !== sessionGeneration) {
mediaStream.getTracks().forEach((track) => track.stop());
return;
}
const mediaRecorder = new MediaRecorder(mediaStream, { mediaStreamRef.current = mediaStream;
mimeType: mimeType,
videoBitsPerSecond: 2500000, // 2.5 Mbps const videoTrack = mediaStream.getVideoTracks()[0];
audioBitsPerSecond: 128000, // 128 kbps if (!videoTrack) {
throw new Error('The selected source did not provide a video track');
}
// Favor smooth motion over static-text quality when the browser needs to
// trade capture resolution for frame rate.
videoTrack.contentHint = 'motion';
mediaStream.getAudioTracks().forEach((track) => {
track.contentHint = 'music';
}); });
const hasAudio = mediaStream.getAudioTracks().length > 0;
const preferredMimeType = getBestSupportedMimeType(hasAudio);
const recorderOptions: MediaRecorderOptions = {
videoBitsPerSecond: getTargetVideoBitRate(videoTrack),
};
if (preferredMimeType) {
recorderOptions.mimeType = preferredMimeType;
}
if (hasAudio) {
recorderOptions.audioBitsPerSecond = 128_000;
}
const mediaRecorder = new MediaRecorder(mediaStream, recorderOptions);
mediaRecorderRef.current = mediaRecorder;
const mimeType = mediaRecorder.mimeType || preferredMimeType || 'application/octet-stream';
const socketBufferLimits = getSocketBufferLimits(mediaRecorder);
console.log(
`Screen sharing at up to ${TARGET_FRAME_RATE} fps, ${Math.round(mediaRecorder.videoBitsPerSecond / 1000)} kbps, using ${mimeType}`,
);
// Connect to WebSocket // Connect to WebSocket
screenShareSocketRef.current = API.startScreenShare(mimeType); const socket = API.startScreenShare(mimeType);
screenShareSocketRef.current = socket;
isStartingRef.current = false;
const resumeRecorderWhenDrained = () => {
if (screenShareSocketRef.current !== socket || mediaRecorderRef.current !== mediaRecorder) {
if (backpressureTimerRef.current !== null) {
window.clearInterval(backpressureTimerRef.current);
backpressureTimerRef.current = null;
}
return;
}
if (socket.bufferedAmount <= socketBufferLimits.resumeAt && mediaRecorder.state === 'paused') {
mediaRecorder.resume();
if (backpressureTimerRef.current !== null) {
window.clearInterval(backpressureTimerRef.current);
backpressureTimerRef.current = null;
}
}
};
const pauseRecorderForBackpressure = () => {
if (mediaRecorder.state === 'recording') {
mediaRecorder.pause();
}
if (backpressureTimerRef.current === null) {
backpressureTimerRef.current = window.setInterval(
resumeRecorderWhenDrained,
RECORDER_TIMESLICE_MS,
);
}
};
// Set up WebSocket event handlers // Set up WebSocket event handlers
screenShareSocketRef.current.onopen = () => { socket.onopen = () => {
if (screenShareSocketRef.current !== socket) {
socket.close();
return;
}
console.log('Screen sharing WebSocket connected'); console.log('Screen sharing WebSocket connected');
setIsScreenSharing(true); setIsScreenSharing(true);
mediaRecorder.start(RECORDER_TIMESLICE_MS);
mediaRecorder.start(100);
}; };
screenShareSocketRef.current.onclose = () => { socket.onclose = () => {
console.log('Screen sharing WebSocket closed'); console.log('Screen sharing WebSocket closed');
setIsScreenSharing(false); if (screenShareSocketRef.current === socket) {
stopScreenShare();
// Stop all tracks when WebSocket is closed }
mediaStream.getTracks().forEach(track => track.stop());
}; };
screenShareSocketRef.current.onerror = (error) => { socket.onerror = (error) => {
console.error('Screen sharing WebSocket error:', error); console.error('Screen sharing WebSocket error:', error);
setIsScreenSharing(false); if (screenShareSocketRef.current === socket) {
stopScreenShare();
}
};
// Stop all tracks on error mediaRecorder.onerror = (error) => {
mediaStream.getTracks().forEach(track => track.stop()); console.error('Screen sharing recorder error:', error);
if (mediaRecorderRef.current === mediaRecorder) {
stopScreenShare();
}
}; };
// Send data over WebSocket when available // Send data over WebSocket when available
mediaRecorder.ondataavailable = (event) => { mediaRecorder.ondataavailable = (event) => {
if (event.data && event.data.size > 0 && screenShareSocketRef.current && screenShareSocketRef.current.readyState === WebSocket.OPEN) { if (event.data.size > 0 && screenShareSocketRef.current === socket && socket.readyState === WebSocket.OPEN) {
screenShareSocketRef.current.send(event.data); socket.send(event.data);
// Browser WebSockets otherwise queue without a limit, turning a brief
// network slowdown into permanently stale playback. Pausing capture
// bounds that queue and resumes at the live edge once it drains.
if (socket.bufferedAmount >= socketBufferLimits.maximum) {
pauseRecorderForBackpressure();
}
} }
}; };
// Handle stream ending (user clicks "Stop sharing") // Handle stream ending (user clicks "Stop sharing")
mediaStream.getVideoTracks()[0].onended = () => { videoTrack.onended = stopScreenShare;
if (screenShareSocketRef.current) {
screenShareSocketRef.current.close();
screenShareSocketRef.current = null;
}
setIsScreenSharing(false);
};
} catch (error) { } catch (error) {
if (sessionGenerationRef.current === sessionGeneration) {
console.error('Error starting screen share:', error); console.error('Error starting screen share:', error);
setIsScreenSharing(false); stopScreenShare();
} }
}, []); }
}, [stopScreenShare]);
useEffect(() => {
return () => stopScreenShare();
}, [stopScreenShare]);
const toggleScreenShare = useCallback(async () => { const toggleScreenShare = useCallback(async () => {
if (screenShareSocketRef.current) { if (screenShareSocketRef.current || isStartingRef.current) {
stopScreenShare(); stopScreenShare();
} else { } else {
await startScreenShare(); await startScreenShare();