implement audio and video basic catalog components

[andrewkolos]

Description

See prior discussion in proposal: https://docs.google.com/document/d/1cRPCuuGLmUYY-PGFXlf7mIhdOuM2rypfP1Cw3nj_myM/edit?usp=sharing

Closes #738. Closes #392. Closes #393. Closes #746.

Implements the audioPlayer and video components in the basic catalog.

audioPlayer is implemented simply, using a slider as the progress indicator/scrubbing control. The component also has your typical play/pause button, reading of the playhead position (e.g. 01:23), length of the audio file (e.g. 02:25), and a basic volume slider.

videoPlayer is likewise simple. Not supported on Linux due to the lack of a lightweight, turnkey video rendering package available on pub; instead, a placeholder is shown to warn developers targeting Linux, who can write their own implementation. I chose to not wrap everything within a card, since I think it's unnecessary since all video controls can fit in a neat control bar directly below the video contents.

Dependencies used and their sizes

audioplayers is used for audio streaming support. video_player is used for video. TL;DR these 0.5-2MB to the app bundle.

I did some crude before-and-after testing to see how the bundle_size of an app could change with these two new dependencies introduced.

I used examples/catalog_gallery as the test app.
I used the following package versions:

audioplayers: ^6.6.0
video_player: ^2.11.1
video_player_win: ^3.2.2
fvp: ^0.35.2 ## Alternative library that bundles in a media library rather than using the native OS' built in APIs.

Here is the data:

Platform	Baseline	audioplayers + video_player	fvp + video_player
Android	17.8 MB	18.7 MB (+0.9 MB / +5.1%)	30.3 MB (+12.5 MB / +70.3%)
iOS	18.0 MB	20.0 MB (+2.0 MB / +11.1%)
Linux
Windows	29.9 MB	30.5 MB (+0.6 MB / +1.9%)	44.9 MB (+15.0 MB / +50.1%)
macOS	44 MB	46 MB (+2 MB / +5%)	68 MB (+24 MB / +55%)

Other things to note

Introducing dependencies carries other burdens other than unnecessarily inflating bundle sizes for those that do not have interest in the audio and video components. At the time of writing, one can be seen just by building for macOS:

swift compiler warnings

andrewkolos-mac:genui_audio_dep_test andrewkolos$ flutter run -d macos --release
Resolving dependencies... 
Downloading packages... 
  matcher 0.12.18 (0.12.19 available)
  meta 1.17.0 (1.18.2 available)
  test_api 0.7.8 (0.7.10 available)
Got dependencies!
3 packages have newer versions incompatible with dependency constraints.
Try `flutter pub outdated` for more information.
Launching lib/main.dart on macOS in release mode...
Running pod install...                                             748ms
/Users/andrewkolos/.pub-cache/hosted/pub.dev/audioplayers_darwin-6.4.0/darwin/audioplayers_darwin/Sources/audioplayers_darwin/WrappedMediaPlayer.swift:205:14: warning: main actor-isolated property 'eventHandler' can not be referenced from a Sendable closure; this is an error in the Swift 6 language mode
        self.eventHandler.onLog(message: "player status: \(status), change: \(change)")
             ^
  private(set) var eventHandler: AudioPlayersStreamHandler
                   ^
/Users/andrewkolos/.pub-cache/hosted/pub.dev/audioplayers_darwin-6.4.0/darwin/audioplayers_darwin/Sources/audioplayers_darwin/WrappedMediaPlayer.swift:211:16: warning: call to main actor-isolated instance method 'reset()' in a synchronous nonisolated context; this is an error in the Swift 6 language mode
          self.reset()
               ^
  private func reset() {
               ^
/Users/andrewkolos/.pub-cache/hosted/pub.dev/audioplayers_darwin-6.4.0/darwin/audioplayers_darwin/Sources/audioplayers_darwin/WrappedMediaPlayer.swift:205:14: warning: main actor-isolated property 'eventHandler' can not be referenced from a Sendable closure; this is an error in the Swift 6 language mode
        self.eventHandler.onLog(message: "player status: \(status), change: \(change)")
             ^
  private(set) var eventHandler: AudioPlayersStreamHandler
                   ^
/Users/andrewkolos/.pub-cache/hosted/pub.dev/audioplayers_darwin-6.4.0/darwin/audioplayers_darwin/Sources/audioplayers_darwin/WrappedMediaPlayer.swift:211:16: warning: call to main actor-isolated instance method 'reset()' in a synchronous nonisolated context; this is an error in the Swift 6 language mode
          self.reset()
               ^
  private func reset() {
               ^
warning: Run script build phase 'Run Script' will be run during every build because it does not specify any outputs. To address this issue, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'Flutter Assemble' from project 'Runner')
Building macOS application...                                           
✓ Built build/macos/Build/Products/Release/genui_audio_dep_test.app (41.0MB)

If such drawbacks prove to be troublesome in the long term, we can move audio and video components into a separate package and deprecate the existing ones.

Pre-launch Checklist

• I read the Flutter Style Guide recently, and have followed its advice.
• I signed the CLA.
• I read the Contributors Guide.
• I have added sample code updates to the changelog.
• I updated/added relevant documentation (doc comments with ///).

If you need help, consider asking for advice on the #hackers-devrel channel on Discord.

[gemini-code-assist]

Code Review

This pull request introduces significant new functionality by implementing audioPlayer and video components, providing actual media playback capabilities across various platforms. However, a critical security concern exists: the handling of URLs from GenUI messages lacks validation, potentially leading to Client-Side Request Forgery or unauthorized access to local files. Strict URL scheme validation and domain allow-listing are recommended. Additionally, general feedback addresses improving asynchronous operations and optimizing logging practices.

[andrewkolos]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces audioPlayer and video components, which is a significant feature enhancement using the audioplayers and video_player packages. However, there are critical security concerns regarding the handling of URLs provided by the GenUI system. Specifically, the lack of URL scheme validation could lead to Server-Side Request Forgery (SSRF) or Local File Inclusion (LFI) if malicious URLs are processed. Additionally, an unhandled exception in Uri.parse within the Video component could lead to application crashes with malformed URLs. Beyond these security issues, consider simplifying the playback logic in the audio player, fixing a state management bug in the video player's volume controls, and addressing code duplication for better maintainability.

[andrewkolos]

Ready for review, but I'll have to look into #816 to get the main CI suite running against this PR again.

The diff here is long, but it's mostly native plugin registration files being updated.