[Arm-netbook] [EOMA68] A20 G2D / VAAPI / VDPAU video acceleration in web browsers

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68


On Wed, Aug 10, 2016 at 8:23 AM, Andreas Baierl <list at imkreisrum.de> wrote:
> Hi Luke,
>
> Am 10.08.2016 um 00:16 schrieb Luke Kenneth Casson Leighton:
>>
>> ---
>> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
>>
>>
>> On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer <russell.hyer at gmail.com>
>> wrote:
>>>
>>> Hi Luke,
>>>
>>> So, if I've understood this post correctly, video playback on vlc
>>> would perhaps be within the bounds of what the A20 hardware / software
>>> could do?
>>
>>   yes.  ffplay, vlc, mpv, mplayer - all work fine, that's what this
>> update was about:
>>    https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
>>
>>   basically by using the full capabilities of the A20's accelerated
>> hardware (both CEDAR and G2D), the CPU usage is somewhere around the
>> 10% mark.  peanuts in other words.
>>
>>   the reason why those work is, through vdpau plugins there's two
>> parts: "decode" part and "actual on-screen display" part.  all those
>> video players have an option to do *both* things, and the vdpau-sunxi
>> plugin that the Cedrus team wrote goes "okay, we're configured to use
>> G2D, let's just go for it".
>>
>>   that's why you end up with the overlay issues, btw, because the G2D
>> hardware is told to write direct to an area on-screen, bypassing the
>> X11 protocol.
>
> Just a minor correction. The G2D (mixer processor) hardware (only on
> A10/A20) is only used within libvdpau-sunxi to blit one rgba surface into
> another one. It has nothing to do with bringing something *really* to
> screen, because it only handles the blit or fill actions on rgba surfaces.
> The only use of rgba surfaces that i know of, is to realize an OSD,
> subtitles etc... as an overlay over the video.

 interesting.... because if i don't enable OSD i get absolutely
nothing (black screen) on mpv and other players.

> NVidia and the others most likely do *real* blits of that rgba into the the
> yuv video. Maybe with the use of OpenGL.
> Our libvdpau-sunxi does separate these surfaces and hold them both on
> separater layers. The display enginge takes care of these two layers and
> pushes the video into layer 1 and then the OSD (if existent) into another
> layer 2 which lays over the video layer. So data keeps separated.
> This is the problem why some things with libvdpau-sunxi do not work exactly
> like someone expects. Especially in windowed mode. We use a colorkey to mask
> the video area and don't render with X11's mechanism, but bypass the whole
> X11 system and write directly to framebuffer as you said.

 .... i knew it was _something_ like that... :)

> So to sum up, overlay issues are caused by the display engine, or the way
> libvdpau-sunxi build his surfaces, but not by G2D.
> As we don't have G2D available on future Allwinner SoCs, the blits there are
> done in software using pixman.

 let's hope there's enough horsepower in those!

>>
>>   anyway, it's *ASSUMED* in the webkit and xulrunner engines that the
>> "on-screen display" bit can be handled by OpenGL or...
>> something-else-don't-know-don't-care-but-basically-it's-not.
>
> Oh. Maybe i completely misunderstood and my previous comment was b*****it,
> but is there really an o-s-d over the video within the browser window? If
> so, they most likely use something with OpenGL, yes.

 google "webkit gtk vaapi gstreamer" there's a blog by the (pretty
much world's only) person dealing with it.

>>
>>   and that means a bit of rewriting of both webkit and xulrunner so
>> that they hand over the display bit to G2D.  that's going to require
>> some specialist work.
>
> It would theoretically be possible to let libvdpau-sunxi use OpenGL/ES under
> the hood and blit everything into one surface/texture. But this maybe would
> drop the performance advantages compared to how we do know. And you depend
> on Mali....

 ... which would be bad.  every person who's running with libre
software at the moment would never be able to view videos.

> There are some efforts to integrate/simulate the nv_gl_interop extension
> with libvdpau-sunxi and OpenGL/ES, which lets OpenGL use vdpau surface and
> vice versa. So it would be possible to let vdpau do the rendering and
> OpenGL/ES do the presentation. Problem is, that it's not finished and it's
> not intended by the spec anyway to either use VDPAU on ARM nor combine it
> with OpenGL/ES, as the nv_gl_interop only talks about i386 and OpenGL. <-
> This is the way kodi works with vdpau for example.
>
> And the whole problem of this all is, that i currently cannot used with
> mainline kernel :)

 i knoooow....

l.