[Arm-netbook] [EOMA68] A20 G2D / VAAPI / VDPAU video acceleration in web browsers

[Andreas Baierl]

Hi Luke,

Am 10.08.2016 um 00:16 schrieb Luke Kenneth Casson Leighton:
> ---
> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
>
>
> On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer <russell.hyer at gmail.com> wrote:
>> Hi Luke,
>>
>> So, if I've understood this post correctly, video playback on vlc
>> would perhaps be within the bounds of what the A20 hardware / software
>> could do?
>   yes.  ffplay, vlc, mpv, mplayer - all work fine, that's what this
> update was about:
>    https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
>
>   basically by using the full capabilities of the A20's accelerated
> hardware (both CEDAR and G2D), the CPU usage is somewhere around the
> 10% mark.  peanuts in other words.
>
>   the reason why those work is, through vdpau plugins there's two
> parts: "decode" part and "actual on-screen display" part.  all those
> video players have an option to do *both* things, and the vdpau-sunxi
> plugin that the Cedrus team wrote goes "okay, we're configured to use
> G2D, let's just go for it".
>
>   that's why you end up with the overlay issues, btw, because the G2D
> hardware is told to write direct to an area on-screen, bypassing the
> X11 protocol.
Just a minor correction. The G2D (mixer processor) hardware (only on 
A10/A20) is only used within libvdpau-sunxi to blit one rgba surface 
into another one. It has nothing to do with bringing something *really* 
to screen, because it only handles the blit or fill actions on rgba 
surfaces. The only use of rgba surfaces that i know of, is to realize an 
OSD, subtitles etc... as an overlay over the video.

NVidia and the others most likely do *real* blits of that rgba into the 
the yuv video. Maybe with the use of OpenGL.
Our libvdpau-sunxi does separate these surfaces and hold them both on 
separater layers. The display enginge takes care of these two layers and 
pushes the video into layer 1 and then the OSD (if existent) into 
another layer 2 which lays over the video layer. So data keeps separated.
This is the problem why some things with libvdpau-sunxi do not work 
exactly like someone expects. Especially in windowed mode. We use a 
colorkey to mask the video area and don't render with X11's mechanism, 
but bypass the whole X11 system and write directly to framebuffer as you 
said.
So to sum up, overlay issues are caused by the display engine, or the 
way libvdpau-sunxi build his surfaces, but not by G2D.
As we don't have G2D available on future Allwinner SoCs, the blits there 
are done in software using pixman.

>
>   anyway, it's *ASSUMED* in the webkit and xulrunner engines that the
> "on-screen display" bit can be handled by OpenGL or...
> something-else-don't-know-don't-care-but-basically-it's-not.
Oh. Maybe i completely misunderstood and my previous comment was 
b*****it, but is there really an o-s-d over the video within the browser 
window? If so, they most likely use something with OpenGL, yes.
>   and that means a bit of rewriting of both webkit and xulrunner so
> that they hand over the display bit to G2D.  that's going to require
> some specialist work.
It would theoretically be possible to let libvdpau-sunxi use OpenGL/ES 
under the hood and blit everything into one surface/texture. But this 
maybe would drop the performance advantages compared to how we do know. 
And you depend on Mali....
There are some efforts to integrate/simulate the nv_gl_interop extension 
with libvdpau-sunxi and OpenGL/ES, which lets OpenGL use vdpau surface 
and vice versa. So it would be possible to let vdpau do the rendering 
and OpenGL/ES do the presentation. Problem is, that it's not finished 
and it's not intended by the spec anyway to either use VDPAU on ARM nor 
combine it with OpenGL/ES, as the nv_gl_interop only talks about i386 
and OpenGL. <- This is the way kodi works with vdpau for example.

And the whole problem of this all is, that i currently cannot used with 
mainline kernel :) So i'd look further in having an open source video 
decoding driver and the needed display driver bits getting mainline ;)

Andreas
> l.
>
> _______________________________________________
> arm-netbook mailing list arm-netbook at lists.phcomp.co.uk
> http://lists.phcomp.co.uk/mailman/listinfo/arm-netbook
> Send large attachments to arm-netbook at files.phcomp.co.uk