Thanks to a tip of my reader, I found a relatively cheap (69Eur) Android box with HDMI input and record ability. I thought that it should be possible not only to record the HDMI input but also to stream it in realtime so I ordered one for experimenting.
The device came without any box, just in bubble foil but it doesn’t matter to me if it keeps the cost low. You can find functional review of the device on cnx-linux page.
Bottom side of PCB:
Upper side with heatsink:
Upper side without heatsink:
As you can see from the pictures, device uses MSO9180D1R SoC for which I couldn’t find any datasheet. The traces from HDMI input go directly to the SoC which is a good sign, that the signal is not converted to analog form (at least outside the SoC).
HDMI recording app is available directly from main menu and offers recording in MP4 or TS (mpegts) format, both with H264 video and AAC audio. MP4 format is not suitable for streaming (at least not until the recording is finished) because the metadata of file is written only after the recording is finished. On the other side, TS is ideal for streaming because data is segmented into properly formated 188bytes packets.
To gain shell access, I installed SSH Server from Google Play which worked without any problems. After connecting, I was easily able to gain root access to device because the devices comes pre-rooted from factory.
$ ssh firstname.lastname@example.org -p60722 The authenticity of host '[10.0.0.116]:60722 ([10.0.0.116]:60722)' can't be established. DSA key fingerprint is 12:45:aa:63:e9:c2:20:f0:e2:f9:dc:0a:b4:25:95:b8. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '[10.0.0.116]:60722' (DSA) to the list of known hosts. email@example.com's password: u0_a58@lemon:/ $ su root@lemon:/ #
Ok, let’s see what do we have here:
root@lemon:/ # cat /proc/cpuinfo Processor : ARMv7 Processor rev 1 (v7l) processor : 0 BogoMIPS : 2189.31 processor : 1 BogoMIPS : 2189.31 processor : 2 BogoMIPS : 2189.31 processor : 3 BogoMIPS : 2189.31 Features : swp half thumb fastmult vfp edsp neon vfpv3 tls CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x4 CPU part : 0xc09 CPU revision : 1 Hardware : napoli Revision : 0020 Serial : 0000000000000000
Ok, my goal was to stream live from HDMI so I thought it might be possible to just pipe tail of recorded TS file to network (preferably multicast) and see if it works. So I installed BusyBox Free from Google Play (to get nc utility). It showed up, that nc from busybox supports only TCP connections (and of course the segmentation to TS packets won’t be good) so this will not be a way.
After a while, ffmpeg came to my mind: you can specify output packet sizes there and in general it’s designed exactly for streaming/encoding and I have seen it used on Android before. So I googled ffmpeg precompiled binary, downloaded it and put it on my USB key to run it from shell.
First, I started VLC on my computer like this:
Then I started recording in “HDMI IN” application (which created recording file in /sdcard/HdmiRecorder/video20151227161832.ts) and then run ffmpeg in device shell:
./ffmpeg -re -i HdmiRecorder/video* -vcodec copy -acodec copy -f mpegts udp://188.8.131.52:1234
The result is here:
The stream played correctly. Great!
But when I stopped vlc, and tried to join the stream again, vlc was unable to start decoding video (audio was playing fine). I did a short analysis of the .ts file and found out, that it contains h264 metadata (SPS, PPS, more on this here) only in the beginning of the file and if these are not received by vlc, it doesn’t know how to decode the h264 bitstream.
but it was throwing some strange errors. I also tried to compile latest ffmpeg on my own (from https://github.com/WritingMinds/ffmpeg-android-java ) but it didn’t help, so I took it from another end: I tried to decompile the application.
Analyzing recording application
In Android, all apps are stored as .apk files in system folder. HDMI IN recording app is not an exception (/system/app/ZidooRecord.apk), so I downloaded it to my computer and decompiled it using http://www.decompileandroid.com/ (great tool, thanks to the author).
After some reading of the code and googling, I found out, that zidoo (maker of the app) has user forum and even more, they provide sources of this app on github (https://github.com/zidootech/zidoorecorder) !
Side note: To avoid confusion here, Zidoo X9 is product from Zidoo company built on top of the same HW platform from company called Mstar who developed the SoC. So Tronsmart is the same HW platform with the same software just sold under other name/company.
Those github sources include binary library integrated with Android using NDK which allows recording from HDMI. The binary is very briefly documented and sources are available only to distributors with signed NDA so I have struggled with cutting the sources to minimum and compiling it myself. But finally I succeeded building my own recording app. https://github.com/danielkucera/ZidoStreamer
I was hoping that I’ll be able to change the recording format so it contains correct extradata before each keyframe but it wasn’t possible (more info here). After start, my app successfully started recording to file and started ffmpeg which streamed it to network but this was little improvement compared to where I was with original app, I still had to catch the beginning of the stream otherwise the stream didn’t work. So I came back to experiment with ffmpeg.
Working ffmpeg command
After careful reading of ffmpeg documentation (https://www.ffmpeg.org/ffmpeg-bitstream-filters.html#dump_005fextra) and some experimentation I found a command which makes working stream:
tail -f -c 1000000000 HdmiRecorder/* | ./ffmpeg -i - -vcodec copy -acodec copy -bsf:v dump_extra -f mpegts udp://184.108.40.206:1234
The point is: tail starts to read recorded file from beginning (-c 1000000000) and keeps reading on (-f) while piping to ffmpeg which reads from stdin (-i -), create correct extradata before each keyframe (-bsf:v dump_extra – this is the key!) and stream to network as mpegts stream (-f mpegts udp://220.127.116.11:1234)
What we have now and where we go
So for now, we have a working android app that starts recording from HDMI input using proprietary binary and working ffmpeg command to create playable stream.
My target is to create app which will be able to start recording without that binary (it should be possible according to some tests I did), set different bitrates/resolutions, omit the recording to file (use just fifo file to avoid flash memory speed bottleneck and limited size – done some successful tests, just need to fix bugs) and allow different output stream types, e.g. rtmp, udp mpegts unicast, tcp mpegts unicast, rtp etc. (this all can be handled by ffmpeg, just needs to be correctly setup). So for now, I have “proof-of-concept” and a plan.
If you are interested in this or want to help, feel free to comment, fork, push commits, whatever.
Bye for now, watch my blog/repo for progress report.