[EPG] MHEG5 EPG Data Collector

Forums › Home Theatre (including HTPC and EPG Collector) › [EPG] MHEG5 EPG Data Collector

SJB

2945 posts

Uber Geek
Inactive user

#58935 23-Mar-2010 17:26

I've noticed that there has been a lot of discussion recently about the availability of EPG data.

As part of a largish application I have code that can process the EPG data that Freeview broadcasts via MHEG5 as well as any data broadcast via the EIT.

The MHEG5 data itself is easy to interpret, getting at it is not so easy. The code that deals with it is CSharp, around 40 classes excluding the Directshow BDA stuff.

Incidentally, last week the data broadcast started including TV Anytime references (ie CRID's). I'm still looking at whether this makes it more reliable to detect the programs to record in a series but it looks promising.

If anybody is interested let me know.

Regards

Steve

[Moderator update (MF): This is the link to download the MHEG5 Grabber and detailed instructtions]

[Moderator update (LC): Copied SJB's update to here.]

SJB:
Hi everyone

We have decided to move the support for EPG Collector from the individual forums we currently monitor in NZ, Australia and the UK to the EPG Collector open discussion forum on Sourceforge which is located here

http://sourceforge.net/projects/epgcollector/forums/forum/1125945.

With around 800 users now and more appearing in different parts of the world it was becoming more and more difficult for us to coordinate support and to make sure users in each country were aware of releases, faults, tips and tricks etc so it makes sense for us to centralise everything in one place.

We can also make use of the Sourceforge bug tracking and enhancement request forums that are available to us.

You will need to create a Sourceforge account if you want to post but it's a fairly painless process. We also suggest you set monitoring on for the forum so you will be notified of any new threads.

If anyone has any concerns or complaints please let us know.

Steve/Brad

1 | 2 | 3 | 4 | 5 | 6 | ... | 44

1819 posts

Uber Geek

Trusted

#310320 23-Mar-2010 17:52

Interesting stuff. What language is it written in?

mentalinc

3244 posts

Uber Geek

Trusted

#310322 23-Mar-2010 18:06

Interested.

CPU: AMD 5900x | RAM: GSKILL Trident Z Neo RGB F4-3600C16D-32GTZNC-32-GB | MB: Asus X570-E | GFX: EVGA FTW3 Ultra RTX 3080Ti| Monitor: LG 27GL850-B 2560x1440

Quic: https://account.quic.nz/refer/473833 R473833EQKIBX

Deev8

481 posts

Ultimate Geek

#310327 23-Mar-2010 18:24

hellonearthisman: Interesting stuff. What language is it written in?

The original post mentioned C#

SJB: The code that deals with it is CSharp, around 40 classes excluding the Directshow BDA stuff

SJB

2945 posts

Uber Geek
Inactive user

#310341 23-Mar-2010 18:56

I don't think it be very difficult to wrap some code around the MHEG5/EIT parsing routines so that you end up with a utility that you could run on a regular basis to acquire whichever EPG data you wanted and output an XML file. I already create an XML file but that's because I chose XML as the format for my applications reference files.

Ideally someone other than me should do it so as to spread the knowledge. It would need a fairly simple Directshow BDA graph (tuner / demod / demux / tif / s+t) at the front configurable for frequencies etc (although you could just scan a range or all the frequencies on a satellite) and a straight forward back end to output the collection of EPG objects that my parsers create to an XML file

You could include or exclude frequencies with scrambled broadcasts (eg Sky) depending on your needs (as you probably know the EIT data on those frequencies is not scrambled so is accessible).

As I said before the code is in CSharp (sorry hash key not working).

Anybody interested?

Steve

eXDee

4032 posts

Uber Geek

Trusted

#310522 24-Mar-2010 08:16

My C# skills aren't advanced enough to help but have you seen this project?
http://code.google.com/p/mheg2xmltv/

Its currently broken, perhaps your program can replace this?

freitasm

BDFL - Memuneh
79319 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#310542 24-Mar-2010 08:42

I would like to see a portable code I could run on my Media Center to get the MHEG5 and load it... Interested.

SJB

2945 posts

Uber Geek
Inactive user

#310592 24-Mar-2010 09:47

I saw the mheg2xmltv some time ago and that's what started me down the mheg path. I didn't use any of the code because it didn't fit in with my application and required some other software to collect the MHEG data. I wanted a completely self contained solution.

I'd be quite happy to supply all the code that collects the data (ie Directshow graphs, MHEG/EIT parsers) and creates a collection of EPG object. Someone else though would have to do the front end that drives the data collection with which frequencies are scanned and the back end which takes the EPG objects and creates an XML file because I don't know how you would want this to fit into your existing setups. I've never used things like Media Centre, I just write my own stuff.

The alternative would be for someone to specify to me what you need and I develop it but that's more time consuming and doesn't really spread the knowledge around. It also means you would be relying on just me to support it whereas the ideal project would have a few developers involved.

I can only test on satellite although the code I have detects the type of tuner (terrestrial/satellite/cable) and tunes accordingly.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

walt12

324 posts

Ultimate Geek

#310624 24-Mar-2010 10:35

Perhaps sub from GB-PVR would be the best person to be involved in this?

MCEabused

30 posts

Geek

#310632 24-Mar-2010 10:46

Having had a brief look at the Mheg standard, You really would need a critical mass of developer to handle the likely ongoing changes to the embedded EPG data structures and java encoding.
If you loosely compare MHEG5's java VM on certified set top boxes to Bluray's java VM on certified players.
You can see the fun and games to come. AnyDVD plays this game but they have a far larger marketplace of customers to support themselves on.

The UK's BBC and their DTG (the MHEG5 creators) are further down the road than NZ, so this arcticle may give you an idea of what the broadcaster are tring to achieve there
http://www.theregister.co.uk/2010/03/23/oft_canvas_probe/

I like my HTPC, so I hope we can sidestep this level of control

SJB

2945 posts

Uber Geek
Inactive user

#310671 24-Mar-2010 12:08

You don't need the Java VM or the downloaded Java app to get at the MHEG EPG data. It is present in the DSMCC object carousel in unencrypted text format. All the Java app does is take that data and display it in a consistent form across all STB's.

My code has nothing to do with Java and I'm using the MHEG data.

Of course, because the broadcaster is in control of the data and the application used to display it on a normal STB they can change both at any time (as TVNZ did last week) but the only thing that would be a complete stopper would be if it became encrypted. And even Sky don't do that.

I'm sure there will be technical challenges down the road but for now the data is accessible and remember we have an advantage - there are always more of us trying to crack this sort of stuff than there are creating it in the first place.

I've decided to knock out a little demo program from the code I have as proof of concept and maybe we can take it from there. It almost certainly won't create the format output file you can use but that can easily be changed. It should only take a day or so.

Regards

Steve

RustyGonad

495 posts

Ultimate Geek

Trusted

#310684 24-Mar-2010 12:31

Hi Steve,

While I'm not discounting anything you do with MHEG, as many/more people on here would get as much/more use from a basic EIT scraper for Windows to get the Sky guide (the Freeview guide is still readily available via multiple download sources, incl Freeview themselves).

In Windows 7 Media Center for example you have to go out of your way to get it to "release" the tuner, so for many people MHEG scraping would be alot harder than say the download app written by Rob V, which doesn't require MCE to release the tuner.

The only thing around at the moment is for Windows is DVBGuide, which while it does a good job of scraping onces it setup, it doesn't support changing the LNB oscillator frequency (10750,11300 etc) which makes it pretty limited on anything but Hauppage cards in NZ. It also needs a registry mod, which puts it into the hard to use category.

You mentioned you'd already written the code to do EIT, so anything you could do to include DVB-S/EIT scraping would be fantastic.

Cheers,

MCEabused

30 posts

Geek

#310735 24-Mar-2010 13:42

SBJ:
While the NZ MHEG EPG data is currently in plain text, that could change if you/someone comes up with a decent MHEG5 EPG utility.
MHEG-5 is far more than a simple set top box Display look & feel, Check out the parts that deal with Pay TV, conditional access, PSI SI signaling(BBC can hide programs & channels).

I really do hope you are right, in that they leave their MHEG5 data carousel EPG data as plain text in the future, as I would be a user of your utility.
(Freeview's stripping of program names in the DVB-T EIT is really annoying as the EPG should just be there per the DVB standard)

Ragnor

8223 posts

Uber Geek

Trusted

#310825 24-Mar-2010 16:41

SJB:

I'd be quite happy to supply all the code that collects the data (ie Directshow graphs, MHEG/EIT parsers) and creates a collection of EPG object. Someone else though would have to do the front end that drives the data collection with which frequencies are scanned and the back end which takes the EPG objects and creates an XML file because

Sounds good, best bet would be to host it as a project Google Code or MS Codeplex (or similar site) so anyone interested could pull it down and have a look at it.

There are plenty of C# developers around fortunately ;)

SJB

2945 posts

Uber Geek
Inactive user

#310933 24-Mar-2010 19:24

MCEAbused:
I know there is a great deal to MHEG5. I have most of the publicly available documentation from ETSI/DTG/ISO on the subject (DSMCC as well) and there is a lot of it. It's definitely the future for STB's so lets get to know it.

Rusty:
The demo program will do EIT and MHEG. I'm using an ini file for frequencies, dish parameters etc. How does Rob's utility get round the tuner problem on W7? Is it an MCE addin of some sort?

At the moment the EPG data is accessible so lets use it.

Regards

Steve

mentalinc

3244 posts

Uber Geek

Trusted

#310943 24-Mar-2010 19:53

SJB: At the moment the EPG data is accessible so lets use it.

When are you planning on releasing the code?

CPU: AMD 5900x | RAM: GSKILL Trident Z Neo RGB F4-3600C16D-32GTZNC-32-GB | MB: Asus X570-E | GFX: EVGA FTW3 Ultra RTX 3080Ti| Monitor: LG 27GL850-B 2560x1440

Quic: https://account.quic.nz/refer/473833 R473833EQKIBX

1 | 2 | 3 | 4 | 5 | 6 | ... | 44