[EPG] MHEG5 EPG Data Collector

Forums › Home Theatre (including HTPC and EPG Collector) › [EPG] MHEG5 EPG Data Collector

1 | ... | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44

6 posts

Wannabe Geek

#372359 24-Aug-2010 20:30

SJB:
boez:
TerminalMan: Hello all

I've got EPG Collector correctly obtaining an XML file from Sky UK and I'm popuilating Media Center via TVSource. All the data seems to come into MC OK but the issue I have is that in Media Center's Movie Browser, non of the extra movie metadata is populated for movie details obtained via XML - ie no cover art, actors, review etc. It obviously knows they are movies as they appear in Movie Browser. My system gets all this info fine for movies on channels that are obtained by my normal DVB-T feed so the mechanism works - just not for the XML obtained by EPG Collector. (Previous to using EPG Collector, I was using another method to get the XML for Sky and with this, the metadata was looked up ok.) Anyone in the UK getting a fully populated movie browser with EPG Collector?

Thanks

Media Center needs a 'date' tag in the EPG data before it will go and look up a movie and then show the extra metadata.

Unfortunately the data from the Sky stream doesn't appear to provide the movie release date other than (usually) somewhere in the program description (or maybe Steve doesn't save it to the XML data?). I've manually added a date tag to a few movies before importing it into MC and confirmed that the extra metadata appears.

HTH
Andy

The Collector does do some editing on the program description for OpenTV, especially for the NZ broadcast. The format of the title and description is different for each location (NZL, AUS, GBR).

If you want to see the unedited data add Debug:LOGTITLES,LOGDESCRIPTIONS to the ini file. This will create EPG Titles.log and EPG Descriptions.log. These files are much easier to read than the xmltv file.

I'm quite happy to put in more editing of the titles and descriptions if it will help. As the Location parameter is mandatory for OpenTV the editing can be specific to a location.

Take a look at the data and let me know if you would like any changes.

Steve

Hi Steve

I looked at the debug and for most movies, the description in the UK (SKY) is the following form:-

"More museum mayhem, as Ben Stiller and his ancient allies battle to keep camp pharaoh Hank Azaria from world domination. Manic family fun with Owen Wilson and Amy Adams. (2009)(101 mins)"

Is there any chance of parsing out the date at the end and adding it as a new tag into the XML for each movie? ie

<date>2009</date>

From what I can see, this would just have to be done for categories 192 to 210 ie all the movie categories.

Whilst not all movie descriptions have the date in the description, it seems like the vast majority do.

This would be brilliant for all the UK SKY users using Media Center.

Thanks

boez

4 posts

Wannabe Geek

#372406 24-Aug-2010 22:49

Steve

This is an example from your XML file grabbed from Sky UK of a movie:

<programme channel="301" start="20100808151000 +0100" stop="20100808163500 +0100"><title>The Crimson Wing</title>
<desc>This dazzling Disney documentary looks at the lifecycle of North Tanzania's lesser flamingo, and its struggle to survive amid vicious predators and an unforgiving habitat. (2008)(75 mins) Also in HD</desc>
<category>Movie</category>
<rating><value>PG</value>
</rating>
<video><aspect>16:9</aspect>
</video>
</programme>

If you could add a tag

<date>2008</date>

to the file them Media Center would look up the movie and provide DVD coverart, etc.

If this date isn't available directly from the stream them its a bit tricky as there is no 'template' over how the date is included in the description (if at all!). I knocked up a quick XML postprocessor in Delphi and got quite good results from parsing the description tag for all strings from '1940'-'2010' and if one is found using that as the basis of the date tag. Of course this is prone to errors if two dates are in the description but generally it works ok (about 97%).

HTH
Andy

boez

4 posts

Wannabe Geek

#372407 24-Aug-2010 22:55

p.s.

I thought it would take too long to parse each entry like this but by restricting it to movies only (from the 'category' tag) it only takes a second or so to process a 7 day Sky EPG (all movie channels included). A typical example would be 2750 movies and 376 not dated at all (hard to gauge the number which are erroneously dated!). Most of the non-dated movies were slightly more obscure anyway!

BornEvil

3 posts

Wannabe Geek

#372438 25-Aug-2010 05:58

Hi

Thanks for this great program, i finally have full Sky UK EPG with 7MC via DVBLink!

Grabbing Sky UK is working great. But I have timezone problems with Sky Italia.

I live in UK, so my timezone is GMT. The data for Sky Italia is grabbed perfectly and uses the ITA dictionary but all programs is shifted by +1hr (using Italian time, GMT +1).

Here is my config:

Output:skyit.xml
Tuner:2,1
Timeouts:10,900
Satellite:130
Location:ITA
Option:ACCEPTBREAKS
Option:USECHANNELID
Dish:9750000,10600000,11700000
Frequency:11880000,27500,3/4,V,OpenTV

And the log says this:

16:05:55:031 Collecting time zone data.
16:05:57:039 Time offset: ITA region 0 offset 02:00:00
16:05:57:040 Time zone count: 1 buffer space used: 74
16:05:57:040 Stopping time offset reader for frequency 11880000 PID 0x14
16:05:57:043 Local time offset set to 02:00:00 for country ITA region 0
16:05:57:044 Time offset will change to 02:00:00 at 28/03/2010 02:00:00

The actual skyit.xml file has CET correction of +0100. (same as my Sky UK grab for some reason)

What can I do to fix this?

SJB

2945 posts

Uber Geek
Inactive user

#372487 25-Aug-2010 09:41

BornEvil: Hi

Thanks for this great program, i finally have full Sky UK EPG with 7MC via DVBLink!

Grabbing Sky UK is working great. But I have timezone problems with Sky Italia.

I live in UK, so my timezone is GMT. The data for Sky Italia is grabbed perfectly and uses the ITA dictionary but all programs is shifted by +1hr (using Italian time, GMT +1).

Here is my config:

Output:skyit.xml
Tuner:2,1
Timeouts:10,900
Satellite:130
Location:ITA
Option:ACCEPTBREAKS
Option:USECHANNELID
Dish:9750000,10600000,11700000
Frequency:11880000,27500,3/4,V,OpenTV

And the log says this:

16:05:55:031 Collecting time zone data.
16:05:57:039 Time offset: ITA region 0 offset 02:00:00
16:05:57:040 Time zone count: 1 buffer space used: 74
16:05:57:040 Stopping time offset reader for frequency 11880000 PID 0x14
16:05:57:043 Local time offset set to 02:00:00 for country ITA region 0
16:05:57:044 Time offset will change to 02:00:00 at 28/03/2010 02:00:00

The actual skyit.xml file has CET correction of +0100. (same as my Sky UK grab for some reason)

What can I do to fix this?

OpenTV programs are broadcast (supposedly) with the time in GMT. As you can see from the log the Collector gets the local time offset for the location that needs to be added on from the broadcast data so for ITA its adding on 2 hours to the broadcast time. If you look at the log for the UK collection that will be 1 hour.

Fortunately you can override the time offset from the broadcast data with an undocumented parameter. Try adding a Timezone parameter to your ini file so for your Italian collection try Timezone:1.00

Steve

SJB

2945 posts

Uber Geek
Inactive user

#372494 25-Aug-2010 09:54

boez: p.s.

I thought it would take too long to parse each entry like this but by restricting it to movies only (from the 'category' tag) it only takes a second or so to process a 7 day Sky EPG (all movie channels included). A typical example would be 2750 movies and 376 not dated at all (hard to gauge the number which are erroneously dated!). Most of the non-dated movies were slightly more obscure anyway!

Parsing each entry is not a problem.

PM me with an email address where I can send you a test version to try.

Steve

BornEvil

3 posts

Wannabe Geek

#372610 25-Aug-2010 12:59

SJB:

OpenTV programs are broadcast (supposedly) with the time in GMT. As you can see from the log the Collector gets the local time offset for the location that needs to be added on from the broadcast data so for ITA its adding on 2 hours to the broadcast time. If you look at the log for the UK collection that will be 1 hour.

Fortunately you can override the time offset from the broadcast data with an undocumented parameter. Try adding a Timezone parameter to your ini file so for your Italian collection try Timezone:1.00

Steve

I looked at the xml file for the Sky UK grab and it is correct to actual time. But bearing in mind we are now in Daylight saving time (i.e. absolute GMT +1).

For example:

Sky UK file says:
20100828124500 +0100 <===this is giving me the correct output (for 28/08/2010 @12:45pm) in MC7 with importing via DVBlink.

looking at the skyit.xml file the Sky Italy's epg broadcast is in absolute GMT.

Sky IT file says:
20100828114500 +0100 <===a programme with this start time gives me GMT +1 (or absolute GMT +2) for the data in MC7.

What part of the xml file is responsible for the timezone offsets? from the above i can tell that EPG Collector did not edit each programme entry.

Anyway, i have tried the timezone:1.00 parameter but the Sky IT xml file still processes as started above. Am i dumb and missing something?

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

SJB

2945 posts

Uber Geek
Inactive user

#372657 25-Aug-2010 13:53

BornEvil:
SJB:

OpenTV programs are broadcast (supposedly) with the time in GMT. As you can see from the log the Collector gets the local time offset for the location that needs to be added on from the broadcast data so for ITA its adding on 2 hours to the broadcast time. If you look at the log for the UK collection that will be 1 hour.

Fortunately you can override the time offset from the broadcast data with an undocumented parameter. Try adding a Timezone parameter to your ini file so for your Italian collection try Timezone:1.00

Steve

I looked at the xml file for the Sky UK grab and it is correct to actual time. But bearing in mind we are now in Daylight saving time (i.e. absolute GMT +1).

For example:

Sky UK file says:
20100828124500 +0100 <===this is giving me the correct output (for 28/08/2010 @12:45pm) in MC7 with importing via DVBlink.

looking at the skyit.xml file the Sky Italy's epg broadcast is in absolute GMT.

Sky IT file says:
20100828114500 +0100 <===a programme with this start time gives me GMT +1 (or absolute GMT +2) for the data in MC7.

What part of the xml file is responsible for the timezone offsets? from the above i can tell that EPG Collector did not edit each programme entry.

Anyway, i have tried the timezone:1.00 parameter but the Sky IT xml file still processes as started above. Am i dumb and missing something?

As far as I know the +0100 field in the xml file is documentary only - it is optional in the xmltv spec anyway and the program time has to be the real program time, not the time before adding on the offset.

The reason that it is +0100 in the Italy collection is that it is taken from the machine it is running on, not the broadcast data (arguably bug number 1).

The reason the Timezone parameter didn't work is because of bug number 2. This parameter was put in just in case the time offset was not available from the broadcast data so it was not tested again after it became obvious that the time offset is always available from the broadcast. The value you put in was ignored.

If you PM me with an email address I'll give you a test version to try that will use the Timezone value.

Steve

BornEvil

3 posts

Wannabe Geek

#372685 25-Aug-2010 14:27

SJB:
BornEvil:
SJB:

OpenTV programs are broadcast (supposedly) with the time in GMT. As you can see from the log the Collector gets the local time offset for the location that needs to be added on from the broadcast data so for ITA its adding on 2 hours to the broadcast time. If you look at the log for the UK collection that will be 1 hour.

Fortunately you can override the time offset from the broadcast data with an undocumented parameter. Try adding a Timezone parameter to your ini file so for your Italian collection try Timezone:1.00

Steve

I looked at the xml file for the Sky UK grab and it is correct to actual time. But bearing in mind we are now in Daylight saving time (i.e. absolute GMT +1).

For example:

Sky UK file says:
20100828124500 +0100 <===this is giving me the correct output (for 28/08/2010 @12:45pm) in MC7 with importing via DVBlink.

looking at the skyit.xml file the Sky Italy's epg broadcast is in absolute GMT.

Sky IT file says:
20100828114500 +0100 <===a programme with this start time gives me GMT +1 (or absolute GMT +2) for the data in MC7.

What part of the xml file is responsible for the timezone offsets? from the above i can tell that EPG Collector did not edit each programme entry.

Anyway, i have tried the timezone:1.00 parameter but the Sky IT xml file still processes as started above. Am i dumb and missing something?

As far as I know the +0100 field in the xml file is documentary only - it is optional in the xmltv spec anyway and the program time has to be the real program time, not the time before adding on the offset.

The reason that it is +0100 in the Italy collection is that it is taken from the machine it is running on, not the broadcast data (arguably bug number 1).

The reason the Timezone parameter didn't work is because of bug number 2. This parameter was put in just in case the time offset was not available from the broadcast data so it was not tested again after it became obvious that the time offset is always available from the broadcast. The value you put in was ignored.

If you PM me with an email address I'll give you a test version to try that will use the Timezone value.

Steve

cheers, pm sent

UPDATE: The timezone paramerter only overrides the broadcast data's time offset as you have said. so the Sky IT EPG is accurate to absolute GMT. which would have been fine if we were not in DST. Our clocks go back (by -1 hr) to GMT on the 31st October.

AnthonyHoward

2 posts

Wannabe Geek

#372930 26-Aug-2010 04:04

Steve,

Thanks for all your hardwork. Are you any closer to being able to release a version that works with the Mystique Satix Dual DVB-S2?

I have two of these cards and I am itching to give your program a go.

Cheers,

Tony.

SJB

2945 posts

Uber Geek
Inactive user

#372936 26-Aug-2010 07:07

AnthonyHoward: Steve,

Thanks for all your hardwork. Are you any closer to being able to release a version that works with the Mystique Satix Dual DVB-S2?

I have two of these cards and I am itching to give your program a go.

Cheers,

Tony.

I had one delivered a couple of days ago but it was damaged. I have returned it but it will take at least a couple of weeks for a replacement to arrive.

It's still the only tuner the Collector doesn't work with that I know of.

Steve

AnthonyHoward

2 posts

Wannabe Geek

#372940 26-Aug-2010 07:28

Thanks for the reply Steve. Thats a shame to hear but it can't be helped. In the meantime if there are any logs/debugs I can gather to help you out just let me know.

Cheers,

Tony.

Deev8

481 posts

Ultimate Geek

#373981 28-Aug-2010 17:04

Could I suggest adding a reference to this support thread to the EPG Collector Sourceforge page. Anyone looking at the Support tab there sees:
Best Way To Get Help
Unfortunately, this project hasn't indicated the best way to get help.

giggles

59 posts

Master Geek

Trusted

#373989 28-Aug-2010 17:31

Deev8: Could I suggest adding a reference to this support thread to the EPG Collector Sourceforge page. Anyone looking at the Support tab there sees:
Best Way To Get Help
Unfortunately, this project hasn't indicated the best way to get help.

Well pointed out :-)

A link to this forum has been added to Sourceforge under 'Support'

GushK

2 posts

Wannabe Geek

#375664 2-Sep-2010 11:29

Hi

I've managed to pull down a complete xml file from the stream for sky uk. 2 questions though.

1) I get no description in the xml file. Every programme has "No Synopsis Available" in the desc tag. is this a bug? e.g

<programme
   start="20100901220500 +0100"
   stop="20100901221700 +0100"
   channel="9004">
   <title>Harry and his Bucket Full of...</title>
   <desc>No Synopsis Available</desc>
   <category>Children - Cartoons</category>
   <rating>
   <value>-</value>
   </rating>
   <subtitles
   type="teletext" />
  </programme>

2) Is there any way to extend the amount of guide data downloaded/included? I seem to be only getting 24hrs worth.

Here's my config file if it helps

Output:C:\Program Files (x86)\DVBLogic\DVBLink2\xmltv\TVGuide.xml
Satellite:282
Dish:9750000,10600000,11700000
Frequency:10773000,22000,5/6,H,OpenTV
Location:GBR
Option:ACCEPTBREAKS
Tuner:1
# Trace:BDA

1 | ... | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44