Tivo epg slices versus xmltv

Forums › Home Theatre (including HTPC and EPG Collector) › Tivo epg slices versus xmltv

lchiu7

6476 posts

Uber Geek

Trusted

#16290 3-Oct-2007 09:26

Most of us running homebrew PVR's are probably using XMLTVNZ courtesy of Reven. Great tool and the PVR community owes a great debt of gratitude to Reven for doing that.

Looking at the tivo community here, they don't use XMLTVNZ but seem to rely on a community of users who use various means to update the epg (from scraping to I am guessing, hand entry) and then run some convoluted process to generate a programme slice which the Tivo units ftp each night. Out of interest I took a look at the slice file but it's in some binary format which I didn't have time to decode (well I had no file layouts!).

Just wondering if folks have any views on this. Some enterprising person could potentially write a program to read the slice file and output xml as a backup in case screen scraping fails again

Larry

Staying in Wellington. Check out my AirBnB in the Wellington CBD. https://www.airbnb.co.nz/h/wellycbd PM me and mention GZ to get a 15% discount and no AirBnB charges.

1 | 2 | 3 | 4

1140 posts

Uber Geek

#89248 3-Oct-2007 14:07

While in the past most of us MediaPortal people have relied on Reven's awesome xmltvnz, it's now come to light that for those of us who have a DVB-S cards for freeview it is actually possible to use the sky transponders as an EPG source, even though the picture is encrypted the EPG data is not. This is by far the most reliable way of getting EPG data.

Help me build a better way of doing politics in Aotearoa New Zealand

political.nz

lchiu7

6476 posts

Uber Geek

Trusted

#89281 3-Oct-2007 15:58

What about those who don't have Sky? Can you still get the epg source using a DVB-S card? I am thinking of getting a DVB-S card for Freeview so that would be a reliable solution

Staying in Wellington. Check out my AirBnB in the Wellington CBD. https://www.airbnb.co.nz/h/wellycbd PM me and mention GZ to get a 15% discount and no AirBnB charges.

itey

476 posts

Ultimate Geek

#89306 3-Oct-2007 17:15

DVB-S cards for freeview it is actually possible to use the sky transponders as an EPG source, even though the picture is encrypted the EPG data is not. This is by far the most reliable way of getting EPG data.

wow that is awesome, can you provide any more info?

cheers

mcraenz

1140 posts

Uber Geek

#89396 4-Oct-2007 08:26

Well if you don't have sky (via SVIDEO in on your PVR) then Prime would be the only channel you'd gain EPG data for, because everything else would be coming from the freeview anyway.

Here's how you do it in Mediaportal TVServer: (I guess other PVRs are the same)

The way TVServer works is that you have 1 Channel with multiple sources, [analog], [DVB-S], [DVB-T], [DVB-C], etc

so take discovery for example you'd Set it up via the svideo input+blaster as usual, then you need to scan in the sky transponders, (which are listed at lyngsat.com). You'll now have all those encrypted channels listed. Go the combinations section under TV Channels and select your analog card which will have your sky channels(SVIDEO from Sky STB) listed on the left select Discovery and you'll see in the right pane that Dicsovery (DVB-S encrtpyed) is listed at the top select it and click "combine".

What you have now is Discovery with 2 sources Analog and DVB-S. So now the DVB-S is a source of EPG for the channel "Discovery"

1 Problem, if you try to try to watch Discovery via DVB-S then it will throw an error "Channel is encrypted".

Solution 1: Under the TV Servers section select your TV Server and move the priority of the analog card above the priority of the DVB-S card. That means it will always be chosen first for the sky channels

Solution 2 (Some people have reported that this works (not for me though)): Un-map the the encrypted SKY channels from the DVB-S card

Hope this makes some sense!

Help me build a better way of doing politics in Aotearoa New Zealand

political.nz

illicit

553 posts

Ultimate Geek

#89586 5-Oct-2007 07:42

For those of you using Bladerunner Pro & xmlTVNZ, i have found a source for epg data thats already 'ripped' from the sky EPG data stream AND converted to xml and available online (nice.net.nz/epg - use the listings-all file) which is updated twice daly!

Much, much more reliable than xmlTVNZ and has Both the freeview & sky data!
Now if someone can it to work with BRP and let us know how - we can all enjoy this reliable source. Currently ALOT of MythTV users use this, but im too much of a novice to get it going with BRP (currently just use xmlTVNZ)

Craig

upsilon

84 posts

Master Geek

#89618 5-Oct-2007 11:48

Thats a good link there illicit, my guess is if we could run an XSLT over it to make it the same as the output from XMLTVNZ, then BRP should have no trouble parsing it. Might have a try this weekend if i get a chance.

lchiu7

6476 posts

Uber Geek

Trusted

#89664 5-Oct-2007 15:49

I looked at it also. Apart from the legality of it, looked like a good source. But to automate it you would need to write a program to download the file each night, and then parse the file to produce the format required by the various pvr's.

I suspect something like Python could do the download and any program do the parsing

Staying in Wellington. Check out my AirBnB in the Wellington CBD. https://www.airbnb.co.nz/h/wellycbd PM me and mention GZ to get a 15% discount and no AirBnB charges.

Samsung

Shop now on Samsung phones, tablets, TVs and more (affiliate link).

rattler

72 posts

Master Geek

#89669 5-Oct-2007 16:24

illicit: For those of you using Bladerunner Pro & xmlTVNZ, i have found a source for epg data thats already 'ripped' from the sky EPG data stream AND converted to xml and available online (nice.net.nz/epg - use the listings-all file) which is updated twice daly!

Much, much more reliable than xmlTVNZ and has Both the freeview & sky data!
Now if someone can it to work with BRP and let us know how - we can all enjoy this reliable source. Currently ALOT of MythTV users use this, but im too much of a novice to get it going with BRP (currently just use xmlTVNZ)

Craig

i had a look at nice.net.nz and noticed that he had (under tools) what could be the tools required to get it into bladerunner.

dunno.

illicit

553 posts

Ultimate Geek

#89804 6-Oct-2007 12:46

upsilon: Thats a good link there illicit, my guess is if we could run an XSLT over it to make it the same as the output from XMLTVNZ, then BRP should have no trouble parsing it. Might have a try this weekend if i get a chance.

If you could get that working - you will be a god!

Ive tried but just dont have the experience/know how to do so.

What i assume needs done is...

1: A way to download the 'listings_all' file from nice.net.nz automatically at set intervals (every day or 3-4 days etc)

2: Then unzip the downloaded file to the BRP directory

3: Get the file working with BRP

4: Have instructions on how to setup with BRP & MCE

I hope this is do-able because I for one would much rather use this source, due to reliablilty and the Freeview guide listings.

Good Luck - I hope it works

illicit

lchiu7

6476 posts

Uber Geek

Trusted

#89806 6-Oct-2007 12:56

I have been thinking about this. Way before xmltvnz was available there was a xmltv script (Perl) that did some scraping etc. But it wasn't supported and stopped working after the sites changed. But I recall from reading the script, it did actually call some Perl functions to read a web site etc. So it would be pretty easy to write a Perl script to download the file from this server and use Windows task scheduler to run the script at some defined interval.

As for transforming the xml data into a format usable by the various PVR's that is just some simple coding. I might have a bash at it when I get the chance

Larry

Staying in Wellington. Check out my AirBnB in the Wellington CBD. https://www.airbnb.co.nz/h/wellycbd PM me and mention GZ to get a 15% discount and no AirBnB charges.

illicit

553 posts

Ultimate Geek

#89808 6-Oct-2007 13:25

There is a monetary reward for those who get this working reliably........

upsilon

84 posts

Master Geek

#89944 7-Oct-2007 15:19

After a quick look, the info from nice.net.nz is very close to the output of XMLTVNZ. Just doing a manual find and replace i have been able to get the data into media centre using BRP as normal. Now just need to find a way to automate it.

Anyone know who operates Nice.net.nz?

tonyhughes

Hawkes Bay
8476 posts

Uber Geek

Retired Mod

Trusted

Lifetime subscriber

#89946 7-Oct-2007 15:31

upsilon: After a quick look, the info from nice.net.nz is very close to the output of XMLTVNZ. Just doing a manual find and replace i have been able to get the data into media centre using BRP as normal. Now just need to find a way to automate it.

Anyone know who operates Nice.net.nz?

Hadlee Rich

lchiu7

6476 posts

Uber Geek

Trusted

#89950 7-Oct-2007 15:41

Since this feed comes from the (I am guessing) Sky satellite and has to be unencrypted else perhaps the STB's can't read it, I wonder if it could ever be legally stopped? The web site could be shut down but it would be just as easy to host this file via bittorrents or via a ftp server a la the tivo slices

Might be the most reliable source to date and pretty much independent of site design.

Since as somebody notes it's pretty easy to transform the file into a form that most PVR's can read, automation of the download is next and I think that is pretty easy.

Staying in Wellington. Check out my AirBnB in the Wellington CBD. https://www.airbnb.co.nz/h/wellycbd PM me and mention GZ to get a 15% discount and no AirBnB charges.

upsilon

84 posts

Master Geek

#89954 7-Oct-2007 16:16

Infact no transformation is required of the xml file, just a modified ChannelInfo.xml file to include the correct channel id's used in the downloaded epg file

This is mine (i only have freeview)

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<NewDataSet>
<Channel>
    <name>One</name>
    <channelID>tv1.freeviewnz.tv</channelID>
    <virtualchannel>1</virtualchannel>
</Channel>
<Channel>
    <name>TV 2</name>
    <channelID>tv2.freeviewnz.tv</channelID>
    <virtualchannel>2</virtualchannel>
</Channel>
<Channel>
    <name>TV 3</name>
    <channelID>tv3.freeviewnz.tv</channelID>
    <virtualchannel>3</virtualchannel>
</Channel>
<Channel>
    <name>C4</name>
    <channelID>c4.freeviewnz.tv</channelID>
    <virtualchannel>4</virtualchannel>
</Channel>
<Channel>
    <name>TVNZ 6</name>
    <channelID>tvnz6.freeviewnz.tv</channelID>
    <virtualchannel>5</virtualchannel>
</Channel>
<Channel>
    <name>Stratos</name>
    <channelID>stratos.freeviewnz.tv</channelID>
    <virtualchannel>6</virtualchannel>
</Channel>
<Channel>
    <name>Maori TV</name>
    <channelID>maori-tv.freeviewnz.tv</channelID>
    <virtualchannel>7</virtualchannel>
</Channel>
<Channel>
    <name>SportExtra</name>
    <channelID>sportextra.freeviewnz.tv</channelID>
    <virtualchannel>8</virtualchannel>
</Channel>
</NewDataSet>

1 | 2 | 3 | 4