Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

Topic # 114552 23-Feb-2013 10:35 Send private message

Hello everyone

In the last few months a couple of people have asked about getting EPG data for some of the channels available from Optus D2. As you may be aware, there is no EPG data broadcast from that satellite. This thread is intended to be a discussion of that situation and possible solutions for it.

//----------------------------------------------------------------------------------
Before I go any further...

1. Anyone who has access to Optus D2 knows that there are a lot of special interest channels broadcast from that satellite. Sometimes when D2 (or even Freeview) are mentioned on GZ there one or two people who take the opportunity to slip in disparaging comments about certain channels. In the interests of encouraging people to freely talk about their interest in EPG data for any channel, I would respectfully request that anyone who post in this thread would refrain from making public judgements about the content or value of channels.

2. Some broadcasters, content rights holders and channel aggregators seem to have an issue with people who provide EPG data sources for use with HTPCs. It is not my intention to attract the anger of such organisations onto myself or anybody else who provides an EPG service or posts in this thread. Nor is it my intention to infringe on any intellectual property. If anybody has an issue with the content that I choose to post in this thread then kindly send me a PM and I'll be happy to take your concerns into consideration.

Thanks.
//----------------------------------------------------------------------------------

So lets talk about possible solutions...

One solution might be to source EPG data from an alternative XMLTV data source. Some of the popular channels on D2 such as RT, NHK World and Press TV are available with EPG in other parts of the world. If you can find an EPG source with the channels that you want then you may be sorted. However non-AU sources can be problematic as the data may require time zone conversion or not reflect the content of the local feed of the channel.

Another solution could be to write a website scraper. This solution would take a little bit more effort to develop but with a bit of work it could prove give the best outcomes (in terms of data accuracy, richness, customisation) in the long run. Of course, websites sometimes change format, so there are maintenance issues.

A third solution might be to redistribute data that is broadcast over-the-air. For example, Daystar and Russia Today are available from both Sky and Optus D2. I'm not sure whether Sky broadcasts include EPG for those channels or how happy Sky would be about redistribution of that data but that is another potential solution for some channels.

I can't think of any other solutions at present. Anybody else?

Moving right along to practical solutions...

valtam kindly shared that you can get EPG data for Russia Today from -->here<-- (http://tvprofil.net/xmltv/).

Some years ago (in the time of XMLTV.co.nz) I started writing a script that aggregated the EPG data for maybe a dozen of the D2 channels. I shared it with the owner of XMLTV.co.nz and for a short time there were public listings available for the channels that my script grabbed. Then Sky came to the rescue and the website had to be taken down. I stopped using and developing the script shortly after as I lost access to D2. However I still have that script and some of the channels still work (hurray for websites not being updated in 3 years!). I've spent a couple of hours updating the script and now have basic data for the following channels:

Russia Today
Press TV
NHK World
TBN
JCTV/Smile Of A Child
The Church Channel

Although I no longer have access to D2, I'd be happy to consider making the script or data available in some form or another. Maybe even maintaining or extending it depending on interest. You can download a sample XMLTV output with data for the next 14 days or so from -->here<--.

Anybody else have any info or solutions that they'd be willing to share?

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2


871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 768363 24-Feb-2013 00:14 Send private message

Had a couple of hours with nothing to do today so added grabbers for:
  • 3 ABN
  • Amazing Discoveries
  • EWTN
I think I might try and cover all the English language channels then find some way to host the XMLTV output if there is interest.

Any requests, or am I talking to myself? Smile

384 posts

Ultimate Geek
+1 received by user: 2


  Reply # 770818 27-Feb-2013 00:04 Send private message

Hi mm1352000, don't hate me but I've moved off Windows and Mediaportal for good now. Using XBMC on Mythbuntu. Anyhoo good luck with the thread, I hope people come forward and share their knowledge.




Support a locally made Linux based Operating System, try Linux Lite.
Download Linux Lite



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 770831 27-Feb-2013 04:49 Send private message

valtam: ...don't hate me...

Why would I hate you? People should be totally free to use whatever software they like. :)

In any case, this thread is entirely about EPG data. XMLTV is an open format. Any HTPC software worth its salt should be able to use/import XMLTV listings. In other words, if you still have those channels in MythTV then I presume there would be some way to import and use this data if you wanted to.



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 775368 6-Mar-2013 00:02 Send private message

Well, I've added God TV and Inspiration TV as well now. Inspiration was tough as they only provide a PDF. I didn't find any easy solutions so I ended up writing my own PDF text and path parser using Perl's PDF::API2. I was able to reconstitute the grid, pull the text content with its relative position in the grid, and hence assemble the schedule info. That was a real programming challenge. 2000 odd LOC later... Perl is awesome! :)
As far as I'm aware Daystar and Hope Channel are the only English-language channels that I haven't yet covered. I'll do them when I have a chance. However, since there has been no interest in this project I won't be doing anything further. No point in publicly releasing if nobody is interested.

135 posts

Master Geek
+1 received by user: 6


  Reply # 775582 6-Mar-2013 12:00 Send private message

Hey, I'm interested. I think what you are doing is very interesting, but way above my head. The channels I use on D2 are RT, NHK, & Hope, but I do know of people interested in EWTN. I would watch some content on the other channels that are supposedly Christian, but I think they are more about dishonestly separating viewers from their money in the guise of religion. Enough said of that. From a purely technical view point I think your idea is excellent.

I use DVBViewer and its RecordingService on my HTPC. I import EPG via Xepg http://www.a123.dk/

I also watch SBS and Channel 7 off Optus C1 and use Xepg to grab the XMLTV from the following website

http://www.sattvguide.com.au/xml/AuroraNZ.xml

Xepg can adjust the time, so even if the timing is out of phase, like when NZ starts daylight saving a week before Australia, time errors are easily corrected.

If I am able to read your EPG from a file such as the one above, I would use it, and I'd post a link to it on the DVBViewer forum so other Xepg users can get at it.

Thanks for you hard work.

Tim

Abo

49 posts

Geek
+1 received by user: 9


  Reply # 776783 8-Mar-2013 11:51 Send private message

I am interested in the script that you have written or the data if the script can't be released, specifically the one for EWTN but the others channels would be very nice to have with some downtime.

Been quite a problem to get epg data ota 




871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 777311 9-Mar-2013 11:59 Send private message

Thanks for the encouragement guys.

Here is a link to a new test XMLTV file.
-->link<--

This one now includes the Hope Channel for you Tim. I haven't yet done DayStar, and I removed the TBN, JCTV/SOAC and Church Channel details from this sample as I need to fix/rework those grabbers.

All times in the file are in NZST, however the script itself takes the target time zone as an input parameter (so the XMLTV file can be generated with native times for any timezone).

There should be data for up to 14 days (depending on data availability from the source websites). The number of days to grab for is also an input parameter so you could attempt to grab more.

Abo: I am interested in the script that you have written or the data if the script can't be released, specifically the one for EWTN but the others channels would be very nice to have with some downtime.

Let's see what happens once I've finished the last few channels. I'm reasonably open to releasing the script but don't know if I can maintain it in the long term.

Been quite a problem to get epg data ota

That would be because there is none on Optus D2, hence this thread.

I'm interested to hear any feedback you have. Whether it can be imported successfully, the data is correct and looks good, etc. etc.

135 posts

Master Geek
+1 received by user: 6


  Reply # 777658 10-Mar-2013 16:00 Send private message

That file worked perfectly with Xepg and DVBViewer. In my channel pairings I see Amazing Discoveries, EWTN, GodTV, Inspiration, NHK World, Press TV, Russia Today, 3ABN.

The timings look correct. It all looks good to me.

I am especially excited about NHK as they do have some interesting News and without EPG its hard to record, so I often miss it.

Thanks again for your skill and effort.

Tim

Abo

49 posts

Geek
+1 received by user: 9


  Reply # 777940 11-Mar-2013 12:57 Send private message

Couple of issues importing it into myth tv.

The first was the tag
</xml>
at the bottom of the file - just removed that and it went through fine.

Mythtv doesn't work with spaces in the channel name by the looks of it - it grabs the first word until the space and then thats it (annoying but simple find and replace works fine)

Apart from that data looks accurate and its good to have some show info rather than a blank screen, good work



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 777955 11-Mar-2013 13:05 Send private message

Thanks for the feedback.

@Tim: Looks like I made a copy paste error on the Hope Channel grabber, so the programmes for Inspiration and Hope have been jumbled up. This will be fixed in the next sample.

@Abo:
The first was the tag at the bottom of the file - just removed that and it went through fine.

I'll remove that tag in the next sample too.

Mythtv doesn't work with spaces in the channel name by the looks of it - it grabs the first word until the space and then thats it (annoying but simple find and replace works fine)

Hmm, okay. I use the channel name as the ID as well, so maybe that is the problem. I'll add a dedicated ID without spaces for the next sample and see if Myth is happier with that.



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 786366 24-Mar-2013 19:57 Send private message

Heya folks

Here is an updated sample:
-->link<--

This one includes TBN, Church Channel and JCTV/SOAC again, as well as Daystar. So you should now have the full compliment of English-language channels:
  • 3ABN
  • Amazing Discoveries
  • Church Channel
  • Daystar
  • EWTN
  • God TV
  • Hope Channel
  • Inspiration/INI
  • JCTV-Smile Of A Child
  • NHK World
  • Press TV
  • Russia Today/RT
  • Trinity/TBN
I've also fixed the issues that you raised as per my previous post.

Let me know if you find any further problems or if you want me to add additional channels.
If you don't find any further problems, I'll start preparing a readme and release notes and put the script up on github or something. Saves finding a server to host and update the data.

Cheers!
mm


384 posts

Ultimate Geek
+1 received by user: 2


  Reply # 786379 24-Mar-2013 20:06 Send private message

Nice work mm :)




Support a locally made Linux based Operating System, try Linux Lite.
Download Linux Lite

135 posts

Master Geek
+1 received by user: 6


  Reply # 787336 26-Mar-2013 16:55 Send private message

Hi mm,

I imported it all into DVBViewer with Xepg. The only channel that seemed to go wrong was Inspiration ... nothing imported for that one. Doesn't worry me as I never watch Inspiration.

Great job!!

I have a request if you have the time. I watch a bit of Arirang TV off Intelsat 19. It is quite interesting at the moment with the heightened tensions on the Korean peninsula.

The guide is here:

http://www.arirang.co.kr/Tv/Tv_Index.asp?MType=S

Thanks again for all your good work.

Tim



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 787377 26-Mar-2013 18:09 Send private message

Heya

Thanks for the encouragement. Smile

KiwiTim: The only channel that seemed to go wrong was Inspiration ... nothing imported for that one. Doesn't worry me as I never watch Inspiration.

You're right. That sample only had one day of Inspiration data - for 25 March. Inspiration release a new PDF at the start of each week that only covers that week. Due to the fact that I grabbed on Sunday I got a PDF with data for 19..25 March. I threw away the expired data for 19..24 March and exported the 25 March data. By the time you imported, I guess the 25 March data was also expired so DVBViewer may have thrown it away. Hence you see nothing for that channel. So its a timing thing that is out of my control.

I have a request if you have the time. I watch a bit of Arirang TV off Intelsat 19. It is quite interesting at the moment with the heightened tensions on the Korean peninsula.

I'll see what I can do.

mm



871 posts

Ultimate Geek
+1 received by user: 21

Trusted

  Reply # 787547 26-Mar-2013 20:41 Send private message

I've added basic Arirang details in this sample:
-->link<--

Getting text for programme and episode descriptions would be desirable, but it looks quite tricky.

Whaddaya think?

 1 | 2
View this topic in a long page with up to 500 replies per page Create new topic




Twitter »
Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:




News »

Trending now »
Hot discussions in our forums right now:

Vodafone TV multicast settings on pfSense?
Created by kenkeniff, last reply by chrispchikin on 22-Aug-2014 22:36 (133 replies)
Pages... 7 8 9


CGA. Is it fair?
Created by BTR, last reply by bazzer on 22-Aug-2014 11:02 (86 replies)
Pages... 4 5 6


Free: Smart Button for your Android device
Created by freitasm, last reply by PhantomNVD on 22-Aug-2014 22:08 (104 replies)
Pages... 5 6 7


Lightbox press event release
Created by freitasm, last reply by Lightbox on 22-Aug-2014 17:04 (469 replies)
Pages... 30 31 32


It was hardly 'hacking' was it?
Created by CB_24, last reply by gzt on 21-Aug-2014 22:26 (97 replies)
Pages... 5 6 7


Warning about Pure Hosting
Created by LostBoyNZ, last reply by Zeon on 23-Aug-2014 00:15 (34 replies)
Pages... 2 3


Vodafone now charging you to receive a bill via post
Created by stocksp, last reply by Linuxluver on 22-Aug-2014 19:45 (86 replies)
Pages... 4 5 6


Does an HRV system actually "dry" the air?
Created by kenkeniff, last reply by mattwnz on 22-Aug-2014 17:32 (17 replies)
Pages... 2



Geekzone Live »
Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.