Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.




483 posts

Ultimate Geek

Trusted

Topic # 59009 24-Mar-2010 21:46
Send private message

This is a basic guide to scraping the EIT guide from the Satellite in windows.  It will get you the same guide you can see in either the standard or MySky decoders.  With a bit of luck the DVBLogic guys will make most of this redundent in the next couple of releases of TVSource.

If your watching Sky using Windows 7 MCE, and you don't know about DVBLogic, then you need to find out.

However, if you want to use things such as the HD flag, add movie dates, add category colours etc in Media Center, then this way will probably remain useful for a bit longer yet.  Theres a couple of guys working on some stuff which may make this alot more user friendly.

You need:
Hardware: Hauppauge DVB-S Card

Software
DVBGuide 0.93

Get it from here:
http://www.dvbguide.com/

Background:
Scraping the guide should be easy, but it not.  Unfortunately, due to the different DVB-S systems used around the world, there aren't many tools, and the ones that exist are either written for Linux, or not exactly NZ aware.

In essence DVBGuide would make this easy, except it doesn't support the LNB type that most of us in NZ use, as a result you can't set the LNB Oscillator frequency, which basically means it won't tune into the bird in NZ.

Please note - if you can't use the good old DOS command line, or run for cover when someone mentions the registry, then DONT DO THIS.

The Basics

To get around this you use the good old Hauppauge DVBS_as_DVBT mod, which if you've been around a while, you will know as the mod the allowed you to use your DVB-S cards prior to the Vista MCE TV Pack.

This is a registry file which "tricks" you DVB-S card into thinking its a DVB-T card.  Its listed as the end of this guide (copy and paste into notepad, and save it as OptusD1.reg).

If you want to undo this mod, change the DVBS_as_DVBT keyword in the registry to a 0.

The downside to this mod is that it effects all Hauppauge cards in a given machine.  To the best of my knowledge it doesn't support DVB-S2 channels, which is a real mongrol for those using HD.  If effectively means that you either need to do it on a seperate machine, or it gets pretty messy.

If your not using HD, you can just tune in the satellite channels as DVB-T channels, and you should be fine.  The only thing to remember is that this won't run if something else is using the tuner, so you need to account for that if you want to script it, by killing/restarting the appropriate process before/after scraping.

Scaping

Once the Hauppauge regmod is done, the scraping itself it pretty simple.

You need to select a transponder and use that with DVBGuide, selecting and xml file for the output.

eg:
DVBGuide.exe -u1 -c578000 -aT -fX -oSky.xml

This will use tuner 1 (-u1), scan DVBT freq 578000 (Which is really DVB-S transponder 12519), and output a xml file (-fx) with the name Sky.xml (-oSky.xml)

Thats basically it.

By changing the frequencies you can also scrape Freeview EIT, and SBS if you want.

DVBGuide.exe -u1 -c562000 -aT -fX -oFreeview.xml - this would do one of the Freeview transponders
DVBGuide.exe -u1 -c634000 -aT -fX -oSBS.xml - would do SBS

If your compentent merging xml files, then it pays to either scan two transponders (for Sky) or the same transponder twice, and merge the results, ditto for Freeview.  Due to the nature of EIT its pretty common to miss some information on a single scan.  Scanning two/twice improves the results.

Cleaning it up...

Once you've got this stuff, all you need to do is clean up all the unwanted channels, rename them to something meaningful, search and replace for the HD flag, remove all the comments, merge the Freeview/Sky files, and you should have something useable!

But seriously - cleaning is a bit of a mission.  You can either use some of Hads python scripts or have a go with tools such as xmltvalter, xmltvmerge etc etc etc.  Once done you end up with some pretty good results in MCE, but thats another whole story...


REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\Parameters]
"DVBS_as_DVBT"=dword:00000001 ; Enable remapping support

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\546000] ; #41
"dvbs_freq"=dword:BD1E10 ; 12394000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\554000] ; #44
"dvbs_freq"=dword:BD8788 ; 12421000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\562000] ; #47
"dvbs_freq"=dword:BE1040 ; 12456000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:2F ; 47
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\570000] ; #50
"dvbs_freq"=dword:BE79B8 ; 12483000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:2F ; 47
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\578000] ; #53
"dvbs_freq"=dword:BF0658 ; 12519000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\586000] ; #56
"dvbs_freq"=dword:BF6FD0 ; 12546000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\594000] ; #59
"dvbs_freq"=dword:BFF888 ; 12581000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\602000] ; #62
"dvbs_freq"=dword:C06200 ; 12608000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\610000] ; #65
"dvbs_freq"=dword:C0EEA0 ; 12644000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\618000] ; #68
"dvbs_freq"=dword:C15818 ; 12671000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\626000] ; #71
"dvbs_freq"=dword:C24E30 ; 12734000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\634000] ; #74
"dvbs_freq"=dword:C0F670 ; 12646000
"dvbs_polarity"=dword:2 ; 2
"dvbs_srate"=dword:3138 ; 12600
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

 

 

Create new topic
1982 posts

Uber Geek
+1 received by user: 257

Subscriber

  Reply # 311236 25-Mar-2010 12:29
Send private message

Piece of p*ss...

257 posts

Ultimate Geek
+1 received by user: 6

Trusted

  Reply # 311250 25-Mar-2010 12:59
Send private message

Thanks Rusty - Got the guide downloaded now.....  now to start the tidy up mission....

Anyone tried xmltvmerge recently?  II have downloaded the XMLTV: Guide Pack from epgstream - and have only installed the tools.

However when I run xmltvmerge it returns to the command prompt - no error messages or anything

Cheers


 
 
 
 


28 posts

Geek


  Reply # 312229 28-Mar-2010 22:19
Send private message

Dude, you're a legend - that worked well.  Now for the formatting, have you got any scripts already for xmltvalter or a guide for using epgsnoop just to format the sky xml for media center?



483 posts

Ultimate Geek

Trusted

  Reply # 312341 29-Mar-2010 11:26
Send private message

I run a bunch of custom scripts, but they're not really fit for public consumtion...

Basically I use alterxmltv, which does the channel ID's, deletes all the unwanted channels, searches for HD, (HD) etc, and turns this into a "HDTV"

mergexmltv merges the Freeview/SBS/Sky listings.

I then do a bunch of seach and replace stuff eg replacing the HDTV category with


HDTV


etc etc

Its a scriting mess of epic proportions, but it works.

Hads has done some much much more elegant stuff with Python, but I couldn't get it to work that well in windows.

What I would love to have is a working imdb lookup solution for Windows, one that takes the Sky Movie name without date, comares against imdb or tmdb and adds simply a field, so that Media Center can populate the rest. There are solutions such as xmltv tv_imdb but it doesn't want to work when the date isn't there, which kinda makes them useless, cause if you had the date you wouldn't need to do the lookup in the first place.

Cheers,

28 posts

Geek


  Reply # 312354 29-Mar-2010 11:50
Send private message

Ok thanks,  the good news is the XML scraped using your guide above imports straight into MCE (Vista) using epgstream perfectly, no mods required.  The next step for me is to find an alternative for epgstream that works in Windows7.  I'm a bit off the pace...


4ga

4 posts

Wannabe Geek


  Reply # 319519 16-Apr-2010 21:37
Send private message

Hey nice guide but fwiw you can also use DVBGuide with DVBS with a bit of fudging the transponder freq.

It appears that DVBGuide uses a hardcoded lof of 9750MHz.  So if for example using the common sky lnb with 10750, take the difference (1000) and subtract from the transponder frequency, so if you want 12519 then command line will look like this:

DVBGuide.exe -as -c11519000 -ph -mQPSK -s22500000 -o sky.xml



Also on my setup found I was missing a few channels when using 12519, changed to 12734 and problem went away.

I use Big Screen EPG to import the raw file from DVBGuide and find it all works quite nicely.

hope this is of help to some.



1677 posts

Uber Geek
+1 received by user: 7

Trusted

  Reply # 320052 18-Apr-2010 21:55
Send private message

4ga: Hey nice guide but fwiw you can also use DVBGuide with DVBS with a bit of fudging the transponder freq.

It appears that DVBGuide uses a hardcoded lof of 9750MHz.  So if for example using the common sky lnb with 10750, take the difference (1000) and subtract from the transponder frequency, so if you want 12519 then command line will look like this:

DVBGuide.exe -as -c11519000 -ph -mQPSK -s22500000 -o sky.xml



Also on my setup found I was missing a few channels when using 12519, changed to 12734 and problem went away.

I use Big Screen EPG to import the raw file from DVBGuide and find it all works quite nicely.

hope this is of help to some.




ah, so much easier than p_ssing about with the registry, thanks for the tip!!

28 posts

Geek


  Reply # 322384 24-Apr-2010 15:10
Send private message

I'm trying to grab SKY epg data to xml using DVBGuide, and tried using frequencies:

12734000
12519000

I've tred several times today and am missing SKY MOVIES, SKY NEWS, 63 and CARTOON NETWORK.

Anyone having the same problem or got any ideas?  Is there a better time of the day to try? I'm not sure what I'm missing, but using DVBGuide with the above frequencies has retreived these channels in the past, any ideas appreciated.

ta

28 posts

Geek


  Reply # 322388 24-Apr-2010 15:24
Send private message

Further to my last message, looking at:
http://www.lyngsat.com/packages/skynz.html

I can see that all the channels I'm having problems with are on:

12519 H
tp NZ13
DVB
SR 22500
FEC 3/

I'm using the method outlined at the top of this page, any reason it wouldn't work? Anyone else having problems.

4ga

4 posts

Wannabe Geek


  Reply # 322948 26-Apr-2010 11:22
Send private message

Those are the same channels I was missing when using 12519, it was working fine then they disappeared from the xml.  I then changed to 12734 and it got all the channels back again.

what command line are you using?  if using reg hack try:

DVBGuide.exe -u1 -c626000 -aT -fX -oSky.xml


I dont use reg hack my command line is:

DVBGuide.exe -as -c11184000 -ph -mQPSK -s22500000 -o sky.xml


that is for lnb with lof 11300, if yours is 10750 (more likely) then use:

DVBGuide.exe -as -c11734000 -ph -mQPSK -s22500000 -o sky.xml

If you want to try the commands I use, you will need to dissable or remove the reg hack and restart your computer or it will not work.

28 posts

Geek


  Reply # 324486 29-Apr-2010 00:34
Send private message

Thanks for your tip - but no matter what I do (even without emulated DVB-T), some frequencies will give me 14 transports at 161 services and others will give me 166 services, but none seem to have the channels:

Sky Movies
Cartoon Network
Sky News
63
Preview
CTV1

Am I the only one? (I've tried all frequencies I can find on Lynsat website and in registry key supplied above)

I welcome all suggestions - thanks in advance...

2865 posts

Uber Geek
+1 received by user: 262


  Reply # 324488 29-Apr-2010 00:48
Send private message

Those first 4 on your list are also missing from haventechs epg at the moment.




Create new topic



Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





News »

CPTPP text and National Interest Analysis released for public scrutiny
Posted 21-Feb-2018 19:43


Foodstuffs to trial digitised shopping trolleys
Posted 21-Feb-2018 18:27


2018: The year of zero-login, smart cars & the biometrics of things
Posted 21-Feb-2018 18:25


Intel reimagines data centre storage with new 3D NAND SSDs
Posted 16-Feb-2018 15:21


Ground-breaking business programme begins in Hamilton
Posted 16-Feb-2018 10:18


Government to continue search for first Chief Technology Officer
Posted 12-Feb-2018 20:30


Time to take Appleā€™s iPad Pro seriously
Posted 12-Feb-2018 16:54


New Fujifilm X-A5 brings selfie features to mirrorless camera
Posted 9-Feb-2018 09:12


D-Link ANZ expands connected smart home with new HD Wi-Fi cameras
Posted 9-Feb-2018 09:01


Dragon Professional for Mac V6: Near perfect dictation
Posted 9-Feb-2018 08:26


OPPO announces R11s with claims to be the picture perfect smartphone
Posted 2-Feb-2018 13:28


Vocus Communications wins a place on the TaaS panel
Posted 26-Jan-2018 15:16


SwipedOn raises $1 million capital
Posted 26-Jan-2018 15:15


Slingshot offers unlimited gigabit fibre for under a ton
Posted 25-Jan-2018 13:51


Spark doubles down on wireless broadband
Posted 24-Jan-2018 15:44



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.