HOW TO: Basic DVB-S Guide Scraping in Windows

Forums › Home Theatre (including HTPC and EPG Collector) › HOW TO: Basic DVB-S Guide Scraping in Windows

RustyGonad

495 posts

Ultimate Geek

Trusted

#59009 24-Mar-2010 21:46

This is a basic guide to scraping the EIT guide from the Satellite in windows. It will get you the same guide you can see in either the standard or MySky decoders. With a bit of luck the DVBLogic guys will make most of this redundent in the next couple of releases of TVSource.

If your watching Sky using Windows 7 MCE, and you don't know about DVBLogic, then you need to find out.

However, if you want to use things such as the HD flag, add movie dates, add category colours etc in Media Center, then this way will probably remain useful for a bit longer yet. Theres a couple of guys working on some stuff which may make this alot more user friendly.

You need:
Hardware: Hauppauge DVB-S Card

Software
DVBGuide 0.93

Get it from here:
http://www.dvbguide.com/

Background:
Scraping the guide should be easy, but it not. Unfortunately, due to the different DVB-S systems used around the world, there aren't many tools, and the ones that exist are either written for Linux, or not exactly NZ aware.

In essence DVBGuide would make this easy, except it doesn't support the LNB type that most of us in NZ use, as a result you can't set the LNB Oscillator frequency, which basically means it won't tune into the bird in NZ.

Please note - if you can't use the good old DOS command line, or run for cover when someone mentions the registry, then DONT DO THIS.

The Basics

To get around this you use the good old Hauppauge DVBS_as_DVBT mod, which if you've been around a while, you will know as the mod the allowed you to use your DVB-S cards prior to the Vista MCE TV Pack.

This is a registry file which "tricks" you DVB-S card into thinking its a DVB-T card. Its listed as the end of this guide (copy and paste into notepad, and save it as OptusD1.reg).

If you want to undo this mod, change the DVBS_as_DVBT keyword in the registry to a 0.

The downside to this mod is that it effects all Hauppauge cards in a given machine. To the best of my knowledge it doesn't support DVB-S2 channels, which is a real mongrol for those using HD. If effectively means that you either need to do it on a seperate machine, or it gets pretty messy.

If your not using HD, you can just tune in the satellite channels as DVB-T channels, and you should be fine. The only thing to remember is that this won't run if something else is using the tuner, so you need to account for that if you want to script it, by killing/restarting the appropriate process before/after scraping.

Scaping

Once the Hauppauge regmod is done, the scraping itself it pretty simple.

You need to select a transponder and use that with DVBGuide, selecting and xml file for the output.

eg:
DVBGuide.exe -u1 -c578000 -aT -fX -oSky.xml

This will use tuner 1 (-u1), scan DVBT freq 578000 (Which is really DVB-S transponder 12519), and output a xml file (-fx) with the name Sky.xml (-oSky.xml)

Thats basically it.

By changing the frequencies you can also scrape Freeview EIT, and SBS if you want.

DVBGuide.exe -u1 -c562000 -aT -fX -oFreeview.xml - this would do one of the Freeview transponders
DVBGuide.exe -u1 -c634000 -aT -fX -oSBS.xml - would do SBS

If your compentent merging xml files, then it pays to either scan two transponders (for Sky) or the same transponder twice, and merge the results, ditto for Freeview. Due to the nature of EIT its pretty common to miss some information on a single scan. Scanning two/twice improves the results.

Cleaning it up...

Once you've got this stuff, all you need to do is clean up all the unwanted channels, rename them to something meaningful, search and replace for the HD flag, remove all the comments, merge the Freeview/Sky files, and you should have something useable!

But seriously - cleaning is a bit of a mission. You can either use some of Hads python scripts or have a go with tools such as xmltvalter, xmltvmerge etc etc etc. Once done you end up with some pretty good results in MCE, but thats another whole story...

REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\Parameters]
"DVBS_as_DVBT"=dword:00000001 ; Enable remapping support

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\546000] ; #41
"dvbs_freq"=dword:BD1E10 ; 12394000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\554000] ; #44
"dvbs_freq"=dword:BD8788 ; 12421000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\562000] ; #47
"dvbs_freq"=dword:BE1040 ; 12456000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:2F ; 47
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\570000] ; #50
"dvbs_freq"=dword:BE79B8 ; 12483000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:2F ; 47
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\578000] ; #53
"dvbs_freq"=dword:BF0658 ; 12519000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\586000] ; #56
"dvbs_freq"=dword:BF6FD0 ; 12546000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\594000] ; #59
"dvbs_freq"=dword:BFF888 ; 12581000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\602000] ; #62
"dvbs_freq"=dword:C06200 ; 12608000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\610000] ; #65
"dvbs_freq"=dword:C0EEA0 ; 12644000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\618000] ; #68
"dvbs_freq"=dword:C15818 ; 12671000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\626000] ; #71
"dvbs_freq"=dword:C24E30 ; 12734000
"dvbs_polarity"=dword:1 ; 1
"dvbs_srate"=dword:57E4 ; 22500
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HCW88bda\DVBS_as_DVBT\634000] ; #74
"dvbs_freq"=dword:C0F670 ; 12646000
"dvbs_polarity"=dword:2 ; 2
"dvbs_srate"=dword:3138 ; 12600
"dvbs_LowOsc"=dword:A40830 ; 10750000
"dvbs_HighOsc"=dword:A40830 ; 10750000
"DiSEqC_Port"=dword:0 ; 0
"demod_ModType"=dword:0 ; 0
"nit_onid"=dword:A9 ; 169
"dvbs_SwitchFreq"=dword:0 ; 0

Satch

1985 posts

Uber Geek

#311236 25-Mar-2010 12:29

Piece of p*ss...

Lenovo

Lenovo computer and accessories deals (affiliate link).

Glazza

279 posts

Ultimate Geek

Trusted

#311250 25-Mar-2010 12:59

Thanks Rusty - Got the guide downloaded now..... now to start the tidy up mission....

Anyone tried xmltvmerge recently? II have downloaded the XMLTV: Guide Pack from epgstream - and have only installed the tools.

However when I run xmltvmerge it returns to the command prompt - no error messages or anything

Cheers

Kellysu

28 posts

Geek

#312229 28-Mar-2010 22:19

Dude, you're a legend - that worked well. Now for the formatting, have you got any scripts already for xmltvalter or a guide for using epgsnoop just to format the sky xml for media center?

RustyGonad

495 posts

Ultimate Geek

Trusted

#312341 29-Mar-2010 11:26

I run a bunch of custom scripts, but they're not really fit for public consumtion...

Basically I use alterxmltv, which does the channel ID's, deletes all the unwanted channels, searches for HD, (HD) etc, and turns this into a "HDTV"

mergexmltv merges the Freeview/SBS/Sky listings.

I then do a bunch of seach and replace stuff eg replacing the HDTV category with

HDTV

etc etc

Its a scriting mess of epic proportions, but it works.

Hads has done some much much more elegant stuff with Python, but I couldn't get it to work that well in windows.

What I would love to have is a working imdb lookup solution for Windows, one that takes the Sky Movie name without date, comares against imdb or tmdb and adds simply a field, so that Media Center can populate the rest. There are solutions such as xmltv tv_imdb but it doesn't want to work when the date isn't there, which kinda makes them useless, cause if you had the date you wouldn't need to do the lookup in the first place.

Cheers,

Kellysu

28 posts

Geek

#312354 29-Mar-2010 11:50

Ok thanks, the good news is the XML scraped using your guide above imports straight into MCE (Vista) using epgstream perfectly, no mods required. The next step for me is to find an alternative for epgstream that works in Windows7. I'm a bit off the pace...

4ga

4 posts

Wannabe Geek

#319519 16-Apr-2010 21:37

Hey nice guide but fwiw you can also use DVBGuide with DVBS with a bit of fudging the transponder freq.

It appears that DVBGuide uses a hardcoded lof of 9750MHz. So if for example using the common sky lnb with 10750, take the difference (1000) and subtract from the transponder frequency, so if you want 12519 then command line will look like this:

DVBGuide.exe -as -c11519000 -ph -mQPSK -s22500000 -o sky.xml

Also on my setup found I was missing a few channels when using 12519, changed to 12734 and problem went away.

I use Big Screen EPG to import the raw file from DVBGuide and find it all works quite nicely.

hope this is of help to some.

allstarnz

1715 posts

Uber Geek

Trusted

#320052 18-Apr-2010 21:55

4ga: Hey nice guide but fwiw you can also use DVBGuide with DVBS with a bit of fudging the transponder freq.

It appears that DVBGuide uses a hardcoded lof of 9750MHz. So if for example using the common sky lnb with 10750, take the difference (1000) and subtract from the transponder frequency, so if you want 12519 then command line will look like this:

DVBGuide.exe -as -c11519000 -ph -mQPSK -s22500000 -o sky.xml

Also on my setup found I was missing a few channels when using 12519, changed to 12734 and problem went away.

I use Big Screen EPG to import the raw file from DVBGuide and find it all works quite nicely.

hope this is of help to some.

ah, so much easier than p_ssing about with the registry, thanks for the tip!!

Kellysu

28 posts

Geek

#322384 24-Apr-2010 15:10

I'm trying to grab SKY epg data to xml using DVBGuide, and tried using frequencies:

12734000
12519000

I've tred several times today and am missing SKY MOVIES, SKY NEWS, 63 and CARTOON NETWORK.

Anyone having the same problem or got any ideas? Is there a better time of the day to try? I'm not sure what I'm missing, but using DVBGuide with the above frequencies has retreived these channels in the past, any ideas appreciated.

ta

Kellysu

28 posts

Geek

#322388 24-Apr-2010 15:24

Further to my last message, looking at:
http://www.lyngsat.com/packages/skynz.html

I can see that all the channels I'm having problems with are on:

12519 H
tp NZ13
DVB
SR 22500
FEC 3/

I'm using the method outlined at the top of this page, any reason it wouldn't work? Anyone else having problems.

4ga

4 posts

Wannabe Geek

#322948 26-Apr-2010 11:22

Those are the same channels I was missing when using 12519, it was working fine then they disappeared from the xml. I then changed to 12734 and it got all the channels back again.

what command line are you using? if using reg hack try:

DVBGuide.exe -u1 -c626000 -aT -fX -oSky.xml

I dont use reg hack my command line is:

DVBGuide.exe -as -c11184000 -ph -mQPSK -s22500000 -o sky.xml

that is for lnb with lof 11300, if yours is 10750 (more likely) then use:

DVBGuide.exe -as -c11734000 -ph -mQPSK -s22500000 -o sky.xml

If you want to try the commands I use, you will need to dissable or remove the reg hack and restart your computer or it will not work.

Kellysu

28 posts

Geek

#324486 29-Apr-2010 00:34

Thanks for your tip - but no matter what I do (even without emulated DVB-T), some frequencies will give me 14 transports at 161 services and others will give me 166 services, but none seem to have the channels:

Sky Movies
Cartoon Network
Sky News
63
Preview
CTV1

Am I the only one? (I've tried all frequencies I can find on Lynsat website and in registry key supplied above)

I welcome all suggestions - thanks in advance...

CYaBro

4091 posts

Uber Geek

ID Verified

Trusted

Subscriber

#324488 29-Apr-2010 00:48

Those first 4 on your list are also missing from haventechs epg at the moment.