How would I distribute lots of 'freeware' data?

Forums › Desktop computing › How would I distribute lots of 'freeware' data?

toprob

126 posts

Master Geek

#223533 4-Oct-2017 19:12

As a member of the NZ flight simulation community, I'm looking to organise some way of distributing freeware scenery for use in desktop simulators. There are existing worldwide file libraries, but they just don't allow for the sort of file size which I'm hoping to be able to handle -- the AVSIM.com library, for instance limits uploads to 500MB, but I'm looking at multiple files 1 -- 5GB. Through my own business I develop and distribute 'payware' scenery for New Zealand, served via AWS/Cloudfront, and while I'm very happy to continue doing that for my own online store, there are plenty of local developers who would produce freeware if there was a free, easy way to get it to users.

A typical release might be of interest to 500 -- 1000 users in the short term, and up to 3000 -- 5000 longterm. Not too sure of the current local interest, there's been a real shakeup in the sim community lately, with our one remaining online community on its last legs... About half my user-base is overseas, so I'd imagine that any freeware would have a similar percentage. Top scenario would be serving 1TB over the first 2 months per release, maybe a couple of new releases per month.

What about torrents? Can someone give me an idiot's intro to this, although I try to do everything myself on my own online store, I'm not that technically competent in anything except flight sim scenery:)

Another option is to build a site which relies on sponsorship/advertising (similar to geekzone!), so what are the cheap options to host this sort of data?

timmmay

20575 posts

Uber Geek

Trusted

Lifetime subscriber

#1877447 4-Oct-2017 19:59

Digital Ocean object storage. US$5/month for 250GB storage and 1TB downloads. The same thing on AWS S3 would be a fairly stunning $93 /month at 1GB/month, though S3 does torrents which could reduce your bandwidth. BackBlaze B2 is cheaper than S3 but much more than DO OS.

1101

3122 posts

Uber Geek

#1877690 5-Oct-2017 10:18

https://mega.nz/

50Gb free
Downloads are fast (compared to some other free file share services)
And its not riddled with ads & malware popups .

if you need more than 50Gb, either buy an a/c or use mulitple free a/c's

yes, its kim.com , but its still a good service .

torrenting would be a pick of last resort (just my opinion) , and that requires others to allow sharing via their PC.
Torrents store & download from other peoples PC's (sort of)
You'll find users would download via torrent, then disable the torrent service, so the whole thing would grind to a crawl.

For files over 1Gb, consider spitting them into two or more smaller files (using zip, RAR etc) .

tchart

2379 posts

Uber Geek

ID Verified

Trusted

#1877830 5-Oct-2017 12:54

Maybe https://koordinates.com/ ?

PANiCnz

990 posts

Ultimate Geek

#1877937 5-Oct-2017 15:29

Surely a Torrent tracker would be a good option? Share the bandwidth and hosting responsibilities across the community.

Dynamic

3866 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#1878043 5-Oct-2017 18:47

I really like the idea of using torrents to share the distribution load, but I think less than 20% of your users would download via this method.

I used to download Linux distributions using torrents whenever possible (thinking was being nice and reducing the load on the distro organisations's servers), but these days I simply can't be bothered. It's easy to click to start the download, minimise it, and walk away.

“Don't believe anything you read on the net. Except this. Well, including this, I suppose.” Douglas Adams

Referral links to services I use, really like, and may be rewarded if you sign up:
PocketSmith for budgeting and personal finance management. A great Kiwi company.

toprob

126 posts

Master Geek

#1878122 5-Oct-2017 20:48

Thanks for the suggestions, much appreciated!

Giving some thought to torrents, I do think that a lot of our users would find it new territory, From my experience with my paying customers, although some have picked up a lot of technical skills from years of simming, most just like something simple. Plenty of 'older generation' folks, I'm pretty much an average age for a simmer, at 61:) Plus there's the general public perception of torrents, I know that they have a lot of uses, but there is a perception that they equate to piracy etc.

I did look at Koordinates when I got the email from them last month, promoting their data hosting service, and although it would be something I'd look at for my business, for this particular purpose it is a bit expensive. Still, I'd love to see a nice simple interface where a user can draw a rectangle over a block of photoscenery tiles to download them....

I've used mega before, receiving single file transfers, and I have no problem with Dotcom! This might be worth a serious look, as each developer could have their own account, but even then for some it would be a learning experience to package it up -- I've been thinking that maybe I could handle all that for them, in which case something like Digital Ocean might be better. I wonder what sort of download speed people could expect here in NZ? That's a bit of a problem with the major flightsim libraries, most are far enough away and/or throttled enough so that a big download can take hours/days.

I would imagine that this is something I would set up myself, and give developers access to allow uploads, and I'd be happy to pay a small monthly cost... I make flightsim scenery for a living, but it's always a struggle in a tiny corner of a niche market.

Regarding limiting file size, I already do this on my own site, I restrict single downloads to 1.4GB. However times change, and with the new simulators coming onto the market this will need to change at some stage -- I'm talking photo scenery here, which is more and more becoming the 'standard'. New Zealand is lucky to have the LINZ service, supplying quality data for free, so at some stage I'm sure we'll see the whole country in photo scenery, for a number of different simulators -- the MS-based sims, such as FSX on Steam, Lockheed Martin Prepar3d, plus the others -- X-plane and Aerofly FS2, which only uses photo scenery. Data-wise, it's a growth industry, although the user base isn't what it was 10 years ago.

raytaylor

4014 posts

Uber Geek

Trusted

#1878147 5-Oct-2017 21:47

Look for a torrent based sync application. Not a standard torrent client.

Its pretty simple - you just share a directory on your computer for 'read only' access, and it gives you a magnet link.

Everyone else just installs the same program, inserts the magnet link and then points it at their game templates directory. In your game forum you can make a sticky post that shows others how to do this with screenshots.

When you save a file into the shared directory, the other users will automatically begin downloading it.

When one user has downloaded some (or all) of the pieces that make up a file in the directory, they begin sharing it to other connected users.

When a user connects, their computer scans the local directory, and will download any missing files from other users.

When you have your computer shut down, other users still online will continue to share the files.

It may interest you to know that many games from major studios have a background torrent application built in which transparently downloads updates and patches, and shares them with others while you play the game - this lowers the burden on the studio to provide bandwidth for serving up the content.

Ray Taylor

There is no place like localhost

Spreadsheet for Comparing Electricity Plans Here

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

bigalow

566 posts

Ultimate Geek

#1878173 5-Oct-2017 21:58

what about something like github

https://github.com/pricing

toprob

126 posts

Master Geek

#1878717 6-Oct-2017 22:12

raytaylor:

Look for a torrent based sync application. Not a standard torrent client.

From a bit of a look at what's available, that would be a great fit -- and really it is the sort of thing which would be easy enough to trial. Initially I would think that we'd be distributing installers, as it's a bit of a process getting everything in the proper location, and activated in the simulator, but I can see a time when we could just drop new scenery straight into the simulator.

I wonder what would represent a healthy number of seeders, given that our users are probably not early adopters of high speed internet...

Thanks!

Shoes2468

785 posts

Ultimate Geek

#1878735 7-Oct-2017 00:16

I’m guessing you can’t dont have fibre available to you? If so you could get an unlimited plan with a high upload speed say 100mb and setup a pc to serve the files? A lot of isps now offer plans with 500mb upload speeds for reasonable prices

toprob

126 posts

Master Geek

#1878886 7-Oct-2017 11:11

Well, fibre is available here, but really I'm looking for something which doesn't rely too much on me long-term. A month ago when I shifted here I chose not to go for fibre, simply because the wait was too long, plus it's a shared driveway, and with the cabinet outside my house I get good VDSL speeds... Upload speed is a major consideration for me, simply because at some stages of development I'm uploading huge quantities of data, but after shifting from 1MB on ADSL last month my 27Mb upload speed seems great:)

jpoc

1043 posts

Uber Geek

#1879428 8-Oct-2017 15:26

usenet?

bender

220 posts

Master Geek

#1879889 9-Oct-2017 13:43

Amazon S3 supports torrent delivery, you could make that the only allowed delivery method and that'd keep costs to a minimum?