Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


Gurezaemon

~HONYAKKER!~
1191 posts

Uber Geek

ID Verified
Lifetime subscriber

#311001 5-Dec-2023 19:56
Send private message quote this post

My company is setting up a local AI machine running PrivateGPT for language model data crunching.
Because of extreme confidentiality requirements, this needs to be on a local machine - not online. Cloud services aren't an option.

 

In terms of graphics cards, I know that faster is better, but we're happy if things aren't super snappy - if the machine can plod through the tasks needed, that should suffice.
We also understand that having more video RAM may be more important that sheer clock speed. 

 

We'd like to keep the budget to no more than NZ$900, and if possible keep power consumption at ~200W or less. 

 

Our thoughts are a RTX 4060 TI 16 Gig that I can source from Japan within our price constraints, but I'm open to any suggestions.





Get your business seen overseas - Nexus Translations


Create new topic
mentalinc
2850 posts

Uber Geek

Trusted

  #3168558 5-Dec-2023 20:01
Send private message quote this post

So you're spending $900 on the processing power?

 

Super concerned about how little you might be spending on everything else involved, policy, process, controls reporting etc etc etc.

 

What's the expected ROI, if you're only spending $900?





CPU: AMD 5900x | RAM: GSKILL Trident Z Neo RGB F4-3600C16D-32GTZNC-32-GB | MB:  Asus X570-E | GFX: EVGA FTW3 Ultra RTX 3080Ti| Monitor: LG 27GL850-B 2560x1440

 

Quic: https://account.quic.nz/refer/473833 R473833EQKIBX 


 
 
 

GoodSync. Easily back up and sync your files with GoodSync. Simple and secure file backup and synchronisation software will ensure that your files are never lost (affiliate link).

gzt

gzt
15531 posts

Uber Geek

Lifetime subscriber

  #3168562 5-Dec-2023 20:18
Send private message quote this post

Why source overseas? Local prices look similar.

Gurezaemon

~HONYAKKER!~
1191 posts

Uber Geek

ID Verified
Lifetime subscriber

  #3168566 5-Dec-2023 20:55
Send private message quote this post

mentalinc:

 

So you're spending $900 on the processing power?

 

Super concerned about how little you might be spending on everything else involved, policy, process, controls reporting etc etc etc.

 

What's the expected ROI, if you're only spending $900?

 

 

This is for a proof or concept machine rather than the full blown operational supercomputer stuff. We HAVE it running on a RTX 2060 and it works OK, but a bit too slow for usability. Since this is a new-ish build, we're looking for a new card, hence the question.

 

For ease of management, this will be in a corner of my office, totally air-gapped, and used for things like processing bilingual glossaries, etc.





Get your business seen overseas - Nexus Translations




eonsim
286 posts

Ultimate Geek


  #3168573 5-Dec-2023 22:37
Send private message quote this post

It's not going to be fast running on a 4060TI but 16GB should help a bit in the size of the model you can run, note you can pick up some 4060 TI 16GBs at around that price point from Aussie https://www.mwave.com.au has a few.

 

I'd recommend you look at what other GPU options can be used, potentially modern Mac's using Metal or AMD cards using ROCm as well (which tend to have more RAM for the same price than NVIDIA).


Create new topic





News and reviews »

Synology Introduces BeeStation
Posted 23-Feb-2024 14:14


New One UI 6.1 Update Brings Galaxy AI to More Galaxy Devices
Posted 23-Feb-2024 10:50


Amazon Echo Hub Available in New Zealand
Posted 23-Feb-2024 10:40


InternetNZ Releases Internet Insights 2023
Posted 20-Feb-2024 10:31


Seagate Adds 24TB IronWolf Pro Hard Drives for Multi-user Commercial and Enterprise RAID Storage Solutions
Posted 19-Feb-2024 16:54


Seagate Skyhawk AI 24TB Elevates Edge Security Capacity and Performance
Posted 9-Feb-2024 17:18


GoPro Releases Quik Desktop App for macOS and Introduces Premium+ Subscription Tier
Posted 9-Feb-2024 17:14


Ring Introduces New Ring Battery Video Doorbell Pro
Posted 9-Feb-2024 16:51


Galaxy AI Transforms the new Galaxy S24 Series
Posted 18-Jan-2024 07:00


D-Link launches AI-Powered Aquila Pro M30 Wi-Fi 6 Mesh Systems
Posted 17-Jan-2024 20:02


Newest LG 4K Lifestyle Projector Doubles as Art Objet
Posted 9-Jan-2024 15:50


More LG Smart TV Owners Set To Enjoy the Latest webOS Upgrade
Posted 9-Jan-2024 15:45


Panasonic Announces the Z95A and Z93A With Fire TV Built In
Posted 9-Jan-2024 15:30


Amazon Echo Pop Review
Posted 8-Jan-2024 14:22


Samsung Tab S9 FE Review
Posted 17-Dec-2023 08:26









Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.