My company is setting up a local AI machine running PrivateGPT for language model data crunching.
Because of extreme confidentiality requirements, this needs to be on a local machine - not online. Cloud services aren't an option.
In terms of graphics cards, I know that faster is better, but we're happy if things aren't super snappy - if the machine can plod through the tasks needed, that should suffice.
We also understand that having more video RAM may be more important that sheer clock speed.
We'd like to keep the budget to no more than NZ$900, and if possible keep power consumption at ~200W or less.
Our thoughts are a RTX 4060 TI 16 Gig that I can source from Japan within our price constraints, but I'm open to any suggestions.