Govt will run "govgpt chat bot"

Forums › ICT Policies and Regulation › Govt will run "govgpt chat bot"

1 | 2 | 3

1561 posts

Uber Geek

#3281247 12-Sep-2024 07:45

gehenna:

Opinions are just that. Objectively and measurably, it has improved.

I'm going to quote an opinion piece:

Googling the question "How much has Google invested in AI?" that same AI, now baked into the search engine, reports that "In April 2024, Google CEO Demis Hassabis said that Google would spend more than $100 billion." Direct cut and paste, dear reader. This will come as news to Google's actual CEO, Sundar Pichai. Google's flagship AI built for Google's flagship product does not know who Google's CEO is – and the company has arranged for this to be the first line of the first result shown on screen. This is not good. This is very far from good.

Remind me again how Google has improved...

My referral links: BigPipe | Mercury

gehenna

8509 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#3281249 12-Sep-2024 07:52

Stop being sensationalist. Objectively it's improved, that's what I said, that's what it has done. I'm not defending any organisation's implementation of AI, or any other features they think are useful or not, I simply challenged the incorrect assertion that Google hasn't improved.

cddt

1561 posts

Uber Geek

#3281252 12-Sep-2024 08:05

Back to "AI" and its relation to the NZ government, I wonder if anyone has read this paper recently publish by some scholars at our very own University of Auckland: https://link.springer.com/article/10.1007/s10994-024-06555-6

My referral links: BigPipe | Mercury

johno1234

2807 posts

Uber Geek

#3281256 12-Sep-2024 08:24

Subjectively speaking, Google most certainly has not improved. It has degenerated to the point of being barely usable and the long despised Bing is now preferable.

Any Google search these days is loaded with sponsored and search history suggested links. By default pi-hole makes about half of Google results unclickable.

Simply awful. Use DuckDuckGo, Bing, anything else and your life will improve.

kiwifidget

"Cookie"
3423 posts

Uber Geek

Lifetime subscriber

#3281264 12-Sep-2024 09:07

The chatbot shouldnt be too hard to program.

All it has to say is "I'm sorry, the Privacy Act prevents me from telling you what you want to know".

That's all I ever hear from a government department person.

Delete cookies?! Are you insane?!

evnafets

537 posts

Ultimate Geek

Lifetime subscriber

#3281319 12-Sep-2024 09:29

cddt:

Back to "AI" and its relation to the NZ government, I wonder if anyone has read this paper recently publish by some scholars at our very own University of Auckland: https://link.springer.com/article/10.1007/s10994-024-06555-6

Took a look at it seeing as you linked it.
Skimmed it.

I don't understand all the technical stuff, but the general concept appears to be that a LLM may be biased / racist / sexist because of the data it was trained on.
And you need to be careful that the training data reflects the characteristics of the outcome you are after.

The paper itself seems to be about measuring/evaluating the bias in LLMs (gets way too technical for my brain) - so doesn't seem that relevant to the topic at hand really.

antoniosk

2358 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#3281323 12-Sep-2024 09:53

jonherries:

Government to launch AI chatbot called Gov-GPT | RNZ News

This feels to me like a "government version of google". Not sure what it would have that the regular Bing or Google wouldn't?

Could agencies choose to change their spiders.txt to stop it scraping data?

Why would Callaghan run it?

Plus the Callaghan press release: GovGPT pilot promises faster access to Government business support | Callaghan Innovation

Jon

"Ultimately, our vision is for a simple, digital front-door where Kiwis can quickly and easily get answers to their questions about Government support and services. Anyone should be able to access important information in a way that’s most intuitive to humans – natural conversations in their preferred language. GovGPT is a first step."

So a wayfinder of sorts? imagine the rules that would need to eventually be implemented for govt grade accessibility, language, culture, privacy etc.... 😉

________

Antoniosk

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

wellygary

8328 posts

Uber Geek

#3281331 12-Sep-2024 10:20

Its gonna be ring fenced to Govt websites, ....

https://www.rnz.co.nz/news/political/527761/government-chatbot-ringfenced-to-specific-websites

Which will make it a very interesting example to watch,

I mean what sort of answers are you gonna spit out if you only got to read govt websites....

I'm sensing a whole lot of public service filler phrases such as "as we go forward" etc,

I think it will be quite difficult for it to get its head around "Te Ao Maori" concepts, but it will be interesting to see how it goes...

johno1234

2807 posts

Uber Geek

#3281335 12-Sep-2024 10:34

My prediction is that it will return answers that on the face of it are sensible but when looked at carefully are nonsensical. After all, it can only work with the material beneath it and it is based on government documents.

BlakJak

1275 posts

Uber Geek

Trusted

#3281352 12-Sep-2024 11:21

So much cynicism.

With no official knowledge whatsoever, I can see how this works:

The model is trained in web content
You ask it a question, it answers based on that web content
It'll provide links and citations for its answers and steer you direct to the authoritive source
It's up to the agencies to ensure that their web content is accurate
So it'll be smarter than doing the equivalent query via Google because it's going to be able to link directly to sources, with a much smaller and less commercially-led set of sources to reference.

Seems like a nice option, to be able to go to an AI instead of having to remember which part of a website to look at - or which agency to go to for areas of fuzz between multiple agencies interfacing over a given subject.

It's an option, it's not taking away anything, so I'd suggest people reserve judgement until it's proven itself as problematic as other AI examples (hallucinations etc) have done in the past.
This has the advantage of an explicitly defined dataset and all-public-info which makes it nice and contained. And CI have been leading a lot of Govt AI initiative so would seem like a good agency to have leading it.

No signature to see here, move along...

KiwiSurfer

1456 posts

Uber Geek

ID Verified

Lifetime subscriber

#3281356 12-Sep-2024 11:31

Have recently started using ChatGPT and have to say very underwhelmed -- especially given the positive rave reviews I've got from (mostly) laypeople. Asked it questions I knew the answer to and it wasn't very good. Did ask some new questions but ended up falling back to a simple DuckDuckGo search which surfaced the info I needed. ChatGPT seem to rely on some very old source materials as some of the questions i asked was Linux networking stuff where it referenced stuff long since depreciated.

Agree with the decline of Google. I remember when >90% of my Google searches had the best result at #1. No longer the case nowdays. DuckDuckGo is my go-to now and it is very impressive at returning what I need without having to massage my query. Whenever I install new machine and forget to switch the default search I am quickly reminded as to why I stopped using Google.

Will be very interesting to see this GovtGPT but am wary given it will be laypeople relying on it for high stakes information. Won't end well I fear but who knows.

freitasm

BDFL - Memuneh
79289 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#3281486 12-Sep-2024 15:22

gehenna: I'm assuming this won't be a conversational model with access outside it's boundaries. More likely a restricted generative answers model that can only supply answers from knowledge that's defined, but can recognise natural language input from the user enough to understand a range of ways to ask for things.

Just an assumption but seems like this type of model use case would require it.

If they are smart, this.

I have implemented a chat service that uses only data from a company's website. Anything outside of that is sent to a rep or a notification is raised for someone to take over.

It's been working quite well.

These things seem to go crazy when you have the whole web to use as your source.

MadEngineer

4291 posts

Uber Geek

Trusted

#3281524 12-Sep-2024 17:29

wellygary:
Its gonna be ring fenced to Govt websites, ....

https://www.rnz.co.nz/news/political/527761/government-chatbot-ringfenced-to-specific-websites

Which will make it a very interesting example to watch,

I mean what sort of answers are you gonna spit out if you only got to read govt websites....

I'm sensing a whole lot of public service filler phrases such as "as we go forward" etc,

I think it will be quite difficult for it to get its head around "Te Ao Maori" concepts, but it will be interesting to see how it goes...

it’s all fun and games until someone puts up a draft or otherwise sensitive document online that’s not linked anywhere thinking noone will see it but the bot picks up on it….

Happens all the time - like when documents were commonly thrown into a folder then the index.html gets dropped and suddenly all is there to see

You're not on Atlantis anymore, Duncan Idaho.

johno1234

2807 posts

Uber Geek

#3281544 12-Sep-2024 19:23

freitasm:
gehenna: I'm assuming this won't be a conversational model with access outside it's boundaries. More likely a restricted generative answers model that can only supply answers from knowledge that's defined, but can recognise natural language input from the user enough to understand a range of ways to ask for things.

Just an assumption but seems like this type of model use case would require it.

If they are smart, this.

I have implemented a chat service that uses only data from a company's website. Anything outside of that is sent to a rep or a notification is raised for someone to take over.

It's been working quite well.

These things seem to go crazy when you have the whole web to use as your source.

I read that ChatGPT is based on knowledge up to 2021 but asking it who the recent winners of the us men’s tennis open it answered up to last year.

mrdrifter

576 posts

Ultimate Geek

ID Verified

Trusted

#3281554 12-Sep-2024 20:47

BlakJak:

So much cynicism.

With no official knowledge whatsoever, I can see how this works:

The model is trained in web content
You ask it a question, it answers based on that web content
It'll provide links and citations for its answers and steer you direct to the authoritive source
It's up to the agencies to ensure that their web content is accurate
So it'll be smarter than doing the equivalent query via Google because it's going to be able to link directly to sources, with a much smaller and less commercially-led set of sources to reference.
Seems like a nice option, to be able to go to an AI instead of having to remember which part of a website to look at - or which agency to go to for areas of fuzz between multiple agencies interfacing over a given subject.

It's an option, it's not taking away anything, so I'd suggest people reserve judgement until it's proven itself as problematic as other AI examples (hallucinations etc) have done in the past.
This has the advantage of an explicitly defined dataset and all-public-info which makes it nice and contained. And CI have been leading a lot of Govt AI initiative so would seem like a good agency to have leading it.

This is what I'm expecting as well and I would barely call it AI as such (I hate that so many of the cognitive services have been caught up in the great AI rebranding), it's simple natural language queries across a set of government websites, rather than having to try and search each one individually. They'll incorporate an LLM to lean on the pretrained transformers and return a set of results/summaries and links/citations for the original source. The technical implementation isn't actually that difficult.

1 | 2 | 3