Should We Be Concerned About Artificial Intelligence in the Doctors Office

Forums › Health and fitness › Should We Be Concerned About Artificial Intelligence in the Doctors Office

Eva888

2868 posts

Uber Geek
+1 received by user: 2524

Lifetime subscriber

#323252 11-Nov-2025 13:58

Some of the very troubling aspects.

Your conversation is listened to by AI
It often makes mistakes and assumptions when note taking.
Your notes are sent to the USA.
It’s under American jurisdiction.

Podcast

https://www.rnz.co.nz/national/programmes/nights/audio/2018947902/ai-at-the-doctor-s-office-the-future-or-should-we-be-concerned

Am too busy today to elaborate further but there is a lot more to this for people to think about before opting in and the privacy aspect and future repercussions is a very large part.

wellygary

8931 posts

Uber Geek
+1 received by user: 5466

#3433380 11-Nov-2025 14:48

The interview is based off the July 2024 Newsroom article, here,

But basically its using AI transcription, its not making medical decisions or judgements,

https://newsroom.co.nz/2024/07/22/im-a-gp-experimenting-with-ai-to-take-patient-notes-heres-what-i-found/

and pretty much all the issues appear to be around "data sovereignty" which TBH could be removed by hosting it on a local LLM...

Its pretty low level AI, I would be interesting to see what the actual plans are for the future after all of this "the data is in the US" gets done with and AI is more of an ingrained product locally,

rb99

3534 posts

Uber Geek
+1 received by user: 1866

Lifetime subscriber

#3433382 11-Nov-2025 14:49

AI does seem to be capable of impressive medical stuff, such as quite possibly being more accurate / reliable at checking scans for lumps and that kind of thing. But don't see any need for AI to take some notes, or constantly hover in the background.

And the cynic in me says a) it'll be there to save money and b) we'll be able to look forward to Google and Meta asking us how the treatment is going and have we considered trying this...

“The modern conservative is engaged in one of man's oldest exercises in moral philosophy; that is, the search for a superior moral justification for selfishness.” -John Kenneth Galbraith

rb99

Bung

6768 posts

Uber Geek
+1 received by user: 2951

Subscriber

#3433383 11-Nov-2025 14:54

https://www.hinz.org.nz/news/706270/Two-AI-medical-scribes-endorsed-by-Health-NZ.htm

The Doctor is still responsible for any notes and should review the output before committing it to your records.

t0ny

418 posts

Ultimate Geek
+1 received by user: 88

Lifetime subscriber

#3433408 11-Nov-2025 15:50

My local clinic used the AI transcription at my last visit. I was curious how well it performed so gave them permission after which it listened to the conversation I was having. At the end, it spit out a transcript and formatted it into nice sections. While it got number of things wrong (lost context, made up some things etc.) it did get most of the info and the doc fixed or removed bits that it got wrong. They had a nice integration which copied relevant parts into the notes section of their app once the doc was happy with the content so it did save him time. I wasnt concerned about my info going out to the web as the transcription was standalone and there was no way to link that to me automatically.

wellygary

8931 posts

Uber Geek
+1 received by user: 5466

#3433414 11-Nov-2025 16:11

Would we be as concerned if there was an actual live professional stenographer note taker doing it instead of AI.?? (Assuming they signed a bunch of confidential NDAs and other privacy agreements,)

-But just like the doctor would still be subject to being dragged before a court if an appropriate legal warrant was obtained)

NZ police can take medical records if they have court endorsed search warrant

https://www.privacy.org.nz/blog/when-is-it-health-professionals/

networkn

Networkn
33022 posts

Uber Geek
+1 received by user: 15610

ID Verified

Trusted

Lifetime subscriber

#3433528 11-Nov-2025 19:40

So, I have family working in Medicine.

There has been a slow and reluctant up take of it. Yes, the note taking is sometimes occasionally error prone, your doctors responsibility is to ensure the facts are accurate and they do spend time on that.

Having said that, it's very good, the number of mistakes it makes is pretty minor and getting better all the time. It's note taking is far better and easier to underrstand for the next visit, or physician, than the shorthand your doctor was likely using before.

From an admin perspective, it's literally a life saver. Your GP is being pushed beyond human limits to get care to all patients right now. Your Dr almost certainly has between 10-15 minutes to hear about your issue, examine you if required, discuss your condition, make a recommendation, get a script if you need it, and write notes.

Overall your standard of care will be better as a result of it. Will things get missed? Absolutely. Will mistakes be made? Yes. Could those be potentially serious? Yes. Do humans make these mistakes? 100%.

My advice to people in this day and age, is book a single appointment for a single issue. If your issue is complicated (mental health, womens health, pregnancy, or otherwise complex), consider booking a double appointment. Give your Dr every chance to succeed when they treat you. If you have more than one issue, book a double appointment. This gives your Dr not only time to deal with your issue completely, but ensures they deal with everyone elses fairly too.

Geekzone

Want to support Geekzone and browse the site without the ads? Subscribe to Geekzone now (monthly, annual and lifetime options).

Nate001

680 posts

Ultimate Geek
+1 received by user: 475

#3433542 11-Nov-2025 20:48

End of the day, the user (Doctor) is responsible for the outputs of AI agent.

I'd rather my doctor give me full attention rather than multitasking notes on the fly, or recall details later and still potentially make mistakes.

Have you ever had co-pilot summarise a teams meeting/recording? The generated outputs are mostly accurate only requiring the odd touch up, this relieves time for the user to do more high value tasks in their day.

Qazzy03

550 posts

Ultimate Geek
+1 received by user: 497

#3433552 11-Nov-2025 22:04

wellygary:

But basically its using AI transcription, .....

and pretty much all the issues appear to be around "data sovereignty" which TBH could be removed by hosting it on a local LLM...

Its pretty low level AI, I would be interesting to see what the actual plans are for the future after all of this "the data is in the US" gets done with and AI is more of an ingrained product locally,

Pretty much this, my work is looking into AI transcription for Call Centres, and are also piloting it with Teams calls as well.
The "data sovereignty" is always going to be an issue with more parties moving into the Cloud.
The Government is going to the cloud, so this is one of the big trade offs, even without AI, as most of the big vendors are multinational using AWS.

I am not a fan of the Cloud TBH, due to loss of sovereignty, and when it goes down, it goes down hard.

Eva888

2868 posts

Uber Geek
+1 received by user: 2524

Lifetime subscriber

#3433700 12-Nov-2025 10:13

I use AI often for exercise programmes when I sprain a muscle etc and it it is helpful in many other situations. I can also type in my symptoms and get an instant diagnosis. One could argue why go to my doctor at all except for medication that isn’t available over the counter.

When you go to a doctor or specialist you choose to based on their skill. You expect them to look at you and consider the many features of your life, culture, intelligence level, job, family situation, other health conditions, current medication, the texture of the skin, agility, do you limp, do you have a tic, is there a tremor, has BP gone up or down, do you speak fast or dither. Do you look anxious or tearful. Have you got a black mole on your ear, are you constantly scratching your leg etc. There are so many subtle clues to health that a doctor is trained to notice.

AI listening to an appointment can only summarise from the words spoken and make a suggestion which could skew the GP in the wrong direction because it is unable to know or see all the other cues. Therefore its assumptions can be grossly wrong. Is the busy doctor going to then re type and correct the information? Takes time to check and correct…

The doctor interviewed pointed out that wrong assumptions were made.

Since we are the Guinea pigs AI is constantly learning from and without the visual clues and background patient information it can gradually skew information and make new assumptions and diagnostics based on its own interpretation and from uncorrected mistakes it’s been fed in the past.

Certainly we will be cajoled into accepting this as amazing innovation that purports to be for the greater good, but there is a downside that a little investigation can find to be troubling enough to pause and consider.

I’m not ready to participate in an experiment I don’t yet trust and prefer one on one exclusively with my GP for something so intimately personal.

I also don’t want my information used or stored in another country by some company selling this as benevolent and which is in this solely for large profit. Will it add value to these profits by selling our information?

Bung

6768 posts

Uber Geek
+1 received by user: 2951

Subscriber

#3433708 12-Nov-2025 10:54

"Heidi Health's AI tool is designed to transcribe and summarize patient consultations for administrative assistance and documentation purposes. It does not make definitive diagnostic suggestions; all professional medical decisions, including diagnoses, remain the sole responsibility of the medical practitioner. "

I'm more concerned at the errors I see in the transcribing old consultation notes into Managemyhealth. Nothing that important but you never know what could affect insurance for example.

eracode

Smpl Mnmlst
9452 posts

Uber Geek
+1 received by user: 6336

ID Verified

Trusted

Lifetime subscriber

#3433711 12-Nov-2025 11:22

Care is needed if talking about AI in the doctor’s surgery. It also stands for Artificial Insemination. 😄

Sometimes I just sit and think. Other times I just sit.

Lego

Shop now for Lego sets and other gifts (affiliate link).

networkn

Networkn
33022 posts

Uber Geek
+1 received by user: 15610

ID Verified

Trusted

Lifetime subscriber

#3433721 12-Nov-2025 12:58

eracode:

Care is needed if talking about AI in the doctor’s surgery. It also stands for Artificial Insemination. 😄

I am pretty sure, I've never heard of that shortened to AI in a Doctors office :)

Are you sure you haven't been going to the Vet by accident ? It could explain some things :)

Bung

6768 posts

Uber Geek
+1 received by user: 2951

Subscriber

#3433728 12-Nov-2025 13:23

You don't go to a vet by accident once you know how much they charge.

duffles

102 posts

Master Geek
+1 received by user: 77

Trusted

2degrees

#3445007 17-Dec-2025 15:44

I went to the GP this week and was asked for permission if notes could be taken by Heidi (https://www.heidihealth.com/)
After the ~10 minute consultation, doctor went over the notes and made a couple of corrections that were likely caused by my hideous southern kiwi accent then posted it on my file. After the copy and paste, the notes were erased. Ready for the next patient.
It was probably the most efficient visit I've ever had. The doctor was fully engaged and looked a little more relaxed. Probably because she didn't have to talk, then type, then talk, then listen, then think, then type, then talk...

There was a little poster on the wall with all of the information and I was asked if I wanted a copy of it.

I asked if many people had declined permission. Unfortunately as of midday that day, I was the only patient who granted permission.

They say he carved that spoon himself.. From a bigger spoon..
Any comment made here is my own and should not be taken as that of my employer. You've seen one of these statements before.

Gordy7

gordy7
2010 posts

Uber Geek
+1 received by user: 505

ID Verified

Lifetime subscriber

#3445011 17-Dec-2025 16:07

eracode:

Care is needed if talking about AI in the doctor’s surgery. It also stands for Artificial Insemination. 😄

Always Interested.

Gordy

My first ever AM radio network connection was with a 1MHz AM crystal(OA91) radio receiver.