Typically in my world with AI medical charting that I am an advisor, for the Gen 2 products don’t hallucinate if built properly or not as much. Gen 1 products are bad. I lecture and teach doctors etc how to over ride the systems to avoid it (prompt engineering I guess lol) and most suggestions. We also teach how to build proper templates etc using normal language models. No coding unless you really want to.
My engineering son @MaineLonghorn said the same as you. His company is looking for a better built mouse trap but most equations etc end up wrong.
I look at AI as an assistant. I only use it for medical charting (notes in 30 seconds and 98% accurate after listening to me and my patients at the same time). I really don’t have other needs for it except for an outline for my lecture on… AI. Lol.
There is a saying and is applied to most fields. “AI won’t replace doctor’s but doctor’s that don’t use AI will be replaced”.
A teacher patient of mine said she say the exact quote but applying to teachers at a conference.