I wish more people were familiar with Gell-Mann Amnesia. It was originally coined to refer to media reporting specifically, but I feel like it applies just as well to ai now.
The phenomenon is this: you see a piece of writing (or an ai approximation) of something in your area of expertise. You see all the mistakes, misrepresentations, inaccuracies. You can see that it's poorly produced, inaccurate trash. You see a piece of writing from the same outlet (or an ai approximation) of something in a field about which your personal experience is limited. It looks plausible enough to pass as probably all right to you, and you give it no further scrutiny, assuming it's correct and fine.
But like... did you forget about the fact that it was produced by the same thing that made an absolute fucking dog's breakfast of the thing you actually know about. Now, more than ever, we all need to hold the idea of this phenomenon in our mental back pocket. Like... please.