Dani AI Overviews

[Dani]

Playing with ChatGPT to organize our tags has sent me down a rabbit hole. I definitely think I'm going to get a lot of flack for this, but I'm experimenting with adding AI Overviews to forum threads (beneath the first post, above replies). Don't be caught off-guard if you spot this in the wild, and please let me know what you think.

They are clearly listed as AI content, which means that they may be incorrect, etc.. I still don't think it's appropriate for AI-generated content to appear within forum posts that are meant to be the words of a person. Especially when used in a deceptive way (e.g. pretending to sound human).

[Salem]

which means that they may be incorrect

So basically like the Hitch Hikers Guide to the Galaxy.
"it has many omissions and contains much that is apocryphal, or at least wildly inaccurate"

Will the summary be guided by things like posts with up-votes carry more weight?

[Dani]

It’s not so much a summary or rephrasing of the existing posts, as much as it is providing complementary content to what exists already.

[rproffitt]

From XKCD. To see if AI is a fine thing, fill in the following with "AI".

[Dani]

Here are some very random examples:

• https://www.daniweb.com/programming/web-development/threads/283030/pass-a-checkbox-value-to-javascript-function
• https://www.daniweb.com/programming/software-development/threads/344201/checking-if-process-is-running-using-c
• https://www.daniweb.com/programming/game-development/threads/502001/pokemon-game-in-python-pygame

[Reverend Jim]

I really don't know what to make of this. A few months ago I posted a SQL solution that was generated by AI and vetted by someone who has decades of SQL programming experience. In reponse you downvoted and flamed me. What am I supposed to take away from this 180 degree turn in policy?

[Dani]

I’m very sincerely sorry. I didn’t mean to offend you. :(

However, I’ve never had a change of heart. It’s against the rules to copy/paste AI-generated content into forum posts. Nothing changed there.

I admit I am embracing AI a bit more here as an experiment to see if it directly has an impact with SEO. But IMHO that doesn’t change my stance on humans posting AI within their forum posts.

[Dani]

Zuul? I don’t get it.

[Reverend Jim]

It’s against the rules to copy/paste AI-generated content into forum posts.

But that's exactly what you are doing except instead of a human it's the AI. I'm thinking more Die Hard 2 than Ghostbusters.

[Reverend Jim]

I didn’t mean to offend you. :(

We had a difference of opinion and I was surprised at the backlash, but I was not offended.

An oldie but a goodie from former heavy metal drummer Steve Hughes on being offended

[Dani]

But that's exactly what you are doing except instead of a human it's the AI. I'm thinking more Die Hard 2 than Ghostbusters.

I'm sorry but I never saw Die Hard 2, and I haven't seen Ghostbusters since the 1980s when I was too little to remember it. I only remember Slimer.

Either way, I feel as if there are some very big differences:

Forum posts are universally understood to be written by community members. Schema.org markup in the HTML identifies posts to Google and other search engine bots as being human-written. The Schema markup sends signals to bots that indicate the profile of the member who made the posting, how many votes the post has received, and other such things. Google then puts their own profile together about the community member, but I digress.

AI-generated writing within posts is quite deceiving to both other community members and Google alike. Members want to trust that they are communicating with other real people, not bots. If they wanted to talk to a bot, they would bring up ChatGPT. Instead, people communicate in forums because they want to share ideas and experiences with each other. That's why you're here, right? You want to demonstrate and show-off a lifetime of learned expertise along with your own personal experiences and opinions. Meanwhile, AI-generated content alludes to confidence and experience. It's simply deceiving when it's copied and pasted into forum posts, pretending to be the words of a real person.

Similarly, it breaks Google's trust when our HTML markup sends clear signals that a post is written by a specific community member, and then it turns out it's written by a bot.

The "Dani AI" overviews, on the other hand, are clearly labeled as AI content, and not designed to resemble the look and feel or to be confused with a forum posting.

[Dani]

Let's put it a different way. Based on the very few example links provided above, can you see any benefit or helpfulness to the AI overviews?

[SCBWV]

I'm not seeing the value here.

1. I thought it was going to give an overview or summary of the entire thread, not just answering the code question.
2. The 3 examples were coding questions. AI answered differently than the humans. Always more than one way to accomplish the task.
3. How will you decide which AI LLM to use? Are some better at coding than others? Are they stronger in one language than another?
4. If it's just providing code, wouldn't people be better off going directly to the AI where they can ask follow-up questions?
5. Will AI be added on non-coding posts? What value would AI add to non-coding posts?

It's cool to see what you've been able to do though.

[rproffitt]

Supply a setting to turn off AI site wide.

It's carp and smells bad.

[Dani]

I'm not seeing the value here.

1. I thought it was going to give an overview or summary of the entire thread, not just answering the code question.

I'm confused. How do you not see the value in answering the question? Isn't that the point of a Q&A site? As far as summarizing the existing contents on the page, that seems a little redundant to me. I wouldn't see the point in that?

1. The 3 examples were coding questions. AI answered differently than the humans. Always more than one way to accomplish the task.

Precisely. Per my previous sentence, I don't see the point in redundancy, so I don't see why it would answer the same way as the posts by humans immediately below. The point is to provide complementary information to what already exists. Not to undermine the existing human interactions.

1. How will you decide which AI LLM to use? Are some better at coding than others? Are they stronger in one language than another?

I don't have much AI experience, but this is using the newly released GPT-5.

1. If it's just providing code, wouldn't people be better off going directly to the AI where they can ask follow-up questions?

It just so happened that the random threads I linked to were programming questions, so it makes sense to provide code as an answer to those. However, people today always have the choice of going directly to the AI nowadays. The purpose of going to a forum, instead, is for human interaction and sharing real world ideas and experiences. My goal is to not undermine that, but simply to provide complementary information, solutions, and references/links to additional information to better serve those stumbling upon the page from a Google search.

I have always been a huge proponent of our responses help the OP once, but then live on to help others thousands of times. For those thousands of people who come across a discussion from years prior as a result of a Google search, they are on their own mission to get an answer to their question. So I figured that incorporating AI that can direct them to additional information and resources on the topic being discussed can't hurt. Again, my primary goal is not to undermine, summarize, or simply repeat the human discussion. The AI overview is meant to be complementary.

1. Will AI be added on non-coding posts? What value would AI add to non-coding posts?

Same as with code posts: Additional solutions and references beyond what is already discussed by the humans on the page.

Here are some examples:

• https://www.daniweb.com/community-center/geeks-lounge/threads/302/word-association-game
• https://www.daniweb.com/programming/web-development/threads/150290/run-an-exe-online
• https://www.daniweb.com/programming/software-development/threads/310948/why-cant-i-import-java-util-scanner

It's cool to see what you've been able to do though.

Thank you!

[rproffitt]

You can do your own research such as https://arstechnica.com/information-technology/2025/08/the-gpt-5-rollout-has-been-a-big-mess

Just turn it off for me.

Clankers not welcome here.

[Dani]

You can do your own research such as https://arstechnica.com/information-technology/2025/08/the-gpt-5-rollout-has-been-a-big-mess

Yeah, I've read quite a handful of such articles. I think the biggest problem people have with it was that the rollout broke existing apps, and I've also spotted numerous complaints that GPT-5 spits out more "sterile" text than GPT-4. In my case, that was just what I was looking for, however.

I also went with GPT-5 because something I was specifically looking for was for Dani AI to spit out links to additional resources for people to be able to do a deep dive into the content. GPT-5 was trained on much newer web data than GPT-4, and I didn't want it to spit out broken links.

Just turn it off for me.

In what way does it being there make DaniWeb unusable for you? It sounds like you have more of a moral issue with it than anything.

[Reverend Jim]

Let me relate a recent experience with ChatGPT 4. I had a question about how to get a wxPython Listview control to display data in subgroups (something I had done in vb.Net). The generated example gave an error. I replied in the same session with the error and got an apology followed by the correct code. Which also resulted in an error. I did the lather-rinse-repeat about 6-7 times before I got an example that worked.

What steps have you taken to ensure that the AI generated responses are correct?

[rproffitt]

1. Dani and I went round and round about AI and GPT. I was ready to write tutorials but the hammer came down. There has been softening on that since the early days but I will not put tutorials here that risk censure. That ship has sailed.

2. The AI generated responses are unwelcome.

3. I don't see how the AI responses will be vetted or kicked out if incorrect.

   ! 4. Wyoming data center will soon use 5x more power than the state's human occupants.

[SCBWV]

I can scroll passed the green box if I'm not interested, so it wouldn't bother me either way. As for vetting AI responses, users often vet human posts, people can do the same for AI posts. I hope people realize AI is not perfect, but then again, my faith in mankind has been dipping lately

As for code, I skip the AI and go to search engines (Google, Bing, etc.), which of course give me links to several sites that I can peruse and compare before selecting the answer I want. AI would give me one (supposedly correct) answer. Many times, my search comes back with links that include DaniWeb. I prefer to Google; maybe I'm just old school.

[Dani]

What steps have you taken to ensure that the AI generated responses are correct?

That’s what I dislike so much about AI. As to ensuring the responses are correct, as mentioned by scbwv, AI is not perfect, but at least by being transparent and declaring it is AI-generated, one can take that information into consideration when reading it. Just as one does when reading Google’s AI overviews or asking something of ChatGPT. And just as one may take into consideration a member’s reputation points and get a sense of their experience level from their member profile when reading and judging the accuracy of human-written posts.

[Dani]

That’s why I think it’s oh so dangerous for AI content, with its super-confident language and false pretense of experience, to seemingly come out of the mouths of human members.

[trcooke]

On a practical note: Are you intending to use an external service to generate these AI Overviews? Or host your own?

Either way it sounds expensive. Is it?

[Dani]

I’m new to AI. Right now I’m just using the OpenAI API. It’s probably going to end up costing me a couple hundred bucks to run a test on enough topics to see if it’s going to give me the SEO improvement I’m looking for.

[Dani]

Oh, I forgot to mention. It’s just over a penny per AI overview. And that’s with the most robust GPT-5 (eg not mini or nano).

[jkon]

Based on the very few example links provided above, can you see any benefit or helpfulness to the AI overviews?

I don't see any benefit for DaniWeb or its users in those examples, I see only potential harm. Why would someone looking for an AI response to a coding question not ask an LLM directly? They would have to Google a query and land on a forum post with an AI-generated answer. Most people who use Google for coding-related issues do so to find human-generated content. When they see AI-generated content on the site, they won't scroll further and will mentally blacklist DaniWeb.

What might be useful is a 'DANI AI summary' on the right side of each post that would summarize each post and its replies. It could be a scrollable box on the right side of the page or a static box at the bottom right that changes as the user scrolls through the answers. This could be especially useful for long posts and answers. However, even this might contain errors that could offend the original poster, so I'm not entirely sure about this idea either.

[trcooke]

Having re-read this thread more carefully I realise I also misunderstood the desired outcome of the AI generated content.

I assumed, probably based on my own idea of what might be useful, that the AI content would create a summary of the responses provided by the real human members. For example: "The most suggested solution was to use composition over inheritance". To me that could be useful as a TLDR summary, instead of reading and filtering 3 pages of replies.

Having GPT-5 provide a response to the original post in a thread seems to be what you are really proposing here as a use case?

There are 2 reasons I wouldn't see value in that:

1. You would be positioning DaniWeb as a proxy for GPT-5. A proxy funded by Dani.
2. It moves away from the spirit of the forum. Wikipedia on "internet forum" describes it as "an online discussion platform where people can hold conversations in the form of posted messages". I hold an emphasis on the "people" component of that statement, and to dilute that component is to me to dilute the value of the forum.

How might you handle the scenario where Dani AI gets it completely wrong, and the human discussion that follows states as such? As a conversation, which is what a thread is, I doubt it would read very well and may not provide the value to the person asking the question that you would have hoped. It might even add confusion where none would have otherwise been without Dani AI / GPT-5.

Lastly, are you planning on updating all threads from all time with Dani AI content? Or just back through some lesser period of history? Or just new threads?

[Reverend Jim]

I think we should be able to vote/comment on the AI responses. If someone sees an AI response with the comment "Tried it. Doesn't work." then they can ignore the response. And too many down votes or negative comments would give you feedback on the success of the deployment.

[SCBWV]

You guys bring up a good point. With the AI I have used, you can point out the incorrect answer and it responds (correct or not is another thing). How would that happen here?

[Salem]

Ignoring all the bots, crawlers and scrapers, who actually reads any of the decade+ old posts?

There are the occasional drive-by posters who somehow inexplicably manage to bump threads close to their anniversary (like nothing knows what year anything was posted). "Hey, it's August, this post from 20 years ago must be super relevant". ISTR Dani doesn't want to blanket close every old thread, but can reply privilege be restricted to established members with positive reputation? The "hey, me too" bumps are not worth anyone's time.

I can see a lot of time, effort and money being spent adding "AI summaries" to 1000's of posts that will never see human eyes again. Maybe a "summary on demand" approach?

Aside from the necromancy, are you proposing to re-run the AI summariser every time a thread gets a reply from a human (you know, someone who might actually know what they're talking about).

TBH, I'd like a check box in my profile to hide the AI summary. It does nothing for me except waste screen space getting to the real conversation.