DaniWeb Auto Answer A.I. feature

[toneewa]

I'm curious on people's thoughts of using DaniWeb's content to train an A.I. model, with all the questions that are marked as Solved. It can be an optional feature, like "Ask DaniWeb AI".

When a user asks a question, the DaniWeb AI will auto answer your question if it's been known to be solved already.

It can also be a reference guide to telling you to maybe take a look at a particular thread(s), if it say matches a threashold of 90%+ content.

With over 541k threads and 2.29 million posts, we could make DaniWeb a place to get all your answers.

[Dani]

Currently, when you ask a question on DaniWeb, as you type in the title and message, we perform a search against all of our existing content to see if a similar question has already been asked.

However, I do admit that I don't have any experience with ChatGPT nor do I know how to train an AI model. From my understanding, the API costs money, although it does look very reasonably priced, especially for something that would make a dent in our traffic or revenue (currently we don't come anywhere close to paying monthly hosting costs).

[toneewa]

The question was inspired from seeing news and videos on Chat with RTX.

[Dani]

That looks interesting. It’s not relevant to us because we would need an expensive GTX video card, Windows computer, and there’s no API … but it looks intriguing :)

[toneewa]

An RTX 30 or 40 series card with 8GB of VRAM. ~$299. There is no support with Wine a user has reported. There was a link to a python api. That was just an example and a thought, but not limited there.

There's creative solutions.

Having only users with an RTX card can use the feature, and processing their own requests. Require a LLM download from DaniWeb.
Remoting and communication routing to other machines that do have Windows. :)

[Dani]

Pardon my cluelessness here. A $300 video card can efficiently perform complex AI tasks against DaniWeb’s entire database in real time?

Missed the link to the Python API.

[toneewa]

I have yet to try it or find it's limitations. Trying to free up some space. Those were the minimum requirements, plus 16GB of RAM and ~100GB free space. I also don't know how large a dataset for DaniWeb would be. Will know more later.

[toneewa]

It indeed seems to be working. The interface is all done in the browser, on your localhost, 127.0.0.1:43905/?__theme=dark, and I stripped it down to just the message box. Just have to gather more data, and it seems good. Took ~50 minutes to get installed. The dataset can be a simple text file with a space between each topic thread.

[toneewa]

I did some sample testing, and parsed a thread topic for the dataset. Reducing the footprint from 220kb to 26.5kb. All in 0.43 seconds. It doesn't use any API. Just the patterns I saw in the code. It took another 6 seconds to embed too Chat with RTX.

Then, not deducting for the deleted threads, this could be a lot smaller, and quicker than expected to parse the pages of DaniWeb in full.

Estimated time: 3 days 3 hours 17 minutes. Expecting a maximum size of 18.144 GB, we can estimate a maximum time of 37.63 days to import at 33.90it/sec. A larger sample would give us better numbers here, instead of worst case.

Chat With RTX uses around 9-16 GB of VRAM depending on the query. You have shared memory (your free RAM) that replaces it if you run out of VRAM.

Food for thought...