Member Avatar for Dani

I've started other threads here recently that basically revolve around how AI generated content can be untrustworthy, and also the frustration to publishers in that AI bots basically crawl/train on their content without a link back to the source.

I have another question that sort of borders on the two of them: Do you think there should be some type of formal association that oversees and certifies AI systems for accuracy, trustworthiness, etc.?

I'm not necessarily saying it has to be a government entity and that AI systems must be certified or else pay huge fines. I'm thinking more of just like a private association that AI systems will want to join because the industry would have evolved to needing the certification to be trusted by end-users.

  • 3 Contributors
  • 8 Replies
  • 44 Views
  • 10 Hours Discussion Span
  • Latest Post Latest Post by Reverend Jim

Recommended Answers

All 8 Replies

Member Avatar for Reverend Jim

I don't see how that would be possible. First of all, who would pay the thousands of people required to do the checking? And at the rate new data, or new arrangements of same are being generated it's even worse than the Red Queen's Race.

Member Avatar for rproffitt

Testing a thing.
giphy_(1).gif

Member Avatar for Dani

I think you guys are completely misunderstanding my question. I'm not talking about federal laws, or laws or regulations of any kidn, really. I'm thinking of a completely independent, private company that happens to be in the business of certifying AI systems.

For example, OpenAI will pay this company to examine their algorithms and data for accuracy, trustworthiness, etc., and in return, they would receive a "Certified a Trustworthy AI by <Company X>" label. My thought is that all of the AI companies are trying to prove that they have fewer hallucinations and higher accuracy than their competitors. I thought maybe out of that an independent certification company might be born. Thoughts?

Member Avatar for rproffitt

Can you share an example of such a certification body that imposes fines?

Edited by rproffitt because: Oops
Member Avatar for Reverend Jim

I understood the first time. I just don't see how what you are suggesting is possible. How do you certify that an algorithm will not produce incorrect results when you can't screen everything that went in to training the model?

Edited by Reverend Jim
Member Avatar for Dani

Can you share an example of such a certification body that imposes fines?

I'm saying not a body that imposes fines.

How do you certify that an algorithm will not produce incorrect results when you can't screen everything that went in to training the model?

Because, in my hypothetical world, this certification body will have access to proprietary data from OpenAI, Claude, etc. and will basically rate them against each other.

I'm comparing it to more established industries where these certification bodies came to exist in order to offer the major companies within the industry the ability to provide greater trust and transparency to the consumer via independent studies.

commented: What did the skull say? "I ain't got no body." +17
Member Avatar for Reverend Jim

I imagine a system where AI packages are developed to assess and rate other AI packages. We get into the whole quis custodes observat (who watches the watchers) spiral. How do you trust that the evaluating software is reliable?

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.