Hmm, I thought that would be okay way to get some free backlinks.
Not anymore. Those backlinks will be worthless. That content be filtered out as duplicate content which means no one will find it to link to it. If no one is linking to their page links from that page are worthless. Add to the fact that those pages not only will be in the supplemental index but they will be duplicate content. For all we know Google discounts those links.
If your article is any good put it on your website. That way you get all of the links from that article. If it is a bad article then it wouldn't have been successful on the other websites anyway and, once again, the links from those sites would be worthless.
Plus if another site republished your article and Google considers them to be the source of the article then your copy will be considered duplicate content and you'll have your page put into the supplemental index.
We had submitted the articles, which are now on the site, to various article submission sites. Would we be better off deleting those articles off our site and writing new ones? Is it affecting our site negatively since the articles have been duplicated?
It won't hurt your site but they won't help it either and thus isn't worth your time. Create unique articles for your site that are useful and informative. Those kinds of articles are good to have because they attract links which is what you are ultimately after.