943,840 Members | Top Members by Rank

Ad:
  • PHP Discussion Thread
  • Unsolved
  • Views: 2126
  • PHP RSS
You are currently viewing page 1 of this multi-page discussion thread
Jul 1st, 2009
1

::Making Daniweb FAQ Bot::

Expand Post »
I was thinking of a new project for making a bot that scans daniweb for information and enters useful information into a database then people can ask the website questions and php+mysql will provide answers in paragraph form for the user to read. Is this ok on the legal side of things as I have written bots for my personal use before and have made a tutorial. Also if this is ok by the mods then any help with the algorithm would be appreciated.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Jul 1st, 2009
0

Re: ::Making Daniweb FAQ Bot::

cwarn23,

Congratulation for the art-work - A sharp, smooth, smart, and intellegent code work.
Moderator
Reputation Points: 2136
Solved Threads: 1228
Posting Genius
adatapost is offline Offline
6,527 posts
since Oct 2008
Jul 2nd, 2009
0

Re: ::Making Daniweb FAQ Bot::

I've posted the question to the moderator lounge:

http://www.daniweb.com/forums/post90...tml#post905956
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Jul 2nd, 2009
-1

Re: ::Making Daniweb FAQ Bot::

Nobody beside mod or admin can follow that link
Moderator
Featured Poster
Reputation Points: 2786
Solved Threads: 873
Code tags enforcer
peter_budo is offline Offline
6,656 posts
since Dec 2004
Jul 3rd, 2009
0

Re: ::Making Daniweb FAQ Bot::

Is the topic on the moderators lounge looking hopeful or haven't there been that many replies yet?
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Jul 3rd, 2009
-7

Re: ::Making Daniweb FAQ Bot::

Not many replies yet, dont think dani has seen it yet. Happygeek (administrator) has and its currently being discussed.
Last edited by jbennet; Jul 3rd, 2009 at 10:58 am.
Moderator
Featured Poster
Reputation Points: 1764
Solved Threads: 574
Moderator
jbennet is online now Online
16,512 posts
since Apr 2005
Jul 3rd, 2009
0

Re: ::Making Daniweb FAQ Bot::

Click to Expand / Collapse  Quote originally posted by cwarn23 ...
Is the topic on the moderators lounge looking hopeful or haven't there been that many replies yet?
still little discussion on that yet.

Is the RSS feeds available to every forum not a better alternative then a spider? It would involve less parsing...
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Jul 3rd, 2009
0

Re: ::Making Daniweb FAQ Bot::

I just checked the rss feeds on forum 17 and the rss feeds seems to be full of questions instead of answers. I was thinking more along the lines where the user could type a question like I did in post #1 or perhaps even send a link to their post and php would be able to string together answers from the database. And the database would be populated from bot scans. For now I'll find another site to scan as a demo while a decision is being made.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Jul 3rd, 2009
0

Re: ::Making Daniweb FAQ Bot::

Some tips for your code cwarn:

Don't use preg_* functions unless absolutely necessary, if you're comparing a static string use strpos or stripos.

Define reused regular expressions in one place (you use /(.*)[.]([^.\?]+)(\?(.*))?/ three times in the same script, define it once in a variable and use it that way, one point of failure is always better.

If you're going to go for speed at the cost of memory usage get rid of in_array. Build an 1-level index and use isset() so you're performing O(1) operations instead of O(n) the in_array and array_key_exists functions are expensive. Example:
php Syntax (Toggle Plain Text)
  1. $some_big_array = array(1 => somestring, ..., 10000 => anotherstring);
  2.  
  3. $strings_index = array('somestring' => 1, ...., 'anotherstring' => 1);
  4.  
  5. $search = 'somestring';
  6. if (isset($strings_index[$search])) // O(1) operation
  7.  
  8. if (in_array($search, $some_big_array)) // O(n) operation

STOP USING global
Don't use unset() right before an assignment of the same variable, ie.,
php Syntax (Toggle Plain Text)
  1. $datac = "somestring";unset($datac);$datac = "anotherstring";
The unset becomes wasted time because reassignment is an implicit flush of previous memory
Last edited by ShawnCplus; Jul 3rd, 2009 at 10:25 pm.
Sponsor
Reputation Points: 520
Solved Threads: 268
Code Monkey
ShawnCplus is offline Offline
1,564 posts
since Apr 2005
Jul 4th, 2009
0

Re: ::Making Daniweb FAQ Bot::

Although that's good advice Shawn and I will take note of that.
The only thing I would disagree with is the unset(). I have noticed in the past when reassigning string to variable in a loop a few million times the server runs out of memory where as if unset is used I have noticed the memory problem does not occur.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in PHP Forum Timeline: Installing php libraries on linux
Next Thread in PHP Forum Timeline: in sql server same unique identifier in 2 tables





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC