Simple script parser - how to ?

[MaestroS]

I really want to know WHERE I can find a simple example of parser for own script language for games.

I found one on gamedev.pl [polish site], but I can't stick together all parts of the code.

It'd be nice if someone can post a links to examples.

I mean about SIMPLE parsers (interpreters) to edit and re-build.

Thanks in advance!

#edit
If someone don't know.

I've file:
monster.mas

and it contents:

setMonsterHP(10);
setMonsterPos(10);

and this parser should just translate this code to be understoodable by game

[Infarction]

If you're reading in from a file, you can write an input method which expects the data to be formatted a certain way, and then parse it yourself by just reading in pieces at a time, as you need them.

[~s.o.s~]

Though reinventing the wheel won't be such a good idea unless your aim here is to learn how to create custom parsers and not the game.

[Infarction]

I'd think that if you kept a simple format, it wouldn't be that much more work. I could easily be wrong though, last time I did something like this was in high school like 5 years ago...

[~s.o.s~]

Also read this for implementation of a scripting language.

[MaestroS]

All I know is that data must be readed from file.

For example:

File monster.mas:

create monster 10 10 2

When the file is loaded, parser is checking each line and is interpreting it.

I got something like this:

Open file

#include <cstdio>
#include <string>
#include <iostream>
#include <fstream>
#include <vector>
 
using namespace std; 
 
string bufor;
 
int main()
{
  ifstream plik;
   plik.open("skrypt.txt");
   char c;
   while(true)
   {
      c = plik.get();
      if(plik.eof()) break;
      bufor += c;
   }
   plik.close();
}

Definitions of global variables for tokens and parameters

const char* WS_SET = " \t\r\n";
const char* BR_SET = "\r\n";
 
#define NOT_FOUND string::npos
#define POS_END string::npos

string bufor, cword, err_str;
int pos = 0, param = 0;

This function is taking each command as token

bool get_token()
{
 int tmp_pos, tmp_pos2;
 
 tmp_pos = bufor.find_first_not_of(WS_SET, pos);
 if(tmp_pos == NOT_FOUND)
 {
  cword.clear();
  pos = POS_END;
  return false;
 }
 tmp_pos2 = bufor.find_first_of(WS_SET, tmp_pos);
 if(tmp_pos2 == NOT_FOUND)
 {
  pos = bufor.length()-1;
  cword = bufor.substr(tmp_pos, bufor.length()-tmp_pos);
 }
 else
 {
  pos = tmp_pos2;
  cword = bufor.substr(tmp_pos, tmp_pos2-tmp_pos);
 }
 return true; 
}

Function that is taking parameter from each command

bool get_param()
{
  int tmp_pos, tmp_pos2;
  string s_number;
 
  tmp_pos = bufor.find_first_not_of(WS_SET, pos);
  if(tmp_pos == NOT_FOUND)
  {
    param = 0;
    pos = POS_END;
    return false;
  }
  tmp_pos2 = bufor.find_first_of(WS_SET, tmp_pos);
  cword.clear();
  if(tmp_pos2 == NOT_FOUND)
  {
    tmp_pos2 = bufor.find_last_not_of(WS_SET, tmp_pos); 
    if(tmp_pos2 == tmp_pos) 
    { 
      s_number = bufor[tmp_pos]; 
    } 
    pos = tmp_pos2; 
  } 
  if(s_number.empty()) 
  { 
    s_number = bufor.substr(tmp_pos, tmp_pos2-tmp_pos); 
    pos = tmp_pos2; 
    } 
  
  param = atoi(s_number.c_str());

Ignoring line for comments

void ignore_line() 
{ 
 int tmp_pos; 
 tmp_pos = bufor.find_first_of(BR_SET, pos); 
 if(tmp_pos == NOT_FOUND) 
 { 
 
  pos = POS_END; 
  return; 
 } 
 tmp_pos = bufor.find_first_not_of(BR_SET, tmp_pos); 
 if(tmp_pos == NOT_FOUND) 
 { 
  pos = POS_END; 
  return; 
 } 
 pos = tmp_pos; 
}
  return true; 
}

Example of a parser code

bool parse() 
{ 
 COMMAND_INFO cmd_info = { CMD_UNKNOWN }; // zaraz wyjaśnimy, co to jest ;-) 
 
 do 
 { 
  if(!get_token()) return true; 
  if(cword.length() > 1) 
  { 
   if(cword.substr(0, 2) == "//") // komentarz - ignoruj resztę wiersza 
   { 
    ignore_line(); 
    continue; 
   } 
  } 
 
  // *** 
  if(cword == "create_monster") 
  { 
   // ... 
   if(!get_param()) return false; // taking first argument
   // ... 
   if(!get_token()) //taking second argument
   { 
    cout << "BŁĄD, spodziewałem się jakiegoś słowa..." << endl; 
    return false;  
   } 
   // ... 
   if(!get_param()) return false; // taking third argument
   // ...  
   if(!get_param()) return false; // taking fourth argument
   // ... 
  } 
  else if(cword == "create_item") 
  { 
   // ... 
   if(!get_param()) return false; 
   // ... 
   if(!get_param()) return false; 
   // ... 
   if(!get_param()) return false; 
   // ... 
  } 
  else if(cword == "equip_monster") 
  { 
   // ... 
   if(!get_token()) 
   { 
    cout << "ERROR: a command expected..." << endl; 
    return false;  
   } 
   // ... 
   if(!get_param()) return false; 
   // ... 
  } 
  else 
  { 
   cout << "Command not recognized: " << cword << endl; 
   return false; 
  } 
  // ... 
  // *** 
 
  
 } 
 while(pos < bufor.length()-1) 
 
 return true; 
}

But now, omg.... can someone stick it together to work functionally or re-build ... ? : /

[vijayan121]

if you are comfortable with c++ (ie. you can do more than write c code using c++ syntax), you would find the spirit recursive descent parser framework (part of the boost libraries) easy to use. internally, it uses template meta-programming techniques; but to use it productively, you just need to be familiar with mainstream c++ (primarily generic programming).

here is an example of parsing a text file:
file key_and_values.txt
------------------------------

phone_numbers = 1234, 5678 : 987,564:0:0
line 2 is a parse error
primes_2_to_19 = 2, 3, 5, 7, 11:13:17:19
primitive_pythagorian_triplet_1 = 8,15 : 7
line = 5 also an error
_12_34 = 1:2, 3:4
_daniweb_thread_numbers = 12345:56789:34567, 78901
primitive_pythagorian_triplet_2 = 9,12 : 15
adds_up_to_12 = -24 : +48 , -12

essentially it contains lines of the form
key = value, value : value.
keys have the same structure as valid c++ identifiers.
values are integers seperated by either a , or a :
white spaces are ignored.

here is a sample program that parses such a file:

#include <boost/spirit/core.hpp>
#include <boost/spirit/actor.hpp>
#include <iostream>
#include <fstream>
#include <vector>
#include <string>
#include <algorithm>
#include <iterator>
using namespace std;
using namespace boost::spirit;

bool parse_it( const char* cstr, string& key, vector<int>& values )
{
    return parse( cstr,
     ( ( ( (alpha_p|'_') >> *(alnum_p|'_') )[ assign_a(key) ] ) >> ( '=' >> 
       ( int_p[ push_back_a( values ) ] >> *(  ( ch_p(',') | ':' ) >> int_p[ push_back_a( values ) ] ) ) ) ),
                           space_p ).full ;
}

int main()
{
    const char* const file_name = "key_and_values.txt" ;
    ifstream file( file_name ) ;
    string str ;
    for( int line=1 ; getline( file, str ) ; ++line )
    {
      cout << "line " << line ;
      string key ; vector<int> values;
      if( parse_it( str.c_str(), key, values ) )
      {
        cout << "\nkey: " << key << "\nvalues: " ;
        copy( values.begin(), values.end(), ostream_iterator<int>(cout," ") ) ;
        cout << '\n' ;
      }
      else
        cout << " *parse error\n" ;
      cout << "-----------------------\n" ;
    }
}

the parsing rule is inlined in lines 15 and 16; the rest is just boiler plate code.
here is the output:

line 1
key: phone_numbers
values: 1234 5678 987 564 0 0
-----------------------
line 2 *parse error
-----------------------
line 3
key: primes_2_to_19
values: 2 3 5 7 11 13 17 19
-----------------------
line 4
key: primitive_pythagorian_triplet_1
values: 8 15 7
-----------------------
line 5 *parse error
-----------------------
line 6
key: _12_34
values: 1 2 3 4
-----------------------
line 7
key: _daniweb_thread_numbers
values: 12345 56789 34567 78901
-----------------------
line 8
key: primitive_pythagorian_triplet_2
values: 9 12 15
-----------------------
line 9
key: adds_up_to_12
values: -24 48 -12
-----------------------

for more information see http://spirit.sourceforge.net/ or the boost libraries documentation.
this example is written using spirit 1.8 version 2 is due in mid-may (during BoostCon '07)

[MaestroS]

@up
thanks for it

#refresh
if someone has easier idea - reply!