What does PyArg_Parse() do in detail??

[sneek]

Hi,

I read the Python/C-API and they say I should not use PyArg_Parse() anymore but use PyArg_ParseTuple(). My problem is that I retrieve an PyObject * out of PyDict_GetItem(...) so this is a single object and not a tuple. PyArgs_Parse() is the only API-Function to get a C-"string" out of this right?
But this not really my main interest. I'm pretty much interested in what these functions do in detail when I specify a format string. Do they copy data from somewhere (wherever the specific object is stored) to a new memory area and sets the pointer to this new area? Do they in fact use "malloc" and friends to copy the data or do they just return a pointer to the place where the PyObject is already stored?

Thanks a lot.

Jonny

[Gribouillis]

If you want to get a C string out of a python object, if the object is a python string you should use

char * s = PyString_AsString(pyobj);

and then use s as a read only C string. If you don't know if the object is a string, you
can use

PyObject* temp = PyObject_Str(pyobj); // convert to python string
char* s = PyString_AsString(temp);
// ... use s read-only
Py_DECREF(temp); // don't use s after this.

About the PyArg_Parse family, the main use of these functions is to convert a tuple of function parameters to C types. They copy small types, like int, double, etc but they don't copy strings, so that the strings read should be used read-only.

[sneek]

Hey thanks :)
Great explanation like always. Thanks a lot.

Do they probably copy short strings sometimes?
I wrote a little test-function that receives a String-Object, uses "PyArg_Parse(args,"s",myString)" and then modifies myString like "myString[1] = 'a'".
Then I did something like that from the interpreter:

a ="Hello World"
x.myTestFunction(a)
print(a)

"Unfortunately" nothing changed. a is still "Hello World" instead of "Hallo World"

[sneek]

Damn there is a new problem. I'm using Python3 so I have do deal with PyUnicode_AsUnicode() but I can't assign that to a char * variable. Is there a way to just get a Python-String into my char *?? I can't find anything in the API-Reference ;(

[Gribouillis]

I think that you should transform your unicode string to a byte array like this

>>> "hello world".encode("ascii")
b'hello world'

and pass the byte array to C. You could then use

char *s = PyByteArray_AsString(pyobj);

instead of the previous PyString_AsString.
The other way is to try to do it from C

temp = PyUnicode_AsASCIIString(pyobj); // must handle NULL return
char *s = PyByteArray_AsString(temp);
...
Py_XDECREF(temp);

[Gribouillis]

I think this (incomplete, 5 years old) [ulr=http://starship.python.net/crew/mwh/toext/toext.html]tutorial[/url] about writing a C extension in python is worth reading.

[Gribouillis]

Do they probably copy short strings sometimes?
I wrote a little test-function that receives a String-Object, uses "PyArg_Parse(args,"s",myString)" and then modifies myString like "myString[1] = 'a'".

The implementation of PyBytes_asString in python 3.1.1 is

char *
PyBytes_AsString(register PyObject *op)
{
	if (!PyBytes_Check(op)) {
		PyErr_Format(PyExc_TypeError,
		     "expected bytes, %.200s found", Py_TYPE(op)->tp_name);
		return NULL;
	}
	return ((PyBytesObject *)op)->ob_sval;
}

It means that this function only returns a pointer to the field ob_sval of the PyBytesObject structure: no new memory is allocated. I don't know exactly what happened in your test. Note that the structure is

typedef struct {
    PyObject_VAR_HEAD
    long ob_shash;
    char ob_sval[1];

    /* Invariants:
     *     ob_sval contains space for 'ob_size+1' elements.
     *     ob_sval[ob_size] == 0.
     *     ob_shash is the hash of the string or -1 if not computed yet.
     */
} PyBytesObject;

Also note that this code shows you how to check the type of an argument, and how to handle a type error.

[sneek]

char *s = PyByteArray_AsString(pyobj);

temp = PyUnicode_AsASCIIString(pyobj); // must handle NULL return
char *s = PyByteArray_AsString(temp);
...
Py_XDECREF(temp);

I thank's another time :icon_cheesygrin:
I tried this like you suggested, but unfortunately it ends up with a Segmentation fault.

PyObject * temp;
char *s = NULL;
temp = PyUnicode_AsASCIIString(name);
s = PyByteArray_AsString(temp);
fprintf(stdout, "Result: %s\n", s);  // <---- this one crashes
Py_XDECREF(temp);

If I just put "name" into PyByteArray_AsString I only receive "null" from fprintf()
May you have another idea why this will crash?

P.s. name is a PyObject * which receives it's value by PyDict_GetItem(). The value from the dict came from a file. This couldn't be the problem right? Python always stores strings in UNICODE, does it?

[sneek]

Hi,

does anybody have a solution to fix this problem. I can't believe that nobody ever wants to printf() a Pyton-String-Object within a C program.

Jonny

[Gribouillis]

Hi,

does anybody have a solution to fix this problem. I can't believe that nobody ever wants to printf() a Pyton-String-Object within a C program.

Jonny

Few people write C programs for python 3. I think you should experiment a while with these functions. I interface C++ and python 2.6, but for the main part, I use Py++ which I discovered recently (it works with gcc and boost python). I think you should start serious C programming with python 2.6.

[sneek]

Hi,

okay. Thank you for your support. I'll take a look at Python 2.6 and use the work-a-round with PyArg_Parse() to get the correct string representation into my char *. That works without any problems for me.

Since I think you really really do know what you are talking about. Did you see my other thread about the interpreter thing? May you have an advise for me? I don't know how to imagine the Embedding-Thing. What role does the interpreter play there? Does it run as a thread inside my C-program after I use Py_Initialize() or does a call to that function starts a new os-process with the interpreter? I imagine an interpreter as a thing where I can type in some commands or put in a script to run but that one doesn't fit here ;(

Thanks another time :o)

[Gribouillis]

No, the interpreter runs in the main thread. Its role is to execute python code. For example, you can call PyImport_ImportModule, and this will run the python code contained in the given module. There is also the PyRun_*** family (PyRun_File, PyRun_String,...) which execute python code. So your C code can execute python functions or python programs, like you would do in an interpreter shell.

[sneek]

Hi,

yeah, I played a little around with these functions but I thought that the interpreter starts with Py_Initialize() and ends with Py_Finalize(). That's how it works to this point right? What happens when I call Py_RunString("b=[/* something */]);
Does the function-call ends up in the same function the interpreter (commandline interpreter) would call if I type it in there? And does this happens because I link my programm against the python-lib?
Oh my lord...I'm a little confused. Sorry :S

[Gribouillis]

Hi,

yeah, I played a little around with these functions but I thought that the interpreter starts with Py_Initialize() and ends with Py_Finalize(). That's how it works to this point right? What happens when I call Py_RunString("b=[/* something */]);
Does the function-call ends up in the same function the interpreter (commandline interpreter) would call if I type it in there? And does this happens because I link my programm against the python-lib?
Oh my lord...I'm a little confused. Sorry :S

When you call Py_Initialize, the program creates the module __main__ where your commands are run, and allocates all the memory it needs for the __builtin__ module, etc, exactly as if you were starting a python session with the command "python". The only difference is that there is no interaction with the user, and your program must issue the calls to python functions and extract the data from the imported modules if it needs it. For example, if you want the value of the python variable 'b', you can try

PyObject* main_mod = PyImport_ImportModule("__main__");
PyObject* b = PyObject_GetAttrString(main_mod, "b");

For each function call, you must think about: Can the call return NULL ? Does it return a new reference to the python object ? (it must be decrefed then).

[lesni.bleble]

I thank's another time :icon_cheesygrin:
I tried this like you suggested, but unfortunately it ends up with a Segmentation fault.

PyObject * temp;
char *s = NULL;
temp = PyUnicode_AsASCIIString(name);
s = PyByteArray_AsString(temp);
fprintf(stdout, "Result: %s\n", s);  // <---- this one crashes
Py_XDECREF(temp);

If I just put "name" into PyByteArray_AsString I only receive "null" from fprintf()
May you have another idea why this will crash?

P.s. name is a PyObject * which receives it's value by PyDict_GetItem(). The value from the dict came from a file. This couldn't be the problem right? Python always stores strings in UNICODE, does it?

correct is:

PyObject * temp;
char *s = NULL;
temp = PyUnicode_AsASCIIString(name);
s = PyBytes_AsString(temp);
fprintf(stdout, "Result: %s\n", s);  // <---- this one crashes
Py_XDECREF(temp);

Few people write C programs for python 3. I think you should experiment a while with these functions. I interface C++ and python 2.6, but for the main part, I use Py++ which I discovered recently (it works with gcc and boost python). I think you should start serious C programming with python 2.6.

hmm, this is advice you "Posting Virtuoso"? go to python 2.6 and Py++? And I have used boost python, swig, Py++ etc.., but I ended in python c api => best choice.

[Gribouillis]

correct is:

PyObject * temp;
char *s = NULL;
temp = PyUnicode_AsASCIIString(name);
s = PyBytes_AsString(temp);
fprintf(stdout, "Result: %s\n", s);  // <---- this one crashes
Py_XDECREF(temp);

hmm, this is advice you "Posting Virtuoso"? go to python 2.6 and Py++? And I have used boost python, swig, Py++ etc.., but I ended in python c api => best choice.

Actually, I don't use Py++ any more. I'm currently interfacing python and D !