Hello, Daniweb --
I have a C++ library, for which I am trying to make a Python interface. I have come across a linking issue which I
haven't been able to solve. The library has a series of header and source files, and I am looking for a way to
add the Python interface without modifying any of the source files, by separately compiling linking in the new Boost Python sources,
rather than adding the additional Boost code directly in the .cpp files.

I have worked out a simple test case for what I am trying to do. There is a simple class called "World", a source file that
implements some of the members, and a source file for the wrapper code. In addition, there is a makefile with two targets, one that
compiles the files in a single translation unit, which works fine, and one that compiles the wrapper and the source as separate
translation units, and then links them, which does not work. There are no compiling or link messages, but on import, Python
reports an undefined symbol.

Here is the header --

#ifndef IG________________WORLD_H______________________

#define IG________________WORLD_H______________________
#include <string>

struct World
  World(std::string msg);// : msg(msg) {} // added constructor
  void set(std::string msg);//{ this->msg = msg; }
  std::string greet(); // { return msg; }
  std::string msg;

#endif // IG________________WORLD_H______________________

Here is the source file

#include <boost/python.hpp>
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
#include "World.h"
using namespace boost::python;

World::World(std::string msg) 
  : msg(msg) 
{} // added constructor

std::string World::greet() 
  return msg; 

void World::set(std::string msg)
  this->msg = msg; 

and here is the wrapper source file.

#include <string>
#include <boost/python.hpp>
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>

using namespace boost::python;

// uncomment this line to compile as a single translation unit
#include "World.cpp"
// OR, uncomment this line to compile as separate translation units. 
//#include "World.h"

  class_<World>("World", init<std::string>())
    .def("greet", &World::greet)
    .def("set", &World::set)
    .add_property("rovalue", &World::greet)
    .add_property("value", &World::greet, &World::set)

The compilation as single or separate translation units is accomplished by including either just the header file
into the wrapper file -- by uncommenting line 11, or including the source file into the wrapper file, by uncommenting
line 9.

The compilation as separate or single translation unit requires separate makefile targets, and this is done with the following

GPP = g++ $(MATHINC:%=-I%) -fopenmp -std=gnu++0x -O3 -Wall -D UBUNTU -D DEBUG
BOOST = -lboost_python -lpython2.7 -lboost_python-py27 -I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu/python2.7 -I/usr/include/boost/python/ -L/usr/lib/python2.7/config-x86_64-linux-gnu/libpython2.7.a  -L/usr/lib/x86_64-linux-gnu/
BOOSTINC = -I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu/python2.7/ -I/usr/include/boost/python/
BOOSTLIB =    -lboost_python -lboost_python-py27 -lpython2.7  # -lboost_python  -lboost_python-py27 -lpython2.7
BOOSTLIBDIRS = -L/usr/lib/python2.7/config-x86_64-linux-gnu/  -L/usr/lib/x86_64-linux-gnu/

# single translation unit target
    $(GPP) -shared -o exe/libWorld.so -fPIC src/WorldWrapper.cpp $(BOOSTLIB) $(BOOSTINC) $(BOOSTLIBDIRS)

# separate translation unit target
    $(GPP) -fPIC -c src/World.cpp $(BOOSTINC) -o obj/World.o
    $(GPP) -fPIC -c src/WorldWrapper.cpp $(BOOSTINC) -o obj/WorldWrapper.o
    $(GPP) -fPIC -shared -o exe/libWorld.so $(BOOSTLIB) $(BOOSTINC) $(BOOSTLIBDIRS) obj/World.o obj/WorldWrapper.o

We have made sure that the libraries, includes, etc are the same, by putting them into variables BOOSTINC, BOOSTLIB, and BOOSTLIBDIRS, and
using these in both targets. To try both of these we simply uncomment either line 9 or 11 in the wrapper file (not both), and then
make the appropriate target, above. Both compile and link without any error message. However, when we try to use the resulting shared object,
libWorld.so, we get the following. Using the following as the python source --

import sys
sys.path.insert(0, "/home/david/finance/mathlib/mathlib/boost/exe")
import libWorld
x = libWorld.World("dave")

we get

david@Thinkpad:~/finance/mathlib/mathlib/boost$ python src/World.py
Traceback (most recent call last):
  File "src/World.py", line 3, in <module>
    import libWorld
ImportError: /home/david/finance/mathlib/mathlib/boost/exe/libWorld.so: undefined symbol: _ZTIN5boost6python15instance_holderE

in the case of the separate translation units, linked together, where as the python code runs without error in the case of single translation units.

Does anyone know what the problem is with my separate translation unit code? By using the same code for both examples,
and nearly the same makefile lines, I claim that the problem can't be in the code itself, rather, it has to be
in the makefile, but I don't see where the problem lies. The libraries included actually DO have the missing symbol,
a class called boost::python::instance_holder, as shown by the single translation unit example.
But nm -a --demangle exe/libWorld.so | grep "instance_holder " | grep " U "
shows that the symbol is definitely missing.

david@Thinkpad:~/finance/mathlib/mathlib/boost$ nm --demangle -a exe/libWorld.so | grep "instance_holder" | grep " U "
                [K U [Kboost::python::instance_holder::deallocate(_object*, void*)
                [K U [Kboost::python::instance_holder::install(_object*)
                [K U [Kboost::python::instance_holder::allocate(_object*, unsigned long, unsigned long)
                [K U [Kboost::python::instance_holder::instance_holder()
                [K U [Kboost::python::instance_holder::~instance_holder()
                [K U [Ktypeinfo for boost::python::instance_holder

Edited by Reverend Jim: Moved to programming forum

4 Weeks
Discussion Span
Last Post by rproffitt

Thanks for having a look. I found a comment on stackexchange that -Xlinker --whole-archive would prevent the compiler from dropping
apparently unused symbols, and this converted the missing-symbol message to a core-dump on python load. Not a big improvement, and nm -a still shows the same symbol (and a bunch of others, actually) to be undefined in the target, when they ARE defined in the libraries i am linking to. Moreover, the man page for g++ says nothing about a "--whole-archive" option for Xlinker. Still mysterious.

Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.