Unix, C++, pipe and fork

[kogorman]

In my current Unix/C++ application, I need to pipe a file through the sort(1) program before reading it. The input is large to huge, so in-memory sorting won't work.

I'm used to using fork(2), pipe(2) and exec(3) in my C programs. I know I can do the same in C++, but this means using C-style rather than C++ style code for pipe I/O operations.

This is not a big deal, but I'd rather stick to a consistent style. Is there an existing library that handles this, or a way to turn a file descriptor from a pipe(2) call into an std::istream?

++ kevin

[Ancient Dragon]

sort() does in-memory sort -- what you need is a file sort program. Or better yet write a file sort-merge function and add it to your program to preprocess the file. That way you can do away with that fork pipe and exec function calls.

you might check the boost libraries to see if it contains anything usefule.

[vijayan121]

...you might check the boost libraries to see if it contains anything usefule.

yes, boost.iostreams could be used.

#include <iostream>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <fcntl.h> // posix open
#include <cassert>
#include <iterator>
#include <stdlib.h> // system
using namespace std ;
using namespace boost::iostreams ;

int main()
{
  int fdin = open( "/usr/local/include/boost/iostreams/stream.hpp", 
                                    O_RDONLY|O_SHLOCK ) ;
  int fdout = open( "/tmp/copy_of_stream.hpp", O_WRONLY|O_CREAT|O_TRUNC ) ;
  assert( ( fdin != -1 ) && ( fdout != -1 ) ) ;

  {
    file_descriptor_source srce( fdin ) ;
    file_descriptor_sink sink( fdout ) ;
    stream<file_descriptor_source> input_file( srce ) ; 
    input_file >> noskipws ;
    stream<file_descriptor_sink> output_file( sink ) ;
  
    istream_iterator<char> begin(input_file), end ;
    copy( begin, end, ostream_iterator<char>(output_file) ) ;
  }

  close(fdin) ;
  close(fdout) ;
  assert( system( "diff /usr/local/include/boost/iostreams/stream.hpp"
                       " /tmp/copy_of_stream.hpp" ) == 0 ) ;
}

unlike most of boost (source only, just include the headers), constructing streams from posix file_descriptors or windows file handles requires linking to the library libboost_iostreams. eg. build with
> g++ -Wall -std=c++98 -I/usr/local/include -lboost_iostreams -L/usr/local/lib mystreams.cpp && ./a.out