Hell, there are shed-loads of distros that can be installed onto a pen-drive.
Depending on the capacity of your pen-drive, it could be anything from tiny distributions like DSL, Slitaz or Puppy, to full fat distros like Fedora or Ubuntu and everything in between.
Take a look at
www.pendrivelinux.com it has instructions for installing many flavours of *nix on pendrives. 4Gb is probably enough to install a live version of virtually any distro. And most of 'em ship with some basic development tools, if only gcc, possibly also g++. But you'll still need to install additional development libraries.
Personally I've been running one of my laptops from an 8GB stick with Ubuntu installed for some time now. (Dead HD, can't afford a new one ATM!)
Alongside the Ubuntu liveCD stuff that went on there, I've added lots of other programs (Code::Blocks, MonoDevelop etc.) and development libraries (Boost, wxWidgets etc). And I've still got a good 2Gb or so left over for my projects/documents.
And it performs quite well too. The boot time isn't quite as fast as a native install of Ubuntu because it's basically a liveCD that can be used on virtually any machine, so it probes the system for hardware etc. each time it boots which takes some time and it also has to mount the virtual file-system, which also takes a little time. But the overall speed of the OS is good. However, because it uses a virtual file-system, read/write operations take a little longer than normal, but this is barely noticeable.
As a development platform it's fine. In fact it's great for any purpose! I've been using it for C, C++, C#, Python, Flash development (yes, that's right flash dev on *nix!) and even some video editing. And video playback is smooth.....Nice!
Anyway, good luck!
Cheers for now,
Jas.