A programmable machine. The programmable part is the key. Thats what seperates a computer from most other basic machines.
The option to run programs is what makes computers different from regular calculators. Also the fact that they follow the stored program concept which says that the instructions to be executed and the data it is to be performed on are held in the same memory.
Ultimately they all perform very basic operations, just lots of them, allowing them to do complex things. Ultimately all a CPU knows is Jump to location in memory, check if something is equal, and add two things.