The difference between using refrences(&var), and pointers(*var) is that using refrences is more efficent because you do not have to derefernce the object.
While there's potential for such an optimization, I wouldn't rely on it. The benefit of references is largely syntactic convenience (for more than you probably expect).
Am I right, and if I am why use pointers then?
Pointers can point to nothing, references cannot. This may seem like a minor thing, but it's actually pretty big. Pointers can also be reassigned to point to another address while references are stuck. Changing the "thing" being aliased is pretty critical when using pointers for things such as linked data structures. And of course, dynamic memory specifically works with pointers, so at the very least you'd need to remove and add indirection to work with it as references (not recommended).