I am trying to use a '#' in the URL to navigate to certain parts of a page and don't even know where to start or what it is called.

I will give you an example:
Most FAQ pages have all the questions as links at the top of the page. When you click the link, it adds a '#' to the URL and navigates the page down to the exact Answer to the Question while staying on the same page.

I cannot figure out if it is done by simple html tricks or maybe javascript?

I want to implement it on my website, but I cannot figure out how it works.

Any ideas or links to tutorials would be extremely helpful. Thanks in advance :-)

You need to place anchors where you want the page to navigate to.

If you wanted to navigate to the top of your page, you would define an anchor at the top with a name.

<a name="top"></a>

The link to navigate to the top of the page where your anchor is defined would be:

<a href="#top">Back to top</a>

further to pwinfrey's reply, answer to the expected follow up question
Yes the link http://www.mysite.com/mypage.html#top works as expected it moves the page to the anchor defined by name='top' when the page is opened

commented: Read my question fully and gave extra input on the problem. Thank you! +1