I don't know a thing about FrontPage, so if there's a nifty button to press to accomplish this, someone else will have to chime in.
You're asking in the HTML/JavaScript/CSS forum... but I'll just assume that's because you're lumping all web development together as "HTML". The best answer for your task, though, is to use a server-side include, rather than a script.
A server-side include looks like a comment. In fact, it's a specially-formatted comment that looks like this, for including an HTML file in every page:
<!--#include virtual="/path/to/your/file.html" -->
Note: your web server must be configured to support SSI, so check with your web hosting company. In almost all cases, part of that configuration requires that your files have an ".shtml" extension, rather than ".html".
Another Note: each page that needs to include the file, will need the include directive. However, when you change the code within the header or footer, every page that includes it will get the new version.