I know that SEO is continuous, but after a couple of years of SEO, let's say 5 years of link building. Do I need to continue doing SEO, or will the traffic that I got the past years will do SEO for me?

You say "I know that SEO is continuous" which kind of answers the question you later pose of "do I need to continue doing SEO" doesn't it?

Link building is not in itself search engine optimization. Link building as the sole tactic in an SEO strategy is like having some cheap icing for a cake. You got to bake cake man!

And about the other part of your question, the "traffic" you may have received in the past is barely related to the traffic you may get in the future, barely. Traffic patterns are important when establishing trust with the search engine but unless things evolve on site the search engine will lose interest. It wants new stuff from a trusted site, fresh brand new stuff man. Take out the bong and contemplate that for a while then get creating content. Out.

Yes, you are ought to continue SEO. It is a long term process.