Without checking your website it is impossible to check the reasons . But still i tell you some following reasons that may impact on off page seo and traffic downfall.
1.Lot of 404 errors . Check it google crawl errors .
Your average content quality is lower . Suppose your website have 10 articles . 3 articles have lots of traffic ,less bounce rate , high page view time . But the other 7 have low page spend time , high bouce rate . In that case your overall website quality dropped and traffic also dropped . To sove this problem Remove the low performing content from website or modify it . And use 410 redirect so search engine can also remove it . If you don't know how to remove content properly you can read this .
One more reason that imacted to lots of website that is create lots of low quality backlinks . Always choose no backlink rather than bad one . Moz also recommend to remove bad backlinkg . Learn how to find bad backlinks
Low quality content and bad backlink are decrease your ranking. One thing is kept remember when add the backlinks the similarity of the your site and backlink site is match, means only put links in related sites.
Site working down is not anything to do through off page optimization. Off page optimization will come into picture if your website is not success listed on Google even if you kind your website url on Google.
However directly typing your website going on browser and if it’s not opening you capacity need to check, configuration, coding, index.html etc.