Member Avatar for tiddster

I have been asked to download the index page from the website, anyone know how to do this using a python script, or just how to do this at all?! Google doesn't seem to throw anything relevant up.

  • 3 Contributors
  • 2 Replies
  • 73 Views
  • 1 Week Discussion Span
  • Latest Post Latest Post by drjohn

Dani AI

Generated

Quick summary for : there are three practical ways to “download the index page” depending on what you actually need and what access you have.

If you only need the raw HTML the server returns for the site root, an HTTP GET is the simplest and most reliable approach. Example using the popular requests library:

import requests

url = "https://example.com/"
resp = requests.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=10)
resp.raise_for_status()
with open("index.html", "wb") as f:
    f.write(resp.content)

This captures the HTTP response bytes (preserves original encoding). See the Requests docs for more options and error handling: Requests documentation.

If you have server-level access (as @rajarajan2017 suggested), fetching the file over FTP downloads the actual file on disk rather than the HTTP-rendered output. Minimal ftplib example:

from ftplib import FTP

ftp = FTP("ftp.example.com")
ftp.login("username", "password")
with open("index.html", "wb") as f:
    ftp.retrbinary("RETR /path/to/index.html", f.write)
ftp.quit()

See Python’s ftplib docs: ftplib — Python docs.

If the page is built client-side (JavaScript-rendered), a simple GET won’t show the final DOM; use a headless browser (Playwright or Selenium) to render then save page.content() (Playwright docs: Playwright Python). Finally, respect site terms and robots.txt, and avoid aggressive scraping without permission.

Recommended Answers

All 2 Replies

Member Avatar for Member #334542

I dont know the script, but why you can't just download from the ftp?

Member Avatar for drjohn

Just visit the web site without adding a page after the domain name. What you see is the index page, the default page shown for the domain. you have been doing this for years when browsing the web.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.