Member Avatar for Ccholic

I want to create a price-comparison website/web application and join affliate programs.My problem is, when I want to join for a company's affliate program,seems like every company asks for my website.

How can I create a price-comparison website if I am not an affliate of any company?Can I create a price-comparison site without being an affliate of any company, is it legal?In this way I will not have access to their developer api, so I can get prices using web-scraping techinuqes.I actually did it with a shopping website but although my small web app (running on local host actually) waited for 20 seconds between each http requests, after about 40 http requests that website started ignoring the connection and started returning "robots" file as response body.

I didn't try it with other online shopping websites , I am actually not sure if it is legal to do that.I have read Amazon's policy about affliate program and from what I understand it says showing/getting information about an Amazon product has to be either via banner/link provided by Amazon or using Amazon's Advertising Api and that they don't allow robots.

If the only legal way of automatically grabbing price information from
another website is to use their api and to use their api I have to join their affliate program and to join their affliate program I have to have a ready website which I would put affliate links,banners, how can I create a price-comparison site,clearly I can't add any store's price information cause I don't have access to their api, so I can't create a price-comparison site without joining their affliate program and I can't join any of the affliate programs because they require me to have a working website...

I hope someone would clear my confusion,thanks.

  • 2 Contributors
  • 2 Replies
  • 543 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by Ccholic

Dani AI

Generated

Short answer: you can build a price‑comparison site before you’re accepted into every merchant affiliate program, but doing it reliably and without legal/contract risk usually means using merchants’ official channels (affiliate links, product feeds or APIs) rather than scraping their pages. Merchants’ associate rules also spell out how product data and reviews may be shown and where links must point. (affiliate-program.amazon.com)

Practical path forward: launch a minimal public site (a static landing + a few sample comparison pages, privacy policy and an affiliate disclosure will do) or use an accepted social channel, then apply to affiliate networks and merchants and list the site/channel where you will publish links. Some networks accept social-only publishers and provide product feeds and tools that comparison sites can consume. For platforms like Amazon you must also meet their signup/sales requirements (for example, three qualifying sales during the initial 180‑day period) before API access and continued account approval are evaluated. (awin.com)

Why scraping is risky: the behavior you saw (many requests, then blocking or a robots.txt response) is exactly how hosts defend themselves. The robots.txt/REP is a protocol publishers use to request crawler behavior (it’s not itself an authorization to access content), and the legal picture around scraping of public data is fact‑specific and unsettled — hiQ v. LinkedIn is a notable example of that complexity. Treat scraping as a last resort, never try to evade explicit blocks, and where possible ask merchants for permission or a feed. As suggested, contact the company; ’s blocked requests are a common early warning sign. (ietf.org)

If you do have permission or are scraping your own data, be polite and robust: cache results, use a descriptive User‑Agent, throttle with exponential backoff, obey any published crawl rules, and keep TTLs on price data so you don’t hammer sources. Example skeleton (for permitted targets only):

import requests, time
from bs4 import BeautifulSoup

session = requests.Session()
session.headers.update({'User-Agent': 'MyPriceCompareBot/1.0 (https://example.com)'})

def fetch(url):
    r = session.get(url, timeout=10)
    r.raise_for_status()
    time.sleep(1.5)   # be polite
    return r.text

html = fetch('https://example.com/product')
soup = BeautifulSoup(html, 'html.parser')
price = soup.select_one('.price').get_text(strip=True)

Quick checklist: publish a real site or social presence, add clear affiliate disclosure, apply to networks/merchants, request official feeds/APIs, and consult counsel if you’re unsure about terms or local law.

Recommended Answers

All 2 Replies

Member Avatar for rproffitt

As to your first question, you put that to the company that asks for your website. Again, if they are not answering your question it's going no where and you are better off without them.

As to Amazon, good to read they don't allow robots.

Member Avatar for Ccholic

Thanks for your answer.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.