Hello, I want help in scrapy i want scrap this page full source code as Inspect Element mode https://www.contechlighting.com/en/products/recessed-lighting/led when i use this code import scrapy class TEST(scrapy.Spider): name = "quotes" start_urls = [ 'https://www.contechlighting.com/en/products/recessed-lighting/led#t=Products&sort=%40wcs_site_tree_rank ascending&layout=card', ] def parse(self, response): filename = 'contechlighting.html' with open(filename, 'wb') as f: f.write(response.body) get html source code and the site use dynamic content with javascript and i cant see the product items and results number on the top how can get full source code with scapy as appear on Inspect Element on firefox ? can any one edit on my code to do this …

Member Avatar
Member Avatar
+0 forum 1

The End.