Member Avatar for FaridMasood

I am a programmer with not more than 2 months experience and facing a very difficult chellange.
I want to read an online page in my aspx page to get the data on that page.
is it possible to read a page not developed by me.

Any one can help me,

  • 2 Contributors
  • 2 Replies
  • 137 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by FaridMasood

Dani AI

Generated

Good start: already has the HTML in a string and pointed out the right two paths (browser control for desktop apps, HTTP request for server code). The next step is to parse the HTML into structured rows and push those rows into a DataSet/DataTable instead of trying to parse raw text with fragile string operations.

Workflow to follow

  • Fetch the raw HTML (use modern HttpClient and set a User-Agent and appropriate cookies/headers).
  • Inspect the page with browser DevTools to find stable DOM markers (headers like "Win-draw-Win" or "Draw No Bet", containing tables or lists of selections).
  • Use an HTML parser (HtmlAgilityPack or AngleSharp) and XPath/CSS selectors to find the market header node, then select the following table/list nodes and extract selection names and prices.
  • Build a DataTable with columns such as MarketName, Selection, Price and add rows as you parse each selection; then add that DataTable to a DataSet.

Example (VB.NET + HtmlAgilityPack)

  • Install HtmlAgilityPack from NuGet and use HttpClient to get HTML.
  • The sample below shows a simple pattern: find header nodes, find the sibling table, iterate rows and fill a DataTable.
Imports System.Net.Http
Imports HtmlAgilityPack
Imports System.Data

Async Function ScrapeMarketsAsync(url As String) As Task(Of DataSet)
    Dim http As New HttpClient()
    http.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0")
    Dim html As String = Await http.GetStringAsync(url)

    Dim doc As New HtmlDocument()
    doc.LoadHtml(html)

    Dim ds As New DataSet()
    Dim dt As New DataTable("Markets")
    dt.Columns.Add("Market", GetType(String))
    dt.Columns.Add("Selection", GetType(String))
    dt.Columns.Add("Price", GetType(Decimal))
    ds.Tables.Add(dt)

    For Each header In doc.DocumentNode.SelectNodes("//h3")
        Dim market = header.InnerText.Trim()
        Dim table = header.SelectSingleNode("following-sibling::table[1]")
        If table IsNot Nothing Then
            For Each tr In table.SelectNodes(".//tr")
                Dim tds = tr.SelectNodes(".//td")
                If tds IsNot Nothing AndAlso tds.Count >= 2 Then
                    Dim sel = tds(0).InnerText.Trim()
                    Dim priceText = tds(1).InnerText.Trim().Replace("£","").Replace(" ","")
                    Dim price As Decimal = 0
                    Decimal.TryParse(priceText, price)
                    dt.Rows.Add(market, sel, price)
                End If
            Next
        End If
    Next

    Return ds
End Function

Notes and cautions

  • If the page renders odds via JavaScript/AJAX, scrape the XHR JSON endpoints or use a headless browser (Selenium) instead of HTML parsing.
  • Respect robots.txt and the site’s terms of service; throttle requests and cache results.
  • Make selectors tolerant (use contains() or class matches) because HTML structure can change and break parsers.

Recommended Answers

All 2 Replies

Member Avatar for ericstenson

There are a few methods by which this can be down. You can control a web broswer control if you are using a windows based application. If it's a web app, it's a little more difficult, but you can create a http request and read the stream back.

The examples of how to do this are quite long. Use google a little to get it going, then when you get stuck, I'll help you out.

Member Avatar for FaridMasood

Hi, Ericstenson
Thank you for you reply,
I got the logic and performed http request and read the data in stream, and showed that on a label.
NOw is the real challenge i want to find some data in that stream and store that in a data set .

Here is the code I used to read the page.

Dim strUrl As String = ""
Dim strPostData As String = ""
Dim strResult As String = ""
Dim wbrq As HttpWebRequest
Dim wbrs As HttpWebResponse
Dim sw As StreamWriter
Dim sr As StreamReader

'Create web request
wbrq = WebRequest.Create(strUrl)
wbrq.Method = "GET"

'Read returned data
wbrs = wbrq.GetResponse
sr = New StreamReader(wbrs.GetResponseStream)
strResult = sr.ReadToEnd.Trim
sr.Close()
lblResult.Text = strResult

I read the page
"http://www.paddypower.com/bet?action=show_event_by_markets&chosen_ev_id=1118182&category=SOCCER&ev_class_id=&disp_cat_id="

you can visit this page i will help you to understand the problem.
now i want to get data from this stream in a dataset such i want to get the "Win-draw-Win" "Draw No Bet' and similar all bet data in dataset how to do so.
I hope you will help me.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.