Website visit automation script

CFu

Baseband Member
Messages
53
The script needs to visit a series of game retailer websites and input some keywords into their search boxes. Then it yields the product results sorted by price in an HTML table in a separate page. The resulting HTML table is a list of the accumulated results.

The goal is to obtain a sorted overview of the total result set, sorted by price so that the cheapest version of the game (lowest price) is visible on the top of the list.

How would I begin coding this?
Which implementation languages are recommended (Javascript, AHK, Selenium, Java, C#, etc.)?
 

root

Site Team
Staff member
Messages
8,181
Location
UK
personally I'd use PHP.

what you want to do is run the script that should have logic as such

start script.
start data stream,
use open page function
send search variables to page. (e.g. site.com/search?keywords=search+terms)
(or your might have to send a post statement rather than using the get method -depends how their search page works)
set up a loop, search through the returned page line by line (for $line0, line<=sizeof($page), line++)
use regex to search regular expressions, (like <tr><td class="product">Product name)
use the split function to split out all the redundant stuff around the line
store your returned results in a database (with other information you require, such as data and time which can be generated locally)

destroy variables
close script

schedule a task

task = "C:\php\php.exe d:\scripts\price_scraping.php"
time = midnight each night.
 
Top