How should I web scrape?

MillField

Daemon Poster
Messages
613
Location
Ireland
I'm working on a project right now and I can't figure out where I should be web scraping from. It is an Android application which will involve web scraping and and a database on a server. What I don't know is where I should be doing the scraping. Should I do it on the server through PHP and push the data to the app, or should I do the scraping right from the app?

Thanks for any help.
 
I would do it on server then push data to the app. The reasoning for this is because using a mobile device to download an entire page and parse the HTML would be a difficult task for a wireless device. I think you would cut down page load time if you simply run that calc on the server then transmit only important data to the app.
 
I would do it on server then push data to the app. The reasoning for this is because using a mobile device to download an entire page and parse the HTML would be a difficult task for a wireless device. I think you would cut down page load time if you simply run that calc on the server then transmit only important data to the app.

If you are concerned with the extra stuff being downloaded then I think a push would be a bad choice. And when it's mobile, i would advice against push. I would have the client connect to the server where the scraping happens, then pull the data down.

Reason for this is because mobile plans have limited download.
 
Thanks for the replays guys. What I am doing is scraping into a MySQL database and fetching the information from it using JSON to the android app.
 
Back
Top Bottom