HTTP Requests with Python 2.7.1

Status
Not open for further replies.

The Python

Solid State Member
Messages
19
I'm trying to use python to rip data off of a website (a website with questions that I would like to have as a text document and I don't feel like copying and pasting 150 questions).
I would also like to know if I can access those questions in that text document using python (like if I print a number in front of it, would I be able to take that long (the question) and print it some where in the IDE).
If your going to ask, I did look at HOWTO Fetch Internet Resources Using urllib2 — Python v2.6.5c2 documentation but I'm getting weird returns saying that urllib2 doesn't exist.
 
What does the
if __name__ == "__main__":
do for the program because I took it out and it seems nothing changed.
Also I had to change the import from "urllib" to "urllib2", any idea why?
 
http://montipythoncode.webs.com/castle_login_table.txt How would I be able to tell python that I want it to look at this:
<input spellcheck='false' type="text" id="login" autocomplete="off" style="width: 150px" maxlength="50" />
<input type="password" id="password" autocomplete="off" style="width: 150px" maxlength="50" />
And fill it in with a username as well as password and log in, which is below:
<input type="submit" id="loginBtn" class="navbtn" value="SIGN IN" style="height: 24px" /><br /><br />
 
Also, thanks a lot already guys, as you can tell I'm a super newbie to python so it's been a little rough understanding.
 
Instead of going to the url of the page with the form on it, you need to go straight to the url specified as the action on the form. Since this is a login form, I'm going to assume that the form method is post:

Code:
#!/usr/bin/python

import urllib

if __name__ == "__main__":
    post_data = urllib.urlencode({'login': 'your_username', 'password': 'your_password'})
    url = urllib.urlopen("form_action_url.php", post_data)
    print url.read()
 
Status
Not open for further replies.
Back
Top Bottom