| View previous topic :: View next topic |
| Author |
Message |
arcanedreams
Voice
Joined: 06 Sep 2007
Posts: 9
|
 Posted: Thu Sep 06, 2007 9:40 pm Post subject: sooo close. Need help with geturl and data functions |
 |
|
Here is what I have so far. Dont worry about the username and password..I just made those up for testing purposes.
| Code: |
bbind pub - !joined joined
proc joined {nick host handle chan text} {
set query "http://forums.shooshtime.com/login.php?vb_login_username=zooshbot&vb_login_password=zoosh&s=&do=login"
set http [::http::geturl $query]
set html [::http::data $http]
regexp {<strong>Welcome, (.*?) </strong><br />} $html - uname
putserv "PRIVMSG $chan :$uname"
|
That link will log you into the site.
From there, I can't get it to find any info on the page. I'm guessing it has something to do with the way it handles the link...perhaps it is taking the info from the actual login.php page
How do I get around this? |
|
| Back to top |
|
 |
rosc2112
Revered One
Joined: 19 Feb 2006
Posts: 1454
Location: Northeast Pennsylvania
|
| Posted: Fri Sep 07, 2007 12:18 am Post subject: |
|
|
Best bet is to debug what you are getting in your data var, by looking at it with putcmdlog lines after getting the webpage, like:
| Code: |
set query "http://somewhere.com/$varwhatever"
set page [::http::geturl $query]
set html [::http::data $page]
set count 0
foreach line [split $html \n] {
incr count
putcmdlog "line$count '$line'"
}
|
It's also helpful to [catch] the geturl errors so you can see them, for example (from one of the movie grabber scripts I did):
| Code: |
set movieurl "http://www.filmspot.com"
catch {set page [::http::geturl $movieurl -timeout $::movietimeout]} error
if {[string match -nocase "*couldn't open socket*" $error]} {
puthelp "PRIVMSG $chan :Error: couldn't connect to filmspot.com..Try again later."
return
}
if { [::http::status $page] == "timeout" } {
puthelp "PRIVMSG $chan :Error: Connection to filmspot.com timed out..Try again later."
return
}
set html [::http::data $page]
::http::cleanup $page
# Now we have data to work with and test the contents, etc..
if {[regexp $regexm $html match moviedata]} {
# the $regexm var is set earlier in the script depending on
# which commandline was given to the script
|
Thats a pretty standard template for any url-grabber script. |
|
| Back to top |
|
|
arcanedreams
Voice
Joined: 06 Sep 2007
Posts: 9
|
| Posted: Fri Sep 07, 2007 2:15 am Post subject: |
|
|
Not sure how to view the putcmdlog, so I just used putlog instead.
only error that showed up was expected. It was an error stating that the $uname variable couldn't be found.
I had it set to spam the channel with all the variables in the entire script.
It will go something like this:
$query: http://forums.shooshtime.com/login.php?vb_login_username=zooshbot&vb_login_password=zoosh&s=&do=login
$http: ::http::5
Then it will stop there and never output the $html variable.
I have tried simply changing the url to just the standard http://forums.shooshitme, and the $html variable will show up like this:
$html: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
So it looks to me like when I use the url that I need to have the bot login to the site, it has trouble at the ::http::data part
If I had to guess I would say the http::data part isn't returning because after logging in, you are redirected to index.php, where the call in the url was to login.php
I also returned the value for ::http::size, and it comes back as 0
For the ::http::code it returns :HTTP/1.1 302 Found
Any idea why, or how to fix it?
Or perhaps an alternative way to get the bot to login to the site and grab info while its logged in? |
|
| Back to top |
|
|
rosc2112
Revered One
Joined: 19 Feb 2006
Posts: 1454
Location: Northeast Pennsylvania
|
| Posted: Sat Sep 08, 2007 12:58 pm Post subject: |
|
|
| There are newlines in the output, so you won't see them, which is why I suggested using foreach [split $html \n] to output each line. |
|
| Back to top |
|
|
arcanedreams
Voice
Joined: 06 Sep 2007
Posts: 9
|
| Posted: Sat Sep 08, 2007 1:59 pm Post subject: |
|
|
well I got it to work. As I suspected, when you log in, it redirects you. So I had to grab the meta data for the url to the re-direction.
Full working code:
| Code: |
package require http
#############################
#######Profile Grabber#######
#############################
bind pub - !profile profile
proc profile {nick host handle chan user} {
set url "http://forums.shooshtime.com/login.php?vb_login_username=zooshbot&vb_login_password=zoosh&s=&do=login&url=/member.php?username=$user"
#putserv "PRIVMSG $chan :$url"
set http [::http::geturl $url]
#putserv "PRIVMSG $chan :$http"
#################
##get meta info##
#################
upvar #0 $http state
array set meta [set ${http}(meta)]
if {![info exist meta(Location)]} {
return $http
}
set redirect $meta(Location)
unset meta
#putserv "PRIVMSG $chan :Redirecting to $redirect"
###################
##Set and cleanup##
###################
set newhttp [::http::geturl $redirect]
set size [::http::size $newhttp]
set code [::http::code $newhttp]
set data [::http::data $newhttp]
::http::cleanup $newhttp
##############
##Debug info##
##############
#putserv "PRIVMSG $chan :Size of html returned, $size"
#putserv "PRIVMSG $chan :Html code returned, $code"
#putserv "PRIVMSG $chan :Html data, $data"
#regexp {To remove (.*?)you must} $data - notloggedin
#regexp {has not registered and therefore (.*?) to view} $data - noprofile
#putserv "PRIVMSG $chan :$notloggedin"
#putserv "PRIVMSG $chan :$noprofile"
###############
##Search Data##
###############
regexp {Total Posts: <strong>(.*?)</strong>} $data - tposts
regexp {Join Date: <strong>(.*?)</strong>} $data - joined
regexp {Referrals: <strong>(.*?)</strong>} $data - referrals
regexp {<td>
<strong>Birthday</strong>:<br />
(.*?)
</td>} $data - bday
regexp {User Notes: <strong>(.*?)</strong>} $data - unotes
##########
##Output##
##########
putserv "PRIVMSG $chan :User Status for $user"
putserv "PRIVMSG $chan :Joined on $joined"
putserv "PRIVMSG $chan :$tposts Total Posts"
putserv "PRIVMSG $chan :$referrals Total Referrals"
putserv "PRIVMSG $chan :$unotes Total User Notes"
putserv "PRIVMSG $chan :Was Born On $bday"
}
|
|
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
