egghelp.org community Forum Index
[ egghelp.org home | forum home ]		 egghelp.org community
Discussion of eggdrop bots, shell accounts and tcl scripts.
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Scraping pages while logged in

 
Post new topic   Reply to topic    egghelp.org community Forum Index -> Scripting Help
View previous topic :: View next topic  
Author Message
pen
Voice


Joined: 23 Apr 2009
Posts: 4
PostPosted: Fri May 01, 2009 2:58 pm    Post subject: Scraping pages while logged in Reply with quote
Hey, I'm trying to figure out how to scrape some text and output it to my irc channel. But the problem is that the text I want is on a page that's only accessible while I'm logged in. Is there a way to do this with tcl?
Back to top
View user's profile Send private message
nml375
Revered One


Joined: 04 Aug 2006
Posts: 2857
PostPosted: Fri May 01, 2009 3:34 pm    Post subject: Reply with quote
Most likely, yes.

I'm assuming you're talking 'bout scraping text off a webpage. "All" you'll have to do, is replicate the login procedure, and you should be all done. Since logons can be done in a multitude of different fashions, that's as good as the answer gets.
_________________
NML_375, idling at #eggdrop@IrcNET
Back to top
View user's profile Send private message
pen
Voice


Joined: 23 Apr 2009
Posts: 4
PostPosted: Fri May 01, 2009 6:31 pm    Post subject: Reply with quote
Yes, it's just a regular web page, but unfortunately, you have to be logged in to see it. I can log in and read it just fine, but I wasn't sure if tcl could be used to do the same.
Back to top
View user's profile Send private message
tomekk
Master


Joined: 28 Nov 2008
Posts: 255
Location: Oswiecim / Poland
PostPosted: Sat May 02, 2009 2:37 pm    Post subject: Reply with quote
remember about cookies, most of login "systems" uses session IDs on the server side and session cookies on the client side (web browser)

some hints,
http://wiki.tcl.tk/3828
http://wiki.tcl.tk/17611
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    egghelp.org community Forum Index -> Scripting Help All times are GMT - 4 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Forum hosting provided by Reverse.net

Powered by phpBB © 2001, 2005 phpBB Group
subGreen style by ktauber