egghelp.org community Forum Index
[ egghelp.org home | forum home ]
egghelp.org community
Discussion of eggdrop bots, shell accounts and tcl scripts.
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Parsing webpages made easy
Goto page Previous  1, 2, 3  Next
 
Post new topic   Reply to topic    egghelp.org community Forum Index -> Tcl FAQ
View previous topic :: View next topic  
Author Message
sKy
Op


Joined: 14 Apr 2005
Posts: 194
Location: Germany

PostPosted: Sat May 06, 2006 2:32 pm    Post subject: Reply with quote

Quote:
Quote:
Does somebody know how to load the core commands of eggdrop into a normal tcl script to then use them with egghttp.tcl (wich needs the eggdrop core functions)?

what are you talking about?


I guess he means. How he can use the eggdrop commands such as putlog (... connect, listen, ....) in a plain tcl environment (perhaps tclsh).

The answer:
As long i know it`s not possible. There is not "package provide Eggdrop" line. Eggdrop`s core commands are written in C and not tcl. However, eggdrop`s source is open source so it could be possible to extract them with some (unless) work.

The http package can be found here aswell too. http://www.tcl.tk/software/tcllib/

Imho we should sort out this thread anyay.
demond: could you update your first post here with a updated website?
_________________
socketapi | Code less, create more.
Back to top
View user's profile Send private message
strikelight
Owner


Joined: 07 Oct 2002
Posts: 708

PostPosted: Fri May 12, 2006 11:56 pm    Post subject: Reply with quote

demond wrote:
domme wrote:
How can I do this with egghttp?

don't use egghttp, it is severily outdated; it had its use long time ago, when Tcl still didn't have the built-in http package, which is superior in any way to egghttp


No, egghttp was written even when the http package for Tcl existed.
However the http package is severely bloated for the function of simply grabbing a web page, not to mention it does not provide async connection across all Tcl versions, and thus all eggdrop versions. Egghttp is still very much a practical utility today.
Back to top
View user's profile Send private message Visit poster's website
De Kus
Revered One


Joined: 15 Dec 2002
Posts: 1361
Location: Germany

PostPosted: Sat May 13, 2006 1:24 pm    Post subject: Reply with quote

refering to the header of http.tcl v2.5.1 from 2005/01/06, http supports callback (truely async) since version 2.1. I found 2.4.2 which was from 2004/04/05. You can probably guess how old 2.1 is...
The biggest diffrence between http and egghttp is, that egghttp uses connect and control and http uses of course socket and fileevent.
_________________
De Kus
StarZ|De_Kus, De_Kus or DeKus on IRC
Copyright 2005-2009 by De Kus - published under The MIT License
Love hurts, love strengthens...
Back to top
View user's profile Send private message MSN Messenger
strikelight
Owner


Joined: 07 Oct 2002
Posts: 708

PostPosted: Sat May 13, 2006 5:28 pm    Post subject: Reply with quote

I didn't say the http package, I said older versions of Tcl itself and thus different eggdrop versions. (There is no event loop to facilitate the required callback mechanism of the http package in these versions)
Back to top
View user's profile Send private message Visit poster's website
sKy
Op


Joined: 14 Apr 2005
Posts: 194
Location: Germany

PostPosted: Fri May 19, 2006 5:36 am    Post subject: Reply with quote

But you always could process the eventloop yourself. Call the update proc every second via utimer.
_________________
socketapi | Code less, create more.
Back to top
View user's profile Send private message
strikelight
Owner


Joined: 07 Oct 2002
Posts: 708

PostPosted: Fri May 19, 2006 11:25 pm    Post subject: Reply with quote

Yes, you could call a utimer event for 'update' every second for certain versions of eggdrop/tcl combination, however, you will be causing your bot to consume heavy cpu usage by doing so. However, also doing update every second will render your bot inoperable for extended amounts of time, negating the purpose of the async mode altogether.
Performance, needless to say, would be sluggish using such a method,
and not be very practical. Calling 'update' every second (or even every few seconds) is not a viable solution.
Back to top
View user's profile Send private message Visit poster's website
sKy
Op


Joined: 14 Apr 2005
Posts: 194
Location: Germany

PostPosted: Sat May 20, 2006 11:31 am    Post subject: Reply with quote

The performance. Well. To call a proc/command-update every seconds doesn`t cause a that high cpu usage. I tested it. Important is there should only run one utimer at once. If this timer is started on evey script load or reash again and again this will cause a high cpu usage for sure.

Update is simple. Do i have work to do? No? -> Return. This doesn`t take that long. And if he has work to do this will block the bot just that long the newer versions of eggdrop blocked aswell too.
_________________
socketapi | Code less, create more.
Back to top
View user's profile Send private message
strikelight
Owner


Joined: 07 Oct 2002
Posts: 708

PostPosted: Sat May 20, 2006 11:14 pm    Post subject: Reply with quote

Not talking about executing any utimer event every second, or every few seconds. We are talking specifically about calling a utimer event that makes a call to the 'update' command. Update will have to process any and all info in the event queue (not just what you think may be there), and it will not return until that has been completed, again as I stated earlier, negating the purpose of async mode altogether. And no, newer versions of eggdrop would not be blocked as their event queue is updated every iteration of the main program loop, so having a utimer event on those bots would be pointless as the event queue would most certainly be empty already. I have tried this on versions that required manual updating, many a time, and it did indeed result in severe performance degredation.
Hence the production of the egghttp.tcl script, and also a patch for certain versions as an alternative solution.
Back to top
View user's profile Send private message Visit poster's website
flashy
Voice


Joined: 01 May 2006
Posts: 24

PostPosted: Tue Jun 13, 2006 7:24 pm    Post subject: Reply with quote

man thats awesome, can someone make a little how-to guide to finding out the nessery things to phrase something off a webpage? finding out table size td row div and such using dom inspector?
Back to top
View user's profile Send private message
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Wed Aug 16, 2006 3:33 am    Post subject: Reply with quote

flashy wrote:
man thats awesome, can someone make a little how-to guide to finding out the nessery things to phrase something off a webpage? finding out table size td row div and such using dom inspector?


that would be your basic HTML primer
_________________
connection, sharing, dcc problems? click <here>
before asking for scripting help, read <this>
use [code] tag when posting logs, code
Back to top
View user's profile Send private message Visit poster's website
salkkus
Voice


Joined: 02 Oct 2006
Posts: 1

PostPosted: Mon Oct 02, 2006 10:20 am    Post subject: Reply with quote

Hello all Smile

I am interested in trying this tclDOM, but my shell doesnt support compiling..
So, is there anything I should know when I compile it on my own machine and then copy to the shell?

Like, should the shell have all those libxml, etc installed? And if yes, how can I find out which modules are installed?

Lots of questions and too few answers Smile ( I checked many websites regarding this, but couldn't find, sort of, basic info )
Back to top
View user's profile Send private message
rosc2112
Revered One


Joined: 19 Feb 2006
Posts: 1454
Location: Northeast Pennsylvania

PostPosted: Mon Oct 02, 2006 2:48 pm    Post subject: Reply with quote

Assuming your shell and your local machine are running the same OS on a similar platform (eg, running freebsd on a 80868 based machine), you can compile the module statically if necessary. You can find out what libs it depends on when compiled dynamically (the usual/default method) by running 'ldd modulename' on it. If your shell isn't running on the same platform, you'll have to cross-compile, which is beyond my own experience, so I'm not much help there.
Back to top
View user's profile Send private message
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Thu Nov 16, 2006 6:17 am    Post subject: Reply with quote

it's not tclDOM, it's tDOM; TclDOM is completely different package that indeed requires TclXML; tDOM does not require any XML package as it itself contains XML engine
_________________
connection, sharing, dcc problems? click <here>
before asking for scripting help, read <this>
use [code] tag when posting logs, code
Back to top
View user's profile Send private message Visit poster's website
johne
Voice


Joined: 19 Jul 2005
Posts: 29

PostPosted: Sun Apr 01, 2007 5:51 pm    Post subject: Reply with quote

some more working examples would be greatly appreciated Smile
Back to top
View user's profile Send private message
karodde
Voice


Joined: 01 May 2007
Posts: 10

PostPosted: Thu May 03, 2007 11:21 am    Post subject: Re: Parsing webpages made easy Reply with quote

The Script works for my shell @ home. Now I want my eggdrop to post this Info by typing !test.

I Made this, but it doesnt work Smile

Code:
#!/bin/sh
# This line continues for Tcl, but is a single line for 'sh' \
exec tclsh8.4 "$0" ${1+"$@"}
package require tdom
package require http
set url "http://www.url.de/"
set page [::http::data [::http::geturl $url]]
set doc [dom parse -html $page]
set root [$doc documentElement]
set node [$root selectNodes {//table[@cellspacing=0]/tr[1]/td[1]}]
set text [[[lindex $node 0] childNodes] nodeValue]

bind pub - !test tester
proc tester { nick uhost hand chan args } {
putserv "PRIVMSG $chan : $text"
}


I got following error msg:
Tcl error [tester]: can't read "text": no such variable

Probably the variable text is not committed,
can somebody help me please? thanks Smile


EDIT:
got it,

Code:
bind pub - !test thefunction
proc thefunction { nick uhost hand chan rest } {

package require tdom
package require http
set url "http://www.url.de/"
set page [::http::data [::http::geturl $url]]
set doc [dom parse -html $page]
set root [$doc documentElement]
set node [$root selectNodes {//table[@cellspacing=0]/tr[1]/td[1]}]
set text [[[lindex $node 0] childNodes] nodeValue]

putserv "PRIVMSG $chan : $text"
}
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    egghelp.org community Forum Index -> Tcl FAQ All times are GMT - 4 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Forum hosting provided by Reverse.net

Powered by phpBB © 2001, 2005 phpBB Group
subGreen style by ktauber