egghelp.org community Forum Index
[ egghelp.org home | forum home ]
egghelp.org community
Discussion of eggdrop bots, shell accounts and tcl scripts.
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

RSS news by demond
Goto page 1, 2, 3 ... 10, 11, 12  Next
 
Post new topic   Reply to topic    egghelp.org community Forum Index -> Script Support & Releases
View previous topic :: View next topic  
Author Message
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Fri Aug 12, 2005 11:31 pm    Post subject: RSS news by demond Reply with quote

since the other thread that people have been posting in about this wasn't actually relevant from the beginning, I'm opening this dedicated to the subject new thread

for those who don't know what the fuss is all about: I made a RSS news script for my own purposes (it still doesn't have an official name, I haven't released it officially yet) and I posted the beta version on several occasions here on these forums, after seeing that people tend to have problems with other RSS scripts; my version differs from all other similar scripts in the mechanism of parsing - it uses the object-oriented and XML facilities of TclLib and TclXML, also the RSS package from BDK, posted on Tcl'ers Wiki; and as it (unfortunately) turns out, most people are not used to installing third-party Tcl packages and therefore have been having problems with my script's requirements

I'm a big proponent of software component re-use, and I think we should encourage usage of good (and de facto standard) component packages like TclLib and TclXML

I'm also inclined to release the script officially, but that would depend on how many people actually manage to successfully install the required packages
Back to top
View user's profile Send private message Visit poster's website
greenbear
Owner


Joined: 24 Sep 2001
Posts: 733
Location: Norway

PostPosted: Sat Aug 13, 2005 11:05 am    Post subject: Reply with quote

Installing aditional packages will probably be a big pita for everyone running bots from their cheap $5 a month shell accounts...
Back to top
View user's profile Send private message Send e-mail
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Sat Aug 13, 2005 1:34 pm    Post subject: Reply with quote

well you certainly have a point here, although if you have at least 5mb quota you should be able to install TclLib and TclXML in your homedir:
Code:

[demond@whitepine demond]$ du -h -d0 lib/tcllib1.7/ lib/tclxml3.0/
3.1M    lib/tcllib1.7/
156K    lib/tclxml3.0/
Back to top
View user's profile Send private message Visit poster's website
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Wed Aug 17, 2005 12:50 am    Post subject: Reply with quote

I might rewrite this thing so that it would no longer depend on external packages, but it will continue to be rather minimalistic RSS solution, I'm not interested in adding bells & whistles
Back to top
View user's profile Send private message Visit poster's website
sKy
Op


Joined: 14 Apr 2005
Posts: 194
Location: Germany

PostPosted: Fri Aug 19, 2005 7:43 pm    Post subject: Reply with quote

I think that`s an good idea.

Well, RSS. I know what RSS means but i think its inconnu as well. Please don`t get me wrong, but if you want some feedback you then should try to explain in some easy sentence what RSS is and what the purpose of your script is.

Quote:
Installing aditional packages will probably be a big pita for everyone running bots from their cheap $5 a month shell accounts...

I think he is right :/.
Back to top
View user's profile Send private message
tonyrayo
Voice


Joined: 31 Jul 2003
Posts: 20
Location: Waldorf, MD

PostPosted: Sun Aug 21, 2005 4:42 pm    Post subject: Reply with quote

I believe demond has been brief because there has been no offical release... or maybe even because no one should try to use the script at this stage that doesn't know what an rss parser is and how it would work based on looking at source... or maybe it's just because demond refuses to wear domed shape hats!!! PORQUE!!!!
Back to top
View user's profile Send private message Visit poster's website
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Sun Aug 21, 2005 4:46 pm    Post subject: Reply with quote

an official release that doesn't require external packages is coming, stay tuned
Back to top
View user's profile Send private message Visit poster's website
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Mon Aug 22, 2005 10:57 pm    Post subject: Reply with quote

ok folks, here it is, feedback is much appreciated:
Code:

#######################################################################
#                                                                     #
# rssnews.tcl - RSS news announcer for eggdrop by demond@demond.net   #
#                                                                     #
#               this will announce the updated news from RSS feed(s), #
#               periodically polling the feed(s); supports multiple   #
#               channels, one feed per channel; you only need to set  #
#               the feeds array, see below                            #
#                                                                     #
#        Usage: !rss <news#|*> (read news# or * for headlines list)   #
#                                                                     #
#######################################################################

package require Tcl 8.3
package require eggdrop 1.6
package require http 2.0

namespace eval rssnews {

# set your feed(s) sources here: channel, poll frequency in mins, feed URL
#
#set feeds(#chan1) {17 http://www.osnews.com/files/recent.rdf}
#set feeds(#chan2) {11 http://news.google.com/news?ned=us&topic=h&output=rss}

variable version "rssnews-1.0"

variable timeout 20 ;# seconds

bind pub  - !rss [namespace current]::news
bind time - *    [namespace current]::timer

putlog "$version by demond loaded"

proc timer {min hour day month year} {
   variable feeds
   if {[info exists feeds]} {
   if {$min} {set min [string trimleft $min 0]}
   foreach {chan feed} [array get feeds] {
      if {$min && $min % [lindex $feed 0] == 0} {
         fetch [lindex $feed 1] $chan
      }
   }}
}

proc fetch {url chan} {
   variable timeout
   variable version; variable token
   set to [expr {$timeout * 1000}]
   set cmd [namespace current]::callback
   ::http::config -useragent "$version by demond"
   if {[catch {set t [::http::geturl $url -command $cmd -timeout $to]} err]} {
      putlog "$version: ERROR($chan): $err"
   } {
      set token($t) $chan
   }
}

proc callback {t} {
   variable version; variable token
   set chan $token($t)
   switch -exact [::http::status $t] {
   "timeout" {
      putlog "$version: ERROR($chan): timeout"
   }
   "error" {
      putlog "$version: ERROR($chan): [::http::error $t]"
   }
   "ok" {
      if {[::http::ncode $t] != 200} {
         putlog "$version: ERROR($chan): [::http::code $t]"
      } {
         process [::http::data $t] $chan
      }
   }
   default {
      putlog "$version: ERROR($chan): got EOF from socket"
   }}
   ::http::cleanup $t
}

proc process {data chan} {
   variable news; variable hash
   set idx 1; set news($chan) {}
   regsub -all {(?i)<items.*?>.*?</items>} $data {} data
   foreach {foo item} [regexp -all -inline {(?i)<item.*?>(.*?)</item>} $data] {
      regexp {(?i)<title>(.*?)</title>}             $item -> title
      regexp {(?i)<link>(.*?)</link}                $item -> link
      regexp {(?i)<description>(.*?)</description>} $item -> descr
      strip title descr
      if {[info exists hash($chan)]} {
      if {[lsearch -exact $hash($chan) [md5 $title]] == -1} {
         if {[botonchan $chan]} {
            puthelp "privmsg $chan :($idx) $title"
         }
      }}
      lappend news($chan) [list $title $link $descr]
      lappend hashes [md5 $title]
      incr idx
   }
   set hash($chan) $hashes
}

proc strip {args} {
   variable html
   foreach a $args {
      upvar $a x
      set amp {&amp; &}
      set x [string map $amp $x]
      set x [string map $html $x]
      regsub -all {<[^<]+?>} $x {} x
   }
}

proc news {nick uhost hand chan text} {
   variable news; variable feeds
   set num [lindex [split $text] 0]
   if {$num == ""} {
      puthelp "notice $nick :Usage: $::lastbind <news#|*>"
      return
   }
   if {$num != "*" && ![string is integer $num]} {
      puthelp "notice $nick :argument must be number or *"
      return
   }
   if {![info exists news($chan)]} {
      puthelp "notice $nick :no news for this channel"
      return
   }
   if {$num == "*"} {
      set idx 1
      puthelp "notice $nick :News source: [lindex $feeds($chan) 1]"
      foreach item $news($chan) {
         puthelp "notice $nick :($idx) [lindex $item 0]"
         incr idx
      }
      return 1
   } {
   if {$num < 1 || $num > [llength $news($chan)]} {
      puthelp "notice $nick :no such news index, try $::lastbind *"
   } {
      set idx [expr {$num-1}]
      puthelp "notice $nick :......title($num): [lindex [lindex $news($chan) $idx] 0]"
      puthelp "notice $nick :description($num): [lindex [lindex $news($chan) $idx] 2]"
      puthelp "notice $nick :.......link($num): [lindex [lindex $news($chan) $idx] 1]"
      return 1
   }}
}

variable html {
   &quot;     \x22  &apos;     \x27  &amp;      \x26  &lt;       \x3C
   &gt;       \x3E  &nbsp;     \x20  &iexcl;    \xA1  &curren;   \xA4
   &cent;     \xA2  &pound;    \xA3  &yen;      \xA5  &brvbar;   \xA6
   &sect;     \xA7  &uml;      \xA8  &copy;     \xA9  &ordf;     \xAA
   &laquo;    \xAB  &not;      \xAC  &shy;      \xAD  &reg;      \xAE
   &macr;     \xAF  &deg;      \xB0  &plusmn;   \xB1  &sup2;     \xB2
   &sup3;     \xB3  &acute;    \xB4  &micro;    \xB5  &para;     \xB6
   &middot;   \xB7  &cedil;    \xB8  &sup1;     \xB9  &ordm;     \xBA
   &raquo;    \xBB  &frac14;   \xBC  &frac12;   \xBD  &frac34;   \xBE
   &iquest;   \xBF  &times;    \xD7  &divide;   \xF7  &Agrave;   \xC0
   &Aacute;   \xC1  &Acirc;    \xC2  &Atilde;   \xC3  &Auml;     \xC4
   &Aring;    \xC5  &AElig;    \xC6  &Ccedil;   \xC7  &Egrave;   \xC8
   &Eacute;   \xC9  &Ecirc;    \xCA  &Euml;     \xCB  &Igrave;   \xCC
   &Iacute;   \xCD  &Icirc;    \xCE  &Iuml;     \xCF  &ETH;      \xD0
   &Ntilde;   \xD1  &Ograve;   \xD2  &Oacute;   \xD3  &Ocirc;    \xD4
   &Otilde;   \xD5  &Ouml;     \xD6  &Oslash;   \xD8  &Ugrave;   \xD9
   &Uacute;   \xDA  &Ucirc;    \xDB  &Uuml;     \xDC  &Yacute;   \xDD
   &THORN;    \xDE  &szlig;    \xDF  &agrave;   \xE0  &aacute;   \xE1
   &acirc;    \xE2  &atilde;   \xE3  &auml;     \xE4  &aring;    \xE5
   &aelig;    \xE6  &ccedil;   \xE7  &egrave;   \xE8  &eacute;   \xE9
   &ecirc;    \xEA  &euml;     \xEB  &igrave;   \xEC  &iacute;   \xED
   &icirc;    \xEE  &iuml;     \xEF  &eth;      \xF0  &ntilde;   \xF1
   &ograve;   \xF2  &oacute;   \xF3  &ocirc;    \xF4  &otilde;   \xF5
   &ouml;     \xF6  &oslash;   \xF8  &ugrave;   \xF9  &uacute;   \xFA
   &ucirc;    \xFB  &uuml;     \xFC  &yacute;   \xFD  &thorn;    \xFE
   &yuml;     \xFF  '      \x27  <p> \x20 <br> \x20 \n \x20
}

}
Back to top
View user's profile Send private message Visit poster's website
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Mon Aug 22, 2005 11:01 pm    Post subject: Reply with quote

' on the last line actually needs to be & #39; (without space between & and #)
Back to top
View user's profile Send private message Visit poster's website
r00tw00t
Voice


Joined: 23 Aug 2005
Posts: 2

PostPosted: Tue Aug 23, 2005 8:24 am    Post subject: It works great!!! Reply with quote

Wow, after two hours of trying to get your 'beta' version working, I found this. One single file does it all. kudos!

Now if I wanned the bot to say the headline + link in the channel, how do I do? Like: <eggdrop> [nameofnewssource]: [headline] - [link]

And everytime a new headline comes up it will say it in the #. (somekind of pseudo-live news feeder.)

Thanks!

EDIT: huh. The script already does that. Smile I was refreshing the RSS until a new headline was up, and checked if the bot had said that. The poll frequency is 3 min but it took 15-20 minutes for the bot to say the headline. Laggy?
Very Happy
Back to top
View user's profile Send private message
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Tue Aug 23, 2005 11:40 am    Post subject: Reply with quote

you set the poll frequency in the feeds array to 3mins but it took 15mins for an update headline(s) to show up? that means the RSS feed got updated 12mins (or more) after the script started polling
Back to top
View user's profile Send private message Visit poster's website
r00tw00t
Voice


Joined: 23 Aug 2005
Posts: 2

PostPosted: Wed Aug 24, 2005 5:01 pm    Post subject: rssnews cant read this rss? Reply with quote

http://www.fragbite.com/xml/headlines_rss.php i use this rss but the script doesnt seem to handle it?
Back to top
View user's profile Send private message
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Wed Aug 24, 2005 5:48 pm    Post subject: Reply with quote

you don't have <description> tags in that feed

I'll fix that in 1.1 by providing default values for <title>, <link> and <description> tags
Back to top
View user's profile Send private message Visit poster's website
demond
Revered One


Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA

PostPosted: Sat Aug 27, 2005 3:49 pm    Post subject: Reply with quote

okay 1.1 is out with that fixed and improved handling of HTML encoding

find rssnews.tcl here, I'll also submit it to Tcl script archive
Back to top
View user's profile Send private message Visit poster's website
sarius
Voice


Joined: 05 Aug 2005
Posts: 20

PostPosted: Sun Aug 28, 2005 2:43 pm    Post subject: Reply with quote

Sweet!

It works for most the the news websites so far. But is it possible to include CDATA support too?

It can't seem to read from this site - http://soccernet.espn.go.com/rss/news

Really appreciate the work done. Cheers! Very Happy
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    egghelp.org community Forum Index -> Script Support & Releases All times are GMT - 4 Hours
Goto page 1, 2, 3 ... 10, 11, 12  Next
Page 1 of 12

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Forum hosting provided by Reverse.net

Powered by phpBB © 2001, 2005 phpBB Group
subGreen style by ktauber