| View previous topic :: View next topic |
| Author |
Message |
KevKev
Halfop
Joined: 03 Oct 2003
Posts: 67
|
 Posted: Fri Sep 03, 2004 8:17 pm Post subject: Increase dequeuing speed of Server module |
 |
|
I've got a bot that is dumping several lines of text to users at a rather speedy clip. Due to the nature of the reuqests coming in (they tend to be in bursts), people are complaining of significant lag in response times.
I'd like to increase the speed at which the queues are flushed to the server. Flooding the server isn't a major issue. I'm aware that this is controled by the server module and isn;t somethign that is terribly straightforward to scale up but i'd like to make whatever changes i can to try to increase the dequeueing speed or increase the allowed size of bursts.
I've seen some other threads regarding similar issues but most resulted in completely skipping the eggdrop queues. With the rest of my project i'd much rather avoid re-inventing the wheel and simply step up the dequeue rate of eggdrop's server module.
Any suggestions? |
|
| Back to top |
|
 |
KrzychuG
Master
Joined: 16 Aug 2003
Posts: 306
Location: Torun, Poland
|
| Posted: Sat Sep 04, 2004 5:56 am Post subject: |
|
|
| Code: |
proc putnow { a } {
append a "\n"
putdccraw 0 [string length $a] $a
}
|
You can find a discution about that procedure on: http://forum.egghelp.org/search.php?mode=results ;)
_________________
Que? |
|
| Back to top |
|
|
KevKev
Halfop
Joined: 03 Oct 2003
Posts: 67
|
| Posted: Sat Sep 04, 2004 11:06 am Post subject: |
|
|
Sure i can use putdccraw to dump it straight out at whatever rate the users make requests. But that's a bit extreme.
I'd like ot use SOME queueing. At this poitn it's feasable for one of my users to request several hundred lines of response from the bot. Add to that there are many people using the bot at the same time and you're in for a very high number of lines per second.
While I said excess flood wasn't a major issue. It IS an issue at some level so i'd like to use the standard queuing mechanism whilst still increasing it's speed or it's burst rate. |
|
| Back to top |
|
|
KrzychuG
Master
Joined: 16 Aug 2003
Posts: 306
Location: Torun, Poland
|
| Posted: Sat Sep 04, 2004 12:08 pm Post subject: |
|
|
You can modify msgrate value in src/mod/server.mod/server.c (line 123 in egg1.6.16):
| Code: |
/* Number of seconds to wait between transmitting queued lines to the server
* lower this value at your own risk. ircd is known to start flood control
* at 512 bytes/2 seconds.
*/
#define msgrate 2
|
_________________
Que? |
|
| Back to top |
|
|
KevKev
Halfop
Joined: 03 Oct 2003
Posts: 67
|
| Posted: Sat Sep 04, 2004 12:40 pm Post subject: |
|
|
results in it dequeuing a little faster but messages are still in the queue for a significant period of time.
is that value required to be an int or can it be a decimal? (i set it to 1 so it shoudl be at double the rate if i'm reading it right) |
|
| Back to top |
|
|
KevKev
Halfop
Joined: 03 Oct 2003
Posts: 67
|
| Posted: Sat Sep 04, 2004 2:29 pm Post subject: |
|
|
OK here's what i'm thinking. I'd like to make the server queue burst along with the mode queue (with the same burst counter of course)
Near as i can tell from the code ( I DO NOT KNOW C so PLEASE correct me if i'm being stupid) this is the block that pops out the messages from the server queue.
| Code: |
if (mq.head) {
burst++;
if (deq_kick(DP_SERVER))
return;
if (fast_deq(DP_SERVER))
return;
write_to_server(mq.head->msg, mq.head->len);
mq.tot--;
last_time += calc_penalty(mq.head->msg);
if (raw_log)
putlog(LOG_SRVOUT, "*", "[s->] %s", mq.head->msg);
q = mq.head->next;
nfree(mq.head->msg);
nfree(mq.head);
mq.head = q;
if (!mq.head)
mq.last = NULL;
return;
}
|
i THINK this should work to make it burst with the mode queue
| Code: |
if (mq.head) {
while (modeq.head && (burst < 11) && ((last_time - now) < MAXPENALTY)) {
burst++;
if (deq_kick(DP_SERVER))
return;
if (fast_deq(DP_SERVER))
return;
write_to_server(mq.head->msg, mq.head->len);
mq.tot--;
last_time += calc_penalty(mq.head->msg);
if (raw_log)
putlog(LOG_SRVOUT, "*", "[s->] %s", mq.head->msg);
q = mq.head->next;
nfree(mq.head->msg);
nfree(mq.head);
mq.head = q;
if (!mq.head)
mq.last = NULL;
return;
}
}
|
have i done anything incredibly stupid in this change / does it make sense?
EDIT: Ok.. update.. the above doesn't work. it simply sent nothing to the server  any pointers to where i might have gone wong? |
|
| Back to top |
|
|
awyeah
Revered One
Joined: 26 Apr 2004
Posts: 1580
Location: Switzerland
|
| Posted: Mon Sep 06, 2004 1:17 am Post subject: |
|
|
Heck, I don't know C. But you were saying you wanted to use some queues and not directly pass it to the sever.
Well you can do a thing:
[1] putquick is the fastest in all the queues.
Use putquick with -next that is very fast, almost the same as putdccraw I have seen. It wouldnt flood off the bot from the client server too. Else use putquick without the -next option, if you don't want an extremely fast output and want it to be queued.
I remember BarkerJr released a code in TCL espescially for handling queues. It is a tcl script code to be loaded within your bot's config file. So you can define, make and use your own queue.
I have subscribed to the eggheads mailing archive list so I get all their threads. If you goto their website and check on the past or the recent archives of this month or the previous one you can find it there. I may have the file but I am not sure.
_________________
·awyeah·
==================================
Facebook: jawad@idsia.ch (Jay Dee)
PS: Guys, I don't accept script helps or requests personally anymore.
================================== |
|
| Back to top |
|
|
awyeah
Revered One
Joined: 26 Apr 2004
Posts: 1580
Location: Switzerland
|
| Posted: Mon Sep 06, 2004 3:34 am Post subject: |
|
|
Hehe, your in good luck. Here I found it... go and use it.
| Code: |
# Copyright (C) 2003 BarkerJr
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
namespace eval put {
# Name the queues here. The further left, the higher priority queues.
set queues [list akill kill echo ctcp2 ctcp msg warn]
# Script begins here (nothing to do below here).
foreach q $queues {
if {![info exists queue($q)]} { set queue($q) {} }
}
unset q
proc serv {q text} {
variable queue
variable errorInfo
if {![info exists queue($q)]} {
set errorInfo "No such queue named: $q"
return 1
}
if {[lsearch -exact $queue($q) $text] != -1} {
set errorInfo "Already exists in queue ($q): $text"
return 2
}
lappend queue($q) $text
return 0
}
foreach timer [utimers] {
if {[lindex $timer 1] == {put::bisecondly}} { killutimer [lindex $timer 2] }
}
if {[info exists timer]} { unset timer }
utimer 2 put::bisecondly
proc bisecondly {} {
variable queues
variable queue
foreach q $queues {
if {[llength $queue($q)]} {
putserv [lindex $queue($q) 0]
set queue($q) [lreplace $queue($q) 0 0]
utimer 2 put::bisecondly
return
}
}
utimer 2 put::bisecondly
}
bind dcc - qstat put::queuestat
proc queuestat {hand idx text} {
variable queues
variable queue
putdcc $idx {Queue Lines}
putdcc $idx {=================}
foreach q $queues {
putdcc $idx "[format %-10s $q] [format %5i [llength $queue($q)]]"
}
putdcc $idx {=================}
}
proc uninstall {} {
unbind dcc - qstat put::queuestat
foreach timer [utimers] {
if {[lindex $timer 1] == {put::bisecondly}} { killutimer [lindex $timer 2] }
}
namespace delete [namespace current]
}
}
|
_________________
·awyeah·
==================================
Facebook: jawad@idsia.ch (Jay Dee)
PS: Guys, I don't accept script helps or requests personally anymore.
================================== |
|
| Back to top |
|
|
KevKev
Halfop
Joined: 03 Oct 2003
Posts: 67
|
| Posted: Wed Sep 08, 2004 12:51 pm Post subject: |
|
|
| Using putquick seems to be the solution. This seems totally counterintuitive to me. To me it would seem that all queues should dequeue at the same rate but in a specific order of priority. |
|
| Back to top |
|
|
nacho
Voice
Joined: 19 Jun 2006
Posts: 2
|
| Posted: Mon Jun 19, 2006 6:50 pm Post subject: |
|
|
how exactly do you use that put script posted above? Loading it into the eggdrop.conf didn't seem to have any effect.
Thanks!
NACHO
*EDIT* Also, I've tried using the `putquick $msg -next` but I was unable to get any kind of noticeable improvement. I've tried this both on my own private ircd-hybrid as well as efnet/linknet.
Thanks again. |
|
| Back to top |
|
|
De Kus
Revered One
Joined: 15 Dec 2002
Posts: 1361
Location: Germany
|
| Posted: Tue Jun 20, 2006 3:53 am Post subject: |
|
|
I use '#define msgrate 1' though I am bound to excess flood. Since if 512bytes/2sec is true, you won't have problems unless you make lines longer than 256bytes (and puthelp will still be slower than 1line/sec)  . I believe I read that '#define msgrate 0' also bursts, but isnt really crazy (does it send only 1 line per main loop cyle?!) and should keep them in order (first putquick, putkick and mode, then server, then help).
_________________
De Kus
StarZ|De_Kus, De_Kus or DeKus on IRC
Copyright © 2005-2009 by De Kus - published under The MIT License
Love hurts, love strengthens... |
|
| Back to top |
|
|
nacho
Voice
Joined: 19 Jun 2006
Posts: 2
|
| Posted: Tue Jun 20, 2006 3:27 pm Post subject: |
|
|
| well im looking for more of a bursty way of doing things here. I want the bot to be able to spam 15 lines quickly, but then maybe nothing really happens for 5 or 30 minutes. Also, it IS important that the bot doesn't get disconnected for flooding.. |
|
| Back to top |
|
|
De Kus
Revered One
Joined: 15 Dec 2002
Posts: 1361
Location: Germany
|
| Posted: Tue Jun 20, 2006 6:34 pm Post subject: |
|
|
| nacho wrote: | | Also, it IS important that the bot doesn't get disconnected for flooding.. |
In that case, setting msgrate to 1 is the fastest I can advise for you... which will just dump 1 line every 1sec... which in normal cases will give the impression of "fluently" output (I doubt a normal person reads faster than 1 line per second )
_________________
De Kus
StarZ|De_Kus, De_Kus or DeKus on IRC
Copyright © 2005-2009 by De Kus - published under The MIT License
Love hurts, love strengthens... |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
