| View previous topic :: View next topic |
| Author |
Message |
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
 Posted: Wed Mar 01, 2006 8:22 pm Post subject: modules and multi-threading |
 |
|
Anyone done that already?
Just curious cause since eggdrop itself is not thread-safe, output seems to be a bit cumbersome:
Either you wrap some mutex around the stuff you need or you do some queue which gets polled by some hook (secondly).
Anyone seen/used any other approach?
P.S.: Is there some thread-safe implementation of TCL avail? |
|
| Back to top |
|
 |
demond
Revered One
Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA
|
| Posted: Thu Mar 02, 2006 12:41 am Post subject: |
|
|
I fail to see a rationale in that
eggdrop is inherently asynchronous, event-driven I/O application and as such, it could hardly benefit from multithreading
maybe a multiserver eggdrop would be better implemented spawning threads (since you have to respond independently, in real time, to events on multiple servers); I thought about starting such project but it would require dedicating a significant amount of my free time (that MT stuff is tricky & hard to get right), too much for a hobby... maybe some day if I'm out of work hehe
of course, I'd applaud you for undertaking an endeavour like this  I'm less than impressed with current direction of eggdrop development (javascript/perl scripting etc. - provided they ever complete that, which is highly unlikely, given their pace)
_________________
connection, sharing, dcc problems? click <here>
before asking for scripting help, read <this>
use [code] tag when posting logs, code |
|
| Back to top |
|
|
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
| Posted: Thu Mar 02, 2006 11:38 am Post subject: |
|
|
It's not about eggdrop benefiting for that kind of multi-threading:
Lets say you want to perform some synchronous I/O in your module or run a lengthy operation.
I assume the bot would be blocked till the call to your module returned.
Easiest way to solve that is to use threads; another way would be to make your module only use async I/O aswell which is not allways possible/desired.
If you use threads then you cannot just call eggdrop's output routines, the module interface functions are just missing locks.
That's my whole issue. |
|
| Back to top |
|
|
demond
Revered One
Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA
|
| Posted: Fri Mar 03, 2006 1:30 am Post subject: |
|
|
the only such realistic scenario I can think of is accessing a database, but if it takes *that* long you were forced to handle it in separate thread, there is something wrong with your db design anyway
_________________
connection, sharing, dcc problems? click <here>
before asking for scripting help, read <this>
use [code] tag when posting logs, code |
|
| Back to top |
|
|
sKy
Op
Joined: 14 Apr 2005
Posts: 194
Location: Germany
|
| Posted: Tue Mar 21, 2006 7:29 am Post subject: |
|
|
Ture, eggdrop isn`t mulitthread safe by standard. But it`s possible.
I am using eggdrop 1.6.17 with tcl 8.4.11. The tcl need to be compiled with --enable-threads
When compiling eggdrop you will have to use the tcl version with enabled threads too. Dunno, but i used
./configure --enable-threads
Then install a threading package. I use this here. Works fine.
http://mkextensions.sourceforge.net/ mkThreads 1.2
_________________
socketapi | Code less, create more. |
|
| Back to top |
|
|
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
| Posted: Thu Mar 23, 2006 8:40 am Post subject: |
|
|
However there is an important restriction:
| Code: |
An important constraint of the Tcl threads implementation is that only the thread that created a Tcl interpreter can use that interpreter. In other words, multiple threads can not access the same Tcl interpreter. (However, a single thread can safely create and use multiple interpreters.)
|
So a threaded Tcl does not help at all when one wishes to access the Tcl-Iterpreter used by Eggdrop in a multi-threaded module (which is what i wanted to do).
I now push my Tcl commands to a thread-safe queue and do a secondly hook which pops the commands from the queue and executes them in the eggdrop main thread.
Regards,
Kappa |
|
| Back to top |
|
|
sKy
Op
Joined: 14 Apr 2005
Posts: 194
Location: Germany
|
| Posted: Thu Mar 23, 2006 8:55 am Post subject: |
|
|
Sorry, I don`t know about modules. :/
I handle this as:
Eggdrop is "globalest level". From there i create a main thread. From the main thread i create all other threads.
By using the tvar command i can:
- show info messages
- show errormessages
- ussing callbacks to the globalest level or other threads
You just write a tvar variable and check and process this on the globalest level. From there you can callback other threads.
This should be a workarround for the code quote you posted. Hope i did get you right and not claim on the wrong three.
_________________
socketapi | Code less, create more. |
|
| Back to top |
|
|
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
| Posted: Fri Mar 24, 2006 12:32 pm Post subject: |
|
|
Ehm no don't think that's a workaround for the quoted statement.
What you say may be perfectly right for Tcl scripts...just got nothing to do with modules or the interpreter itself...
That is unless i got something terribly wrong 
You refer to the usage of threads "inside" a tcl interpreter, but the quoted text says that access to a Tcl_Iterp struct is only safe when done from the thread that created it (using Tcl_CreateInterp()).
This means that you cannot use any Tcl_Eval/Set/Get/... functions on the eggdrop tcl interpreter from another thread.
If one would like to do so that would require guarding the interpreter struct with some synchronization primitive (e.g. a mutex). |
|
| Back to top |
|
|
De Kus
Revered One
Joined: 15 Dec 2002
Posts: 1361
Location: Germany
|
| Posted: Fri Mar 24, 2006 2:37 pm Post subject: |
|
|
if you want to run a modular task threaded, just run a new process by running either the TCL interpretator or directly the compiled executable. Communication will be done via sockets (redirecting stdin/out). This is the way I use my server status (port scanner) script which might be ready in 1s or take 2min. Of course thats not the solution for modules like stats or seens where the data is stored within a huge memory table. But piping data up to ~1mb should be fast enough.
_________________
De Kus
StarZ|De_Kus, De_Kus or DeKus on IRC
Copyright © 2005-2009 by De Kus - published under The MIT License
Love hurts, love strengthens... |
|
| Back to top |
|
|
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
| Posted: Fri Mar 24, 2006 4:05 pm Post subject: |
|
|
Yeah, that's also a nice idea to handle that topic.
Didn't even think about that
Ohh just by the way here's the code I currently use to dispatch Tcl commands (just in case anyone cares).
Might be incomplete and missing some #includes, hope no mem-leaks, just a quick copy&paste...
| Code: |
/* mymodule.c */
static void TclCommandQueueHook()
{
CheckTclCommandQueue();
}
char* mymodule_start(Function* global_funcs)
{
/* ... */
InitTclCommandQueue();
add_hook (HOOK_SECONDLY, (Function)TclCommandQueueHook);
/* ... */
}
|
| Code: |
/* cmdqueue.h */
#ifdef __cplusplus
extern "C"
{
#endif
int InitTclCommandQueue();
void DispatchTclCommand( const char* aCommand );
void CheckTclCommandQueue();
#ifdef __cplusplus
}
#endif
|
| Code: |
/* cmdqueue.c */
struct CmdQueueEntry
{
char* Command;
struct CmdQueueEntry* Next;
};
struct CmdQueueEntry* CmdQueueStart = NULL;
struct CmdQueueEntry* CmdQueueEnd = NULL;
/* mutex for the command queue */
pthread_mutex_t* TclCommandQueueGuard = NULL;
int InitTclCommandQueue()
{
pthread_mutexattr_t aMutexAttribute;
pthread_mutexattr_init( &aMutexAttribute );
pthread_mutexattr_setpshared( &aMutexAttribute, 0 );
pthread_mutexattr_settype( &aMutexAttribute, PTHREAD_MUTEX_RECURSIVE );
// initialize mutex
TclCommandQueueGuard = nmalloc(sizeof( pthread_mutex_t ));
if (pthread_mutex_init( TclCommandQueueGuard, &aMutexAttribute ) != 0)
{
TclCommandQueueGuard = NULL;
return -1;
}
return 0;
}
void DispatchTclCommand( const char* aCommand )
{
if (TclCommandQueueGuard == NULL) return;
pthread_mutex_lock( TclCommandQueueGuard );
{
struct CmdQueueEntry* aNewCommand = nmalloc( sizeof( struct CmdQueueEntry ) );
aNewCommand->Command = nmalloc( strlen( aCommand ) + 1 );
strncpy( aNewCommand->Command, aCommand, strlen(aCommand) );
aNewCommand->Command[strlen( aCommand )] = '\0';
aNewCommand->Next = NULL;
// no end set yet?
if (CmdQueueEnd == NULL)
{
// start from the beginning
struct CmdQueueEntry* Next = CmdQueueStart;
if (Next != NULL)
{
while (Next->Next != NULL)
{
Next = Next->Next;
}
CmdQueueEnd = Next;
}
// no start yet?
else
{
CmdQueueStart = aNewCommand;
CmdQueueEnd = aNewCommand;
}
}
else
{
CmdQueueEnd->Next = aNewCommand;
CmdQueueEnd = aNewCommand;
}
}
pthread_mutex_unlock( TclCommandQueueGuard );
}
void CheckTclCommandQueue()
{
if (TclCommandQueueGuard == NULL) return;
pthread_mutex_lock( TclCommandQueueGuard );
{
while (CmdQueueStart != NULL)
{
if (CmdQueueStart->Command != NULL)
{
Tcl_EvalEx( interp, CmdQueueStart->Command, -1, TCL_EVAL_GLOBAL );
nfree( CmdQueueStart->Command );
}
struct CmdQueueEntry* Next = CmdQueueStart->Next;
nfree( CmdQueueStart );
CmdQueueStart = Next;
}
CmdQueueEnd = CmdQueueStart;
}
pthread_mutex_unlock( TclCommandQueueGuard );
}
|
|
|
| Back to top |
|
|
demond
Revered One
Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA
|
| Posted: Tue Mar 28, 2006 1:47 am Post subject: |
|
|
| Kappa007 wrote: |
Ohh just by the way here's the code I currently use to dispatch Tcl commands (just in case anyone cares).
|
I'd care if your efforts serve a purpose other than exercising your MT skills - that is, an eggdrop purpose (and you are able to demonstrate how eggdrop benefits from that)
_________________
connection, sharing, dcc problems? click <here>
before asking for scripting help, read <this>
use [code] tag when posting logs, code |
|
| Back to top |
|
|
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
| Posted: Fri Mar 31, 2006 6:08 am Post subject: |
|
|
Well benefits are: you can use (foreign) code which uses blocking calls in your modules w/o having to worry about your eggdrop beeing stalled.
And that's enough for me
And if that does not count for you then it's just about the possibility to use them at all  |
|
| Back to top |
|
|
De Kus
Revered One
Joined: 15 Dec 2002
Posts: 1361
Location: Germany
|
| Posted: Fri Mar 31, 2006 6:44 am Post subject: |
|
|
| Kappa007 wrote: | Well benefits are: you can use (foreign) code which uses blocking calls in your modules w/o having to worry about your eggdrop beeing stalled.
And that's enough for me |
Did you read my post? Its absoludly possible to run asyncron tasks. Btw. the reason eggdrop is so popular is, because its single threaded. That way shells can restrict eggdrop shells to 1 bg process. For further background tasks possible. Furthermore 1 bg process eggdrop = 1 IRC connection. For hosting on a general shell account with multiple back processes and connections allowed it might be usefull, but that wouldnt be all. Eggdrop is and will always keep single threaded and single server for these reasons. You can however still fork into a forground thread to execute async calls as mentioned above by me.
_________________
De Kus
StarZ|De_Kus, De_Kus or DeKus on IRC
Copyright © 2005-2009 by De Kus - published under The MIT License
Love hurts, love strengthens... |
|
| Back to top |
|
|
Kappa007
Voice
Joined: 26 Jul 2005
Posts: 38
|
| Posted: Fri Mar 31, 2006 2:32 pm Post subject: |
|
|
Ehhh yes I read it, i even replied in case you didn't notice...
| Kappa007 wrote: |
Yeah, that's also a nice idea to handle that topic.
Didn't even think about that
|
|
|
| Back to top |
|
|
demond
Revered One
Joined: 12 Jun 2004
Posts: 3073
Location: San Francisco, CA
|
| Posted: Sat Apr 01, 2006 3:31 am Post subject: |
|
|
| Kappa007 wrote: | Well benefits are: you can use (foreign) code which uses blocking calls in your modules w/o having to worry about your eggdrop beeing stalled.
And that's enough for me
And if that does not count for you then it's just about the possibility to use them at all |
well, you're still on the proof-of-concept stage
I'd applaud if you roll out, say, a non-blocking mysql module
and I'd admire you for producing multi-server MT eggdrop
_________________
connection, sharing, dcc problems? click <here>
before asking for scripting help, read <this>
use [code] tag when posting logs, code |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
