[Topaz-dev] Mulgara Performance Woes

[Life is hard, and then you die]

On Sat, Mar 01, 2008 at 10:05:20PM +1000, Andrae Muys wrote:
> 
> On 01/03/2008, at 7:49 AM, Russell Uman wrote:
> 
> >> This is why having a large acceptCount is better. In fact, you may
> > want to cut down the number of threads to, say, 10, and just make >  
> > the
> > acceptCount very large (2000?). If a client drops the connection  
> > before
> > it's been accepted, then no Session is allocated and it > won't get  
> > the
> > write-lock and then wait for it to time-out.
> >
> > yes. we've been discussion ways to get long running pub-app  
> > sessions to
> > timeout WITHOUT opening and orphaning a mulgara session. some way for
> > the pub-app to check how long it's been running when it's woken up and
> > before it opens a session.
> >
> > if mulgara is single-threaded, why have 10 maxThreads? why not 1 or 2?
> > (i'm sure it's a bad idea, i just don't know why :)
> 
> Mulgara is definitely not single-threaded.  Every insert/delete/query/ 
> setAutocommit/etc call is handled by a separate thread; Jotm, which  
> is used internally to manage transactions, runs several; several are  
> spawned for every update operation to handle the IO; and of course  
> the JVM runs several threads for its own mysterious purposes.   
> Mulgara places only two restrictions on concurrent access.  First the  
> Session object is non-reentrant, so if you want to perform multiple  
> operations in parallel you have to use a separate session for each  
> operation.  Second only one session at a time can be accessing the  
> 'current' (ie. uncommitted) phase, and this is enforced via the write- 
> lock --- as many sessions as you like can access the 'committed'  
> phase, but that phase is strictly read-only.

Since (I think) I first used the term "single-threaded", let me
clarify. Yes, mulgara is not single threaded at all. What I said was
"effectively single-threaded", and that's due to the fact that we use
a transaction for everything (and production is not using the JTA
branch yet), so hence from an app perspective mulgara appears
single-threaded in the sense that only one transaction (and therefore
session) is processed at a time.

> To be perfectly honest 10 threads seems rather small, I would expect  
> 3-4 for the JVM; 3-4 for JOTM; ~10 for the block-buffer and object- 
> pool; and 1 thread per active concurrent Session.

No, we're talking about the thread-limit in tomcat, which is worker
threads, not the total number of threads in the JVM.

> Now I'm about to go black for a couple of days while I make my way to  
> your fair city - maybe we'll get a chance to talk about this stuff  
> when I get there.

Sure. But just remember that we've wrapped mulgara in a webapp and are
running that inside of tomcat, so some things are related purely to
our wrapping and to tomcat, not to mulgara per se (and the max-threads
and accept-count discussion is one of those). I can understand that
what is what is not necessarily obvious to either of you, so apologies
for the confusion.


  Cheers,

  Ronald