04:10:23 <stepcut> dons: exactly! 04:10:55 <stepcut> dons: most of the dependencies now should be in HP or fairly trivial, or very relevant (like BlazeHtml) 19:31:33 <Gracenotes> whoooaa, how did my happstack-state database bloat up to 900MB :| 19:33:43 <Gracenotes> I'm not storing that much data... I think... 19:34:34 <Gracenotes> yeah, the database is ~20MB before I start the mirroring processes. hmm. *looks* 19:35:54 <burp> my irc bot with 1.5MB database is blowing up, up to ~150M in memory after some usage :> 19:36:35 <Gracenotes> well, memory usage besides, this is actually the size of events and whatnot.. 19:36:44 <Gracenotes> on the disk 19:37:27 <Gracenotes> oooh... maybe I'm passing huge arguments to update functions 19:37:54 <Gracenotes> now why would I be doing a silly thing like that 19:42:42 <mightybyte> burp: I've seen behavior like that is well. 19:43:02 <mightybyte> burp: My database was like 10 megs on disk, and 700 megs in RAM 19:43:18 <burp> :/ that's why I switched to sqlite now for the irc bot 19:43:27 <mightybyte> But a few months ago that improved substantially. 19:43:38 <burp> hm ok, haven't tried newest 19:43:54 <mightybyte> Now it's only 350 megs. 19:44:05 <burp> oh nice ;) 19:44:14 <mightybyte> Yeah. Much more reasonable. 19:44:21 <mightybyte> Although still bigger than I'd like. 19:46:14 <stepcut> happstack 0.7 will really focus on figuring out memory usage stuff 19:46:48 <Gracenotes> I'm mainly concerned about the database being big on disk, as in-memory is something I'm used to 19:47:00 <Gracenotes> I wonder why a few operations would bring it from 20MB to 900Mb 19:47:24 <Gracenotes> as it's also caused memory to grow by about 500MB 19:47:43 <stepcut> your checkpoint file went from 20MB to 900MB ? 19:48:53 <Gracenotes> 25 Aug 13 02:53 current-0000000000 / 29M Aug 13 03:02 events-0000000000 / 94k Aug 13 11:00 events-0000000001 / 200M Aug 13 12:01 events-0000000002 / 672M Aug 13 20:56 events-0000000003 19:49:22 <Gracenotes> hm, 30MB in this case I think 19:51:24 <stepcut> looks like you definitely have some big events there. Would be interesting to see the size of a checkpoint file after those events 19:52:13 <Gracenotes> I didn't add much data, really, so I'm guessing I passed a big arg to an update function somewhere 19:52:17 <Gracenotes> many times 19:52:38 <stepcut> yeah 19:53:28 <Gracenotes> unless there actually is that much new data 19:55:58 <Gracenotes> erg 19:57:08 <Gracenotes> don't see anything.. 20:07:30 <Gracenotes> stepcut: is there any way to see which events have lots and lots of data attached to them? 20:08:03 <Gracenotes> maybe some kind of strings+grep.. except the serialized data also has plenty of strings. 20:14:04 <Gracenotes> hm. the serialized data overwhelmingly has huge lists of all packages with all package versions. 20:14:06 <stepcut> Gracenotes: not yet 20:14:11 <Gracenotes> but not from the main index.</detective> 20:14:25 <Gracenotes> maybe it's the revdeps index.. *checks* 20:14:35 <stepcut> Gracenotes: 0.7 will have better tools for examining the contents of the event and checkpoint files 20:21:13 <Gracenotes> woooooo Happstack 0.7 20:21:25 <Gracenotes> I mean wooooooooooo Happstack 0.6 20:24:06 <siracusa> This is a bad hack to quit a Happstack server in GHCi, any ideas why it just doesn't quit if cancelListen is set to true? http://hpaste.org/fastcgi/hpaste.fcgi/view?id=29018#a29018 20:24:50 <siracusa> It's in module Happstack.Server.HTTP.Listen 20:28:25 <Gracenotes> ooh I see, the problem is reverse dependencies. There are three options in this regard: pass the update functions the main package index; query the main package index outside the update functions and replace the entire revdeps index; keep a copy of the main package index inside and update it incrementally. it seems only the 3rd would prevent 600MB event files. ah well.