Experimental IRC log happs-2008-03-03

Available formats: content-negotiated html turtle (see SIOC for the vocabulary)

Back to channel and daily index: content-negotiated html turtle

These logs are provided as an experiment in indexing discussions using IRCHub.py, Irc2RDF.hs, and SIOC.

00:46:09<Saizan_>mightybyte: are you on unix? have you installed HAppS-Server separately or you use searchpath to compile?
00:46:20<Saizan_>mightybyte: that shouldn't happen if -DUNIX is passed
01:06:30<mightybyte>Saizan_: Yes, on unix.
01:07:02<mightybyte>I discovered that when I removed the forkIO and waitForTermination, the problem went away.
01:08:22<Saizan_>yeah it's waitForTermination that waits for input on non-unix
01:09:16<mightybyte>I found that before I removed that, it was giving me the same message when I ran it in the foreground and hit CTRL-D
01:12:23<Saizan_>sure, C-D it's end of file, no?
01:13:22<mightybyte>Yeah
01:13:29<mightybyte>I thought that was strange
01:14:08<mightybyte>waitForTermination obviously catches C-c, but also C-d
01:15:11<Saizan_>how did you build HAppS-State? with cabal?
01:17:53<mightybyte>Yeah, I built the whole thing with cabal.
01:17:59<mightybyte>Didn't use sp.
01:18:20<mightybyte>Although, I was on a different box when I got that error and there I had used sp.
01:21:57<Saizan_>well the important bit is that the flag -DUNIX is passed when building HAppS.State.Control, if not it expects an 'e' on stdin instead of handling signals
01:23:30<Saizan_>you can see it in cpp-options in the .cabal
01:26:30<mightybyte>Ok
02:29:54<mightybyte>I have posted some links to a live demo of my tutorial.
02:30:07<mightybyte>If anyone is interested, it's at http://softwaresimply.blogspot.com/2008/03/happs-demo.html
04:59:47<dbpatterson>is it normal for it to take a very long time (and use a lot of memory) to build Facebook.hs?
05:00:22<dbpatterson>I didnt notice last time I built it, but now I'm building it on a little VPS, so it is more noticable
05:02:05<dbpatterson>and also be at nearly 0 cpu.
05:05:15<porrifolius>dbpatterson: building certainly took me a long while and a lot of memory last time I did it. I wasn't paying attention but I suspect Facebook.hs is the culprit, there's a ticket filed for it.
05:12:12<dbpatterson>porrifolius: okay, guess I'll just stare at htop a little more (never seen mem full and swap half before :P)
07:55:41<MarcWeber>mightybyte: Thanks. I'll have a look at it later
13:37:54<MarcWeber> mightybyte I've read that you don't have you're own server? If you want I can give you an account as well. Then you can put up your examples as darcs repository ? If you're interested give me a note.
15:06:38<mightybyte>What data structure would be appropriate to use in a Component that stores a large array (random access, add to end) of items (like blog comments)?
15:07:10<mightybyte>Data.Array seems the obvious choice, but I'm not familiar with all the different variants.
15:07:29<Saizan_>well Array copies on each update
15:08:25<Saizan_>but you also can't store one of the mutable arrays before freezing it first in an immutable one
15:08:55<Saizan_>even if there are operations like unsafeFreeze/unsafeThaw that could avoid copying
15:09:13<Saizan_>hpaste used Data.Sequence.Seq to solve this.
15:15:26<mightybyte>Doesn't the same copy happen for maps as well?
15:16:43<Saizan_>maps can share most of the structure
15:17:04<Saizan_>while arrays being contiguous chunks of memory can't
15:17:10<mightybyte>Oh, I didn't realize that
15:18:13<mightybyte>Is there any way to get a truly immutable contiguous chunk of memory in haskell?
15:18:14<Saizan_>well for boxed arrays it's still copying pointers
15:19:02<Saizan_>mightybyte: isn't that Data.Array?
15:19:19<mightybyte>Oops, I meant mutable.
15:20:48<mightybyte>Or is IOArray the answer...and one that can't be used in a Component?
15:22:46<Saizan_>well you can also directly mess with pointers using FFI, but storing a pointer in a Component won't help much no?
15:23:28<mightybyte>Of course. I was wondering if there was anything native that could be used in a Component.
15:23:41<mightybyte>...native to haskell that is.
15:25:28<Saizan_>i don't think so, mutation involves the IO monad usually
15:25:50<mightybyte>Ok
15:28:38<mightybyte>So it looks like Data.Sequence is similar to Data.Map with better endpoint insertion performance.
15:29:14<Saizan_>yes, if you use Int as keys in Data.Map
15:32:09<Saizan_>the structure is quite different.
15:32:18<mightybyte>Hmmmm, except sequence indexes by Int, which is of limited size.
15:32:33<mightybyte>Is Int represented with 32 or 64 bits?
15:34:53<mightybyte>...or is it system-dependant?
15:35:05<Saizan_>the latter
15:35:16<Saizan_>it's the size of a machine-word
15:35:55<mightybyte>Ok
15:43:02<mightybyte>Has hpaste been upgraded to use 0.9.2?
15:43:22<Saizan_>no
15:43:47<Saizan_>i don't know if it will, glguy is rewriting it without happs
15:44:01<mightybyte>Oh, interesting
15:47:38<mightybyte>Grrr, Seq doesn't have a Serialize instance.
15:48:09<Saizan_>that's to be expected
15:48:33<mightybyte>And the standard "instance (Serialize a) => Version (Seq.Seq a)" and "$(deriveSerialize ''Seq.Seq)" isn't working.
15:48:57<Saizan_>that's because the contructor aren't exported
15:49:28<Saizan_>*constructors
15:49:37<Saizan_>so you need to go via a list
15:50:10<mightybyte>What do you mean?
15:51:20<mightybyte>Do I just have to create the instance manually?
15:51:49<Saizan_>yes
15:51:58<mightybyte>Ok
16:35:00<dbpatterson>has anyone done stress testing on happs apps? I just ran httperf on it, and with a static page (ie, just some xhtml), it was fine, cpu is high, but mem is constant and low. however, hitting a page that is pulling stuff out of state the memory goes way up
16:35:18<dbpatterson>is there some optimizing I need to do?
16:35:30<dbpatterson>is that what IxSet is for?
16:40:06<mightybyte>I think IxSet is more for allowing RDBMS-like queries.
16:40:26<mightybyte>What data structure are you using to store your state?
16:43:51<Saizan_>dbpatterson: compiling with -O?
16:43:57<Saizan_>or O2
16:45:57<dbpatterson>Saizan_: I just did ghc --make
16:46:03<dbpatterson>what is the default flag?
16:46:24<dbpatterson>and mightybyte it is just a structure with three strings and an integer...
16:46:42<Saizan_>default is no optimization
16:47:00<Saizan_>so try with ghc --make -O at least
16:47:01<dbpatterson>Saizan_: reccommendation?
16:48:21<Saizan_>-O is reccomended because it doesn't increase compilation time by much and not many programs benefit from -O2
16:48:42<dbpatterson>and -O woudl likely make it handle memory usage better?
16:48:58<dbpatterson>like, from the short test, even after the test was finished, the memory usage kept increasing..
16:49:09<dbpatterson>pretty steadily. like a memory leak.
16:49:57<mightybyte>dbpatterson: Ok, I was just asking to make sure you weren't using Array like Saizan had been telling me required a complete copy.
16:50:16<Saizan_>it can solve some leaks yes, because of strictness analysis etc..
16:50:50<dbpatterson>mightybyte: hmm.. so in general, maps are preferred to arrays?
16:51:26<mightybyte>dbpatterson: Apparently so. :)
16:51:38<dbpatterson>very good to keep in mind...
16:53:49<dbpatterson>oo... wait, I take back my earlier statement, I now AM using an array ..
16:53:57<dbpatterson>at least that isolates the problem.
16:54:30<Saizan_>do some tests :)
16:54:54<dbpatterson>also, is there _any_ way to decrease the memory needed for linking?
16:55:14<dbpatterson>it goes into swap, and so of course takes forever...
16:55:35<Saizan_>yeah same problem here..
16:59:06<Saizan_>i think the only way is to remove Facebook.hs from HAppS-Server
16:59:41<Saizan_>but.. it may not solve it
17:01:12<dbpatterson>Saizan_: oh, I already did that :)
17:01:18<dbpatterson>and FlashMsgs / HelpReqs
17:01:26<dbpatterson>because I'm not doing anything with facebook...
17:33:16<dbpatterson>and I guess it is a well known thing that I shoudl use ByteStrings instead of Strings, huh?
17:35:41<Saizan_>if you plan on storing large ones yes, String uses a lot more memory
17:36:16<Saizan_>ByteStrings are 8-bit so pay attention to encodings
22:35:10<dbpatterson>do any of the examples deal with ByteStrings? I started switching things over to them (just a few really long strings) and I started getting mis-typing issues, mostly involving lookBS returning a bytestring from Data.ByteString.Lazy.Internal - which from what I looked up about it, is supposed to be the internal version, that you shouldnt be using...
22:35:45<dbpatterson>this meant that using the ByteString from Data.ByteString didnt work..
22:36:42<dbpatterson>maybe I should say, does anyone have examples that work, because there aren't any (at least in HAppS-HTTP)
22:39:38<dbpatterson>or is it Data.ByteString.Char8 I need to use?
22:45:14<dbpatterson>nevermind. Data.ByteString.Lazy.Char8 seems to work
22:46:30<Saizan_>there are two distinct ByteString types
22:46:38<Saizan_>one .Lazy and one not
22:46:51<Saizan_>both are defined in the respective .Internal module
22:47:28<Saizan_>but the type is re-exported by the safe module
22:48:49<dbpatterson>and HAppS uses the lazy type
22:48:55<dbpatterson>at least in the look* functions
22:49:14<dbpatterson>(I misread the L as a B in SimpleHttp.hs at first)
22:50:10<Saizan_>yes
22:55:52<dbpatterson>can you pattern match against bytestrings?
23:02:49<Saizan>no
23:03:05<Saizan>you can use uncons

Back to channel and daily index: content-negotiated html turtle