12:28:25 <rostayob> stepcut, stepkut: http://mazzo.li:8000/
12:29:16 <rostayob> i fixed it to work with acid-state and ixset
12:31:25 <rostayob> oh great, a friend just posted goatse.
12:31:59 <Lemmih> haha
12:32:37 <Lemmih> rostayob: How do you store the passwords?
12:32:44 <rostayob> Lemmih: pwstore
12:32:56 <rostayob> but I still have no admin interface
12:32:59 <rostayob> or editing in general
12:33:23 <rostayob> apart from that, it works
12:34:33 <rostayob> and there's something wrong with the initial scoring....
12:35:37 <rostayob> oh good, you posted ehe
12:35:43 <rostayob> oh also, I need to add markup with pandoc
12:35:45 <Lemmih> (:
12:35:46 <rostayob> there's some stuff to do ehe
12:39:32 <Lemmih> I hope you don't pass plaintext passwords to AcidState events.
12:39:58 <rostayob> Lemmih: I don't
12:40:17 <rostayob> actually, wait....
12:40:30 <rostayob> no I don't.
12:40:34 <Lemmih> Phew.
12:40:46 <rostayob> aha
12:41:19 <Lemmih> The comment count seems off.
12:41:41 <Lemmih> Does it only count top-level comments?
12:42:06 <rostayob> Lemmih: it does, I have to find a nice way to do that
12:42:11 <rostayob> also, to count karma
12:42:24 <rostayob> those are classical map/reduce stuff
12:43:40 <Lemmih> Do you need to recompute those numbers?
12:44:41 <rostayob> Lemmih: well the thing is that nested comments are tricky to count, you have to traverse the whole thing
12:47:01 <rostayob> I could, easily, but it'd be slow...
12:47:45 <Lemmih> I mean, wouldn't it be easier to increment/decrement the numbers as needed?
12:49:20 <rostayob> Lemmih: yes actually now with macid it would be easy
12:49:39 <rostayob> no actually i'm an idiot
12:49:46 <rostayob> I can count them really easily
12:51:28 <rostayob> mh no... maybe it's just easier to keep track...
12:52:22 <rostayob> Lemmih: how would you solve the karma problem? adding up all the votes that a certain user's post have received. i was just thinking of updating it every tot hrs
12:54:05 <Lemmih> I'd make it real-time.
12:54:53 <rostayob> Lemmih: how? just adding the karma as the posts get voted? the problem is that posts and users are on different AcidStates
12:56:08 <rostayob> so I wouldn't be able to have an Update that includes voting the post and updating the user
12:56:24 <Lemmih> Well, then you can't guarantee correctness (in case of power outages) but it would be fine if you recomputed the info every N hours.
12:57:28 <rostayob> Lemmih: it's not only power outages, it's that if i have something like vote post -> update user, I may not have consistent values
12:57:45 <rostayob> but yes I could have the two solutions together
12:58:17 <Lemmih> Why would that lead to inconsistent values?
12:58:53 <rostayob> Lemmih: the vote post -> update user's karma action is not atomic
13:00:35 <Lemmih> You don't do: user <- query FindUser; update PutUser (user{ karma = karma user + 1})
13:00:38 <Lemmih> do you?
13:01:05 <rostayob> no, I don't for that reason :P. reacalculating the karma checking all the posts every time would be too expensive
13:02:28 <Lemmih> My point is that the order of transactions is somewhat irrelevant. As long as they complete, your state should be consistent.
13:03:28 <rostayob> my point is that if i want to do it in real time i have to exploit the voting I get, since recalculating it all each time it too expensive
13:05:45 <Lemmih> Right, would that be a problem?
13:06:30 <rostayob> if would, because "exploting the voting" roughly transates to user <- query FindUser; update PutUser (user{ karma = karma user + 1})
13:06:36 <rostayob> *it would
13:07:13 <Lemmih> Not necessarily. update IncUserKarma would do fine and be perfectly atomic.
13:08:20 <rostayob> that's if the karma is always just incrementing stuff, but if it's based on the post in some way (like the score) it doesn't work
13:08:39 <rostayob> but yeah that would work
13:09:10 <rostayob> but it'd be nice to have a karma that gives a picture of "hot" users, as scoring gives you hot posts
13:09:19 <rostayob> which is something that reddit & hacker news don't do
13:14:49 <rostayob> also in general, it's hard to keep track of stuff like that
13:15:05 <rostayob> like when you delete a post you have to handle the karma changes for the usre that posted that post and all the comments
13:20:28 <rostayob> also, you may have various kind of karma etc etc, I'd prefer not to have to keep track of all that stuff eheh
13:34:39 <stepkut> rostayob: nice!
13:41:57 <rostayob> stepkut: thanks (:
13:54:42 <rostayob> I do 300 req/sec. mhm.
13:55:17 <stepkut> spiffy
13:55:23 <rostayob> stepkut: not that much :P
13:55:26 <stepkut> now you just need to get 300 req/sec
13:55:33 <rostayob> ahah
13:55:52 <stepkut> it's about 300 req/sec more than you get :)
13:55:53 <rostayob> but the bottleneck is not in the database, it's in the page generation i think
13:56:00 <stepkut> ah
13:57:00 <rostayob> I'm sure there are vast margins of improvement with templating
13:57:29 <stepkut> yeah
13:57:57 <rostayob> but maybe it's just the huge monads stack i have mhm.
13:58:08 <rostayob> i wonder how ghc compiles monad transformers
13:58:12 <rostayob> that must be great fun
13:58:26 <HugoDaniel> :)
13:59:54 <HugoDaniel> rostayob: are you using blazehtml ?
14:00:14 <rostayob> HugoDaniel: no, hsp
14:04:31 <rostayob> does anybody know something about the performance of pandoc?
14:10:59 <stepkut> no idea. I don't use it because it is GPL :-/
14:13:34 <rostayob> stepkut: oh. that's not a problem for me. but they use it in gitit, so i hope it's good for real time stuff
14:14:18 <rostayob> i'd like to use this to be honest: https://github.com/tanoku/upskirt
14:14:22 <rostayob> but no bindings
14:15:26 <stepkut> neat
14:15:31 <stepkut> maybe i will switch to that
14:15:34 <rostayob> i'll probably write the bindings
14:15:40 <rostayob> i mean that's made to be secure and fast
14:15:40 <Lemmih> First goatse and now some upskirt url? Geez.
14:15:45 <rostayob> ehe
14:15:59 <stepkut> right now I just use runProcess and call the perl markdown script :p
14:16:10 <rostayob> stepkut: nooooo
14:16:12 <rostayob> (:
14:16:14 <stepkut> I do cache the results though so that I only have to do that once :)
14:16:29 <rostayob> that's slow as hell iirc
14:16:36 <stepkut> also, I use xss-sanitize to make sure that no one is trying to inject anything
14:16:40 <rostayob> how do you cache?
14:16:51 <stepkut> I just store the output in my database
14:16:59 <rostayob> ok, i wouldn't do that
14:17:06 <stepkut> not very space efficient. but more time efficient :)
14:17:15 <rostayob> no it's not about space
14:17:26 <stepkut> ?
14:17:37 <rostayob> but out of experience you often change the markdown
14:17:55 <stepkut> yes, I store the original and the rendered version
14:17:56 <rostayob> if you store the output in the database, there's no turning back
14:17:59 <rostayob> ah ok
14:18:01 <rostayob> that's better
14:18:05 <stepkut> hence the space usage :)
14:18:36 <stepkut> I'm sure well get sharding working before that becomes an issue though :p
14:18:54 <rostayob> yeah that makes more sense. but with a proper library, i'm sure the performance impact is trascurable
14:19:00 <rostayob> is trascurable an english word?
14:19:11 <rostayob> no.
14:19:31 <rostayob> well you get the point :P
14:19:33 <stepkut> yeah, a library would be nice.. I started to write one until I realized how wretched parsing markdown is
14:19:56 <rostayob> stepkut: the only safe parsing library seems to be upskirt
14:20:02 <rostayob> github version in particular
14:20:06 <stepkut> safe?
14:20:30 <rostayob> as in reacts well to malicious input
14:20:41 <stepkut> ah. I use this, http://hackage.haskell.org/package/xss-sanitize-0.2.6
14:20:56 <rostayob> it's not only that
14:20:57 <rostayob> Upskirt has been extensively security audited, and includes protection against all possible DOS attacks (stack overflows, out of memory situations, malformed Markdown syntax...) and against client attacks through malicious embedded HTML.
14:20:59 <rostayob> We've worked very hard to make Upskirt never crash or run out of memory under any input. Upskirt renders all the Markdown content in GitHub and so far hasn't crashed a single time.
14:21:03 <rostayob> i mean
14:21:15 <stepkut> nice
14:21:35 <stepkut> wish it was debianized already :-/
14:21:36 <rostayob> probably using haskell half of those problems go away temselves
14:21:45 <stepkut> :)
14:21:45 <rostayob> but not all of them
14:21:52 <stepkut> yeah
14:22:17 <rostayob> how the hell does github manage to survive using ror?
14:22:18 <stepkut> more importantly, running the markdown through a perl script first doesn't really give you any haskell benefits
14:22:52 <rostayob> no ehe
14:23:06 <rostayob> pandoc gives you Html
14:23:08 <rostayob> which is nice
14:23:30 <stepkut> yep
14:23:49 <stepkut> I keep waiting for them to fix that bug where they licensed it under GPL ;
14:23:50 <stepkut> ;)
14:24:04 <rostayob> i like gpl
14:24:16 <rostayob> i don't like rms but I like GPL
14:24:22 <Lemmih> What's the license of upskirt?
14:24:33 <Lemmih> Ah, see it.
14:25:00 <Lemmih> BSD/MIT like.
14:25:30 <stepkut> yeah
14:25:43 <rostayob> why wouln't you release the source anyway?
14:25:55 <stepkut> rostayob: does upskirt have an API? or does it just create a binary?
14:26:10 <rostayob> stepkut: it's alibrary
14:26:13 <stepkut> nice
14:26:17 <stepkut> I would binutils was a library :(
14:26:20 <rostayob> https://github.com/tanoku/upskirt/blob/master/examples/upskirt.c
14:26:43 <rostayob> it's got plugins as well! ehe
14:26:49 <stepkut> heh
14:27:26 <rostayob> i want a javascript DSL :(
14:27:43 <rostayob> jmacro mhm
14:27:49 <rostayob> i'm going to cook something, ciao
14:37:20 <stepkut> k
14:55:34 <stepcut> rostayob: you should add a link to the source at the bottom of the page maybe?
15:00:42 <Lemmih> Are there any decent templating languages that aren't language specific?
15:01:02 <mightybyte> HStringTemplate/
15:01:30 <mightybyte> Whether it's decent or not is another question.
15:09:08 <Lemmih> Hm, doesn't look bad, actually.
15:09:26 <Lemmih> Never really considered it before. Mostly because of the name, I think (:
15:09:33 <stepcut> heh
15:11:04 <Lemmih> mightybyte: How complete is the port?
15:11:22 <Lemmih> StringTemplate looks extremely complex.
15:18:15 <Lemmih> Hm, looks like it can't deal with nested data.
15:19:57 <Lemmih> Or can it? Damn, this documentation is hard to follow.
15:26:54 <mightybyte> Yeah, I've never been crazy about it.
15:27:28 <mightybyte> But I'm also mostly just interested in HTML templating
15:28:09 <Lemmih> Can HStringTemplate do nested loops?
15:28:50 <Lemmih> Googling it doesn't show any solutions that aren't language specific.
15:28:58 <Lemmih> (For StringTemplate, that is)
15:30:18 <mightybyte> No idea.  I've never actually used it...just read a few docs doing reasearch for Heist.
15:30:34 <Lemmih> )-:
15:32:11 <Lemmih> This is silly. I might as well just fix heist-aeson.
15:32:19 <stepcut> heh
15:35:25 <mightybyte> Is there anything you think Heist is lacking?
15:36:35 <Lemmih> It is language specific. That alone is a show-stopper.
15:38:07 <Lemmih> Hiring Haskell programmers to do templating is like hiring PhDs to do your grade-school homework for you.
15:39:25 <stepcut> :)
15:39:29 <Lemmih> Heist-aeson is exactly what I need except that it feels like it was programmed by a drunk monkey /-:
15:41:54 <Lemmih> The lexical scoping is slightly annoying in heist, though.
15:44:27 <mightybyte> For splices?
15:45:01 <Lemmih> Yeah.
15:45:51 <mightybyte> How would you suggest it be done?
15:46:27 <Lemmih> I do not know. /-:
15:46:36 <mightybyte> heh
15:46:54 <mightybyte> When you know, you know where to find me. :)
15:52:53 <stepcut> Lemmih: what are some improvements in acid-state? cleaner code, explicit handle to update/query, no more component class, less boilerplate for creating Serialization and migration instances..
15:54:17 <stepcut> safecopy only has two type classes (SafeCopy and Migrate) instead of three (Serialize, Version, and Migrate)
15:55:44 <stepcut> oh, separate event/checkpoint files for each 'component'
15:56:02 <Lemmih> stepcut: Better safety.
15:56:04 <stepcut> in acid-state, you have a single event file now ?
15:56:17 <stepcut> oh right, much more robust in the face of things like 'error' and 'fail'
15:56:23 <Lemmih> No, one event file for each acid-state.
15:56:32 <stepcut> right
15:56:39 <Lemmih> stepcut: And in the face of pretty much any other kind of failure.
15:56:49 <stepcut> but before there was (at worst) one event file per event
15:57:04 <Lemmih> Oh, I see what you mean.
15:57:13 <Lemmih> Yeah, far fewer files.
15:57:34 <Lemmih> It may be a problem that acid-state uses several files. Proper databases don't do that.
15:57:54 <Lemmih> But it shouldn't be a problem for solid state disks.
15:58:22 <stepcut> yeah
15:58:42 <Lemmih> Anyhow, it won't be a concern for quite a while and can be fixed without (m)any interface changes if it ever becomes one.
15:58:51 <stepcut> in happstack-state you can (perhaps not entirely safely) create a checkpoint and then purge old event files
15:59:02 <stepcut> can you still do that in acid-state ?
15:59:14 <Lemmih> Yes, I'm writing a tool for doing that.
15:59:34 <stepcut> k
15:59:44 <Lemmih> It will archive unnecessary files in a, say, tarball.
15:59:47 <stepcut> nice
16:00:04 <Lemmih> Whether to delete it or keep it around will be up to the user.
16:00:07 <stepcut> one issue with MACID is that it is very opaque when something goes wrong.. there is not simple way to 'browse' your state
16:00:35 <stepcut> do you have plans for any tools to examine / modify an acid store ?
16:00:45 <Lemmih> Well, kinda.
16:00:56 <mightybyte> Yeah, I also think tools are important
16:01:04 <stepcut> obviously you can write custom code in all your applications to do that.. but it is tedious
16:01:09 <Lemmih> It's hard to examine something which is 100% application specific.
16:01:14 <stepcut> yes
16:02:19 <Lemmih> But you could write 'state -> String' and 'event -> String' which a tool could then use to give decent information about the state.
16:02:32 <stepcut> I have pondered that you could write a library using generics that you would link against your application that would provide some simple CLI for quering the state
16:03:17 <Lemmih> I don't like generics. Would rather use ghci.
16:03:27 <stepcut> generics and/or TH
16:03:39 <Lemmih> I'm not sure that's necessary.
16:04:26 <stepcut> ACTION tries to remember why ghci is not great for this..
16:05:26 <stepcut> ghci would be more interesting with multimaster, if you could examine the state of a running server :-/
16:06:12 <Lemmih> That /will/ be in multimaster.
16:06:20 <stepcut> spiffy
16:06:24 <Lemmih> Connecting to a cluster without copying the state, that is.
16:06:28 <stepcut> right
16:06:43 <Lemmih> You'd be able to connect and send queries and even updates with ghci.
16:06:49 <stepcut> neat
16:07:10 <Lemmih> Inspecting the history of the state is a little bit more tricky.
16:08:49 <stepcut> inspection tools are pretty low on my personal wishlist
16:09:24 <stepcut> below multimaster, sharding, improved IxSet, and ways to graceful exceed the bounds of RAM
16:09:33 <Lemmih> (:
16:10:23 <stepcut> I am  hoping the new safecopy will fix some code structuring issues with state migration... I could never figure out how to make my code nicely organized with out getting circular dependencies
16:11:07 <stepcut> gotta run
16:14:19 <rostayob> stepcut: yeah i'll add a source link
16:14:29 <stepcut> rostayob: nice
16:14:43 <rostayob> stepcut: i'm using safecopy and i'm having the same problem in organizing the code with regards to circular dependencies
16:14:51 <stepcut> another form of 'safety' in the new acid-state is that it doesn't accidentally delete your components
16:14:57 <stepcut> rostayob: suck
16:15:11 <stepcut> rostayob: we need to indentify the root cause of that and see how to fix it
16:15:42 <rostayob> stepcut: wait a second what do you mean with circular dependencies problem? I just have problems in organizing the various data types and the functions in separate files
16:15:54 <rostayob> i end up putting the data type and the queries/updates in the same file
16:17:01 <stepcut> rostayob: gotta run, ttyl
16:17:20 <rostayob> later
16:17:26 <Lemmih> What's the problem with keeping things separated?
16:20:58 <rostayob> Lemmih: i'm trying to remember what the problem was, but yesterday night I had a State/ and a Types/ folder, and at some point the State/ folder was merged with the Types/ folder :P
16:22:13 <Lemmih> (:
16:22:43 <rostayob> i have to refactor that code...
16:23:08 <rostayob> and my exams start in 4 days
16:26:04 <rostayob> what was that stack based language that had a raptor as a logo?
16:26:08 <rostayob> i think it was stack based
16:29:40 <rostayob> factor.