00:18:30 <donri> do you mean hsx-xhtml
18:31:33 <stepcut> donri: yeah, that is what I meant
18:31:59 <stepcut> donri: I have never used it myself -- I just know that it exists
18:44:25 <donri> ACTION discovers that darcs allows subcommands to be shortened if unambiguous
18:45:40 <stepcut> :)
18:45:45 <donri> but not aliases, heh
18:48:05 <stepcut> :)
18:49:23 <donri> also the second bug i found in jmacro was a feature :)
18:49:58 <stepcut> :)
18:50:01 <donri> apparently antiquotes can access local javascript variables somehow
18:50:09 <stepcut> :p
18:50:32 <donri> which confused me because i was doing things like var node = `(node)` which got errors for conflicting names
18:51:20 <stepcut> ah
18:53:05 <donri> >>> renderJs [jmacro| var x = 5; alert(`(show x)`); |]
18:53:05 <donri> var jmId_0;
18:53:05 <donri> jmId_0 = 5;
18:53:05 <donri> alert("ValExpr (JVar (StrI \"jmId_0\"))");
18:53:17 <stepcut> heh
18:53:23 <donri> not sure how to use that
18:53:39 <donri> i guess if you splice in something based on the variable?
18:54:50 <donri> JExpr is a Num instance so you can do things like x + 2 in haskell xD
18:58:35 <donri> i guess that's particularly useful due to the hygienic names, you can't just pass "x" as a string in haskell because it renders as jmId_0 and you don't know what at the time
19:00:03 <donri> [jmacro| var x = 5; var xs = `(replicate 5 x)`; |] --> jmId_1 = [jmId_0, jmId_0, jmId_0, jmId_0, jmId_0];
19:46:57 <stepcut> yay ?
19:58:29 <donri> :)
21:49:58 <Lemmih> stepcut: I like that penance. (:
21:58:24 <Lemmih> stepcut: Can I see your SQLite benchmark code?
22:37:55 <stepcut> Lemmih: sure
22:37:57 <stepcut> one moment
22:52:36 <stepcut> darcs get http://patch-tag.com/r/stepcut/acid-state-benchmarks
22:53:16 <stepcut> ultimately they should criterion or something for measuring the timing
22:54:04 <stepcut> and there should be other tests
22:55:50 <stepcut> I added you as a repo member in case you want to push a patch :p
22:58:19 <stepcut> I wonder how often people use a relational programming model that is not backed by a database
23:07:31 <stepcut> Imagine you write a program that works with a bunch of data and does some processing on it. You write it in Haskell creating your types and functions the way you normally would. Then after it is working, your boss says, "Thats great but we need to sure sure that we does lose the state if the server crashes in the middle."
23:07:50 <stepcut> Surely you would not think, "Well, obviously I must use a relational approach now"
23:08:47 <stepcut> but if persistence requirement was given up front.. then I think a lot of people would just reach for the relational solution?
23:27:08 <donri> maybe less so in this NOSQL era
23:27:29 <stepcut> yeah
23:27:38 <donri> also people sometimes avoid relational when they really should be using it
23:27:47 <stepcut> happstack-state was NoSQL before it was cool :p
23:27:47 <donri> or they use a relational DB like it was a key value store
23:28:13 <stepcut> a lot of people use relation DB like a key/value store because the database are too slow to be used properly :p
23:28:47 <donri> that's part of why i'm skeptical of ORMs or whatever corresponds to it in haskell: if you use SQL, don't throw away the power of raw SQL
23:29:12 <stepcut> facebook does that (last I heard). They only put key/value pairs in their mysql databases, then stick a memcached farm in front of that, and then do al their 'joins' in PHP
23:29:58 <donri> like, how often do people use stored procedures with SQL?
23:30:00 <stepcut> even though PHP is slooowww, they can through a huge number of PHP servers at the problem.. but scaling beyond 800 mysql servers was given them fits
23:30:35 <donri> (yes, enterprisey professionals use such things, but not really "mere web developers")
23:31:26 <donri> how many people who regularly use SQL directly or indirectly know about things like "index on"
23:32:15 <stepcut> i try not to be friends with people who use SQL
23:32:26 <donri> :)
23:32:40 <donri> i think SQL is cool if *used right*
23:32:49 <donri> which it rarely is
23:32:54 <stepcut> SQL is the right tool for jobs
23:33:00 <stepcut> but they are not jobs I am willing to do :p
23:33:06 <stepcut> oops
23:33:08 <stepcut> for *some* jobs
23:34:16 <donri> what we need is a relational data structure we can put in acid-state!
23:34:33 <donri> as noted before, ixset et al aren't truely relational
23:34:34 <stepcut> http://patch-tag.com/r/stepcut/relational/home
23:34:37 <donri> just indexed maps
23:34:49 <stepcut> I don't have time to work on that right now
23:34:56 <donri> is that *efficiently* relational or just a DSL for queries?
23:35:14 <stepcut> the next step is to refactor it so that you can eithehr use Dynamics (like it does now) or use a restricted sets of types (aka, SQL types)
23:35:34 <stepcut> it could eventually be effient
23:35:45 <stepcut> it does not have any indexes at the moment, so everything is a full linear scan
23:36:41 <stepcut> if you want indexes, then you could make the table be a [Map Key [Row]], instead of [Row]
23:36:49 <donri> relational databases can efficiently look up data by cross-examining between tables and fields and then get the ten first after sorting the whole tables by some arbitrary column ...
23:36:51 <stepcut> (aka, IxSet :p)
23:37:13 <stepcut> yes
23:37:33 <stepcut> that implementation at the moment uses a very lame implementation for MultiSets
23:37:42 <stepcut> but that is fixable
23:37:44 <donri> in the pastebin i'm writing for practice i have a "recent pastes" list that is an ixset sorted by the Key. i realized this probably is insanely inefficient with many pastes ...
23:37:46 <stepcut> it was the best I could do in a few hours
23:38:38 <donri> because it has to sort the whole ixset, i would guess? i don't think a relational DB would have to, used correctly.
23:39:13 <stepcut> donri: you have a ixdex on that Key ?
23:40:21 <donri> IxSet (Key,Paste)... ixFun $ \(k,_) -> [k]
23:40:39 <donri> take 10 . toDescList (Proxy :: Proxy Key)
23:40:58 <stepcut> which part do you think is inefficient?
23:41:13 <donri> well, if i have a gazillion pastes, doesn't it have to sort the whole set?
23:41:45 <donri> hm maybe a reldb would too?
23:41:56 <stepcut> well, it basically just calls, Map.toDescList on the Map that contains that key
23:42:17 <stepcut> so, the sorting was mostly already done when you created that index
23:42:40 <stepcut> A Map is basically just an ordered binary tree
23:42:46 <donri> obviously the better way would be to just take the current nextKey to get the latest key and count down from that
23:42:49 <stepcut> so if you walk it in the right order you get the keys sorted
23:43:12 <stepcut> count down? How would you do that ?
23:43:23 <stepcut> take 10 . toDescList kind of does that already
23:43:28 <donri> well i have a GetPaste Key method
23:43:44 <donri> OK
23:43:52 <stepcut> you would call GetPaste 10 times?
23:43:59 <stepcut> that would definintely be slower
23:44:04 <donri> yea, or make it a method itself
23:44:22 <donri> it's currently: recentPasteState = pastes %. take 10 . toDescList (Proxy :: Proxy Key)
23:44:41 <donri> where %. is flip liftM (asks . getL) i think
23:44:43 <stepcut> GetPaste would have to start at the root of the tree and walk to each node
23:44:56 <stepcut> walking almost the same distance and covering most of the same ground each time
23:45:07 <donri> OK
23:45:13 <stepcut> toDescList will just walk to the end of the tree and then start unwinding
23:45:25 <stepcut> take 10 $ will cut it after after it's done enough
23:45:31 <donri> laziness yay
23:45:35 <stepcut> so you get your pastes in a single traversal
23:45:39 <donri> cool
23:45:58 <donri> and this works for all indices?
23:46:05 <donri> because each index is a Map in ixset?
23:46:07 <stepcut> implementing an in-memory relational database is much easier than one that is on the disk.
23:46:15 <stepcut> yes
23:46:55 <donri> btw... are the indices recalculated every time you open acid-state?
23:47:01 <stepcut> yes
23:47:06 <donri> seems like they would need to if you add or change indices
23:47:13 <stepcut> yes
23:47:21 <donri> so that could potentially slow down start up
23:47:36 <stepcut> the SafeCopy instance for IxSet just converts the IxSet to/from a list and serializes that
23:47:39 <donri> if say you have full-text search where all text is tokenized and stemmed ...
23:48:10 <stepcut> if you have a full-text search with all the text tokenized and stemmed, then you probably don't want an IxSet
23:48:14 <stepcut> that is my guess
23:48:29 <donri> i guess you could store that in the data directly
23:49:17 <stepcut> added it to the wish-list, http://code.google.com/p/happstack/wiki/AcidStateWishlist
23:49:21 <donri> :)
23:49:50 <stepcut> I think full text search is likely to require its own special datatype, and the SafeCopy instance for that can do something sane
23:50:00 <donri> perhaps you could even calculate that in the methods... or would that be bad?
23:50:12 <donri> assuming the stemming etc is pure
23:51:15 <donri> if you have checkpoints they don't re-run every time right
23:51:27 <stepcut> one advantage of recalculating the indexes when reloading a saved IxSet is that it preserves sharing, so that each index Map points to the same copy of the value in memory. If we serialized each Map separately, then it would take way more space when you reloaded the state from disk.
23:51:44 <stepcut> what doesn't re-run every time?
23:53:14 <donri> addPost text = put $ Post { text = text, searchTerms = map stemWord $ tokenize text }
23:53:25 <donri> is it bad practice to write code like that?
23:54:19 <stepcut> don't think so, why might it be?
23:54:38 <donri> not sure either way, thus asking :)
23:55:01 <donri> i thought it might be bad to do "heavy computations" inside acid-state methods
23:55:08 <donri> that they should better be done elsewhere first
23:55:12 <stepcut> ah
23:55:40 <stepcut> it will need to evaluate that expression entirely before it can serialize the event to disk
23:57:06 <donri> not sure what that means
23:57:10 <stepcut> I am not really clear if the database is locked while that happens
23:57:16 <donri> it stores a compiled copy of the whole stemming library? :P
23:58:01 <stepcut> when you run an event like, Post, it writes that event to disk first
23:58:28 <stepcut> so Post is just a type with a constructor like, data Post = Post Text SearchTerms
23:58:46 <donri> you mean AddPost?
23:58:55 <donri> i didn't take searchTerms as an argument there which was sorta the point
23:58:56 <stepcut> yeah
23:59:07 <stepcut> oh
23:59:13 <stepcut> sorry, brain not on
23:59:15 <donri> :)