07:39:10 <Lemmih> Yay, it didn't break.
15:24:22 <stepcut> Lemmih: :)
15:29:08 <Lemmih> stepcut: Gracjan would like to migrate from happstack-state to acid-state.
15:29:21 <stepcut> ok
15:29:26 <stepcut> migrate what?
15:29:34 <Lemmih> His state, I guess.
15:29:42 <stepcut> ah
15:30:07 <Lemmih> Does happstack-state have a maintainer?
15:30:16 <stepcut> me
15:30:23 <stepcut> and anyone else that wants to help
15:30:34 <stepcut> Gracjan did a lot of work on IxSet
15:31:54 <Lemmih> A 'happstackStateToAcidState' function would be appreciated, I think.
15:32:03 <stepcut> yeah
15:34:26 <stepcut> there are two ways of doing that.. either import both happstack-state and acid-state to do the migration -- or write an acid-state function that can read the old state directly.. since you added a function to safeCopy that can read the old happstack-data format already, that shouldn't be too hard
15:35:43 <Lemmih> It can ready the happstack-data format, not the happstack-state format.
15:36:24 <Lemmih> You need the entire transaction apparatus to get the state out.
15:40:12 <stepcut> if there are unplayed events -- definitely
15:40:52 <stepcut> if you have a checkpoint -- then I think the checkpoint file is just a serialized  Map
15:41:02 <stepcut> though there are some tricky type issues to deal with
15:41:54 <Lemmih> Oh?
15:43:32 <Lemmih> Isn't it pretty hard to access the checkpoints without emitting a GetCheckpoint event?
15:43:36 <stepkut> the serialized state contains multiple components, each with a different type. In acid-state, those will each be different acid handles?
15:44:12 <stepkut> Lemmih: you can easily access a checkpoint by having the user specify the name of the checkpoint file
15:46:07 <Lemmih> For some value of easily, I guess. You'd have to understand the file format and then fetch the latest checkpoint.
15:46:07 <stepkut> I am leaning towards just creating a new page that imports happstack-state and acid-state and does a transition that way
15:46:37 <Lemmih> Yeah, I think that would be best.
15:48:03 <Lemmih> What do you mean by "a new page", exactly? A new page in the documentation?
15:48:06 <stepkut> Lemmih: for happstack-state the developer just has to look for the file named 'checkpoint-xxxx' with the highest xxxx. The actual checkpoint file format is simple as well. it is jsut a list of the component types (as Strings) and their serialized values.
15:48:13 <stepkut> oops. I meant new package.
15:49:22 <stepkut> or maybe all it really needs is a migration guide
15:49:54 <stepkut> hmm
15:50:37 <stepkut> I guess the tricky part is that a lot of existing apps do not already have methods written for each component that simple return the entire state from that component
15:51:47 <stepkut> but I could add a function to happstack-state that does that perhaps
15:53:18 <stepkut> I gotta run, but I will think about this later
15:53:33 <stepkut> well, I will try to implement so code, that usually clears the issues up
15:54:48 <Lemmih> (:
20:59:07 <rostayob> stepcut, stepkut: if I have a data type with multiple constructor, and I have an index on that, can I do a query to retrieve all the items with a certain constructor in IxSet?
20:59:34 <stepcut> nope
20:59:50 <stepcut> you would need to create a special index just for values with that constructor
21:00:00 <rostayob> In my case I have data SContent = Ask String | Link String String
21:01:00 <rostayob> stepcut: ok, thanks
21:02:09 <rostayob> i'm really in doubt between using persistent and state+ixset. ixset still has the sorting problem, because I realized that I can't always narrow things down enough. i'll probably just do it anyway without caring about performance...
21:03:25 <stepcut> at least IxSet makes it more obvious when you are going to do a linear search
21:04:30 <rostayob> stepcut: well it's basically after you "use" the first map and you have to rebuild, right?
21:04:42 <rostayob> i mean that's when you don't get sorting for free
21:05:35 <stepcut> not clear what you mean
21:06:14 <stepcut> if you query on one key, you can not then sort on a second key for free
21:06:39 <stepcut> but that is probably true of SQL as well.. they just hide it from you :)
21:07:38 <rostayob> stepcut: toDescList (Proxy Foo) set, toDescList (Proxy Foo) (set @= (Boh "boh"))
21:08:14 <stepcut> yeah, it will have to rebuild the index for Foo
21:08:33 <rostayob> in the first case i call it "for free" since
21:08:44 <rostayob> in the second you rebuild the index, so you basically sort it
21:08:48 <stepcut> yep
21:09:07 <stepcut> do you have an example of a system that can do a better job ?
21:09:19 <rostayob> lazyness would work as expected in those case right? or is there some strictness in Map/Set?
21:09:27 <stepcut> it is lazy
21:09:47 <rostayob> stepcut: with mongodb you can define compound index
21:10:02 <stepcut> rostayob: you can define a compound index with IxSet..
21:10:43 <rostayob> yeah but how do you do equality on a compound IxSet index?
21:11:01 <stepcut> same way as normal..
21:11:10 <rostayob> like if you want to index by equality on the first thing on the compound, and then order by the second thing
21:12:03 <rostayob> no wait a second: if I want to order something after Foo and then after Boh, and then I want to get all the items in which Foo = something and order them by boh
21:12:21 <stepcut> so in mongod,b if you have a compound index on (a,b,c), you can then query that index on a or a,b or a,b,c, but not b,c, right ?
21:12:34 <rostayob> correct
21:13:38 <stepcut> I'll have to think about that
21:14:33 <rostayob> also, I don't know exactly how SQL dbs work, but I doubt that queries that select and order work like ixset works
21:23:18 <stepcut> rostayob: http://dev.mysql.com/doc/refman/5.0/en/order-by-optimization.html
21:23:49 <stepcut> rostayob: according to that page, there are many cases where MySQL can not use the indexes to help order by. For example,
21:23:49 <stepcut> The key used to fetch the rows is not the same as the one used in the ORDER BY:
21:23:50 <stepcut> SELECT * FROM t1 WHERE key2=constant ORDER BY key1;
21:25:05 <stepcut> IxSet could do a better job with compound keys
21:27:13 <stepcut> if the compound key was stored as, Map key1 (May key2 (Set a)), then you could do a lookup on key1 and a sort on key2 for free
21:28:06 <stepcut> really, it would be nice if Data.Map itself added better support for compound keys
21:30:07 <rostayob> stepcut: oh, cool. My expectations were too high it seems (:
21:30:15 <stepcut> rostayob: :)
21:35:45 <rostayob> I just worry too much about these things I guess
21:37:25 <stepcut> http://code.google.com/p/happstack/issues/detail?id=148
21:38:00 <stepcut> well, thanks to your worrying we do at least have groupAscBy/groupDescBy and toAscList/toDescList functions :p
21:38:07 <stepcut> anyway, now is the time to worry about this
21:38:27 <stepcut> the goal is to have a solid MACID eco-system for happstack 7
21:43:51 <rostayob> stepcut: what's not satisfying you right now?
21:44:21 <stepkut> in terms of MACID being solid?
21:44:45 <rostayob> yes
21:45:11 <stepkut> mostly ixset efficiencies and scaling beyond the RAM on a single machine
21:45:43 <stepkut> the ideas for addressing those obstacles exist, but the code does not yet
21:46:05 <rostayob> oh right, so you'd like to have a way to scale horizontally
21:46:07 <stepkut> there was other stuff, like the global IORef that query / update used -- but that is fixed in acid-state
21:47:06 <rostayob> right, in the current state of things there is no way to use acid-state on more than one machine
21:48:17 <stepkut> yeah.. happstack-state has experimental support for that, but it has no been added to acid-state yet
21:48:37 <stepkut> though adding it to acid-state should be cleaner/easier than adding it to happstack-state was
21:48:53 <stepkut> the multiserver stuff is where things get real interesting
21:49:04 <stepkut> but best to have a solid foundation first
21:49:04 <rostayob> stepkut: how would you do that cleanly?
21:51:17 <stepkut> rostayob: the essense of multiple server MACID is that update and query events can be serialized and sent around the network, and the response can be aggregated. There are a lot of variations on how to do that which can yield different behaviors
21:51:31 <stepkut> Lemmih knows more that I do
21:52:20 <rostayob> that's interesting... don't you lose a lot of nice properties that way?
21:52:25 <stepkut> like what ?
21:52:44 <stepkut> MACID also needs better tools for dumping (and modifying) the state of a database
21:52:58 <rostayob> i mean how do you distribute a single data structure on various machine?
21:53:01 <rostayob> *machines
21:53:02 <stepkut> right now it is incredibly opaque, which can make it difficult to debug your code
21:53:09 <rostayob> like a big huge ixset
21:53:16 <rostayob> stepkut: doesn't safecopy partly address that?
21:53:32 <stepkut> not that i know of (regarding safecopy)
21:53:56 <rostayob> actually no I tought you were referring to migrations
21:54:07 <rostayob> also, don't you lose lazyness with the multiple server thing?
21:54:43 <stepkut> for distributed MACID there are two things you can look at. One is a network where every machine has a complete copy of the database. That increases the number of query events you can do per second. But does not really increase the number of update events, nor does it allow you to spread the ixset across multiple machines.
21:54:55 <stepkut> MACID is not really lazy
21:55:03 <rostayob> stepkut: yeah that's replication, I'm talking about sharding
21:55:15 <rostayob> ah... ok
21:55:24 <stepkut> for sharding, that can happen at a few levels
21:56:03 <stepkut> one option is to shard components across different servers. For example if you have a user component, an image component, and a blog component, you can run those each on different machines
21:56:17 <rostayob> yes, that makes sense
21:56:37 <stepkut> if you want to shard an individual ixset, then you need a special version of ixset that understands sharding
21:56:44 <rostayob> yeah
21:57:01 <stepkut> but at the core, all of those are implementing via the same mechanism of sending Events over the network and collecting the responses
21:57:42 <stepkut> one issue I am not sure about regarding sharding is what happens when you make a request and the resulting dataset is larger than available ram
21:58:03 <stepkut> for example, if you shard your ixset across 10 servers, and then request the entire ixset.. that is not going to be pretty
21:58:25 <rostayob> yeah that's what I was trying to picture
21:59:25 <rostayob> mhm. keeping things type safe and having a pure interface makes everything more challenging
22:01:00 <stepkut> making things impure makes other things more challenging
22:01:21 <rostayob> yes, but I wonder if in the end, if you have to get something done
22:01:37 <rostayob> is it really worth it to go through the "trouble" of keeping everything correct?
22:01:51 <rostayob> I love haskell for that reason
22:02:15 <rostayob> but I wonder if it'll ever become the preferred way of doing things
22:03:05 <stepkut> it is my preferred way.. some people still prefer fortran
22:03:20 <stepkut> some people think php is great
22:03:34 <rostayob> some people are fucking stupid (cit. george carlin)
22:04:27 <rostayob> no but jokes aside, a lot of smart people advocate more pragmatic methods
22:05:52 <rostayob> i think that the thing that will make people switch to stuff like haskell is the fact that concurrency is impossible to get right in java & co
22:06:35 <stepkut> they can recommend 'more pragmatic methods' and be correct if they pick the right set of 'things to optimize for'
22:06:35 <stepkut>  
22:07:09 <stepkut> but if you care about different things, then their solution may not apply to you
22:07:39 <rostayob> I mean, I have high consideration of linus torvalds, and I bet he thinks of haskell what he thinks of darcs: http://markmail.org/message/vk3gf7ap5auxcxnb
22:08:21 <stepkut> linus also thinks monolithic kernels are a great idea
22:09:52 <rostayob> yeah but look what he did. the whole "worse is better" thing seems to work well for some people eheh
22:11:03 <stepkut> no, worse is just worse.. I have fixed bugs in mainstream opensource code due to people not checking for eAGAIN more times that I wish to remember
22:11:26 <rostayob> stepkut: so you'd rather be waiting for hurd or whatever instead of having linux?
22:12:36 <stepkut> no, I wish he had listened to andrew and done things right
22:12:59 <rostayob> if he had listened to ast, he probably wouldn't have done things at all, that's the point
22:13:10 <stepkut> I don't think the popularity of Linux is a direct result of Linus choosing monolithic over microkernels
22:13:48 <rostayob> i think that the fact that he choose to go "simple" is definitely part of linux success
22:15:09 <rostayob> anyway. it's an old discussion. i'm on the haskell side of things :P
22:16:10 <stepkut> meh. Linux still doesn't have reliable sound support
22:16:34 <rostayob> I mean linus is a beast in getting things done. he's done two things, linux and git, with a similar philosophy, and both are the best choice for me in their fields
22:17:41 <stepkut> linus is great at creating a following.. if he pooped in a bowl and called it chocolate pudding, people would declare it was the greatest pudding ever :)
22:17:48 <rostayob> stepkut: no? i never had problems really, but i'm not that picky with sound i guess
22:18:07 <rostayob> no cmon, that's not fair to say. he's not even that of a public figure
22:18:25 <rostayob> and linux and git are great
22:19:09 <stepkut> i suppose
22:20:09 <rostayob> no, they are (:
22:21:23 <stepkut> FreeBSD seems just as good as Linux to me :)
22:22:04 <stepkut> and all revision control systems suck.. some just suck less for your particular needs
22:22:16 <rostayob> stepkut: maybe. it has less hardware support I guess (like for laptops) but that's kind of a chicken and egg problem. and freebsd is monolithic as well
22:22:29 <stepkut> yep
22:22:53 <rostayob> so you have a mac or what?
22:23:03 <stepkut> yep
22:23:08 <rostayob> eehhhh
22:23:57 <rostayob> i mean if you're happy. i bought and sold a macbook in less than 6 months lol
22:24:15 <rostayob> main reason: no tiling wm. but it's just a symptom of the lack of freedom
22:24:25 <rostayob> anyway it's an old discussion ehehe
22:24:32 <stepkut> I use xmonad with OS X. but only for running X apps
22:25:02 <rostayob> what do you think of the fact that if you want gcc you have to install that monster that XCode is?
22:25:13 <stepkut> linux sucks for doing video, music production, and photography, so..
22:25:20 <stepkut> I use xcode anyway
22:25:52 <rostayob> but doesn't just pisses you off that you're at their mercy if you want THE C compiler :P?
22:25:57 <stepkut> no
22:26:16 <stepkut> it pisses me off when my soundcard doesn't work ;)
22:26:20 <stepkut> or my video card
22:26:44 <rostayob> my soundcard & video card has always worked with linux
22:27:00 <stepkut> consider yourself lucky
22:27:02 <rostayob> but yes if you want to do video and music production a mac is better suited i guess
22:27:21 <rostayob> stepkut: no that's not true. nowadays making your hw work with linux is too easy...
22:27:30 <stepkut> perhaps I am more pessimistic because I used to work for a linux distro company and have seen a lot more hardware issues
22:28:21 <stepkut> if you have an app that uses OSS (/dev/dsp) I am pretty sure that still blocks any other apps from using sound
22:28:22 <rostayob> mah. we're just different on those issues. i simply can't stand the fact that i can't do what i please
22:28:39 <rostayob> stepkut: well oss doesn't have software mixing no
22:28:53 <stepkut> right, but it blocks the ALSA driver
22:29:09 <stepkut> that is, if you are using the alsa provided OSS emulation, it still blocks the ALSA driver
22:29:30 <rostayob> i've never went into that. you are probably right
22:29:44 <rostayob> i still prefer to have control. that's my n 1 priority
22:30:35 <stepkut> then you should use acid-state, because you can at least understand and control all the code involved.. there is no way you will be able to understand the MySQL codebase enough to ever control it :)
22:30:41 <rostayob> and apple doesn't give me control, so i'll just be constantly pissed off that I can't do what I want
22:31:23 <rostayob> stepkut: I'm not using mysql, but still with mysql i'd have control if i wanted to. i'm an idealist i guess eheh
22:33:11 <stepkut> you only have the illusion of control with a codebase as large as MySQL :p
22:33:43 <Entroacceptor> rostayob: OSS4 does have quite good software mixing
22:33:54 <Entroacceptor> that it blocks the Alsa driver is alsa's fault
22:34:07 <rostayob> yeah but I don't care about me actually using the fact that I have control, but the fact that if a software is os, probably someone else has done what I want to do
22:34:33 <rostayob> the window manager stuff is a good example, i hate the whole mac os x interface. i can't understand how people can like that
22:34:33 <Entroacceptor> and if you think about mysql...
22:34:37 <rostayob> Entroacceptor: oh, ok
22:34:41 <Entroacceptor> use postgres if at all possible
22:35:35 <stepkut> rostayob: there are other window managers besides the default OS X one
22:36:18 <rostayob> stepkut: I tried to have a tiled environment in mac os x, and I failed
22:36:19 <stepkut> for example, http://www.irradiatedsoftware.com/sizeup/
22:36:35 <rostayob> there is some sw that tiles windows
22:36:38 <rostayob> but that's not the point
22:37:12 <rostayob> stepkut: that's not a window manager...
22:37:37 <rostayob> it doesn't replace mac os x window manager at all, it justs places the windows in a tiling manner
22:38:35 <rostayob> and I mean paying for something like that is ridicolous
22:38:49 <rostayob> *ridiculous
22:38:55 <rostayob> i can't spell english ehe
22:40:13 <stepkut> i see no issues with paying for it
22:41:11 <rostayob> yeah it' just kind of funny that there are solutions 100x more powerful on linux, not only for free but open source
22:42:11 <stepkut> yeah.. linux has dozens of window managers, and 0 usuable replacements for photoshop, final cut, and logic pro :)
22:43:00 <rostayob> stepkut: that's the only area in which linux is behind imho
22:43:05 <stepkut> gaming?
22:43:10 <rostayob> there are way more examples the other way around
22:43:33 <rostayob> stepkut: that's quickly changing, and the battle there is mostly vs windows, not mac
22:44:31 <stepkut> for pretty much everything I do on my mac, the only place linux is on par is for running chrome :) Though I can use netflix on mac and not linux..
22:45:18 <rostayob> well i can assure you you can program in haskell on linux, and i'm sure you do that eheh
22:45:32 <stepkut> that is pretty much the only thing I do on linux
22:45:41 <rostayob> oh so you dual boot?
22:45:45 <stepkut> no
22:46:59 <rostayob> oh ok
22:47:06 <rostayob> still, you use linux and not freebsd ehe
22:47:13 <stepkut> yep
22:48:22 <rostayob> i just can't use a mac or windows. but again a lot of smart people do, i'm curious to see how it'll end
23:26:36 <rostayob> stepkut: I don't need the Happstack.Data.Serialize stuff with safecopy, right?
23:27:01 <stepkut> rostayob: yep
23:27:15 <stepkut> rostayob: do you do not need happstack-data or happstack-state
23:27:35 <stepkut> rostayob: though you will need to make a SafeCopy instance for ixset (which depends on happstack-data)
23:28:23 <stepkut> http://hpaste.org/46089/safecopy_instance_for_ixset
23:28:52 <rostayob> stepkut: thanks
23:30:16 <stepkut> eventually we will have Data.IxSet which will probably depend on safecopy and provide a similar instance
23:35:49 <rostayob> what is this: http://hackage.haskell.org/packages/archive/happstack-data/6.0.0/doc/html/src/Happstack-Data-Proxy.html#Proxy ??
23:36:22 <stepkut> just a type, data Proxy a = Proxy
23:36:40 <rostayob> yeah but why the TH stuff?
23:36:58 <stepkut> that derives an instance of Default
23:37:07 <stepkut> which I am not a fan of
23:37:14 <stepkut> (and which does not exist in safecopy anyway)
23:38:12 <rostayob> because I was trying to understand why IxSet indexes by type
23:38:23 <rostayob> what are the rules? that that type must appear once in the data type?
23:38:52 <stepkut> the first index in the list of indexes must appear at least once in the data typ
23:38:54 <stepkut> type
23:38:57 <stepkut> the other indices do not have to
23:39:52 <rostayob> ok, so I don't have to newtype stuff, right?
23:40:18 <rostayob> i can simply do ixFun (\submission -> [sAuthor submission])
23:40:37 <stepkut> you will need the newtypes for the queries possibly
23:40:45 <rostayob> oh, right
23:41:10 <rostayob> so does the querying works with the constructors or with the type themselves?
23:41:18 <stepkut> both..
23:41:37 <rostayob> so I can do @= ("francesco" :: Author), without newtyping
23:41:42 <stepkut> no
23:41:55 <stepkut> because "francesco" is a String, so it won't have the type
23:41:56 <stepkut> Author
23:42:20 <rostayob> no if I define type Author = String
23:42:53 <stepkut> well, then that would be the same as, @= ("francesco" :: String), which works fine if there are no other Strings being indexed
23:43:18 <rostayob> what if there is a String and one Author, with author being a type synonim?
23:43:21 <rostayob> *synonym
23:43:44 <stepkut> won't work, because they are the same thing
23:44:06 <stepkut> we want to get away from the type based queries.. it's mostly just annoying
23:44:53 <stepkut> sometimes I create newtypes for the Indices, but I add them via the calcs function.. I do not put them in the actual data type
23:45:16 <rostayob> what's the calcs function?
23:45:41 <stepkut> data Post = Post String UTCTime ; calcs (Post author _) = (Author author) ; $(inferIxSet "Posts" ''Post 'calcs ['Author])
23:46:08 <stepkut> the 3rd argument of inferIxSet is a function that takes a value a creates additional indices (more or less)
23:46:29 <stepkut> though.. I have not tried using that with ixFun
23:46:51 <stepkut> maybe with ixFun you can just do, ixFun (\submission -> [Author $ sAuthor submission])
23:47:43 <stepkut> that keeps the Author newtype out of the Submission type itself, but still allows you to do, @= (Author "francesco")
23:47:50 <rostayob> yes, that's cool
23:47:53 <rostayob> thanks again
23:48:49 <stepkut> doing things that way gives you a way to name fields that have the same type
23:57:44 <rostayob> i'm actualy thinking of having fields named like sAuthorT
23:57:53 <rostayob> with sAuthorT :: Submission -> Author
23:58:07 <rostayob> and then defining functions like sAuthor :: Submission -> Strng
23:58:12 <rostayob> maybe it's better, boh