09:09:14 <Lemmih> Man, figuring out the semantics of multimaster is tough.
09:55:17 <etarasov> how to implement    abc :: ServerPartT (ErrorT String (StateT SomeState IO)) a -> ServerPartT (ErrorT String IO) a    ?
09:55:52 <etarasov> I'm confused with monad stack
09:58:12 <rostayob> etarasov: http://hackage.haskell.org/packages/archive/happstack-server/6.0.3/doc/html/Happstack-Server-Monads.html#g:2
09:58:18 <rostayob> check mapServerPartT out
09:58:24 <rostayob> it's what you're looking for
09:58:45 <rostayob> if you got the state too
09:59:12 <rostayob> you simply have to run the state to
09:59:14 <rostayob> *too
09:59:20 <rostayob> with evalState, or something like that
10:00:10 <rostayob> https://github.com/rostayob/reskell/blob/master/src/Types/App.hs , I have ServerPartT (ErrorT AppError (ReaderT Context IO)) so it's similary, check unpackApp
10:03:12 <etarasov> rostayob: thanks
10:05:29 <etarasov> so complicated
12:03:34 <etarasov> it looks like I'm trying to add monad into bottom of transformer stack
12:03:42 <etarasov> it seems impossible =\
12:16:14 <etarasov> trying mapErrorT ...
12:54:37 <gienah> cd
14:58:51 <stepcut> etarasov: did you figure it out ?
15:08:23 <etarasov> stepcut: yes, first time in my life I don't fully understand a code that I wrote myself. http://hpaste.org/46285/add_state_to_monad_stack
15:10:26 <stepkut> :)
15:11:20 <stepkut> you could probably just use evalState instead of runState. evalState :: State s a -> s -> a
15:11:48 <stepkut> which part of that code do you find most confusing?
15:12:29 <etarasov> all of them =)
15:13:14 <stepkut> :)
15:13:16 <etarasov> I'm not very experienced with type level programming. possibly I should learn more about it
15:14:40 <stepkut> are you familier with fmap?
15:15:16 <etarasov> I only know how to use <$> from Control.Applicative
15:15:54 <etarasov> Didn't even mind about it's signature
15:16:23 <stepkut> any idea how you might implement, Functor Maybe /  fmap :: (a -> b) -> Maybe a -> Maybe b
15:21:10 <etarasov> > fmap fun a = maybe Nothing (Just . fun) a in fmap (+1) (Just 3)
15:21:11 <lambdabot>   <no location info>: parse error on input `='
15:21:17 <etarasov> > let fmap fun a = maybe Nothing (Just . fun) a in fmap (+1) (Just 3)
15:21:19 <lambdabot>   Just 4
15:21:24 <etarasov> > let fmap fun a = maybe Nothing (Just . fun) a in fmap (+1) Nothing
15:21:26 <lambdabot>   Nothing
15:22:43 <stepkut> if we are more explicit, we get something like, let fmap f a = case a of Nothing -> Nothing ; (Just a') -> (Just (f a'))
15:22:56 <etarasov> yeah
15:23:02 <stepkut> looking at your original question you asked:
15:23:21 <stepkut> how to implement: abc :: ServerPartT (ErrorT String (StateT SomeState IO)) a -> ServerPartT (ErrorT String IO) a
15:23:46 <stepkut> which is pretty similar to fmap in a lot of respects
15:24:05 <stepkut> instead of, f a -> f b, you want, ServerPartT m a -> ServerPartT n b
15:24:44 <stepkut> everything else falls out automatically from trying to implement a function that will let you do, ServerPartT m a -> ServerPartT n b
15:26:24 <stepkut> the mapServerPartT and mapErrorT functions are very much like fmap
15:27:06 <etarasov> so, is my code is good enough?
15:27:07 <stepkut> in this case you have nested things like, [Maybe a], so you need to use two fmap functions like, fmap (fmap (+1)) [Just 1, Nothing, Just 2]
15:27:34 <stepkut> your code is exactly right. though using evalState would make it a little shorter
15:27:52 <etarasov> ok, it seems I got it
15:27:56 <stepkut> yep
15:28:06 <stepkut> if you just follow the types you can't really go wrong
15:28:49 <stepkut> the types are, unfortunately, fairly complex and scary looking. But the actual code is pretty straight-forward
15:41:10 <stepkut> i do plan to add a section to the crash course on this though
15:41:15 <etarasov> all my code can be replaced with mapServerPartT (mapErrorT $ evalStateT myState) statefulAction, but I'm afraid I will not be able to understand it next year
15:41:22 <stepkut> :)
15:41:37 <stepkut> there is nothing wrong with keeping it long form with more type signatures
15:41:41 <stepkut> I do that sometimes
15:42:28 <stepkut> type signatures are often faster to read and understand than the code, so I will often split things into smaller chucks so that I can add more type signatures
15:44:25 <stepkut> after all, the point of haskell is to not have to remember so much or think so hard
18:05:06 <stepkut> Lemmih: did acid-state get 100k transactions per second or 55k (for the event that updated an integer)
18:20:00 <stepcut> http://happstack.blogspot.com/2011/05/macid-haskell-persistent-data-store.html
18:56:36 <stepkut> jaspervdj: There is an issue with the Environment type in digestive functors..
19:12:04 <stepkut> or maybe we should be using FormInput [Inputs]
19:55:14 <stepkut> ok git lovers. I have the lastest version of digestive-functors from git. I have made some changes to the working directory. How do I send these changes to jaspervdj ?
20:16:06 <Lemmih> stepkut: You can test it.
20:16:15 <Lemmih> stepkut: It was less than that, I think.
20:16:31 <stepkut> 14:48:03<Lemmih> Yay, ~55000 transactions per second without modding cereal. Better than expected.
20:16:45 <stepkut> what did that refer to?
20:17:57 <Lemmih> That's the theoretical maximum.
20:19:06 <Lemmih> 100k transactions in 7.5s for real transactions.
20:19:11 <stepkut> ah
20:19:23 <Lemmih> So, 13.3k transactions per second.
20:20:25 <stepkut> updated the post.
20:21:17 <Lemmih> Performance will only go up.
20:21:33 <stepkut> right
20:21:44 <stepkut> that will be an excuse to make more blog posts ;)
20:22:25 <Lemmih> As it now, multimaster replication will increase the possible number of updates you can do.
20:22:34 <Lemmih> (updates per second, that is)
20:22:37 <stepkut> really?
20:22:40 <stepkut> interesting..
20:22:47 <stepkut> in the old version it could only slow it down..
20:22:53 <Lemmih> Updates from replication are already serialized and will be nearly as fast as replaying updates from the log.
20:22:58 <stepkut> because it had to wait for everyone to play the event
20:23:31 <Lemmih> And updates from the log play at hundreds of thousands per second.
20:23:35 <stepkut> ah
20:24:19 <stepkut> so if you have two servers the hard work of doing an update event can be split across them, and then when they do the replication part that is fast
20:24:27 <Lemmih> Replication will still add latency.
20:24:34 <stepkut> right
20:25:57 <Lemmih> "the hard work of doing en update event" == serializing the event.
20:26:10 <Lemmih> Right now, serializing an event is way slower than executing it.
20:26:29 <stepkut> right
20:27:21 <stepkut> what work do you still have to do on single server MACID?
20:28:07 <Lemmih> Nothing vital.
20:28:35 <stepkut> are you going to look at IxSet improvements first or multimaster?
20:28:40 <stepkut> or something else?
20:28:42 <stepkut> or nothing else?
20:28:48 <Lemmih> IxSet is your domain (:
20:29:06 <Lemmih> Some auxiliary tools would be to have but I'll probably start on multimaster first.
20:29:35 <stepkut> did you have ideas about how to get IxSet away from type based queries?
20:29:38 <Lemmih> IxSet is very low on my wishlist. I'm pretty much fine with just using maps.
20:29:56 <stepkut> you were talking about using mmap with IxSet somehow?
20:30:18 <Lemmih> Using something like compact-map should be fairly easy.
20:31:04 <Lemmih> I'm not sure how to move away from type based queries without being really evil.
20:32:10 <stepkut> :)
20:32:28 <stepkut> in the current IxSet type based queries can be a little less annoying
20:32:58 <Lemmih> Maybe making them less annoying would be ok.
20:34:22 <stepkut> Basically if you have, data Foo = Foo { bar :: String, baz :: String }. You can then create types (aka names) for your keys like, newtype Bar = Bar String, newtype Baz = Baz. Then you do, foos @= (Bar "some string")
20:34:39 <stepkut> but those newtyped values do not have to actually be in the the Foo type itself
20:35:04 <stepkut> they pretty much only appear in the queries
20:36:00 <stepkut> and, in that context, they 'feel' more like naming fields than specifying types
20:36:41 <stepkut> and they don't require you to modify all you types to have newtypes in them like, data Foo = Foo { bar :: Bar, baz :: Baz }
20:36:51 <Lemmih> Is it annoying to use? I haven't myself.
20:37:23 <stepkut> it is annoying to have to add lots of newtypes to your datatypes just so you can do queries on fields that have the same type
20:37:37 <stepkut> but this usage pattern gets around that part
20:37:40 <Lemmih> Ah.
20:37:51 <stepkut> you still have boilerplate to create the newtypes though
20:38:19 <stepkut> the other pattern that is fairly annoying is dealing with id values and autoincrementing them
20:39:00 <stepkut> for example, if you have, data User = User { userId :: UserId, username :: String }
20:39:13 <stepkut> when you create a new user, you need someplace to get the next unused UserId from
20:40:03 <stepkut> right now I just have another field in the state like, nextUserId :: UserId. But it is tedious (and some what error prone) to have to maintain and increment that field by hand. Especially when you have lots of id type things
20:41:50 <Lemmih> Isn't that the right way to do it?
20:42:29 <stepkut> mostly
20:43:32 <stepkut> it might be nice if there was a $(deriveId "ItemId") function that created this:
20:43:33 <stepkut> newtype ItemId = ItemId { unItemId :: Integer }
20:43:34 <stepkut>     deriving (Eq, Ord, Read, Show, Data, Typeable)
20:43:34 <stepkut> $(deriveSafeCopy 0 'base ''ItemId)
20:43:37 <stepkut>  
20:43:40 <stepkut> instance Inc ItemId where
20:43:43 <stepkut>     inc (ItemId i) = ItemId (succ i)
20:43:46 <stepkut>  
20:43:55 <stepkut> because I am tired of typing that in by hand :p
20:45:55 <stepkut> it is also conviceable that the IxSet type itself could have some sort of auto-increment field. So that if I did, IxSet.insertNew (Item { ... }), it would automatically set the ItemId (in Item) to the next available ItemId and return the modify Item to me. (insertNew :: (AutoIndex Item, IxSet Items) => Item -> IxSet Items -> (Item, IxSet Items), or something
20:46:19 <Lemmih> Why use a counter for each thing? Why not a single one for everything?
20:46:27 <Lemmih> It's not like you're gonna run out of Integers.
20:46:39 <stepkut> that could work too
20:47:07 <Lemmih> I don't really like it doing it automatically. Generating unique ids has to be done properly if you wanna scale.
20:47:29 <stepkut> have a single counter vs per-item does not really reduce much boilerplate at the moment
20:49:15 <Lemmih> How about a unique-id component?
20:49:51 <stepkut> hmm
20:50:08 <Lemmih> It is somewhat necessary to break those transactional lines to get good performance.
20:50:10 <stepkut> I'll try that and see
20:50:25 <stepkut> I tend to have a lot of components already, so that is no big deal :)
20:50:35 <Lemmih> A unique-id component could scale 100%.
20:50:40 <stepkut> yeah
20:54:14 <stepkut> i am not clear how that is more scalable than a unique id per id-type though
20:55:16 <Lemmih> It isn't if you kept the per type id in separate components. But that would be bothersome.
20:55:42 <Lemmih> Keeping the id counter together with the data doesn't scale, though.
20:56:21 <stepkut> why not?
20:56:27 <Lemmih> But this is very academical. Sharding isn't really a concern.
20:56:44 <Lemmih> It doesn't lend itself to partitioning.
20:57:14 <stepkut> ah
20:57:39 <jaspervdj> stepkut: Got your mail
20:57:53 <jaspervdj> will fix
20:58:09 <stepkut> Lemmih: I can see how that could be trouble. Hard to think about since I don't have a clear vision for how sharding will actually work.
20:58:50 <stepkut> Lemmih: for now, I just try to split things into lots of components, since that reduces update contention (and allows simple partitioning of different components to different servers)
20:59:09 <stepkut> jaspervdj: is that why you adde getInputStrings?
21:01:52 <jaspervdj> yes
21:02:59 <stepkut> cool
21:03:31 <stepkut> so having, Environment (FormId -> m (Maybe i)), with i == [Input] is the intended idea?
21:04:48 <stepkut> that results in two ways to express 'zero', Just [] and Nothing. In my patch I convert Just [] to Nothing
21:07:30 <stepkut> jaspervdj: also, maybe we should add a transformer, required :: (Monad m) => e -> Transformer m e (Maybe a) a
21:08:24 <jaspervdj> stepkut: errm right, that would be handy
21:08:35 <jaspervdj> but I'm a bit busy at the moment <_<
21:08:42 <jaspervdj> stupid university stuff
21:09:33 <stepkut> jaspervdj: I can send a patch, I already wrote it ;)
21:10:48 <stepkut> required err = transformEither $ maybe (Left err) Right
21:11:13 <jaspervdj> stepkut: cool, feel free to make a github pull request :-)
21:11:25 <stepkut> though, maybe it should be called, notOptional to go with optional ;)
21:11:48 <jaspervdj> required seems more clear to me
21:12:00 <stepkut> agreed ;)
21:12:17 <stepkut> will you be making a hackage release soon? or should I build from git for now ?
21:13:11 <jaspervdj> already done
21:13:15 <stepkut> epic!
21:13:16 <stepkut> thanks!
21:13:22 <jaspervdj> np