--- Log opened Tue Apr 14 00:00:23 2009
03:20 < alexj_> anyone here know how to build hslogger?
11:17 < alexj_> checking in after a long time away...  can anyone here talk about changes from happs to happstack?
11:17 < alexj_> do I just pull?  does it work with 6.10?
11:17 < alexj_> 6.10 definitely breaks happs.
11:22 < aRcatan> i wonder if there's any comparison of Ocsigen and Happstack
11:25 < stepcut> alexj_: happstack works with 6.8 and 6.10
11:27 < alexj_> stepcut: were the repos consolidated into one big darcs repo?
11:27 < stepcut> alexj_: the biggest change is that ServerParts no longer take a lists of children. It's all based around MonadPlus. So instead of, dir "foo" [ dir "bar" [ anyRequest $ ok "hello" ]], you do, dir "foo" $ dir "bar" $ anyRequest $ ok "hello"
11:27 < stepcut> alexj_: yes, there were consolidated, much to my dismay
11:27 < stepcut> alexj_: but, different .cabal packages still
11:28 < alexj_> ok that is sort of weird.
11:28 < stepcut> alexj_: yeah, mae though that it would make it easy to move code around from one module to another
11:28 < alexj_> I like the monadplus change.  I recall discussing it in the design process and for some reason we decided gainast, but I don't remember why.
11:29 < stepcut> alexj_: SimpleHTTP is really different on the inside now -- the rest of the stuff is mostly the same but with haddock comments and type signatures
11:30 < alexj_> awesome.
11:30 < stepcut> alexj_: there is a test suite now too and a buildbot.
11:30 < alexj_> very cool.
11:30 < stepcut> alexj_: the testsuite is mostly just the old tests, but integrated into HUnit
11:30 < stepcut> alexj_: unforntuantely there are a bunch of failures that we don't understand
11:31 < alexj_> installing darcs, ghc, etc onto my new machine.  wanting to get back into this stuff in some form.
11:31 < alexj_> failures in which code?
11:31 < alexj_> perhaps I can help with assumptions we made in writing?
11:32 < stepcut> alexj_: in happstack-data, a bunch of XML conversion tests fail. The interesting part is that they seem to have been failing ever since they were checked in
11:32 < alexj_> that code was very alpha.  there were a bunch of planned fixes that never went in.
11:32 < stepcut> alexj_: so, we did not know if the tests were right and the code is wrong, or the code is right and the tests are wrong
11:33 < alexj_> I don't remember the details.  I do remember that we were using bleeding edge language features to make that code work.
11:33 < stepcut> alexj_: more disturbing those is that the congested checkpoint test in happstack-state oftens fails (but not always)
11:33 < alexj_> the bleeding edge probably made it not work when it changed behavior slightly.
11:34 < alexj_> what is the congested checkpoint test?
11:34 < stepcut> alexj_: one moment
11:35 < stepcut> happstack/happstack-state/tests/Happstack/State/Tests/CheckpointProperties:prop_runRestoreCongestion
11:36 < aRcatan> hmm, happs tutorial would be nicer if the code/ghci blocks would have different style than text
11:36 < aRcatan> the tutorial.happstack.com one
11:36 < stepcut> alexj_: mae and I looked into it a bit once. It basically checkpoints a list of numbers. When the test fails, one of the numbers from the middle of the list is missing..
11:37 < stepcut> alexj_: ./bin/build-install-all.sh will build and install everything as the local user
11:38 < alexj_> stepcut: would like to have everything working with searchpath as well.
11:38 < stepcut> alexj_: talk to mae about that
11:39 < alexj_> no worries on that front.  more concerned about the functional problems.  trying to install modern darcs now.  been out of the loop for over a year and a lot has changed.
11:41 < stepcut> actually, do, ./bin/build-install-test-all.sh, which will build with tests enabled
11:41 < alexj_> the failure you are describing is weird.
11:41 < stepcut> and then do ~/.cabal/bin/happstack-state-tests to run the compiled tests
11:41 < stepcut> or, cd happstack-state/tests ; make local, to run the tests against the local source directory
11:42 < stepcut> alexj_: yeah
11:43 < alexj_> is it a bug in binary serialization?
11:43 < alexj_> does the test serialize in read/show format or in binary?
11:44 < stepcut> alexj_: I don't think so. It does not happen everytime. I think it is a race condition
11:44 < stepcut> alexj_: my unsubstantiated hypothesis is that when the checkpoint saver makes the cut for what to save, sometimes the event right after the cut gets lost
11:45 < alexj_> ok weird.
11:46 < stepcut> alexj_: oh, happstack now defaults to using UTF-8 for everything. Before it didn't really do anything sensible for non-ascii
11:47 < alexj_> these americans, they are so provincial.
11:47 < alexj_> ;- )
11:47 < stepcut> alexj_: well, in your app, a String is always a proper Unicode string. But we convert to/from utf-8 when coverting to/from the binary that is sent over the wire, etc.
11:48 < alexj_> yes, that is good.  was on my mental agenda way back when, but not a high priority for me personally.
11:48 < alexj_> not understanding the repo structure.
11:49 < alexj_> why the consolidation into one darcs repo for what appears to be multiple separate projects?
11:49 < alexj_> (I had been hoping that some of those projects would be sucked into haskell and out of happs)
11:51 < alexj_> also appears to have the hierarchy changed to Happstack.
11:51 < stepcut> alexj_: mae felt that it would be easier for developers if there was only one repo. Especially since it is hard to move things from one package to another if they are in different repos. However, we didn't want to consolidate into one giant cabal package, because we also hope to split some of those things out of happstack as well...
11:51 < stepcut> alexj_: short version: because all version control systems suck
11:51 < sm> wb alexj_! thanks for creating happs, it's cool
11:52 < alexj_> sm: :- )
11:53 < stepcut> alexj_: there has actually been some attempt to make the different projects more independent. for example, happstack-server no longer depends on happstack-state.
11:53 < stepcut> alexj_: the new 'happstack' package includes a bunch of 'kitchen-sink' type integration stuff.
11:53 < sm> there was debate about the repo merging and loss of history, after the fact, but the community seemed to go with the merged version
11:54 < stepcut> sm: there was debate before the fact to, but mae had his mind made up about that
11:55 < alexj_> I am fine with either view.  but we have something in between.  why not have one source hierarchy in the darcs repo?
11:56 < alexj_> some build scripts could then generate individidual cabal packages as needed.
11:56 < stepcut> alexj_: because happstack is modular, and we want to be able to upload indivdual packages like, happstack-state, happstack-server, etc to hackage.
11:57 < sm> sounds like you both just said the same thing, and that is the current practice
11:58 < sm> (one source repo, multiple hackage packages)
11:58 < alexj_> stepcut: the modularity is fine.  but wouldn't that be expressed as different build scripts to produce different packages?
11:58 < alexj_> that would work even with the source repo had one source hierarchy rather than six.
11:58 < stepcut> alexj_: that could work. I wonder if it would cause people to be confused about what cabal package a module belong to when developing
11:59 < alexj_> oy, this is exactly why I was opposed to cabal.
11:59 < alexj_> module names should identify functionality
12:00 < alexj_> you want a build system that recursively finds all the modules you need.
12:00 < alexj_> (which is what searchpath does)
12:01 < alexj_> stepcut: so you are right, this is an issue of version and packaging systems.
12:02 < stepcut> alexj_: the current solution was quick and easy, was minimally different from what happs did making it easier for people to transition, and didn't require much developement time, so we could focus on more important things
12:02 < stepcut> alexj_: like haddock comments :)
12:02 < alexj_> yes, that makes sense.
12:02 < alexj_> :- )
12:03 < stepcut> alexj_: the original release of happstack  was 100% backwards compatible with happs
12:04 < stepcut> alexj_: the biggest missing piece is still sharding. I understand pretty well how the current multimaster works at the code level.
12:04 < stepcut> alexj_: oh, one moment.
12:04 < alexj_> stepcut: except for the change in the module hierarchy from HAppS to Happstack
12:04 < stepcut> alexj_: that did not happen until later
12:05 < alexj_> ok.  do we stlll have a dependency on hslogger?
12:05 < stepcut> alexj_: yes, we still use hslogger
12:05 < stepcut> alexj_: the author of this paper, http://github.com/jsnx/members-only/blob/master/notes/Consistent%20Logging%20Algorithm, had some 'serious' concerns about the use of spread
12:06 < stepcut> alexj_: he initial concern was that no one really knew anything about how well it would actually scale, or what really happens when a node goes offline
12:07 < alexj_> what is the current state of spread dependency?
12:07 < stepcut> alexj_: same as before. spread is the entire basis of multimaster
12:08 < alexj_> yes, lemmih and I discussed this extensively.
12:08 < alexj_> multimaster needs logic to decide how to handle network segmentation.
12:08 < stepcut> alexj_: happstack-state uses the spread mode that ensures all the messages from all servers are delivered to all clients in the same order
12:08 < alexj_> it was the next step on that front.
12:09 < alexj_> before thinking through sharding
12:09 < stepcut> alexj_: interesting
12:10 < alexj_> the gist of the correct solution is that you need to have an odd number of multimaster servers.
12:10 < alexj_> a supermajority needs to be connected to allow progress.
12:10 < stepcut> alexj_: interesting
12:12 < stepcut> alexj_: progress on the multimaster / sharding stuff is currently stalled because no one really knows what was already figured out / tried / discarded. So, we have been focusing on the low handing fruit
12:12 < stepcut> alexj_: s/handing/hanging/
12:13 < alexj_> makes sense.
12:13 < mightybyte> mae and I were discussing sharding the other day
12:13 < mightybyte> He sounded like he might push it back from the currently listed 0.4 release.
12:13 < alexj_> sharding has to follow getting multimaster working properly/reliably.
12:14 < stepcut> alexj_: agreed
12:14 < lanaer> is the use-case of multimaster redundancy or scaling?
12:14 < mightybyte> lanaer: Both in my understanding
12:14 < stepcut> alexj_: any guidance (or code) you can provide would be highly appreciated
12:15 < lanaer> ah cool
12:15 < alexj_> multimaster gives you scaling on reader monad stuff and reliability on writes
12:15 < alexj_> stepcut: have no code at this point just theory.
12:16 < mightybyte> I've been starting to think a little more about sharding.  Nothing substantial though.
12:16 < stepcut> alexj_: the happstack team doesn't even have theory yet :)
12:16 < lanaer> so it's pretty similar to a master/slave replicated rdbms, except without a specific master
12:16 < stepcut> alexj_: OT: this is a pretty interesting presentation on the facebook architecture, http://www.infoq.com/presentations/Facebook-Software-Stack
12:17 < stepcut> alexj_: they clearly need happs :)
12:17 < mightybyte> I'm getting closer to deploying my Happstack app.  Even though I may not ever need to scale it, it's still been crossing my mind.
12:18  * mightybyte agrees
12:19  * stepcut notes that they do use a bit of Haskell at Facebook
12:19 < mightybyte> Oh really?  I hadn't heard that.
12:20 < stepcut> mightybyte: http://www.facebook.com/careers/puzzles.php
12:20 < stepcut> mightybyte: all the languages list on the right are there because they are used somewhere at facebook
12:21 < stepcut> mightybyte: in the presentation I pasted above he talks about it a bit. You can use whatever language you want -- but if no one else knows it, then *you* are going to be the one supporting that app for the rest of time :)
12:22 < mightybyte> Nice
12:23 < stepcut> mightybyte: how's facebook connect looking?
12:24 < mightybyte> I duplicated Facebook.hs, made some mods, and called it FBConnect.hs
12:24 < mightybyte> ...just got it compiling.
12:24 < stepcut> spiffy
12:24 < mightybyte> Nothing's tested yet, but I can push if you're interested.
12:24 < stepcut> I'm still finishing up my taxes
12:25 < stepcut> so, I won't be looking at it until at least tomorrow
12:25 < mightybyte> Just sent mine this morning.
12:25 < mightybyte> I thought I'd go the duplication route until the right way to refactor becomes more apparetn.
12:27 < mightybyte> My initial thoughts were to have modules for Input, Output, and Types/Common.
12:28 < mightybyte> ...to make it more obvious how to use the library.  What do you think about that?
12:31  * alexj_ headed to lunch will be back this afternoon
20:59 < alexj_> trying to build happstack-data
20:59 < alexj_>    Could not deduce (Data DefaultD (M.Map a b))
20:59 < alexj_> ?
21:01 < stepcut> :-/
21:02 < stepcut> alexj_: how did you try to build it ?
21:02 < alexj_> just trying to load the code in interpreter mode
21:03 < alexj_> the cabal file does not ask for any particular flags that should be set.
21:04 < stepcut> alexj_: which code?
21:04 < alexj_> Happstack.Data.Default
21:04 < alexj_> (gets pulled in by Happstack.State)
21:04 < stepcut> alexj_: in theory, all the flags are supposed to be set in the cabal file and in the specific .hs files, though in practice some may be missing
21:04 < gwern> sure the cabal file isn't enabling any extensions?
21:05 < gwern> nor opts like -fglasgow-exts?
21:05 < stepcut> alexj_: what version of ghc? That file loads into ghc fine for me
21:05 < stepcut> ghci
21:06 < alexj_> apparently ghc is ignoring the flags in the file.
21:06 < alexj_> just added -XFlexibleInstances to the caommand line and it is working properly.
21:06 < stepcut> alexj_: hrm, that file has FlexibleContexts but not FlexibleInstances
21:07 < alexj_> ah ok well the misread worked.  FlexibleInstances on the command line solved the problem I think.
21:07 < alexj_> FYI, hspread also had some build issues.
21:07 < stepcut> alexj_: oh?
21:07 < alexj_> walking through the requirement stack and finding small build bugs like this.
21:08 < alexj_> Line 92 of Spread.Client.Connection needs "SomeException"
21:08 < alexj_>    Could not deduce (Data DefaultD (M.Map a b))
21:08 < alexj_>     in handle (\(SomeException _) -> return ()) $ loop `finally` (hClose h >> C.closeChan c)
21:08 < stepcut> alexj_: what version of ghc ?
21:08 < alexj_> (ignore the deduce)
21:08 < alexj_> 6.10
21:08 < alexj_> the update of the exception lib in 6.10 is breaking stuff all over teh place
21:09 < stepcut> alexj_: we automatically build against 6.8 and 6.10 everytime someone commits a patch...
21:09 < stepcut> alexj_: 6.10.1 or 6.10.2 ?
21:10 < alexj_> hmm this may be my fault.
21:10 < alexj_> upgrading ghc aparently did not take in one of my directories.
21:10 < alexj_> weird.
21:10 < stepcut> alexj_: weird
21:11 < stepcut> alexj_: I believe we autobuild against 6.10.1, but not yet 6.10.2. So, there could also be some new bugs lurking there
21:11 < alexj_> using 6.10.2
21:12 < stepcut> alexj_: I heard some chatter about updating the buildbot to support 6.10.2, but I don't think it has happened yet
21:12 < alexj_> ok well walking through it myself now.
21:13 < stepcut> alexj_: cool. Fixes for 6.10.2 are certainly desired. Currently I think we are aiming for out of the box support for 6.8.3, 6.10.1 and 6.10.2
21:17 < stepcut> alexj_: if you want commit access create an account at, http://patch-tag.com/, and then ask mae to give you write access to the happstack project
21:18 < alexj_> ok thanks.  hslogger appears to be having exception issues.
21:18 < stepcut> seems 6.10.2 broke a lot given that it is a minor release
21:19 < alexj_> this is the updated exception lib
21:19 < alexj_> it may be that proper building with cabal solves, but I can't tell.
21:20 < alexj_> I don't like being forced to use cabal just to load files and interact with them.
21:21 < stepcut> alexj_: right, we want everything to be easily loadable in GHCi, which is why all the files have (or should have) LANGUAGE pragmas at the top.
21:21 < alexj_> the issue is that the exception lib was changed and that is breaking code.
21:21 < stepcut> alexj_: though, if the new extensible exceptions is not compatible with the old one, then I am not sure what to do
21:22 < stepcut> is the new extensible exceptions shipped with 6.10.2?
21:22 < stepcut> (the release notes seem to indicate as such)
21:23 < alexj_> I think so.
21:23 < stepcut> since 6.10.2 is the latest stable release, I think the right thing to do is to fix happstack to work with the latest extensible exceptions. And people on 6.10.1 will just have to upgrade extensible exceptions
21:23 < alexj_> s/Control.Exception/Control.OldException works.
21:23 < stepcut> unless there is someway to modify the code to work with both versions of extensible exceptions. though that hardly seems worthwhile
21:24 < alexj_> the change is s/Control.Exception/Control.OldException/  I think that is completely bad form on the part of ghc folks.
21:24 < alexj_> but that is what they did.
21:24 < stepcut> alexj_: yeah
21:24 < stepcut> alexj_: I started a thread on the mailing list.
21:24 < stepcut> (the happstack mailing list)
21:25 < alexj_> seeing this error on xm: /Users/alex/.SearchPath/darcs_get_--partial_--tag=0.9.2_http_happs.org-repos-HAppS-Data/HAppS-Data/src/HAppS/Data/Xml/Base.hs:281:23:
21:25 < alexj_>     GADT pattern match with non-rigid result type `t1'
21:25 < alexj_> do you know what that is?
21:26 < alexj_> oh that is my bad.  nevermind
21:26 < stepcut> no idea
21:26 < alexj_> mixing old code and new.
21:26 < stepcut> ah
21:28 < alexj_> now using the correct *.Data and stll getting error...  /src/Happstack/Data/Default.hs:123:9:     Could not deduce (Data DefaultD (M.Map a b))       from the context (Data DefaultD a, Data DefaultD b, Ord a)       arising from the superclasses of an instance declaration
21:29 < stepcut> alexj_: if you add FlexibleInstances to the LANGUAGE pragma does that help?
21:30 < alexj_> nope.  had that on the command line
21:30 < stepcut> odd, I wonder if it is ghc 6.10.2 specific
21:31 < alexj_> this is the two instance declarations at the end.
21:31 < alexj_> I remember this stuff as syb-with-class bleeding edge back when I last touched it.
21:31 < alexj_> igloo magic iirc
21:32 < stepcut> alexj_: dunno. with ghci 6.10.1 I can do, :load "src/Happstack/Data/Default.hs", no problem...
21:34 < alexj_> interesting...
21:35 < alexj_> do you have commit privelidges on hslogger?
21:36 < stepcut> alexj_: no, I think only john goerzen does because it is his project?
21:36 < alexj_> ok...
21:36 < stepcut> alexj_: he's been good about applying patches in my experience.
21:37 < stepcut> alexj_: though, his grandmother just died
21:43 < alexj_> ok well I assume he will take care of it when he can....  sorry to here about his grandmother.  just put a copy updatted with the one change at http://searchpath.org/hslogger-1.0.7.tgz
21:44 < stepcut> cool
21:45 < alexj_> do you have patch priveledges on happstack?
21:47 < stepcut> yes
21:50 < alexj_> if you comment out the last two lines of Default.hs it works with 6.10.2
21:50 < stepcut> alexj_: interesting... those were just recently added :)
21:51 < stepcut> alexj_: have you looked at hyena at all?
21:52 < stepcut> alexj_: http://code.google.com/p/happstack/issues/detail?id=11&can=1&q=Map
21:54 < stepcut> alexj_: what version of syb-with-class do you have installed?
21:55 < alexj_> ah this may be it.
21:55 < alexj_> have to run now.  will be back in 2 hours
21:56 < stepcut> alexj_: ok. I will probably be asleep then
21:56 < alexj_> gnihht
21:57 < alexj_> well thanks.  I just got it all built (commentnig out those two lines).  will try syb-with-class update shortly.
21:58 < stepcut> cool
21:58 < alexj_> where does syb-wtih-class lvie these days?
21:58 < stepcut> http://hackage.haskell.org/cgi-bin/hackage-scripts/package/syb-with-class
21:58 < stepcut> not sure where darcs is, Saizen is the maintainer now
21:58 < stepcut> (he is often on this channel)
21:58 < alexj_> ok cool
21:59 < stepcut> http://code.google.com/p/syb-with-class/source/checkout
21:59 < stepcut> ...perhaps ?
--- Log closed Wed Apr 15 00:00:25 2009