02:07:02 <stepcut> Gracenotes: hackage appears to build fine against rss 3000.1.0.. should I send a patch ?
02:08:41 <Gracenotes> stepcut: patches are great, but.. for what?
02:08:58 <stepcut> Gracenotes: the cabal files specifies 3000.0.*
02:09:27 <stepcut> I can either change to to, 3000.1.*, or change it to allow either..
02:10:08 <stepcut> also, I think initialize is spelled with a 'z' not an 's'? Or is that the queen's english ?
02:10:39 <Gracenotes> the source had initialise to start with, british spelling
02:10:59 <stepcut> can I submit a patch that accepts either ?
02:11:06 <dcoutts> stepcut: oh it's because we used to require the stable haxml
02:11:13 <dcoutts> as did happstack itself
02:11:31 <Gracenotes> the main different between newer and older Text.RSS is that CFilter is declared to be parametrized
02:11:47 <stepcut> I think happstack still does, but I am going to fix that for the next release ( possibly by removing it altogether)
02:11:51 <Gracenotes> which, as far as I can tell, does represent a change in haxml
02:13:25 <Gracenotes> and then depending on HaXml (≥1.19.2 & <1.20) vs. HaXml (1.13.*). stepcut: does the patch involve a cabal flag?
02:13:28 <stepcut> I also had to change the dependency in rss to allow HaXml 1.21.. so maybe I should start there
02:13:52 <Gracenotes> or just a different dependency range, maybe..
02:14:01 <dcoutts> Gracenotes: I would not bother with complex alternative dep rages for hackage-server
02:14:15 <dcoutts> pick either the old haxml-1.13  or the new 1.20+
02:14:28 <stepcut> this is the only change I made to hackage-server:
02:14:30 <stepcut> -    rss == 3000.0.*,
02:14:30 <stepcut> +    rss == 3000.1.*,
02:14:31 <dcoutts> if happstack still requires the old one then we have no choice
02:15:02 <stepcut> dcoutts: happstack can unrequire the old one tomorrow if that helps.. dons already mentioned it and it is on the TODO list for 0.6
02:15:18 <dcoutts> stepcut: I don't think it affects hackage-server that much
02:15:53 <stepcut> anyway, I got hackage-server to build :) I am using ghc 6.13, so that makes everything annoying
02:15:53 <dcoutts> stepcut: but you should not have been able to build hackage-server using rss == 3000.1.*, since that does require a later haxml
02:16:54 <stepcut> dcoutts: oh, I had to use, runhaskell Setup.hs configure --user, on everything because ghc comes with a newer version of Cabal than is on hackage, and that really confuses cabal
02:17:05 <stepcut> so I ended up linking against two versions of HaXml :-/
02:17:10 <dcoutts> ohh ok
02:17:20 <dcoutts> stepcut: btw, how does cabal get confused?
02:17:30 <dcoutts> ACTION would hope that it still works
02:18:07 <stepcut> ~/n-heptane/projects/haskell/csv-0.1.1 $ cabal configure
02:18:07 <stepcut> Resolving dependencies...
02:18:07 <stepcut> <command line>: cannot satisfy -package Cabal-1.9.1:
02:18:07 <stepcut>     Cabal-1.9.1-11f4c5be55a95b301eaa2dc9f9abad42 is unusable due to missing or recursive dependencies:
02:18:07 <stepcut>       directory-1.0.1.1-25246d95984e637ec7bee77cfca4fcfe process-1.0.1.2-58ac66a46160bde7e4a9816cfa36017c
02:18:08 <stepcut>     (use -v for more information)
02:18:45 <dcoutts> stepcut: and does ghc-pkg say there's anything wrong?
02:19:13 <stepcut> after I install some stuff, directory and process get rebuilt in in user package configure, but process is the same version that is in the global package config.. and I think that has something to do with it
02:19:13 <dcoutts> like the Cabal-1.9.1 package being busted
02:19:32 <dcoutts> ah
02:19:32 <stepcut> ghc-pkg check says nothing
02:20:09 <dcoutts> I think it's a ghc bug
02:20:15 <stepcut> could be..
02:21:25 <stepcut> hope it gets fixed.. I am tired of run, runhaskell Setup configure --user, <look at missing dependencies>, cabal unpack <dependency>, cd <dependency>, runhaskell Setup configure --user, etc..
02:21:36 <dcoutts> possibly this one http://hackage.haskell.org/trac/ghc/ticket/4072
02:22:02 <dcoutts> stepcut: but you can avoid it by avoiding installing core packages in your user package db
02:22:21 <dcoutts> i.e. blow away the user versions of directory and process
02:22:42 <dcoutts> and don't use cabal upgrade
02:23:02 <stepcut> well, cabal keeps trying to reinstall directory and process for whatever reason it decides to do those things
02:23:14 <stepcut> I never asked it to explicitly
02:23:22 <dcoutts> ok that's the source of the problem
02:23:42 <dcoutts> --dry-run -v often gives some hints
02:24:10 <Gracenotes> oh, gr, laptop
02:24:35 <stepcut> directory-1.0.1.1 (reinstall) changes: filepath-1.2.0.0 -> 1.1.0.4
02:24:35 <stepcut> process-1.0.1.2 (reinstall) changes: filepath-1.2.0.0 -> 1.1.0.4
02:24:50 <stepcut> I think the problem is that ghc comes with a newer version of filepath than is on hackage ?
02:25:53 <stepcut> I think the older filepath got installed this time because the older Cabal forced it
02:26:02 <stepcut> and I needed the older Cabal to build hackage-server :p
02:26:08 <dcoutts> ahh
02:26:20 <dcoutts> since it otherwise will happily pick an installed version
02:26:22 <stepcut> GHC 6.13 is a cruel mistress
02:26:28 <dcoutts> even if there's no corresponding hackage vers
02:26:30 <dcoutts> version
02:26:33 <stepcut> yeah
02:27:14 <stepcut> anyway, now that I got it to build, I can look at the feature stuff
02:27:51 <dcoutts> so if you forced it with --constraint='filepath==1.2.0.0' then it'd complain about Cabal-1.8 being incompatible
02:27:57 <stepcut> yep
02:28:17 <stepcut> anyway, seems like the first thing I can do is to fix happstack-data to not require HaXml
02:29:01 <stepcut> I hope BlazeHtml turns out well
02:29:18 <dcoutts> aye, hope they're sensible with design
02:29:19 <stepcut> not because it is fast, but because it can generate HTML 4.01
02:29:32 <dcoutts> ah yes
02:29:46 <dcoutts> we were using xhtml in hackage-server for that reason
02:29:51 <stepcut> yeah
02:29:53 <dcoutts> it generates something vaguely modern
02:30:05 <stepcut> yeah, but ie still doesn't support it :)
02:30:30 <stepcut> and if you start adding inline <script> stuff it will probably break
02:30:51 <dcoutts> and it seems the world is not going xml for html
02:31:20 <dcoutts> looked that way 5 years ago
02:31:27 <stepcut> no, html 5 (the non-XML version) seems like the feature.. which is, perhaps, better
02:31:44 <stepcut> the w3C declared html dead and xhtml the future in 1999 :p
02:32:20 <stepcut> on my machine the big-table benchmark was slower with blazehtml than with plain-old Text.Html though :-/ not sure what I did wrong (if anything)
02:33:29 <stepcut> how general purpose do you think the Distribution.Server.Users stuff is ?
02:34:12 <dcoutts> stepcut: oh lemme take another look
02:34:57 <dcoutts> stepcut: so some variation on the Distribution.Server.Users.Users is ok
02:35:11 <dcoutts> the Group could also be reusable, though perhaps not just yet
02:35:20 <dcoutts> I'm not satisifed at all with the Permissions
02:35:33 <stepcut> Permissions are really tricky
02:35:35 <dcoutts> Gracenotes and I have been talking about scrapping it
02:35:51 <dcoutts> we want to use a model where group membership bestows permissions
02:36:02 <dcoutts> and there is no real user -> group mapping
02:36:08 <stepcut> yeah..
02:36:13 <dcoutts> just a bunch of groups that are sets of users
02:36:27 <dcoutts> and the permissions are what you do with the group
02:36:37 <dcoutts> what action the group membership guards
02:36:50 <stepcut> yeah..
02:37:21 <dcoutts> one reason it's not quite perfect yet is that you often want to merge groups temporarily for permissions checks
02:37:37 <dcoutts> but the current Group code forces that to be a Set.union which is expensive
02:38:04 <stepcut> the capabilities people say the ACLs are fundamentally flawed.. but implementing a capabilities based solution is a Ph.D thesis probably :p
02:38:48 <dcoutts> stepcut: the idea of the ServerPart for editing a Group is that features can embed a Group in their state and then get an editor that supports GET/PUT/DELETE for free
02:38:57 <dcoutts> stepcut: it's probably a little specialised
02:39:12 <dcoutts> since it's imposing a resource view
02:39:19 <stepcut> yeah
02:39:28 <dcoutts> and happstack seems neutral when it comes to REST stuff
02:39:46 <stepcut> the web is not very modular.. so to make anything reusable, you have to impose some restrictions.
02:39:49 <Gracenotes> also s/neutral/flexible/ though
02:39:56 <dcoutts> we're trying explicitly to use a REST approach, but that's not necessarily right for happstack
02:39:57 <stepcut> my feeling has been that the happstack layer should be pretty neutral/flexible
02:40:03 <dcoutts> indeed
02:40:16 <dcoutts> since people can build resource abstractions/helpers on top
02:40:18 <Gracenotes> sorry, though, what did I miss while my laptop went berserk?
02:40:34 <stepcut> but, some people obviously want higher level things too, like a CMS system
02:41:04 <stepcut> Gracenotes: http://happstack.com/irc-logs/happs-2010-06-01.txt
02:43:13 <Gracenotes> and the digest stuff, which needs some structural revamping to be less monolithic, is in Distribution.Server.Auth.Basic
02:43:56 <stepcut> yeah, I was just looking at that
02:44:55 <dcoutts> stepcut: the one sensible thing about the Users collection is separation of UserId for internal maps and Groups from UserName for external use
02:45:12 <stepcut> yeah
02:45:17 <dcoutts> allowing user names to be reused if accounts are deleted, and never getting user ids confused
02:45:22 <dcoutts> since they're never reused
02:45:22 <stepcut> right
02:45:29 <stepcut> or allow users to change their username ?
02:45:40 <dcoutts> that's fine
02:45:52 <dcoutts> it presumably changes resource names
02:46:00 <dcoutts> so it's up to the app if they want to allow that
02:46:03 <dcoutts> but it's possible
02:46:40 <stepcut> right.. if you index everything by username, then you have to update a bunch of stuff when the user changes their name. But if it is just a UserId, then only one place needs to be updated
02:46:51 <Gracenotes> there should be some semblance of system-wide configurable preferences.. which I need to be careful not to conflate with enabling/disabling Features generally
02:47:13 <dcoutts> Gracenotes: what sort of preferences are you thinking of?
02:48:33 <Gracenotes> well, different 'modes', e.g. a stripped-down tarball-only mirror, an immutable web interface mirror, different allowed actions for different groups, ...
02:49:05 <dcoutts> ah right
02:49:29 <dcoutts> so you can decide if you want to do that at compile time or runtime
02:49:34 <dcoutts> e.g. using cabal flags
02:49:39 <Gracenotes> the regular haskell.org instance as well... and then the enabling/disabling of features, e.g. I was thinking of changelogs editable on the server
02:49:41 <dcoutts> or runtime flags / config
02:51:53 <dcoutts> Gracenotes: note that some features will need corresponding special clients
02:52:06 <stepcut> I noticed you created your Serialize instances by hand instead of TH. Any reason ? (It's not bad, just wondering)
02:52:31 <Gracenotes> dcoutts: hm, executables?
02:53:36 <dcoutts> Gracenotes: e.g. the obvious way to do mirroring is with a special client
02:53:54 <dcoutts> it gets stuff from the old server, and does authenticated uploads to the new server
02:54:06 <dcoutts> and then follows the rss feed of the old server for incremental updates
02:54:36 <dcoutts> then the only thing you need in the server is a way to do direct package uploads, bypassing the normal validation
02:54:38 <Gracenotes> ah, yeah
02:54:46 <dcoutts> using suitable authentication
02:54:57 <dcoutts> since we don't want authors doing that :-)
02:55:25 <Gracenotes> stepcut: ahm, does the default deriveSerialize instance look for Data.Binary instances?
02:55:29 <dcoutts> and indeed the special client needs to be authorised to "fake" the upload so that we retain the original uploader info
02:56:53 <stepcut> Gracenotes: probably not? Why ?
02:57:49 <dcoutts> stepcut: it's useful to restrict TH to a few modules, it really makes the compile times longer
02:58:15 <stepcut> dcoutts: yeah
02:58:27 <dcoutts> and it's a bit nicer to have only one or two modules in each feature actually depend directly on happs stuff
02:58:36 <Gracenotes> not sure. it seems many of the serialize declarations are just pointers to Data.Binary - even when the component types themselves implement Serialize
02:58:41 <dcoutts> have the rest be pure haskell ADTs etc
03:00:41 <stepcut> spiffy
03:10:18 <stepcut> ok. tomorrow I will deal with HaXml in happstack-data. Though you won't actually benefit from that until 0.6 is out
03:18:34 <stepcut> so in going with a REST style API, do you intend to have a 3rd party API for hackage ?
03:18:50 <stepcut> I guess you do, in some since already, since cabal talks to hackage somehow..
03:18:53 <dcoutts> stepcut: yes exactly
03:19:12 <dcoutts> we want to give out as much info as possible, in useful formats
03:19:16 <stepcut> right
03:19:43 <dcoutts> cabal-install should not be special
03:19:44 <stepcut> and be able to serialize the internal state (aka checkpoint) as a XML file or something?
03:20:15 <dcoutts> oh you mean complete backups/dumps ?
03:20:19 <stepcut> yeah
03:20:26 <stepcut> in a 'friendly' format
03:20:34 <dcoutts> yeah that too, but I'm not sure we were planning to make that public
03:20:39 <dcoutts> just for backups
03:20:44 <stepcut> right
03:21:01 <dcoutts> individual resources will be available in whatever formats seem most useful
03:21:06 <stepcut> yeah
03:21:30 <dcoutts> the dump/restore mechanism that we've already got partially implemented just uses a big tarball
03:21:39 <dcoutts> of simple text files
03:21:44 <stepcut> ah
03:21:50 <dcoutts> or of package .tar.gz files etc
03:23:26 <stepcut> I have done a little bit of experimentation with REST web services and happstack. One thing I looked at was having the API calls return a normal Haskell value. And then you can just have classes which turn those values into JSON/XML/etc, depending on some query parameter
03:29:28 <stepcut> I wonder if HackageFeature having a phantom type for it's Component type would make things better or worse :-/
03:33:10 <stepcut> anyway, I have a plan for how to answer that
03:33:41 <stepcut> but now it is time to do something else
03:35:56 <dcoutts> stepcut: right, it makes sense for resources to be typed
03:36:11 <dcoutts> they are of some canonical type/representation
03:36:30 <dcoutts> and converted/serialised to the representations provided in the web ui
03:36:45 <dcoutts> even if the canonical type is just a record of multiple stored representations
03:37:32 <dcoutts> stepcut: the other thing a REST abstraction would provide would be a resource abstraction that you implement and in turn that implements the ServerPart
03:37:49 <dcoutts> providing Etags, conditional GETs, OPTIONS, HEAD etc
03:38:06 <stepcut> I think what we should do is just walk through the process of starting up the server, deserializing the state, and enabling the components, and see what features/behaviour we are trying to achieve, and what issues come up. Then we can figure out what needs to be done to actually support that..
03:42:25 <dcoutts> sounds sensbiel
03:42:35 <dcoutts> but I cannot spell and thus must go to sleep
03:42:46 <Gracenotes> night
03:42:55 <stepcut> me too
03:42:57 <dcoutts> g'night
03:42:58 <stepcut> going to stretch first
06:22:52 <ybit> i've a slight problem
06:22:54 <ybit> "Server error: fd:10: hFlush: resource vanished (Broken pipe)"
06:23:00 <ybit> any idea how to fix this?
07:07:22 <Gracenotes> ybit: oh, did it cause the server to crash?
07:07:46 <ybit> Gracenotes: no i just can't get gitit to send a user his/her password
07:07:54 <ybit> which is unfortunate since i've forgotten mine
13:40:56 <stepkut> ybit: that looks like the client-side closed the connection before the server was done.. but it could be something else..
23:48:19 <Gracenotes> HasHeaders doesn't seem to be exposed.. it is?
23:48:31 <stepkut> hmm
23:48:57 <Gracenotes> *is it, that is.
23:49:09 <Gracenotes> well, so long as I can find out whatever implements it :)
23:49:13 <stepkut> doesn't seem to be
23:49:18 <stepkut> what are you trying to do ?
23:49:44 <Gracenotes> well. the type signature for getHeader :: HasHeaders r => String -> r -> Maybe ByteString looked nice and alluring
23:50:07 <stepkut> go on..
23:50:10 <Gracenotes> perhaps getHeaderM would work as well
23:50:34 <Gracenotes> since it's the same thing monadified
23:50:50 <stepkut> you apply getHeader to Response, Request, or Headers..
23:50:55 <stepkut> what would the monad version do ?
23:51:49 <Gracenotes> I mean, getHeaderM :: ServerMonad m => String -> m (Maybe ByteString)
23:51:54 <Gracenotes> is indeed the function I was looking for
23:52:02 <stepkut> getHeader <$> askRq ?
23:52:19 <Gracenotes> stepkut: since I assume Request is a HasHeaders, yes
23:52:32 <stepkut> oh, I see that getHeaderM already exists :)
23:52:40 <Gracenotes> but made unnecessary by my rudimentary API-searching abilities
23:52:59 <stepkut> and the (currently) poor structing of that stuff I think
23:53:05 <stepkut> I am doing some jiggering right now
23:53:35 <stepkut> looking into the possible removal of HaXml, or at least some #ifdefs so you can use 1.13 vs >= 1.20
23:54:14 <stepkut> it looks like happstack-server uses HaXml as well, so it is a little trickier than I originally thought