--- Log opened Sat Jan 24 00:00:33 2009
00:02 < mae> " _
00:02 < mae> : )
00:03 < mae> gwern: you like that? :P
08:46 < dcoutts> stepcut: you know that .cabal packages can specify their corresponding source repos including if they're in a subdir of a repo
09:16 < stepcut> dcoutts: I'll have to think and see if that helps anything
09:17 < dcoutts> stepcut: it should provide enough info to automatically download and locate the package sources
09:18 < dcoutts> there's support for specifying 'head' branches and also 'this' to identify particular releases (including branch and tag)
09:18 < stepcut> dcoutts: right, but the autobuilder knows nothing about .cabal, only debian packages
09:19 < dcoutts> ah, sorry, I didn't catch the context
09:29 < Saizan> stepcut: you seem to have used happs quite a lot, do you find HAppS-Data's Xml class and SYB instances useful?
09:32 < dcoutts> Saizan: it would be useful to be able to easily render Haskell data in xml form, if that's the question
09:36 < Saizan> dcoutts: the question is more which is the better way to get that
09:37 < Saizan> i'd find TH more appropriate
09:38 < dcoutts> Saizan: yeah, I don't really mind
09:38 < chessguy> 'morning
14:50 < gwern> sm: there's now a public demo of filestore gitit: http://gitit.johnmacfarlane.net/
14:51 < sm>  gwern: cool
14:51 < sm> excellent
14:55 < gwern> john is pondering whether to include your switch in the release
14:56 < gwern> sm: thoughts?
14:57 < sm> gwern: oh.. I don't see why you wouldn't
14:57 < sm> not everyone likes markdown
14:57 < sm> twb, eg
14:57 < sm> I don't want to hear his rants :)
14:58 < gwern> I suspect he doesn't like rst either given how similar they are...
14:58 < sm> I envision gitit supporting everything pandoc supports, and that list growing
14:58 < dcoutts> haddock markup! :-)
14:59 < dcoutts> good point, someone should add haddock format to pandoc
14:59 < sm> and org-mode
15:00 < sm> ah, he's pushed it
15:02 < gwern> dcoutts: how would that work?
15:02 < dcoutts> gwern: haddock markup as an input format at least
15:03 < dcoutts> it's obviously rather limited as an output format
15:03 < gwern> but haddock markup requires source to be sensible...
15:03 < dcoutts> gwern: na
15:03 < dcoutts> .cabal file descriptions are parsed as haddock markup
15:04 < dcoutts> gwern: so it's just the simple paragraph level markup
15:04 < gwern> hm. what could one do with pandoc supporting haddock as in/out?
15:04 < dcoutts> it'd be much easier to implement a haddock wiki
15:05 < dcoutts> let users edit the markup and map it back to the original code to construct darcs patches
15:05 < sm> you might want to convert haddock markup in source to some better format
15:05 < dcoutts> or that
15:06 < sm> but if the structure of haddock and pandoc's canonical document is too different, I guess it's no go
15:07 < dcoutts> translating from haddock into pandoc must be trivial
15:08 < dcoutts> the other way would be very lossy
15:08 < dcoutts> haddock structure is deliberately very simple
15:08 < dcoutts> there's an existing standalone haskell parser for it so it'd not be a hard task
15:08 < dcoutts> it's used in the hackage server
15:09 < dcoutts> just a couple modules
15:11 < gwern> hm. maybe I should file a bug report
15:12 < gwern> dcoutts: persoanlly my opinion on haddock markup with gitit is that it should follow the literate haskell mode of gitit - showing the source with the haddocks treated the same way as literate - interpreted, but as haddock rather than as markdown
15:12 < gwern> dcoutts: could you send me a link to the haddock parser?
15:13 < dcoutts> gwern: http://code.haskell.org/hackage-server/Distribution/Server/Pages/Package/
15:14 < dcoutts> gwern: that's should be the same as the modules in http://darcs.haskell.org/hackage-scripts/
15:15 < dcoutts> gwern: do you have an example of the literate haskell mode of gitit that shows the treatment of haddock?
15:15 < dcoutts> I'm not sure I really follow the description, an example would be best
15:15 < gwern> dcoutts: as far as I know none of the public gitit have a literate haskell file I could show you
15:16 < gwern> I could send you a screenshot of my local example?
15:17 < dcoutts> sure
15:18 < dcoutts> gwern: so are you really hacking on a tool to let us make small contributions to package docs? (eg on hackage)
15:18 < gwern> hm. http://gitit.johnmacfarlane.net/Curry-Howard-Lambek Correspondence is not quite right unfortunately - it's markdown with haskell mixed in, not haskell with markdown mixed in :(
15:18 < dcoutts> or something that might be going in that direction anyway
15:18 < gwern> dcoutts: not really, I'm more gathering and dispersing ideas and suggestions to see what the 'killer app' of gitit will be for haskell development
15:19 < dcoutts> right'o
15:19 < gwern> perhaps it will be the DWN model, or perhaps it will be this haddock model, or perhaps it will be 'web-based haskell development' with a gitit slapped down over a secondary repo
15:19 < dcoutts> I think there is a really useful niche in between darcs/git management and a wiki
15:19 < gwern> you still at duncan.coutts@worc.ox.ac.uk ?
15:19 < dcoutts> gwern: that'll do
15:20 < gwern> I'm convinced that there's *something*
15:20 < dcoutts> something where users can make quick changes, but where the authors still retain control
15:20 < dcoutts> so users basically end up sending changes to the author to review & apply
15:20 < dcoutts> the tricky bits are in how interactive it is
15:20 < dcoutts> are user changes reflected immediately
15:21 < dcoutts> what happens to rejected patches etc
15:21 < gwern> my proposal on -cafe was that the gitit repo be 'secondary', with a reviewer who every day or so reviews edits and either forwards/pushs or obliterates them
15:21 < gwern> they'd be reflected immediately in the secondary repo of course
15:21 < dcoutts> ah I see yet
15:21 < dcoutts> personally I'd be happy to receive emails like I get other darcs patches via email
15:22 < dcoutts> though if there is a secondary repo then I have to explicitly reject patches rather than just ignore them
15:22 < dcoutts> so they'll be obliterated from the secondary repo
15:22 < gwern> so it'd be like the wiki was someone's private repo - they can make and record changes, and these are real first-class changes, but that doesn't mean they show up in the master repo. and if the private repo gets too far out of sync, that's that author's problem
15:23 < gwern> in thiscase, the wiki admin's
15:23 < dcoutts> that part is a bit like darcswatch
15:23 < dcoutts> mm, that's fine if there is an admin with time
15:23 < dcoutts> I'd like to have this for every darcs repo I maintain on code.h.o for example
15:23 < dcoutts> I don't want to have to spend lots of time
15:23 < gwern> dcoutts: one of the nice things about DVCS repos is they are very scriptable. one could add a hook to darcs to, before/after a record, pull from the master repo and obliterate every local patch older than 7 days
15:24 < gwern> so now the administrator is reduced to 'push?' and he can leave the rejected patches to be eventually 'garbage-collected' by the script
15:24 < dcoutts> perhaps something like tracking not-yet-pushed-upstream patches and obliterating user suggestions if they're not applied upstream within a certain time
15:24 < dcoutts> gwern: ah, that sounds like the same thing, obliterate conflicting or old patches
15:24 < gwern> you see what I mean about this simple model being powerful and understandable
15:25 < dcoutts> gwern: yes, keeping it as a copy and then scripting what to do with patches being pulled/pushed
15:25 < dcoutts> really takes advantage of the darcs model
15:25 < dcoutts> gwern: presumably this will use the darcs library api?
15:25 < gwern> mmhmm. I don't know how well it'd work with git, but that's git users' problem :)
15:25 < dcoutts> heh heh
15:26 < gwern> dcoutts: hmm, it may not need to. there are even simpler methods: every 7 days, say, 'rm -rf wikidata/ && get master-repo'
15:27 < dcoutts> true
15:27 < gwern> (hmm, this is a little confusing. I see no way to get a .lhs file's comments interpreted as markdown, although I was sure that's what could happen.)
15:27 < dcoutts> gwern: maintaining multiple suggestions on a single area would be interesting
15:28 < dcoutts> gwern: eg consider what would have happened with the recent Monoid documentation
15:28 < dcoutts> lots of people with different suggestions
15:28 < dcoutts> and do we want a single 'current' suggestion to send to the package author or multiple independent alternatives from different contributors?
15:29 < dcoutts> or is it just a sequence of edits from different authors
15:29 < dcoutts> in traditional darcs style and darcs works out if the commute or not
15:31 < gwern> being a wiki I imagine bunch of edits on the latest, with occasional reverts perhaps
15:32  * sm catches up.. interesting!
15:32 < sm> gitit + filestore + gwern's darcs backend is definitely opening up this area
15:33 < gwern> it's a little annoying waiting for sebastiaan
15:33 < sm> the filestore guy ?
15:33 < gwern> no, the orchid guy
15:33 < sm> who wrote filestore ?
15:33 < gwern> john and I are 'the filestore guy' :)
15:34 < sm> yay, thank you
15:34 < sm> I think the name is just a tad obscure mind you
15:35 < sm> and I can't really think of anything better
15:37 < sm> but why do you have to wait for sebastiaan ?
15:38 < gwern> we want a coordinated release
15:38 < sm> I see
15:38 < gwern> we have release notes and everything: http://haskell.org/haskellwiki/User:Gwern/releasenotes
15:40 < sm> nice. I take back what I said about the name, this isn't just about a generic api for rcs's
15:40 < sm> it's a file store for versioned files
15:41 < gwern> yeah. I am still unsure what good an sqlite backend is, but it does demonstrate that
15:41 < sm> I could make a filestore backend for the zope object database and my zope wikis would show up in gitit
15:42 < gwern> sounds like a great idea to me
15:43 < sm> someone could make it work with sql, and wikipedia could show up in gitit. maybe :)
15:44 < gwern> mediawiki isn't supported by pandoc as an input
15:44 < Lemmih> gwern: Does gitit support some kind of page inclusion? I'd like to add computer generated contents to the lhc wiki.
15:46 < gwern> page inclusion? no. I'm sure you hack something up going through the actual files tho
15:46 < gwern> 'sed -e 's/{{template}}/expanded content stuff//s' * && darcs record --all' etc.
15:47 < gwern> Lemmih: templates are on the todo
15:57 < gwern> Lemmih: one of the issues is we don't have a good template plan
15:57 < gwern> we don't want to do shitty mediawiki templates, but we can't just call ghci -e
15:57 < gwern> if you had a good use-case for templates, that'd certainly help with design
15:57  * gwern suspects a good templating system for wikis is worth at least a master's thesis :)
15:59 < Lemmih> I want to show the recent darcs history and the latest testsuite results.
16:17 < sm> gwern: yeah.. definitely make it pluggable
16:18 < sm> Lemmih: should be pretty easy to hack gitit's recent activity view
17:51 < stepcut> gwern: I hacked up some code once to allow you to use hsplugins to load templates
17:53 < Saizan> i think it's easier to use hint these days
17:54 < stepcut> gwern: actually, it is preprocessor, similar to trhsx, which converts pandoc documents which contain snippets of haskell code into straight haskell.
17:54 < stepcut> And then I think I was going to use hsplugins to compile and load the template into the application...
17:55 < stepcut> yeah, this work predated hint
17:55 < stepcut> no..
17:56 < stepcut> I take that back, i'm not sure why I didn't use hint. I think I was just modeling things after happs-hsp-template
17:57 < stepcut> using the pre-processor to turn the markup+haskell code into straight haskell code, and then hint to execute the code and get out a String would be better than loading foreign code into the server space :)
17:58 < stepcut> unless it's trusted code, and you want that...
18:06 < stepcut> does anyone understand the suggestion of using BDB as the basis for MACID?
18:09 < Saizan> the one in the issue tracker?
18:10 < stepcut> dunno, Rick R suggested it on the mailing list, one moment
18:11  * sm didn't
18:11 < stepcut> ugh, I can never figure out how to link to a specific post on google groups
18:12 < sm> stepcut: don't bother, use news.gmane.org
18:12 < sm> you can get the thread too if you prefer
18:13 < stepcut> http://article.gmane.org/gmane.comp.lang.haskell.happs/1179
18:15 < Saizan> ah, yeah, i think he has misunderstood Lemmih or MACID or both
18:16 < stepcut> is that the same person that thought if a node went down all the data was lost/
18:17 < Saizan> that one was Andrey Chudnov according to my gmail
18:18 < stepcut> ah
18:18 < stepcut> lots of confusion ;)
18:18 < stepcut> I think I know the origin of that misconception though
18:19 < Saizan> which is?
18:19 < stepcut> Lemmih was once talking about the dream of not doing any logging to disk and only having things in RAM. Ideally on a large enough, geographically distributed cluster, which replication, you were never have all of the nodes which contained the data down at once.
18:19 < Saizan> ah, i see
18:19 < Lemmih> stepcut: I was?
18:19 < stepcut> the 'ideal' was to be able to make a HAppS clustor that was resiliant enough that 'disk backup' was not required
18:20 < stepcut> Lemmih: I think so? Maybe that was someone else?
18:20 < stepcut> maybe that was me?
18:21 < sm> heh
18:21 < stepcut> No I think was not me. I think this was 6 months ago...
18:21 < gwern> stepcut: I'm afraid I don't really know how trhsx and such work would serve in templating gitit
18:21 < Lemmih> Well, writing the log to disk shouldn't impact performance.
18:21 < sm> that was skynet infiltrating your dreams from the future :/
18:21 < stepcut> Lemmih: right. But the idea that it would be pointlessly redundant is an interesting idea...
18:22 < gwern> stepcut: but it seems john just added some sort of plugins to pandoc HEAD, so I guess whatever gitit does will have to start from that
18:22 < stepcut> gwern: depends on what you mean by 'templating' I guess
18:24 < stepcut> gwern: some template systems you can do $VARIABLE_NAME, and then something replaces all the variables with values to fill in the template. In a more advanced template system, instead of just $VARIABLE_NAME, you could right <% arbitrary haskell expression %>, which would produce a value that gets inserted into the template ?
18:25 < stepcut> Saizan: regarding your earlier question. I try to avoid XML at all costs ;)
18:27 < Saizan> stepcut: heh, good rule of thumb i guess :)
18:27 < stepcut> Saizan: in theory, the syb stuff for automatic form generation is neat. But it seems like it is difficult to have fine enough control over the layout in real applications when you do that.
18:27  * Saizan nods
18:28 < stepcut> Saizan: and the syb-with-class RJson library has been a bit troublesome as well. Often times the generic toJson output it creates can not be parsed by fromJson.
18:28 < stepcut> or, worse, if I have custom toJson/fromJson instances, but I forget to import them, I don't a compiler error -- just bogus json
18:28 < gwern> stepcut: yes, arbitrary code would be nice, but the problem is, is it safe?
18:29 < Saizan> yeah.. the apprach of defining a "fallback" instances using generics just looks wrong to me
18:29 < stepcut> gwern: yes and no. If you use hsplugins to compile the code and load it into your application then no. If you use hint to only run 'safe' code which produces a String, then yes?
18:31 < stepcut> Lemmih: I have a few minor updates to the validation patches, should I send them your way? Or are the happs.org repos externally frozen now ?
18:31 < gwern> how could you use hint to run only safe code?
18:32 < stepcut> gwern:  isn't that what hint does? (that's what lambdabot uses now?)
18:33 < gwern> no. hint is merely a pleasant wrapper around the ghc api
18:34 < gwern> mueval is an executable which uses hint and a number of techniques to safely evaluate a small subset of haskell
18:34 < stepcut> ah, mueval is what I meant then
18:34 < stepcut> I knew hint didn't sound quite right
18:35 < gwern> mueval is not particularly suited for the task in multiple respects
18:35 < Lemmih> stepcut: I've heard the project is being continued by a bunch of guys not affiliated with HAppS LLC.
18:35 < gwern> hee hee
18:36 < stepcut> Lemmih: right, I will submit the patches there as well. But those jerks decided to put everything in a single repository, so my darcs patches won't apply cleanly
18:37 < Lemmih> Well, you can send me your patches but I can't say when they'll be applied (if ever).
18:37 < stepcut> ok.
18:37  * stepcut thinks being able to stick Serialize in the deriveAll clause should be on the roadmap
18:39 < stepcut> maybe I should revive my HAppS based distributed IRC bot if this channel is going to pick up in traffic...
18:39 < stepcut> a bot for everyone!
18:39 < Saizan> we can adjust Data.Derive's Binary derivation for Serialize
18:40 < stepcut> as long as there is a migration path, I have real data using the existing format
18:41 < stepcut> but, in my ignorance, using the new Binary libraries instead of the HAppS specific Serialize instances sounds like a good idea
18:41 < Saizan> Serialize ensures that the versioning/migrating works well, afaiu
18:41 < stepcut> ah yes
18:42 < stepcut> I should know that, I just wrote a blog on exactly how that works, including hexdumps
18:42 < Saizan> heh
18:43 < stepcut> oh, HAppS needs some improvements to static file serving as well... it's really hard to take an incoming url, and use some direction to serve a file from the disk which has a different name
18:43 < Saizan> we could write an "instance (Typeable a,Version a, Binary a) => Serialize a where ..." but that has the same problems of RJson above
18:44 < stepcut> like -> http://example.org/photos/me.jpg --> /srv/happs/pictures/<md5sum>
18:45 < Saizan> a more flexible interface for fileServe
18:45  * stepcut should make a list
18:45 < Saizan> it should be easy
18:46 < stepcut> Saizan: yeah, refactoring the code that fileServe depends on would do the trick. I had to copy and modify a lot of code out of HAppS-Server to do it, but it seemed like fileServer could have been built on top of my modified code easily
18:47 < stepcut> It's also suprising tricky to use syb to extract all the Strings from you datatype so you can index them for search
18:48 < Saizan> really? isn't that something like "listify (const True) foo :: [String]"?
18:49 < stepcut> Saizan: nope!
18:49 < stepcut> *Main Data.Generics> listify (const True) "hello" :: [String]
18:49 < stepcut> ["hello","ello","llo","lo","o",""]
18:49 < stepcut> :)
18:49 < Saizan> hah :)
18:50 < stepcut> it finds too many strings, because a String is made up of *more* Strings. It's Strings all the way down.
18:51 < stepcut> gTypes :: forall a. (Typeable a) => GenericQ [a]
18:51 < stepcut> gTypes x = ((foldl (++) [] (gmapQ gTypes x)) `mkQ` (\ (a::a) -> [a])) x
18:51 < stepcut> that function does it 'right'. Once it finds a type that matches, it does not try to recurse into the children of that type to find more.
18:55 < Saizan> uhm, why foldl?
18:57 < stepcut> dunno
18:57 < stepcut> I think I was copying some existing code or something
18:57  * stepcut does not use foldr as often as he should
18:59 < stepcut> Saizan: because 'everything' in Data.Generics.Scheme use foldl, and that was the basis of my function
18:59 < stepcut> and, I figure the people who wrote everything are smarter than me, so who am I to go changing their foldls to foldrs
18:59 < stepcut> :)
19:02 < Saizan> :)
19:02 < Saizan> i guess it's for returning results in order
19:03 < stepcut> could be
19:32 < stepcut> maybe alexj was the one talking about clusters so big you wouldn't bother with backing up to disk
19:36 < gwern> 'ANN: gitit 0.5' :)
19:37 < stepcut> :)
19:37 < stepcut> when will gitit have blogging support ?
19:38 < stepcut> I don't need a wiki. But a blog that I could post to via darcs would be nice. Especially if it supported wordpress templates
19:40 < gwern> dunno. what does blogging support mean?
19:41 < stepcut> hrm. An rss feed, comment posting, tags, a front page with the latest articles?
19:41 < stepcut> an cronological archive
19:42 < gwern> a chronological archive could just be a page with a list of links in order
19:43 < gwern> as for rss feed, well, that wouldn't be too hard to add with all the rss libs on hackage
19:43 < gwern> you could add that yourself
19:43 < gwern> comment posting? == the discussion page
19:43 < gwern> tags... well, I guess the wiki equivalent is categories. that's on the todo list anyway
19:44  * gwern needs to make a full list of all these ideas
19:46 < gwern> ah, rss is already in the todo
19:56 < sm> ikiwiki was great for blogging.. and aggregating
19:57 < sm> I already speak of it in the past tense.. sorry ikiwiki
19:58 < gwern> heh
19:59 < gwern> I always wondered whether 'ikiwiki' was meant to be prounounced 'icky'
20:03 < mae> gwern: I like your idea of gitit wiki
20:03 < mae> for documentation
20:03 < gwern> of course you do. only a fool could dislike it!
20:03 < mae> hehe
20:03 < mae> i just dislike the git part :)
20:04 < mae> but it is refreshingly pragmatic
20:04 < mae> I must say
20:04 < sm> it would be using darcs
20:04 < mae> I thought about doing a cms using mercurial as a back end before
20:04 < sm> gwern is not *that* pragmatic
20:04 < mae> : )
20:04  * gwern is actually very ideological when it comes to licensing
20:05 < mae> the mailing lists have died down a bit
20:05 < mae> I need some responses to my sheriff thread
20:05 < sm> gosh, look at the time
20:05 < gwern> mae: you're very fond of those grandiose names aren;t you?
20:05 < mae> gwern: why not it makes it fun, doesn't it ? :)
20:06 < sm> mae, shouldn't patches be showing up on the list ? you might get a bit of review
20:06 < mae> of course they will go on the list
20:06 < mae> but someone has to apply them
20:06 < mae> I can do that, but when I'm not around I shouldn't be holding things up
20:06 < sm> well that part's easy
20:07 < sm> can't those people you've joined to the repo do it, also ?
20:07 < sm> excuse me if these are dumb questions :)
20:07 < mae> well, they can yes, but I want to tighten down security, that is, the # of people applying to the repository
20:07 < sm> new project, new tools..
20:08 < mae> my first thought was to go bohemian style and give everyone commit access
20:08 < mae> but then when I saw the concern for destabilization of the codebase, I revised my road map
20:08 < mae> and with that I revised this bohemian committer plan
20:08 < mae> Instead we have sheriffs who will apply the patches
20:08 < mae> everyone goes through them
20:08 < mae> I thought it was pretty clear but I guess thats only in my own head :)
20:09 < gwern> that often happens to me
20:09 < mae> Well how could I clarify it then gentlemen? I really do need all your support :)
20:09 < gwern> I had a beautiful idea, and it comes out sounding like the black language spoken by the Orcs of Mordor!
20:10 < sm> heh
20:10 < sm> maybe their ideas were beautiful too
20:10 < sm> strange
20:11 < sm> mae: the mail is clear, I'm just not clear why the existing committers can't start.. you are calling for more experienced happs folk to step up ?
20:12 < sm> for me, the less formality the better.. until you need it
20:13 < mae> yes exactly
20:14 < sm> in the darcs project, we have one repo maintainer who applies patches, and the whole list is the reviewers
20:14 < sm> so it's up to the list how fast something gets approved (+ some amount of judgement by the maintainer)
20:15 < sm> if there's a bottleneck, everyone can see.. patches don't get stuck "in so and so's queue"
20:15 < sm> but your call may attract some good volunteers, you just sent it
20:19 < mae> well i wasn't thinking in terms of scaling at this point
20:19 < mae> I was more thinking that more people would be able to take an active part in maintaining the codebase
20:20 < mae> and also redundancy, other people can take on the role of "patch applier" besides one
20:22 < sm> whatever you think best.. I think the work is in patch review.. you really want as many eyes as you can get.. then applying is easy. Except coming up to release, when you need a strong release manager
20:22 < mae> right
20:23 < mae> well, I was just thinking in terms of going into short release cycle mode
20:23 < mae> this is what linux does
20:36 < stepcut> as part of the cleanup, I would like to see all functions have type signatures. It's hard to read the source with out them IMO
20:40  * sm agrees
22:54 < sm> hurrah! g'day, sheriff
23:36 < mae> stepcut: I agree, also stops you from accidental borks when refactoring functions
23:37 < mae> : )
--- Log closed Sun Jan 25 00:00:02 2009