Experimental IRC log happs-2007-07-30

Available formats: content-negotiated html turtle (see SIOC for the vocabulary)

Back to channel and daily index: content-negotiated html turtle

These logs are provided as an experiment in indexing discussions using IRCHub.py, Irc2RDF.hs, and SIOC.

03:03:25<shapr>yay, my new box builds happs in 2.5 minutes!
03:04:03<blackdog>schwanky.
03:04:35<blackdog>hey, does happs have a user agent for sending off requests to other servers?
03:07:23<blackdog>oops, of course it does. sorry.
04:03:57<sorear>shapr: How many for GHC? :)
04:15:04<shapr>Oh, haven't tried that yet!
04:15:36<shapr>Heck, I might be able to use 4gb on that...
13:47:58<dinoMI>msouth: Did you pull latest source of our thing? Do you have your own changes to push up to it?
13:48:57<dinoMI>Perhaps today I should change the response body to be an XML document.
13:50:37<msouth>dinoMI: doing my travel expense report, I'll come by right after that. how did the training go friday?
13:52:24<dinoMI>msouth: Like shit.
14:01:47<msouth>sorry to hear that
14:02:05<msouth>people not receptive?
14:08:40<dinoMI>msouth: Some of that maybe. But I fear more that I was really pretty awful with speaking to everyone.
14:08:46<dinoMI>First time for anything like that, so not unexpected.
14:09:48<dinoMI>I tried to show that kminima example. Maybe a bad idea. There's like 9 crazy concepts in that one line of code.
14:10:38<dinoMI>pair tuples, sortBy (comparing ...), function application.
14:10:47<dinoMI>People looked fearful when I tried to describe ($)
14:11:28<dinoMI>I did not try to explain (.)
14:11:31<msouth>I saw tlittle on saturday and he said he was impressed with $
14:11:44<dinoMI>I did say that it's like backwards shell pipes.
14:11:47<msouth>glad that he didn't have to deal with closing all the parens like in lisp
14:12:02<dinoMI>Yeah, this is the inheritance from ML and NOT LISP.
14:12:14<dinoMI>ML is syntactally like this, but impure and noisier.
14:13:05<msouth>andrew warmed up to the idea a bit on the trip
14:13:16<dinoMI>Yeah, he wrote me something about being excited about TM and the parallel stuff.
14:13:31<msouth>future still in doubt though, I would say
14:13:35<dinoMI>You know, I talked about the TM idea and parallel processing with Roger and Andrew both on my first day at this job out at lunch.
14:13:47<dinoMI>did not get received well at that time.
14:14:11<msouth>spj did a keynote on the STM
14:14:28<dinoMI>ya. I thought they were changing the name to "TM"
14:14:41<msouth>could be
14:14:56<msouth>andrew also went to the ndp talk
14:15:07<dinoMI>Also, the book that strike sent us email about, Beautiful Code, the SPJ chapter is all about TM.
14:15:23<dinoMI>ndp?
14:17:28<msouth>nested data paralellism
14:17:31<dinoMI>ah
14:17:45<dinoMI>Now, that's something different from (S)TM?
14:18:01<dinoMI>I'm not up on it all yet, just did light reading about (S)TM.
14:18:01<msouth>yeah
14:18:33<dinoMI>After Brad asked, I did show off the work we have so far with that soa-dict server.
14:18:44<msouth>it's a way to abstract out the mapping of data for parallel processing to different processors
14:18:48<dinoMI>Probably a good thing, to show something actually doing real work after all the abstract madness.
14:19:07<dinoMI>ah
14:19:09<msouth>I'm really sorry I didn't get more done while I was out there
14:19:27<dinoMI>Nah, I was busy being stressed out about Friday myself.
14:19:32<msouth>I think we need to do a database thing next
14:19:52<dinoMI>Ok. We will also need to reply in XML to whatever is asking.
14:20:11<msouth>my understanding is that the main thing to do there is figure out how to create the serializable thing that state is maintained by.
14:23:02<msouth>if we did something that took a user name and password and looked them up, pushed it up to six million users and tested the responsiveness, that would be a good thing
14:23:13<dinoMI>That's a different service, credential lookup.
14:23:23<dinoMI>Oh, for the actual credential lookup service. :)
14:23:56<dinoMI>Now, we want to use real SQL db? Or that spookypants S3 stuff?
14:24:04<dinoMI>Which you'll have to explain to me, what all that means.
14:26:02<dinoMI>When you say state, you mean happs state?
14:26:05<dinoMI>Why do we need that at all?
14:26:12<dinoMI>The db connection?
14:26:39<msouth>I meant no database
14:26:45<dinoMI>I see.
14:27:03<msouth>i guess I have a clearer picuture in my head of the value of happs when it doesn't need to talk to a database
14:27:04<dinoMI>Well, if we have more than like 8G data, we're going to screw the machine over.
14:27:09<dinoMI>I still don't get how that's going to work.
14:29:06<dinoMI>The way we've seen code changes make the state stale, we're going to need a way to rapidly populate the "db" when we have to delete the state dirs.
14:31:09<msouth>probably we'll have a script that reads the state in, converts it, saves it in a new directory, then we restart it with it pointing to the new directory instead of the old one.
14:31:41<msouth>maybe we only have uname, password, session information in the happs store, and all the profile stuff is maintained in a database.
14:31:44<dinoMI>Sounds like mucking in the way the happs does its internal stuff.
14:32:14<dinoMI>Probaby easier to make another happs thing that make http requests to the "real" one to feed data in a large loop.
14:32:35<dinoMI>mm
14:32:37<dinoMI>yeah
14:37:33<dinoMI>I don't know if the value is all in the storage here. There's value in the way happs allows you to code up these handlers and deal with the rq data.
14:37:54<dinoMI>I say that even though it's been difficult to get a handle on. Clearly the code we have now working is pretty terse and cool.
14:40:32<tuukkah>have you heard about the big changes ahead for happs?
14:43:37<dinoMI>tuukkah: Not really, no. I've been spending most of my time on the basics yet.
14:44:39<tuukkah>i don't know about them either, but we might want to change our plans when we hear about them :-)
14:45:05<tuukkah>something about making it easier to do io
14:45:32<tuukkah>but data structures that are stored partly on disk have also been mentioned
14:45:42<dinoMI>Hm. I was getting some help with the more modern happs API where IO was much easier than I originally thought.
14:46:44<dinoMI>Making handlers that :: .... -> m (Either Request (IO Result))
14:47:23<tuukkah>yeah, although i don't know if i can change state within the io part
14:47:38<dinoMI>Because of the two monads involved?
14:47:46<dinoMI>s/two/more than one/
14:48:17<tuukkah>yes, as the io is run outside the state
14:48:17<dinoMI>Apologies if I ask something stupid, sadly still have much to learn about monads.
14:48:31<dinoMI>mm
14:49:38<dinoMI>Ok, did some quick reading about S3. Sounds neat. But also, and I guess I show my age here, is anyone scared of storing confidential data with these big guys like G and A?
14:50:41<tuukkah>perhaps you could encrypt the confidential parts?
14:53:37<dinoMI>Google would really get my attention if they buy Sealand, heavily arm it and put the hosting in it.
14:54:15<tuukkah>wouldn't it be all too small for them ?-)
14:54:39<dinoMI>Probably. They can build a moon-based facility then. Armed with defensive lasers.
14:56:05<dinoMI><- paranoid
14:56:10<tuukkah>=)
14:56:38<dinoMI>I do nightly backups of all systems at home and monthly backup DVDs locked in a safe.
14:57:43<tuukkah>that doesn't sound paranoid
14:58:17<tuukkah>i hope the safe is at least in a different city
15:00:05<msouth>tuukkah: the paranoid part is where he has the safe suspended over a vat of acid that he can drop it into if they do a dawn raid
15:00:23<tuukkah>msouth, i see :-)
15:01:59<msouth>week before last they were saying that they hoped to get the road map out last week.
15:02:30<msouth>now I'm going to start asking for a road map to the release of the road map
15:02:48<dinoMI>msouth: heh
15:03:09<msouth>:). Or I could help, I guess.
15:03:19<msouth>but at this point I would probably only set things back.
15:03:55<tuukkah>msouth, shapr said yesterday that the roadmap would very likely be out this week
15:03:59<shapr>right!
15:04:14<tuukkah>shapr :-)
15:06:28<tuukkah>girlfriend-caused setbacks =)
15:07:01<msouth>dinoMI: the thing is, re your terse-and-cool comment above, that soa-dict thing could be done in a lot fewer lines in perl, I think.
15:07:20<dinoMI>Really? Even with reading the file?
15:07:28<dinoMI>Half that code is not happs.
15:08:22<shapr>tuukkah: yes :-)
15:08:22<dinoMI>And that machinery to dispatch to handler based on URI/method, does that readily exist in Perl?
15:09:43<tuukkah>dinoMI, you would use a framework in perl too, like catalyst
15:09:58<msouth>dinoMI: you can get an Apache object that will give you everything about the request
15:11:38<msouth>If there are problems that will fit into memory, or if the new stuff will have a way to not have that restriction or whatever, then we could get significant complexity reduction by not having to do the language<->db translation.
15:12:32<msouth>that's a pain in the butt in any framework, trying to map it well onto the database's idea of how data should look.
15:12:55<msouth>so I was looking at this as a way that we could just not have to worry about that at all.
15:13:58<dinoMI>I mean no happs disrespect, but I'm skeptical. I wonder if we can dump the entire data store to text like you can with SQL tables, for instance.
15:14:18<dinoMI>Or go the other way to populate
15:14:19<dinoMI>.
15:15:34<shapr>What about dumping to bytestrings or something?
15:16:29<shapr>Read/Show isn't fast, but Data.Binary is very fast as compared to read/show.
15:17:01<shapr>I admit, I am stung by the idea that perl might be more concise... if it really is, I'd like to fix that.
15:17:37<msouth>shapr: let me look, it's been a while since I have looked at the code.
15:19:29<dinoMI>A lot of stuff is like that with Perl. That Quicksort example from _Why Haskell Matters_ can be done in one or two more locs than Haskell, basically because you don't have the gorgeous pattern match binding syntax.
15:19:58<dinoMI>Perl can be very list-oriented.
15:20:43<shapr>I sure liked Pugs.
15:21:05<shapr>I only played with it for a few days, but it was very nice.
15:21:19<shapr>Audrey is amazing.
15:21:51<dinoMI>Sure is. :o I wonder if the whole Parrot thing could be done with Haskell.
15:22:17<shapr>The VM part?
15:22:21<dinoMI>ya
15:22:28<shapr>hm, yeah it could
15:22:36<dinoMI>Not just P6, but the whole ball of wax.
15:23:35<shapr>Yeah, there would be major advantages to doing it that way, but unfamiliarity could be a big disadvantage.
15:24:24<dinoMI>shapr: What you said earlier, yes we could write stuff to extract/populate data. But db engines have this nut cracked in some ways, is why I feel scared turning away from them.
15:24:57<dinoMI>They deal with failover, scalability and concurrent data access, abstracting the data away from apps, etc..
15:25:12<shapr>In that case, I can barely wait till the roadmap comes out...
15:25:21<dinoMI>ya
15:25:33<shapr>Since failover and scalability are directly addressed.
15:25:39<dinoMI>cool
15:32:12<msouth>dinoMI, shapr: this is approximate, I haven't even tried to compile and I don't exactly remember the Apache syntax, but it's something like this: http://pastie.textmate.org/83482
15:34:24<shapr>Hm, I should try that...
15:35:53<shapr>It'd be interesting to get a more detailed comparison.
15:35:56<msouth>although if you wanted stuff to be dispatched to a handler succinctly you would probably do it in mod_perl. Apache would handle the url matching (and I think splitting on request method too, not sure)
15:36:35<shapr>With HAppS, I'd suck up the dictionary into a Haskell ADT and use that.
15:38:11<msouth>in perl you could just suck it into a hash (which would be one line, and run at server startup) and do "exists $DICT{$word}"
15:39:09<msouth>but this isn't really where you want to do battle with perl
15:39:46<shapr>Where then?
15:39:48<msouth>perl is going to win for a lot of small things in terms of being concise and expressive
15:40:58<msouth>well, I would say that the more complex the algorithms are, the more haskell's ability to express complex things succinctly is going to start winning out
15:41:14<msouth>also, the peace of mind about knowing that your code is side-effect free.
15:41:43<shapr>yeah
15:42:15<msouth>for example, I went to the higher order perl talk at oscon that mjd did, and it was immediately clear to me that there were things you just wouldn't have to worry about getting right if you were doing it in haskell instead of perl.
15:42:50<msouth>(Mark Jason Dominus has a book called "Higher Order Perl" where he basically presents the results of looking at perl lispishly)
15:43:15<dinoMI>msouth: I'll not that I could have made the Haskell code a lot shorter by doing string constructions "inline" instead of writing the format function. And also doing the same thing with the file lookup.
15:43:20<shapr>Yeah, I talked to him when he showed up on the #haskell channel for awhile.
15:43:39<shapr>I also joined an extreme programming in Perl list for a few weeks.
15:43:40<dinoMI>msouth: So, not so short if you actually wrote it the same way in Perl, with separate functions for each of those things.
15:43:46<msouth>On one slice he asserted that you could do X and it wouldn't change Y, and I was like "really?". I was looking at the code, and there was so much trickery in it that I would have had to run it in the debugger to be sure what it was doing.
15:44:08<msouth>but in haskell I would know that the stuff he said couldn't change, couldn't
15:44:18<shapr>Yeah, Perl was designed to leverage knowledge of awk, sed, and all the sharp unix tools.
15:45:05<msouth>he spoke highly of haskell in his talk
15:45:15<shapr>He's a smart guy ;-)
15:45:34<msouth>we could turn him
15:45:35<shapr>But seriously, it was fun talking to him when he hung out on #haskell for awhile.
15:46:21<dinoMI>Be interesting to see what happens in the future.
15:46:44<msouth>perl6 is going to be a very interesting beast
15:46:47<dinoMI>As I understand it, P5 is not going to be able to deal with threading well. Everybody says the VM code is madness.
15:46:49<shapr>Yeah, I'd be happy if Haskell becomes a popular language. I'd also be happy if most of its features show up in other languages.
15:47:06<dinoMI>On the other side, I personally feel like they worked too much on OO in P6 and not enough on making it functional.
15:47:17<shapr>You could always tell Audrey :-)
15:47:49<shapr>Write up a clear description of your thoughts and email it to audrey, I'm sure she'll listen.
15:47:53<shapr>Audrey is an amazing person.
15:49:00<dinoMI>Very disappointing to me to see the partial-eval notation in P6. I feel it's really awful. It works by creating a dependency on the original function impl's bind variable name strings.
15:49:19<shapr>I haven't seen that.
15:53:40<msouth>one thing is that larry wants it to be as understandable as possible to novices. so stuff like you have in haskell that is very elegant but takes a long time to wrap your head around is less likely to show up in perl.
15:54:35<msouth>on the other hand, perl tries to be good about adopting good ideas from everywhere, so one can imagine that side-effect-controlled code might show up in it.
15:55:40<shapr>Yeah, audrey is putting a lot of cool stuff.
15:57:01<dinoMI>Unfortunately, the imperative nature at the core of a given lang -- difficult to un-imperative it.
15:57:11<dinoMI>Maybe easier to go the other way a-la monadic sequencing.
15:58:24<dinoMI>msouth: I will find the partial-eval example after lunch. It looks to me like modifications to the var names used in a function will break all consumers who are using .assuming
15:58:38<dinoMI>I should try it in Pugs, if .assuming is implemented.
15:59:30<dinoMI>Maybe it's also possible to use .assuming without a hash, positions of args.
16:01:50<dinoMI>ACTION goes to lunch.
16:09:33<kaol>I hope my IO code won't be made useless by that new branch that I keep hearing of...
16:10:36<kaol>ACTION happily does almost everything after a respond $
16:28:03<shapr>I'll see if I can get the usage examples cleared for release.
16:34:03<kaol>I'm keeping my code in a public darcs repo. Mind taking a look? Is what I'm doing so far anything that would still work after the new branch has been brought in?
16:38:44<kaol>though I guess my code's uninteresting
16:46:09<shapr>Sure, where's the repo?
16:47:50<kaol>darcs get http://piperka.net/darcs/
16:48:48<kaol>the code can't really be ran without the DB, but it's correct at least
17:11:47<shapr>ACTION installs hdbc
17:22:59<dinoMI>ACTION has returned.
17:30:10<msouth>dinoMI: modifications to var names in functions breaking things is pretty similar to modifications of positions of arguments breaking things right?
17:30:39<msouth>but I agree with you on the fact that haskell's currying is much more elegant
17:30:59<msouth>but normal humans are probably going to be able to "get" the .assuming() syntax.
17:33:36<dinoMI>msouth: here: http://pastie.textmate.org/83512
17:34:11<dinoMI>msouth: I actually think it's similar but worse to create a string literal dependency. But perhaps that's just our positional-args upbringing.
17:34:32<dinoMI>I was able to make our happs thing less LOC than that Perl from earlier.
17:34:39<tuukkah>("given" would sound less uncertain than "assuming" :-)
17:35:01<dinoMI>By doing the same things: no separate functions
17:36:18<dinoMI>msouth: One might also argue that var names being significant for partial application breaks the Principle Of Least Surprise.
17:36:26<dinoMI>partial eval, I mean
17:37:03<msouth>i don't know--if it's called with names, I would expect the names to be part of the api
17:37:24<dinoMI>I was just going to say, sounds like the prevailing mentality will be "stop calling anything ever without a hash"
17:37:32<shapr>Hm, MissingH won't build...
17:38:47<msouth>tuukkah: "given" does sound like a better word to me
17:46:44<kaol>shapr: it's enough if you remove import MD5 and the call to md5s (if you're still trying to check my code)
17:55:07<dinoMI>msouth: I do think normal humans can get this syntax: (+ 5) is addition with one arg supplied.
17:58:35<dinoMI>Or (functionThatTakesTwoArgs "Dino") it looks like a function call already and doesn't have any special keywords.
19:03:27<kaol>generic-xml and syb-with-class tarballs could really use a license text along with them. I know that it's fairly obvious what BSD3 means, but I still can't upload them to Debian without one.
19:04:35<kaol>I could email alexj and ask if the same license text that happs itself has applies to them too and put that in debian/copyright, but having it in the tarballs themselves would be even better
19:05:28<kaol>we're rather pedantic about licensing. ;-)
19:16:11<kaol>ACTION emailed

Back to channel and daily index: content-negotiated html turtle