Experimental IRC log happs-2007-11-07

Available formats: content-negotiated html turtle (see SIOC for the vocabulary)

Back to channel and daily index: content-negotiated html turtle

These logs are provided as an experiment in indexing discussions using IRCHub.py, Irc2RDF.hs, and SIOC.

11:02:52<Sizur>hslogger has problems with the new cabal so i cannot install it. HAppS-Util depends on it. what to do?
11:11:18<Sizur>ok, i got that solved. now im' building HAppS-Util and i get Could not find module `Data.Array.Unboxed' but i have just installed array-0.1.0.0
11:12:24<Sizur>correction: sjanssen got the cabal conflict solved
11:12:39<Saizan_>Sizur: the .cabal descriptions have not been updated to work with 6.8.1
11:12:53<sjanssen>Sizur: add array to build-depends in the .cabal file
11:13:16<sjanssen>Sizur: this will be a problem with many third party packages for a little while
11:13:25<Sizur>sjanssen: kk thanks
11:23:19<Sizur>ok, i have another problem base-3 does not have Data.ByteString.Base that's needed by Util
11:24:20<Sizur>neither does bytestring package
11:25:53<Saizan_>ah, that module has been renamed to .Internal, (however Igloo has yet ported everything to 6.8.1, he just need to push the patches afaik)
11:39:56<Sizur>thanks for tips! makign progress here... now i have Data/Generics/SYB/WithClass/Basics.hs:27:17: parse error on input `*'. should i simply wait until the patches will merge?
11:48:21<Saizan_>maybe yes, i'm not sure how to fix that, maybe with data Proxy a = forall c. P (a c)
11:56:08<Sizur>that worked, i had to enable MultiParamTypeClasses
12:22:10<Sizur>how about this one: http://pastebin.org/7318 ?
16:50:28<shapr>good morning!
19:01:21<perspectivet>I've got a happs question
19:02:09<perspectivet>I haven't found anything that uses the happs UDP client-server stuff.
19:03:07<perspectivet>what's it used for that I haven't found or supposed to be used for that is isn't currently implemented?
19:30:06<shapr>It was used in a previous project.
19:35:00<perspectivet>what was it used for in that previous project?
19:35:40<shapr>pass.net was using UDP for faster connections.
19:36:03<perspectivet>ah, ok.
19:38:02<perspectivet>So the Multi-master replication happens over custom TCP protocol?
19:38:15<shapr>Yup, multimaster is over TCP.
19:39:21<perspectivet>ok, I'm a little fuzzy on the terms, but you guys are planning on adding some stuff for sharding in the near future?
19:40:01<shapr>Yeah, it's in the queue after multimaster.
19:41:01<perspectivet>ok, here's where my fuzziness is going to get me into trouble...
19:41:29<perspectivet>presumably there will be some communication mechanism for coordination between the shards, right?
19:42:00<shapr>I assume so.
19:42:48<shapr>This is where you should talk to alexj :-)
19:43:07<perspectivet>yoohoo, alexj
19:43:37<alexj>hey
19:43:49<perspectivet>from what you know, would it be reasonable to build on top of the multimaster protocol or would they be mostly independent
19:44:03<perspectivet>alexj: wow. I can't believe that worked
19:44:09<alexj>?
19:44:16<perspectivet>my yoohoo
19:44:31<shapr>ACTION grins
19:44:40<alexj>my presence here is intermittent, but my commitment is high :-)
19:45:00<perspectivet>happs is testament to that I would say.
19:45:01<alexj>am writing up an email about multimaster for the happs list right now.
19:45:30<perspectivet>My question really boils down to this:
19:46:21<perspectivet>I'm building a fairly tightly coupled distributed system and I need an efficient simple Haskell-friendly system to my IPC
19:46:47<perspectivet>I'd like to avoid doing my own TCP protocol if possible.
19:46:56<alexj>have you looked at spread?
19:47:20<perspectivet>do you have a link?
19:48:09<perspectivet>googling spread + haskell doesn't turn up anything that looks like what I expect you're talking about.
19:48:24<alexj>spread.org
19:48:35<perspectivet>ouch
19:48:41<perspectivet>ok, should have tried that first
19:48:42<alexj>saizan is working on a haskell client to spread right now.
19:49:08<alexj>were you writing your own alternative to spread?
19:49:46<perspectivet>Spread looks like what I would want to build on.
19:50:17<perspectivet>I'd be interested in getting in on the haskell client dev
19:53:16<alexj>right now its Saizan's project. I think we started on it yesterday.
19:53:26<alexj>Saizan_: you here?
19:54:18<perspectivet>ok
19:54:25<alexj>what do you want to do?
19:55:00<alexj>my general plan is to use spread to implement multimaster and then sharding.
19:55:32<perspectivet>yeah, looks like that would be a good way to go
19:56:08<perspectivet>alexj: you're question is, what problem do I want to solve or what part of the work do I want to help with?
19:56:10<Saizan_>i'm here
19:56:31<alexj>perspecttivet: both
19:57:02<Saizan_>i'm reading the user guide at the moment, to have a better idea of how to structure the api for haskell
19:58:09<alexj>perspectivet: where are you located?
19:58:19<perspectivet>vancouver, bc, canada
19:58:31<alexj>ah we are in the same timezone.
19:58:38<alexj>i love vancouver.
19:58:38<perspectivet>alexj: for the former, let me take a look at the spread manual. I should be able to comment better after that.
19:58:44<perspectivet>alexj: me too :)
19:58:51<perspectivet>alexj: your in sf?
19:58:53<alexj>yes
19:58:55<perspectivet>*you're
19:58:58<perspectivet>blech
19:59:24<perspectivet>I've only spent time in sf for work. Haven't really gotten to know the city too well.
19:59:50<alexj>are you using happs for work?
20:00:30<perspectivet>I'm planning on building my next project with it.
20:00:37<perspectivet>but yes, for work
20:00:44<perspectivet>for my company specifically
20:03:59<perspectivet>but the spread stuff is stuff would be used to coordinate pieces of what is effectively a custom cluster database for a particular query type
20:04:39<perspectivet>can't really say more than that
20:04:54<alexj>is it your company?
20:04:59<perspectivet>yes
20:05:14<alexj>cool
20:06:11<alexj>funded?
20:06:15<perspectivet>not yet
20:06:22<perspectivet>proof of concept
20:06:40<perspectivet>hence the desire to reuse as much existing stuff as possible
20:07:16<alexj>you raising money yet?
20:07:44<perspectivet>as soon as the first part of the proof of concept is finished I will start with the fundraising
20:08:37<perspectivet>which, if all goes well, should be middle/end of jan next year
20:08:48<alexj>ok cool. it would be really neat to invest in a company that is actually using haskell (and even happs). i'd love to be on the distlist when you start that process.
20:09:09<perspectivet>definitely
20:09:12<alexj>(I am an angel in a few other companies)
20:09:25<perspectivet>cool
20:09:48<perspectivet>I could always use an angel :)
20:10:01<perspectivet>atheism aside
20:11:09<perspectivet>but yeah, I'm planning to make heavy use of both haskell and happs
20:11:26<alexj>there are no atheists in the trenches.
20:11:33<alexj>very cool.
20:11:36<perspectivet>hehe, nice one
20:21:46<shapr>hah
20:23:39<shapr>Hi Mark
20:23:43<shapr>How's code?
20:23:44<markluffel>hello
20:23:48<markluffel>hmm?
20:23:57<markluffel>are we acquainted?
20:24:03<shapr>I dunno, maybe.
20:24:16<markluffel>is searchpath worth using?
20:24:17<shapr>If you've been in the Haskell community much, you probably know me.
20:24:29<shapr>I'm Shae Erisson, disorganizer extraordinaire!
20:24:56<markluffel>yeah, i recognize your handle, i'm a lurker though
20:25:12<shapr>Anyway, I was just asking how HAppS was treating you, if you had any questions, problems, that sort of thing.
20:25:34<markluffel>oh, well i tried installing it two months-ish ago
20:25:47<markluffel>and could get all the dependencies working
20:25:49<shapr>I put some new instructions on the happs.org frontpage today, want to try them?
20:25:59<markluffel>oh, that's new today
20:26:03<markluffel>is was just looking
20:26:18<markluffel>had used cabal for dependencies before,
20:26:23<markluffel>is searchpath the future?
20:26:40<shapr>I don't know.
20:26:54<shapr>It's definitely the future for HAppS.
20:26:54<kaol>does it authenticate what it downloads?
20:27:16<shapr>No
20:28:27<alexj>kaol: it calls out to curl so it does whatever curl does.
20:28:51<alexj>if you used map files and repos hosted via https then you would get curls handling of bad certs.
20:30:04<alexj>kaol: does that make sense?
20:30:22<markluffel>the searchpath instructions could include "cd HAppS-Begin" before "sp runhaskell -ihaskell haskell/Main.hs"
20:33:18<markluffel>also, would prefer names of modules downloading rather than progress fishes (or in addition to fishes)
20:40:06<kaol>It's just me... I tend to get twitchy when a piece of software downloads other software automatically. Though, in this case, it's just source code and anyone who cares can read through it themselves. And I suppose auditing Haskell code is a lot more pleasant than anything written in just about anything else.
20:40:37<kaol>assuming one knows haskell reasonably well
20:47:39<markluffel>how long should i expect the happs compile to take on a recent core 2 duo?
20:47:44<markluffel>im at: " Warning: `mixCommands' is exported by `module HAppS.State.EventTH' and `module HAppS.State'"
20:48:20<alexj>markluffel: what are you importing?
20:48:31<alexj>You probably just want to import HAppS.Server.State
20:48:56<markluffel>i'm following the instructions from happs.org
20:49:08<markluffel>building HAppS-Begin
20:49:22<alexj>shapr: can you modify HAppS-Begin to use the new stuff in AllIn?
20:49:30<shapr>Yeah
22:02:47<perspectivet>Saizan_: have you gone over the user guide?
22:07:16<markluffel>shapr: i canceled the HAppS-Begin build (a few hours ago), will "the new stuff in AllIn" make the compilation faster/possible?
22:09:13<shapr>It's not in yet, I'm still working on docs.
22:09:21<shapr>Is the build broken?
22:09:26<shapr>ACTION checks
22:09:32<markluffel>well, i couldn't get it to complete
22:09:49<shapr>Let me try to build it first... if it works for me I'll ask you for the error.
22:10:28<markluffel>i didn't see an error, but it was taking more that 45minutes, and not showing any progress
22:10:45<shapr>That's really weird.
22:11:21<markluffel>it was churning on something, turning on my fan, not paging though
22:14:19<shapr>um, huh...
22:15:41<shapr>I think some of the cabal packages are missing exports or deps.
22:15:50<shapr>ACTION goes bughunting
22:17:25<shapr>Yup, looks like HAppS-State is missing something.
22:26:51<Saizan_>perspectivet: yes
22:28:37<alexj>happs-begin is building for me using searchpath.
22:28:52<alexj>markluffel: how were you building?
22:29:25<markluffel>sp runhaskell -ihaskell haskell/Main.hs
22:30:03<Saizan_>markluffel: did you check if the server was running?
22:30:04<perspectivet>Saizan_: I didn't give it a thorough going over, but I'm comfortable I understand the arch, and basics of the api.
22:30:41<perspectivet>Basic support would be pretty simple
22:31:44<perspectivet>just skimming ocamlspread right now
22:32:43<Saizan_>uuh, where is it?
22:32:59<perspectivet>http://sourceforge.net/project/showfiles.php?group_id=59814&package_id=55985&release_id=111221
22:33:18<perspectivet>pretty old. written for 3.16.2
22:34:26<Saizan_>ah, i was looking at the java code, to port the lowlevel part of the protocol
22:35:39<perspectivet>Yeah, I was thinking of wrapping the c api
22:36:16<Saizan_>with FFI?
22:36:25<perspectivet>yeah
22:36:58<Saizan_>i think it's not worth it, the structure of the messages is very simple
22:37:13<Saizan_>even if there's an ugly bitfield
22:38:13<perspectivet>I haven't done any TCP stuff in haskell so I was thinking about the lowest-risk path
22:38:50<perspectivet>but if you want to take the lead on a haskell-native spread binding I'd help where I can
22:40:36<Saizan_>sounds good :)
22:41:33<Saizan_>mmh, we need a repo then, but first a design of the api
22:41:47<perspectivet>is there any advantage to using network-bytestring over network?
22:44:31<perspectivet>Does there need to be much more design than just a 1-1 mapping between C client functions and the 0th layer of haskellspread
22:44:54<Saizan_>by network-bytestring you mean a specific package or simply read from a network handle with bytestring primitives?
22:45:04<perspectivet>the latter
22:45:40<perspectivet>but I assumed the latter required the former (I haven't looked into it enough to know either way)
22:46:30<perspectivet>but continuing on, I figure I would want to build an RPC type layer on top of the 0th layer
22:46:44<Saizan_>well, with standard IO you get a [Char] which is space consuming and not suited for an octect-stream
22:47:46<perspectivet>Yeah, that was what I understood from the blog bits I read about the problem with Network.HTTP
22:51:09<perspectivet>Is there a replacement for network.socket that uses Data.ByteString?
22:53:40<Saizan_>i don't think so, but we can use socketToHandle and then read from the handle
22:55:33<perspectivet>ah, cool, didn't see that first time through Network.socket
22:56:49<perspectivet>so do you think the Oth layer requires more than a reimplementation of the C api?
22:58:45<Saizan_>maybe less
22:59:31<Saizan_>well that depends on what we think the 0th layer is
23:00:25<Saizan_>i imagined that for each connection you'd have a thread fetching messages from the net and putting them on a Chan
23:00:33<Saizan_>where others can listen on
23:01:09<Saizan_>then you have to keep track of the joined channels
23:01:21<Saizan_>s/channels/group
23:01:26<Saizan_>s7
23:02:51<perspectivet>one Chan for every joined group then
23:06:15<perspectivet>But I was thinking above that you would provide something like the c client api
23:06:55<Saizan_>ok, i'll look more at it then
23:06:59<perspectivet>the Chan per group construct would be required to enforce the ordering semantics between threads
23:08:01<perspectivet>ie for the c api if one thread read and get message X, the next thread would read the message after X.
23:08:08<perspectivet>*read and got
23:10:35<Saizan_>yes, and you can also dupChan it so everyone get every message if you want
23:10:47<Saizan_>the C api is essentially receive
23:11:02<Saizan_>looking at the manual
23:11:49<perspectivet>receive + 4 group and system management functions
23:12:50<perspectivet>hmmm, yeah the dupChan effect would be pretty nice
23:13:02<Saizan_>join,leave, and [group]multicast?
23:14:33<perspectivet>there's also the connect + disconnect, but those could potentiall be managed internally
23:14:52<perspectivet>I was calling those 2 system management
23:15:31<perspectivet>and I was pairing the [group]multicast stuff with the receive
23:18:05<perspectivet>so operating at the chan level, the chan's are typed
23:19:41<perspectivet>so we need a nice way to construct message type that takes care of whatever is needed marshall and unmarshalling
23:19:49<perspectivet>I don't have any ideas on how to do that
23:21:10<Saizan_>you just need a pair of functions :: Message -> ByteString , ByteString -> Message
23:22:13<perspectivet>Yeah, it would be nice to have those constructed automagically by the api
23:22:42<perspectivet>similar to the template haskell haxr impl
23:23:09<perspectivet>grinding out trivial mapping functions makes me cranky
23:23:19<Saizan_>heh :)
23:23:56<Saizan_>Data.Binary could help
23:24:38<Saizan_>the C api is way cleaner than the java one..
23:25:04<perspectivet>Is that possible? C is so much more primitive than java ;)
23:26:03<perspectivet>so in some ways this almost turns into a group == RPC function type system
23:26:58<perspectivet>so that you would have one group per remote function
23:27:30<Saizan_>i'm not sure where RPC enters here
23:28:08<perspectivet>well ok, don't want to get off track but...
23:28:49<perspectivet>It appears that each group would effectively have a single message type
23:29:29<Saizan_>from what?
23:31:08<perspectivet>ok, perhaps I don't understand chan properly, but it appears to me that each chan can only be used for a single type
23:31:36<Saizan_>yes
23:32:22<Saizan_>but you are also saying that messages from different groups have different types?
23:32:30<perspectivet>so unless we have a spread group mapping to more than one Chan we can only send one type of message
23:32:57<perspectivet>That type could be fairly complicated, but it would be a single haskell type
23:34:03<Saizan_>what do you mean by "type of message"? are you referring to the mess_type field?
23:34:52<Saizan_>or regular vs. membership type?
23:48:15<perspectivet>Saizan_: Sorry, on the phone
23:49:24<Saizan_>k
23:49:36<perspectivet>by type of message I mean our top-level haskell message type
23:49:45<perspectivet>So lets say:
23:50:49<perspectivet>data foo = { Int a } deriving (Eq, Show, SpreadMessage)
23:51:17<perspectivet>oops
23:51:26<perspectivet>s/Foo/foo
23:52:55<Saizan_>data Foo = Foo { a :: Int } ... you mean?
23:53:16<perspectivet>yeah, sorry
23:53:17<Saizan_>a is supposed to be the body of the message?
23:53:45<Saizan_>or what?
23:54:06<perspectivet>I'm thinking that Foo would be some type that we used at the application level
23:54:49<perspectivet>we would provide the Foo -> ByteString and ByteString -> Foo functions
23:55:16<Saizan_>ah, i see you're already thinking about serialization of custom datatypes inside messages?
23:55:39<perspectivet>yes, I'm a top-down kind of guy
23:57:06<Saizan_>ok, i was still supposing the Chan contained a record Message with the various fields you get from receive, and the actual data still in ByteString form
23:58:30<perspectivet>so then it would be up to the API user to parse the message contents in the Bytestring
23:59:40<perspectivet>that's probably a good place to start in all fairness

Back to channel and daily index: content-negotiated html turtle