03:04:11 <McManiaC> http://blog.n-sch.de/2010/11/21/work-in-progress-monad-comprehension-1/ :D
03:05:11 <stepcut> nice
20:28:37 <sannysanoff> Hello gentlemen, I am going to implement file upload handler, on server. I want to monitor file size progress as it being saved in temp dir. To do this, I am going to modify code in ./happstack-server/src/Happstack/Server/Internal/Multipart.hs and around this. Should I copy/paste it and do it side-by-side or my patch is going to be incorporated? Who do I talk to about design of this feature?
20:29:35 <stepkut> you can talk to me
20:29:59 <stepkut> why do you want to monitor the file size ?
20:30:43 <sannysanoff> Ok, as file is being uploaded, my ajax client would ask server to report progress to client. I want to do use it in this ajax app. I am going to upload large video files.
20:30:52 <stepkut> that seems sensible
20:31:01 <stepkut> the client does not have that information already available client-side ?
20:31:28 <sannysanoff> in modern browsers, answer is "probably". I don't want to use it this way.
20:31:40 <stepkut> ok
20:32:33 <sannysanoff> so, I would think about adding notification handler to BodyPolicy, but in case when policy is shared (constant), this may not be the best Idea.
20:33:24 <sannysanoff> or it is? as in:    inputs <- bodyInput (myBasePolicy { handler = mySpecialHandler .. }) req
20:35:21 <stepkut> In BodyPolicy, you can define your own InputIter. It sounds like you want to use a modified version of defaultInputIter... That would allow you to achieve what you want with out having to modify happstack itself.. though the feature you want may be worth adding to happstack.
20:35:51 <sannysanoff> yes, I can, but I don't want to copy/paste around 100 LOCs
20:36:20 <sannysanoff> I have enough pain of maintaining modified packages (own forks)
20:37:03 <stepkut> defaultInputIter is only 38 lines of code. Unless modifications go beyond that... but let's talk about the actual modifications you want to make
20:37:40 <stepkut> so, in order for the client to get the progress update, it is going to have to make a separate HTTP request. So, this means you also have some idea in mind how to share that information across requests ?
20:37:44 <sannysanoff> let me see. The notification handler  must be called in hPutLimit' (inside)
20:38:23 <stepkut> when in hPutLimit? Can't it just use the value return by hPutLimit ?
20:38:29 <sannysanoff> yes, maintaining all infrastructure (tickets) is separate issue, orthogonal to the main problem ;)
20:38:30 <stepkut> (trunc, len) <- hPutLimit (maxDisk - diskCount) h b ?
20:38:55 <sannysanoff> main iteration (chunked write to disk) happens in hPutLimit' (inside)
20:39:22 <stepkut> ah right, hPutLimit will not return until the whole thing is saved
20:40:16 <sannysanoff> another option is to notify the handler about creation of temp file, and then the monitor task is able to check file size.... but I like it less.
20:40:45 <stepkut> how come? Doing a stat in another thread seems a lot less complex?
20:42:04 <sannysanoff> doing stat looks not elegant, although it doesn't require modification of depths of hPutLimit' ...
20:42:23 <sannysanoff> extra syscall, anyway :)
20:42:57 <stepkut> yeah, but that extra syscall could still be a lot less overhead of maintainig a running counter
20:43:24 <sannysanoff> nope, IORef is cheaper, isn't it?
20:43:36 <stepkut> so, you want to modify hPutLimit' so that takes another argument (Int64 -> IO ()) which gets called after every hPut ?
20:43:44 <sannysanoff> yep.
20:44:14 <stepkut> sannysanoff: depends, you are going to modify that IORef everytime you write a chunk. But an ajax request might only happen once per second (or even less frequently)
20:44:52 <sannysanoff> yes, I agree. Having this in mind, please advise.
20:45:06 <stepkut> and then defaultInputIter would take an argument like (FilePath -> Int64 -> IO ()) ?
20:46:07 <sannysanoff> I am not sure I saw the mime-part size field parsing in code, but seems correct.
20:46:38 <stepkut> oops, the type I meant was, FilePath -> IO (Int64 -> IO ()), actually
20:46:57 <stepkut> assuming your callback might want to know the name of the temp file for some reason
20:47:50 <sannysanoff> hmm. Interpreting... For stat, it definitely wants the filename. but I cannot comprehend this signature.
20:48:19 <sannysanoff> my callback returning callback with int parameter/
20:48:19 <sannysanoff> ?
20:48:21 <stepkut> yeah a name would help, mkCallback :: FilePath -> IO (Int64 -> IO ())
20:48:50 <stepkut> that is just a function that generates the callback that hPutLimit would call
20:49:38 <sannysanoff> hpaste.org helps, i will do there.
20:51:27 <stepkut> so, what if instead, defaultInputIter took an argument like, (FilePath -> Int64 -> IO (Bool, Int64)), so that you could just pass in your own custom version of, openBinaryTempFile >> hPutLimit >> hClose?
20:51:49 <sannysanoff> http://hpaste.org/41681/modification_of_defaultinputit
20:52:35 <stepkut> so far that notifyHandler, you are only getting the name of the tmp file ?
20:53:00 <sannysanoff> this is minimalistic version, of course your idea with custom version of .... seems more flexible etc.
20:54:30 <sannysanoff> but in your case, you forgot Lazy bytestring in args, no?
20:55:04 <stepkut> I did .. I remember but somehow it did not actually make it into the type sig
20:55:46 <sannysanoff> and also, it wants to return filename too.
20:55:57 <sannysanoff> should I do this way?
20:58:46 <stepkut> what you really want is the arrow version of RqData :-/
20:59:24 <sannysanoff> ah, these things are called arrows ;) . Seriously, I don't think i got them right yet.
20:59:39 <stepkut> no.. arrows are different
20:59:59 <stepkut> I am not sure what 'these things' refers to.. but there are current no arrows in happstack
21:00:06 <sannysanoff> i know.
21:01:23 <sannysanoff> it was a joke about these things, anyway. So, should I do it this way? Or we want to generalize it even more?
21:01:39 <stepkut> I'm not sure yet
21:03:05 <stepkut> your solution solves your particular issue -- it provides a callback that tells you the name of the temp file (and maybe also how many bytes have been transferred so far). A more general solution is to allow the user to supply a custom 'saver' function. In which case, you could just provide your own custom saver that tracked the bytes.
21:03:31 <sannysanoff> yes, I am talking about implementing the latter.
21:05:12 <stepkut> right now I am leaning towards allow the user to supply a custom saver. And the defaultSaver would just be those three lines we have now
21:05:33 <stepkut> In the arrow version this is a lot more straight-forward
21:06:27 <stepkut> there are no tmp files, etc. The file gets saved directly to the final destination. And the saver function would be (L.ByteString -> IO ()) I think.
21:07:02 <stepkut> no, L.ByteString -> IO (Maybe (String, Input String))
21:07:15 <stepkut> but, it could then do things like save the files to S3, etc.
21:08:03 <stepkut> but, it needs some work still
21:08:31 <sannysanoff> yes, becasue we're passing some RamLimits which want to be used, etc.
21:08:53 <stepkut> anyway, if you submit a patch that adds defaultInputIter', defaultSaver, and defaultInputIter = defaultInputIter' defaultSaver, I would take that.
21:09:07 <stepkut> in the arrow version, you can specify those limits on a per-field basis
21:09:19 <stepkut> and have different savers or a per-field basis
21:10:17 <sannysanoff> stepcut, i still don't understand the arrow version.. maybe if i see this implemented like that, I will understand it.
21:10:27 <sannysanoff> i will do non-arrow, right?
21:10:32 <stepkut> yes
21:11:46 <stepkut> the arrow version is something I am working on, but is not released yet
21:12:29 <sannysanoff> got it.
21:12:45 <sannysanoff> question: why are you storing to RAM if there's no "filename" header supplied?
21:12:58 <sannysanoff> why don't create tmp file with generic prefix?
21:13:24 <sannysanoff> does not matter to my case, just a question.
21:13:36 <stepkut> sannysanoff: because things with out a filename header are probably not files. They are probably just normal form values like, <input type="text" name="foo" value="bar" >
21:13:46 <sannysanoff> ah, right.
21:15:33 <stepkut> how's happstack working for you overall ?
21:19:40 <sannysanoff> I could do this project in Java, but I'm a little bit tired of it (since 1996). In short: Developing in haskell is slower and takes more effort but gives joy, Happstack starts faster than Tomcat ;) Mostly, it is haskell that I feel is working for me overall. Happstack is great project, but I use only simple web server part, no MACID&stuff.
21:20:18 <sannysanoff> From haskell learning point of view, Happstack is a great tutor.
21:20:23 <stepkut> cool
21:23:41 <stepkut> hmm, I just put out a survey asking what template system people are using with happstack, and what template system they want to use
21:27:26 <sannysanoff> i personally don't use any. I use GWT.
21:27:50 <stepkut> how does that work ?
21:28:01 <stepkut> everything is rendered client-side ?
21:28:15 <sannysanoff> how gmail works? yes, using javascript+dom manipulation.
21:28:19 <stepkut> yeah
21:28:38 <stepkut> does that work well ?
21:29:45 <sannysanoff> you're talking to GWT adept. Yes, it works well. But don't use javascript frameworks for this, or it will be too complex.
21:31:13 <stepkut> do you have to do anything special to use GWT with happstack ?
21:33:00 <sannysanoff> created DbMonad to interface the DB + couple of functions for data to JSON conversion + custom auth layer
21:34:51 <stepkut> would be interesting to see a blog post about what it takes to get started with happstack+GWT :p
21:35:22 <sannysanoff> what it takes? forget about server-side debugging first ;) All the rest is easy.
21:36:05 <sannysanoff> do you use any debugger? leksah? vim? emacs?
21:38:38 <stepkut> no
21:38:42 <sannysanoff> GWT has good integration between client and server, when server is in java too. But when not, you just make XMLHttpRequests, receiving json, parsing it, all using standard GWT libs... On happstack side, you receive "/ajaxcall/listCustomers?q=john%20doe" and return [{name: "John"...},{..}]
21:38:44 <stepkut> printf
21:38:54 <sannysanoff> printf for debugging, too.
21:39:18 <stepkut> nice
21:43:55 <sannysanoff> if you are just curious, there's a generation of java programmers... java giveth everything, in turn taking your mind away. As part of it, GWT gives you: compacted obfuscated js code compiled from java, with unused symbols stripped, and dynamic code loading; strongly-typed language for browser-side development (i.e. java), DEBUGGING !! of your GUI in native browser (!!), all the pleasures of java (enormous refactoring)..... but you need to
21:43:55 <sannysanoff>  stay fit, so do your haskell/C/whatever in spare time or on other projects ;)
21:44:17 <stepkut> heh
21:44:50 <stepkut> I have only used java a little for android programming
21:44:57 <stepkut> though I heard rumors you can use scala
21:46:19 <stepkut> I pretty much only use Haskell
21:46:26 <stepkut> I do Agda to keep my mind fit :p
21:46:27 <sannysanoff> what else you do?
21:47:10 <stepkut> ?
21:47:46 <sannysanoff> you're programming full daytime? is happstack your main project?
21:48:41 <stepkut> my day-job involves developing using happstack, so I work on happstack in support of that. But I am not (yet) paid to develop happstack full-time.
21:49:47 <sannysanoff> i understand this model. And what kind of project are you using happstack for?
21:50:58 <stepkut> Right now we are working on launching some websites
21:51:45 <stepkut> nothing publicly visible yet
21:51:56 <stepkut> but hopefully some stuff in December
21:52:07 <stepkut> upgrading to GHC 7 has been a bit time consuming
21:52:15 <stepkut> and upgrading to mtl-2
21:52:27 <stepkut> (our tool chain that is, happstack itself was easy)
21:56:43 <sannysanoff> some web sites -- what kind of?        Full of input forms / admin sites? Or simplistic, like twitter? Or maybe like facebook? Or Flickr? Or text-based (livejournal, newspapers)? Or maybe like gmail / GWT ?
21:57:59 <sannysanoff> what scale (code size)? What target scale (text hits / minute) ?
21:59:56 <sannysanoff> I am asking this, because I have no haskellers who do it fulltime to ask interactively ;) Sorry, If I distract you.
22:00:45 <stepkut> one site is similar in complexity to fmylife.com, (a little more complex). Currently around 4000 lines of code including whitespace and comments
22:01:52 <stepkut> the other one is more complex, with more javascript & ajax fanciness. As well as what it is actually trying to do..
22:03:14 <stepkut> probably closer to 30,000
22:03:46 <stepkut> though, it still has a ways to go
22:04:15 <stepkut> will likely grow to 100,000+
22:05:32 <sannysanoff> 100K haskell LOCs?
22:05:43 <stepkut> yeah
22:05:57 <stepkut> but not at launch
22:06:09 <sannysanoff> javascript generated or handcrafted?
22:06:39 <stepkut> a mixture at the moment. Haven't found a satisfactory way of doing javascript yet :-/
22:06:41 <sannysanoff> (i just was wandering about uses for HJavaScript)
22:07:15 <stepkut> some code did use HJScript, but it didn't really work out that well. So then more was doing using javascript+jquery
22:07:38 <stepkut> HJScript works well as an 'FFI' for calling javascript from Haskell. But not so great as a DSL for generating a lot of javascript.
22:07:47 <stepkut> haven't tried jmacro yet
22:08:31 <sannysanoff> HJScript as FFI for using JS on server-side?
22:09:20 <sannysanoff> (cannot imagine browser-side / haskell integration)
22:12:13 <stepkut> if you are generating an html template in Haskell, and you want that template to call some javascript functions with non-static arguments, you can use HJScript to take care of converting haskell values to javascript values, and ensuring that you are passing in values of the right type.
22:14:09 <sannysanoff> ah, ok. Question: 1) how do you annotate type signatures in definition type FileSaver = Int64 -> FilePath -> ... -> IO (...)
22:14:25 <sannysanoff> (comments?)
22:15:53 <stepkut> http://www.haskell.org/haddock/doc/html/ch03s02.html#id565220
22:16:06 <stepkut> ?
22:17:38 <sannysanoff> great! first time contributing, did not need that before!
22:21:48 <stepkut> heh
22:22:19 <sannysanoff> what are you using to navigate the code? Say, you downloaded package (cabal install). Then you edited your project.cabal file, then you added "import Some.Package.Part", then you need to look in the details of types. Are you using only hackage db for haddocks? Locally generated them? Something else? ctags (forgive me)?
22:22:50 <stepkut> I usually just use hackage, or open the source files in emacs
22:23:01 <stepkut> leksah might do something fancier
22:23:50 <sannysanoff> but which source files? One type references another... there are default mkSomething functions, how do you find them? Remembering sort of where to look? Grepping all the time?
22:24:11 <sannysanoff> leksah is what I use, but it is too raw.. no good navigation.
22:25:14 <sannysanoff> your ~/.cabal/packages/haskell.org/packagename/version/ is too much to grep through, narrowing to necessary directory is quite slow..
22:25:35 <sannysanoff> (if you use 20 packages)
22:25:40 <aavogt> ghci's :info
22:26:14 <sannysanoff> same here. Than you look where the symbol is defined, then :m +That.Long.Package, then :info again?
22:26:20 <stepkut> a lot of grepping I guess
22:26:31 <stepkut> or the haddock docs on hackage
22:26:42 <stepkut> I do also build and install all the documentation locally, but I don't actually use it :-/
22:26:57 <aavogt> stepkut: is it fixed in darcs that some happstack classes are not exported?
22:27:22 <aavogt> so that you have no idea which types can actually be accepted by some functions
22:28:20 <stepkut> aavogt: I have no idea what you are talking about, so I don't know :)
22:28:22 <aavogt> ACTION doesn't have a specific example, but is looking now
22:30:03 <aavogt> stepkut: getHeader has a constraint involving Happstack.Server.HTTP.Types.HasHeaders
22:30:54 <aavogt> so at that point, I needed to look in the source to figure out which types that one could be used at
22:31:48 <aavogt> if the concern is to keep those classes 'closed', they could have a superclass that's not exported. I think there are other ways to get the same result too
22:34:30 <stepkut> aavogt: not sure if it is intended to be closed. Certainly we should make the haddock output for that useful
22:35:23 <aavogt> quite likely there are other classes doing the same. One way to check is by looking at haddock's warnings
22:35:35 <stepkut> aavogt: that specific issues is not fixed yet
22:35:40 <stepkut> aavogt: k
22:36:07 <stepkut> aavogt: I am going to be looking at the haddock docs in detail soon and make sure they don't suck
22:36:30 <aavogt> :)
22:36:38 <stepkut> aavogt: there are 2-3 code issues still remaining first