02:05:22 <stepcut> mightybyte: that could be really good or really bad ;)
02:06:56 <stepcut> what do you think the right way to handle server timeouts is ? It appears that happstack currently imposes a 30 second hard limit on reading a request and sending a response.. that doesn't seem very good
02:08:06 <JuanDaugherty> 5 is the ancient standard for user patience, doubling it gives 10
02:08:47 <stepcut> JuanDaugherty: but what if you are streaming a 30 minute video?
02:09:33 <JuanDaugherty> obviously standards based on human interaction don't apply to processes where the user isn't at attention
02:09:47 <stepcut> right, this is a question about how to actually implement the timeout
02:10:41 <stepcut> in theory, setting a per socket timeout and letting the OS take care of it would be nice.. but not all OS's support that socket option..
02:10:42 <JuanDaugherty> 30 s is a long time but not too long unless a lot of them pile up
02:11:30 <stepcut> JuanDaugherty: a long time for what? if someone is downloading a 30GB file, 30s is pretty darn fast..
02:12:03 <JuanDaugherty> so who is the server?
02:13:18 <stepcut> some happstack program
02:13:32 <JuanDaugherty> why are you even imposing a limit on the client?
02:14:05 <stepcut> JuanDaugherty: because if the client decides to never finish we can't keep the connection open forever
02:14:39 <JuanDaugherty> so maybe 120 s would be more reasonable
02:15:04 <stepcut> JuanDaugherty: eventually we have to decide that the client is gone and will never receive more data from us.. but if the client does not explicitly close the connection, then we have to simply timeout
02:15:08 <JuanDaugherty> standard 2 min warning
02:15:50 <stepcut> JuanDaugherty: 120s since when ?
02:15:50 <JuanDaugherty> that's what I do on IRC
02:16:06 <JuanDaugherty> google two minute warning
02:16:41 <stepcut> JuanDaugherty: right now the client gets 30s from the time it connects to the server to send it's request and receive the entire response
02:17:22 <JuanDaugherty> so if they're in the process of sending and the 30s runs out you disconnect in the middle?
02:17:27 <stepcut> yep
02:17:33 <JuanDaugherty> that's stupid
02:17:39 <stepcut> I believe that is what I said
02:17:55 <stepcut> the question is, how do you implement a non-stupid solution
02:18:19 <JuanDaugherty> cancel the timeout once the response is received
02:19:02 <stepcut> the response might be gigabytes long, so you need to reset the timeout everytime you successfully send more data. But how do you actually do that
02:19:28 <JuanDaugherty> maybe haskell makes that more difficult than in other langs
02:19:53 <stepcut> perhaps.
02:20:20 <JuanDaugherty> in a c/c++ environment you would be able to cancel the timeout at the start of processing the response
02:21:19 <stepcut> well, it is a continual thing, yes? You might be sending data fine, and then in the middle of the file the client stops responding
02:22:57 <JuanDaugherty> so you need to distinguish between a failure to respond and a failed response
02:23:22 <stepcut> what is a failed response?
02:23:23 <JuanDaugherty> with different values for the timeouts
02:24:21 <JuanDaugherty> one that once started doesn't respond further for some length of time
02:24:38 <stepcut> then what is failure to responsed ?
02:24:45 <stepcut> failure to respond ?
02:25:17 <JuanDaugherty> but as you say it's definitely stupid to set an arbitrary short time frame for an operation that could be moving 10s of gigabytes
02:25:26 <stepcut> yes
02:25:45 <stepcut> I am just not clear on the best way to actually write the code that implements the timeout.
02:26:01 <JuanDaugherty> either don't move gigabytes or don't set the arbitrary limit
02:26:16 <stepcut> right
02:26:32 <stepcut> but we still need a limit for how long we spend sending no data before we give up
02:27:08 <stepcut> you are not addressing the question I actually care about, let me try again
02:27:30 <stepcut> to send a response we currently do, Data.ByteString.Lazy.hPut socket response
02:27:58 <stepcut> but, by itself, there is no way to tell if that hPut is actually successfully putting data or not. It just blocks until it is done.
02:28:31 <stepcut> if the hPut is successfully moving data, then we can just leave it alone
02:28:55 <stepcut> but if it has not sent any data for over 120s (or some other time limit), then we want to kill that connection and free up the resources
02:29:06 <stepcut> the question is, how can we tell if the hPut is doing anything or not
02:29:38 <JuanDaugherty> i don't know hPut so can't comment on that
02:30:15 <stepcut> one approach is to set the timeout on the socket itself, so that the OS will automatically close the socket if nothing is happening. Then we just catch the EOF in haskell and clean things up.
02:30:26 <JuanDaugherty> but 120s is definitely enough time to decide the connection is dead
02:30:35 <JuanDaugherty> 30s maybe not
02:30:47 <stepcut> But, I believe that per-socket timeout configuration is not a POSIX standard and is not available on all OSes
02:31:10 <JuanDaugherty> well that's another matter entirely
02:31:17 <stepcut> so, another option is to implement a version of hPut that includes a timeout feature
02:31:44 <stepcut> I wonder if there are any other solutions I am missing
02:46:45 <stepcut> JuanDaugherty: if you were using C, how would you make write() timeout ?
02:48:56 <JuanDaugherty> i don't know enough about this situation to comment conclusively but 30s is definitely short to decide an arbitray net peer has died and 120s definitely is not
02:49:45 <JuanDaugherty> *arbitrary
02:51:44 <JuanDaugherty> ACTION sleep.
02:53:08 <stepcut> JuanDaugherty: well, the timeout will eventually be specificied in Conf anyway, with a default of 120s probably
12:25:07 <Gracenotes> what's the preferred expression :: Response for saying there's no entity-body?
12:55:48 <stepkut> toRespose (), perhaps ?
12:56:02 <stepkut> what context ?
12:58:29 <Gracenotes> ah.. probably better than toResponse "", which does text/plain, I think
12:59:02 <Gracenotes> for various things which simply don't have an entity body, like head/options/sometimes delete/client caching
13:00:33 <stepkut> toResponse () does text/plain as well
13:00:58 <stepkut> there is some built in support for HEAD
13:01:35 <stepkut> because for HEAD you still need to send the mime-type and content-size
13:01:57 <Gracenotes> ah. good point.
13:02:18 <stepkut> so, when it goes to actually send the response, it checks the Request, and if it is 'head' then it only sends the headers, but no body
13:02:22 <Gracenotes> at the moment, I just run the get request (which would set the mime-type) but don't return the Response
13:03:04 <Gracenotes> hm. so methodSP HEAD not needed?
13:03:28 <stepkut> not usually
13:04:07 <Gracenotes> methodSP GET would let HEAD in then
13:04:08 <stepkut> but if you do, methodSP GET $ .. then you would never actually get to the code that handles HEAD
13:04:27 <stepkut> I think you need to use, methodSP [GET, HEAD] $ if you want to allow the internal mechanism to work
13:04:50 <stepkut> if you want to supply your own, then I think you can just  generate your response, and then do resp { body = L.empty }
13:04:58 <stepkut> hmm.. not quite
13:05:22 <Gracenotes> ah. can do that. and I suppose the actual response needs to be rendered anyway to do the content-length stuff
13:05:31 <stepkut> ... because then it would think the body size was 0
13:05:41 <stepkut> right
13:06:35 <stepkut> so, if you want to do it yourself, you would need to render the body, add the content-length header, and then set the flag in the Response that says it already has a content-length, and then set the body to L.empty
13:08:36 <Gracenotes> ACTION will trust it to the reliable mechanisms of the server
13:08:49 <stepkut> :)
13:09:14 <stepkut> not sure what the content-type is supposed to be for other things that don't return a body
13:09:36 <stepkut> maybe you just don't have a Content-Type header ?
13:11:22 <Gracenotes> I think so
13:11:32 <stepkut> you can create a response with no body and no content-type like, result 200 ""
13:12:37 <Gracenotes> this guy rooted through the specs about what signals the inclusion of an entity-body. http://stackoverflow.com/questions/299628/is-an-entity-body-allowed-for-an-http-delete-request/299696#299696
13:13:27 <Gracenotes> with a special clause on HEAD..
13:15:24 <stepkut> spiffy
13:15:51 <stepkut> I'm off to the park for a bit, bbl
13:16:03 <Gracenotes> have a good time
17:25:00 <dschoepe> When I run and then quit the first macid example from the tutorial(http://tutorial.happstack.com/tutorial/yourfirstmacid), it doesn't delete the lockfile. Am I doing something wrong?
17:25:37 <mightybyte> How did you quit the program?
17:25:43 <dschoepe> by typing 'q'
17:25:59 <mightybyte> Hmmm, not sure.
17:26:14 <mightybyte> You didn't have to Ctrl-C or anything?
17:26:19 <mightybyte> ...because that could do it.
17:26:43 <dschoepe> no, it exited normally and after the deleting the lockfile it correctly restores the state
17:26:59 <mightybyte> Does the example call the "shutdownSystem" function before exiting?
17:27:20 <dschoepe> no, it doesn't
17:27:28 <mightybyte> That's the problem.
17:27:49 <mightybyte> What's on the startSystemState line?
17:28:09 <dschoepe> main = startSystemState rootState >>= commandLoop
17:28:18 <dschoepe> and rootState = Proxy
17:28:26 <mightybyte> What's the first parameter to commandLoop?
17:28:42 <dschoepe> an MVar TxControl
17:29:01 <mightybyte> Ok you should change main to something like this
17:29:03 <mightybyte> main = do
17:29:20 <mightybyte>   control <- startSystemState rootState
17:29:34 <mightybyte>   commandLoop control
17:29:48 <mightybyte>  shutdownSystem control
17:29:50 <dschoepe> ah
17:29:56 <dschoepe> that fixed it
17:29:56 <mightybyte> (oops, that needs one more space)
17:30:23 <mightybyte> In most real-world deployments, you'll also want to call "createCheckpoint control" right before the shutdownSystem line.
17:30:34 <dschoepe> I'll check whether that's already fixed in the repository for the tutorial and send in a patch if not
17:30:47 <mightybyte> Ok, cool.
17:37:47 <dschoepe> okay, patch sent.
17:38:10 <dschoepe> (Assuming that this is the correct repository: http://patch-tag.com/r/wchogg/happstack-tutorial/home)
17:39:08 <mightybyte> That would be my best guess.
18:02:37 <dschoepe> mightybyte: apparently the same problem is present in some other macid examples in the tutorial as well. Did some recent change break those examples or has this just not been noticed yet?
18:03:30 <mightybyte> I don't know.  shutdownSystem has cleaned up the lock file for as long as I can remember, so I'm guessing that it just got missed somehow.
18:12:09 <dschoepe> am I right to assume that startSystemMultimaster from the tutorial is now named startSystemAmazon, or is this something else(It has almost the same type signature except for an ApplicationName)?
18:12:41 <mightybyte> It sounds like those two are different.
18:13:06 <mightybyte> But it could be that they just started using Amazon as the place to test multimaster because it provided the easiest means to do so.