00:00:47 <stepkut> ACTION slaps himself with a large trout
00:00:55 <stepkut> ?slap stepkut
00:00:55 <lambdabot> ACTION hits stepkut with a hammer, so they breaks into a thousand pieces
00:02:50 <Lemmih> stepkut: It should fail to load, I think. Check sum error.
00:03:00 <luite> oh clientsession uses some mutable global state in an IORef btw
00:03:13 <stepkut> luite: I'm sure it does
00:03:41 <luite> it keeps a cryptographic RNG there
00:04:01 <luite> which it periodically reseeds from the system entropy source
00:04:15 <luite> it used to not do that, but that made yesod 100x slower
00:04:32 <luite> so some IORef was deemed to be an acceptable tradeoff :p
00:04:58 <stepkut> Lemmih: we usually see garbage versions numbers I think
00:05:22 <stepkut> Lemmih: would it make more sense to write it to a temporary file and and rename it into place when it is done?
00:05:35 <Lemmih> stepkut: No.
00:05:40 <stepkut> how come?
00:06:06 <Lemmih> The checksum is enough.
00:06:12 <stepkut> what is the correct thing to do when you have a corrupted checkpoint like that?
00:06:32 <Lemmih> To fail with a checksum error.
00:06:39 <stepkut> and then what?
00:06:51 <Lemmih> Tell the user to fix it.
00:06:56 <stepkut> how do they fix it ?
00:07:06 <Lemmih> Delete the checkpoint.
00:07:32 <stepkut> and why is that better than having the checkpoint not be in the way at all? Now my server sometimes fails to restart automatically ..
00:07:38 <Lemmih> ACTION is in his bed and halfway sleeping. His answers are correct but somewhat shunted.
00:08:29 <stepkut> so far my experience is --> server dies in middle of checkpoint, fails to restart because checkpoint is base, users get pissed, I delete the bogus checkpoint and start it by hand.
00:09:06 <stepkut> I am not clear how the broken checkpoint getting in the way is benefitting me yet
00:09:11 <stepkut> only how it is causing pain
00:09:17 <Lemmih> hm
00:10:04 <stepkut> if it was written to checkpoint-xxxx.tmp then it would still be there in case I wanted to try to recover something from it (even though all the data should already be in the event logs) but it won't cause my server to fail to start
00:10:39 <Lemmih> no no, not an option.
00:11:00 <Lemmih> run acid tool, solution is
00:12:07 <stepkut> the primary reason I see it is not an option is that the rename could fail ?
00:12:57 <stepkut> acid tool better be a library I can call from my app
00:13:21 <Lemmih> it could be that too
00:13:34 <stepkut> it should be a library, with an optional cli built on top of it
00:13:55 <stepkut> as should everything in binutils.. I filed a bug with gnu about that.. they haven't gotten back yet
00:14:47 <stepkut> so.. acid tool would look at the system, see the bogus checkpoint, and then what?
00:14:59 <Lemmih> i like having same format for events and checkpoints. renaming not an option
00:15:16 <stepkut> so.. events could fail in the same way then
00:15:34 <Lemmih> you'd just tell it to delete all invalid entries in the 'checkpoints' log.
00:15:49 <Lemmih> yes, absolutely. that's the point
00:16:00 <stepkut> didn't there used to be a file that recorded what the last successfully written log file was ?
00:16:13 <stepkut> do we still have that ?
00:16:51 <Lemmih> why would we want that?
00:17:08 <stepkut> I'm not saying we do.. just asking
00:17:18 <Lemmih> we don't. no need.
00:17:27 <stepkut> so we determine invalid checkpoints/events by looking at the checksum
00:17:35 <Lemmih> yes.
00:18:17 <Lemmih> and fixing errors by deleting data doesn't and shouldn't happen automatically
00:19:42 <stepkut> deleting corrupted event log files seems dodgy .. though in theory checkpoint files contain only redundant data
00:19:50 <stepkut> but... I guess that is only half a fix then
00:20:53 <Lemmih> what is?
00:21:24 <stepkut> deleting corrupted checkpoints automatically .. that only fixes the problem when there is a corrupted checkpoint
00:21:46 <stepkut> with a corrupted event file.. you might have valuable data that can be recovered in it still
00:21:48 <luite> sounds scary
00:22:01 <stepkut> like... if 99 out of 100 events were written.. you would want those 99
00:23:11 <luite> take one down, pass it around
00:23:33 <stepkut> well, 'deleting' the checkpoint file is perhaps a bit strong.. moving it to a broken checkpoint directory maybe..
00:23:43 <tazjin> How do checkpoints get corrupted?
00:24:00 <stepkut> tazjin: if you pull the plug while the checkpoint is being written
00:24:05 <stepkut> or kill -9
00:24:12 <Lemmih> tazjin: That or rock 'n roll.
00:24:36 <tazjin> That's pretty unlikely though, right? And if somebody does it with -9 it's probably intentional
00:25:15 <stepkut> a call to update should never return until the data is safely on disk.. so recovery there is a bit.. interesting. If the transaction never completed, should it really be recovered?  But clearly, there could be useful data in there sometimes..
00:26:02 <Lemmih> ACTION has lost consciousness and won't be available for a couple of hours.
00:26:22 <stepkut> tazjin: in our case, we create a checkpoint on shutdown so that if we install a new server that adds/removes acid-state methods everything works dandy. But, the shutdown script was calling kill -9 after 2 seconds.. so it was not entirely intentional
00:26:34 <stepkut> Lemmih: later!
00:27:37 <stepkut> this is no worse that any SQL database, of course. You can not change the laws of physics
01:13:25 <stepkut> ok.. happstack-clientsession is ready for a new release unless someone points out why it is not
01:13:33 <stepkut> but.. I will do that tomorrow when I am more awake
01:14:33 <stepkut> I wonder if we should set HttpOnly on the clientsession cookie
01:22:04 <stepkut> added a sessionHttpOnly flag and set it True by default
01:22:37 <stepkut> i'll like I spent all week writing haddock documentation for happstack-clientsession
01:23:19 <stepkut> the happstack-clientsession library is twice as long as clientsession itself :-/
01:32:11 <tazjin>  but we have SafeCopy!
01:33:38 <stepkut> :-/
01:34:49 <tazjin> More like http://ragefac.es/43
01:35:38 <stepkut> :)
01:41:43 <stepkut> so.. here is the big problem with clientside sessions..
01:41:49 <stepkut> they are not atomic
01:42:12 <stepkut> if the browser makes multiple page requests at once, and they each try to modify the session.. who knows what you will end up with
01:42:31 <stepkut> whichever Response is sent last will get the final say
01:42:45 <stepkut> but.. they will all get unspecified starting states
01:42:56 <tazjin> How does Yesod handle that?
01:43:02 <stepkut> so.. it would not work as a request counter
01:43:24 <stepkut> perhaps it doesn't
01:43:30 <stepkut> that is just the way cookies work
01:43:49 <stepkut> aren't they great?
01:44:02 <tazjin> Yes, so .. modern and .. reliable
01:44:13 <stepkut> one of the many awesome things that the w3c came up with
01:44:19 <tazjin> Did I mention safe?
01:45:24 <tazjin> Well, if I ever had to use clientsession for something in production I wouldn't put any mutable data into the session
01:45:34 <stepkut> a very wise idea
01:45:43 <stepkut> I am updating the docs as we speak
01:46:44 <tazjin> Who implemented the cryptography behind clientsession?
01:46:58 <stepkut> various people
01:47:39 <tazjin> Ah, I see it on hackage. Hmm, have these packages been audited?
01:47:45 <stepkut> nope
01:48:01 <stepkut> there is the clientsession package itself, and then all the crypto libraries it depends on.
01:48:12 <tazjin> I have kind of a weird attitude towards storing anything besides some basic information in cookies, even if it's encrypted
01:48:28 <tazjin> I'd probably tape some own security layers on top as well, like weird checksums or something
01:48:44 <tazjin> though that would at some point just end up being security by obscurity
01:49:02 <stepkut> :)
01:49:14 <stepkut> yeah, I would prefer serverside sessions
01:49:26 <stepkut> clientside sessions were supposed to easy.. not take all week!
01:52:39 <tazjin> Does Yesod have serverside sessions?
01:53:18 <stepkut> no idea
01:54:11 <tazjin> Hmm, if the book is accurate then it doesn't
01:54:30 <stepkut> we'll have to beat them to the punch!
01:55:07 <tazjin> But they store mutable values in the clientsession as it seems (the Messages thing)
01:55:25 <stepkut> the trickiest part of serverside sessions that I know of so far is making them expire with out doing a database update on every request
01:55:47 <stepkut> what is the Messages thing?
01:56:26 <tazjin> Any site can throw in a message with the current client session and that message will be displayed in a designated place in the default layout on the next page
01:56:30 <tazjin> if I got it right
01:56:48 <tazjin> for example error messages
01:57:15 <stepkut> why does that need to be in a cookie?
01:57:31 <tazjin> Don't ask me :P
01:57:44 <stepkut> ok
01:58:14 <tazjin> and @ expiring sessions: I don't really see a problem here. Assuming you have some kind of sessionGuard function that checks the validity of the session on every page request (I guess there's no other way?) this could check the sessions age, and if it's too old just fail
01:58:25 <tazjin> on the DB side old sessions could just be cleaned up once per hour or so
01:58:36 <tazjin> it doesn't really hurt to have invalid sessions lying around for a bit, does it?
01:59:17 <stepkut> where do you store the sessions age ?
01:59:47 <stepkut> the easy answer, is you store its last access time in the database.. but that requires a write on every access
02:00:11 <tazjin> Why access time? A session could just have a fixed expiry date and boom
02:00:14 <tazjin> that's how Google does it
02:00:42 <stepkut> a fixed expiration date works in some cases
02:00:42 <tazjin> if you use the text message authentication thing where you enter a pin code that's just stored for 30 days and then it's gone, no matter when you last logged in
02:00:57 <stepkut> but other people want a short expiration timeout on inactivity.. like banks
02:01:08 <stepkut> but they don't want to log you out every 5 minutes even if you are doing stuff
02:01:30 <tazjin> Hrrm
02:02:10 <tazjin> Well, I think the sessions wouldn't have to be stored in an acid-state
02:02:18 <stepkut> why not?
02:02:47 <stepkut> you gotta store the information somewhere
02:02:58 <tazjin> That would make disk I/O necessary, but there's really not a huge amount of reasons to keep sessions alive between server restarts (imho, there might be some use case where this is necessary)
02:03:16 <stepkut> well. it depends on what you consider a session I guess
02:03:32 <stepkut> what we really want isn't so much sessions, but clientid tracking
02:03:37 <luite> hmm, i think keeping sessions alive is kind of important
02:03:59 <luite> logging out all users when you fix a small bug is rather annoying for example
02:04:14 <stepkut> we want someone to be able to authenticate, and get an auth token that is good for some period time, but can expire through inactivity
02:04:16 <luite> especially if you've made a nice zero-downtime takeover setup
02:05:15 <stepkut> and then .. built on top of that you can have data which gets expired when the auth token expires
02:05:55 <stepkut> and if you have multiple servers and need to replicate the session data across multiple servers, then you might well use acid-state
02:06:52 <tazjin> yeah, hmm
02:07:30 <stepkut> but, the central part, I think is having some way to give the user a token so you can track them, and then be able to expire the token on inactivity, and trigger events to run as well.
02:07:52 <stepkut> once you have that, you can built a session data library pretty easily
02:09:39 <tazjin> I feel like this should be a solved problem in 2012, there has to be some kind of go-to solution
02:10:01 <stepkut> I think clientsession data is most useful when you have a high-volume of infrequent users, who don't have accounts on your site, but who could be better served by storing some information across their infrequent visits
02:10:09 <stepkut> but, if you lose that data.. no big deal
02:10:49 <tazjin> Hmm, TIL clientsession is the default in Rails
02:12:20 <stepkut> i wonder if there is any sane way to coordinate clientsession data across multiple simultaneous requests
02:12:45 <luite> waddayamean?
02:13:53 <luite> pipelined reqs?
02:13:55 <stepkut> luite: are you familiar with the idea that modifications to the client sessiondata are isolated ?
02:14:02 <stepkut> luite: yes..
02:14:12 <stepkut> doesn't even have to be pipelined
02:14:59 <stepkut> if a client submits multiple simultaneous requests that modify the clientdata.. it is a bit .. interesting
02:15:30 <luite> right, for some reason this doesn't appear to be a problem in practice though
02:15:35 <stepkut> they would each start with the same session data, make independent modifications, and then overwrite each other when the request is sent
02:15:47 <stepkut> luite: not enough users :)
02:15:57 <tazjin> luite: That's probably luck up until now
02:15:59 <luite> dunno, i mean that rails uses the same mechanism
02:16:06 <tazjin> I'll ask on #rubyonrails how they handle this
02:16:56 <stepkut> one solution is to only store things in clientsession data where that doesn't matter
02:16:58 <luite> stepkut: i think yesods implementation is a bit suboptimal in this respect since it sets a new cookie every new request, even if it's not modified
02:17:12 <stepkut> yeah, that makes it even worse
02:17:19 <tazjin> Wow what, why?
02:17:26 <luite> tazjin: to set the new expiration date
02:17:39 <stepkut> because a real modification could go out, and then a delayed response could overwrite it
02:18:00 <tazjin> Hmm
02:18:14 <tazjin> Could somebody who's logged into a Yesod app modify the expiry date of his cookie
02:18:21 <tazjin> and thus stay logged in indefinitely?
02:18:22 <luite> tazjin: it should work with some grace period, and the api has been updated to allow that
02:18:25 <luite> no
02:18:43 <stepkut> if you do not have frequent clientdata modifications, then the problem would not appear very often. Like.. if you only update the clientsession when they change their username.. they are not likely to make two conflicting updates at once
02:19:10 <stepkut> tazjin: only if they can decrypt and re-encrypt the cookie
02:19:36 <tazjin> So there's an additional check in the session data itself? Okay
02:19:45 <tazjin> and #rubyonrails doesn't seem very talkative  :p
02:19:47 <stepkut> but.. if you have a very dynamic sessiondata.. then I think problems will start popping up
02:20:27 <stepkut> tazjin: the session data in the cookie is encrypted.. so the addition check is 'can it be decrypted'
02:20:44 <stepkut> additional
02:21:35 <stepkut> if the user has managed to create a fake cookie with valid encryption data.. you probably have more serious issues already
02:21:40 <stepkut> like.. someone stole your encryption key
02:22:08 <tazjin> So when Yesod rewrites the cookie on every request
02:22:24 <tazjin> does it re-encrypt the actual session data as well or just reset the expiry time?
02:22:34 <luite> both
02:22:37 <tazjin> That's the part that confuses me a bit, I think I have to look at the implementation
02:22:41 <tazjin> ok
02:22:44 <luite> the date is part of the enctypted payload
02:22:53 <stepkut> tazjin: if you change the session data, you have to re-encrypt it..
02:23:29 <stepkut> that's what makes it encrypted..
02:25:29 <tazjin> I just looked at how it works on haskellers.com, got it now
02:26:12 <stepkut> :)
02:26:52 <tazjin> Hrrm, the Rails channel stepped over my question and is now talking about clearing browser caches
02:26:53 <stepkut> so I added more caveats and warnings to clientsession
02:27:00 <stepkut> tazjin: :)
02:27:36 <stepkut> there is clearly a use case for clientsessions.. but you just have to be clear what the conditions are
02:28:54 <stepkut> in yesod, the requests probably write all over each others modified session data all the time.. but since the only real difference is a tiny variation in the expirey time.. that is fine
02:28:58 <luite> looks like pipelining is disabled by default on all browsers
02:29:24 <luite> and POST shouldn't be pipelined
02:29:36 <stepkut> but a browser can still make multiple simultaneous requests without pipelining
02:29:44 <luite> that leaves multiple connections yes
02:30:22 <stepkut> so.. if you only modify the session during a POST, then you are likely to be pretty safe
02:30:24 <luite> oh wait, opera has pipelining enabled
02:31:13 <stepkut> because there are not a lot of cases where you would be submitting multiple POST requests at once..
02:31:29 <luite> hm it's a bit harder to google for that
02:34:12 <tazjin> All googling lead me to threads like "My Rails session is getting reset when I have concurrent requests", hrrm
02:34:53 <stepkut> haha
02:37:17 <tazjin> Hmm, apparently Rails doesn't encrypt the cookie
02:37:18 <tazjin> D:
02:37:31 <luite> all googling sends you to the same stackoverflow question? :p
02:37:41 <tazjin> luite: Yep
02:37:52 <tazjin> luite: My Google-fu is decreasing at this time of the day ;-)
02:40:24 <luite> tazjin: hm, that must be why so many hackernews commenters insisted that encrypting the cookie was useless
02:40:53 <tazjin> Well, I got only one response on the rails channel
02:41:06 <tazjin> [04:34:42] <alindeman> tazjin: By default session data is stored in a digitally signed cookie, but it's not encrypted. Users can read it, but it's hard to tamper with.
02:41:39 <tazjin> I want to see this in action, what's a popular website running on Rails?
02:41:47 <luite> github i think
02:42:19 <tazjin> Ah yeah, right
02:42:34 <tazjin> I'd not be surprised if they use a different session system though, lets see
02:45:18 <tazjin> Throws something that does look like either a session ID that got way out of hand or an encrypted cookie
02:48:56 <stepkut> ok, now there are lots of discouraging words in the happstack-clientsession header
02:49:03 <stepkut> my work is done!
02:49:33 <tazjin> :D
02:50:11 <tazjin> I read tons of articles about storing sessions now, there's no general consensus. Apparently lots of people use memcached for high performance sites, but that leaves the shutdown problem again
02:57:08 <stepkut> acid-state is like memcached.. but solves the shutdown problem! just not yet the replication problem
02:59:16 <tazjin> Well, the issue with our sessions is the disk I/O, right?
02:59:38 <tazjin> if you have to write on every request
03:01:10 <stepkut> yes.. or network IO for distributed
03:01:31 <stepkut> if I didn't care about restart, I could just use the memory backend for acid-state
03:01:43 <stepkut> if I didn't care about restart *or* replication
03:02:00 <tazjin> Yes, so wouldn't it be possible to keep the session data in a purely in-memory state and sync this up with the disk every few minutes or so?
03:02:07 <tazjin> This would take some load off of individual requests
03:02:10 <tazjin> but replication is still an issue
03:02:54 <stepkut> plus you could lose several minutes worth of data
03:03:20 <tazjin> How? You would serve requests out of the in-memory session state
03:04:04 <stepkut> my solution is to look at how long a session lasts, and when the last access time was, and only update the last access time every few minutes (variable based on the length of the timeout)
03:04:38 <stepkut> if you are only syncing to disk now and then, you could lose data if you ran for several minutes without syncing
03:06:15 <stepkut> so.. if the last access time was 2 seconds ago, don't bother updating
03:06:23 <stepkut> if the last access time was 1 minute ago.. then refresh it
03:06:44 <stepkut> then you have, at most 60 updates per hour instead of potentially thousands
03:08:55 <tazjin> Hrmm
03:09:41 <stepkut> the longer the expiration time, the longer you can wait between updates
03:10:47 <stepkut> so.. if you timeout is 24 hours.. you might only update the access time if it was more than 20 minutes ago
03:11:07 <stepkut> worse case, that means your timeout was 23:40 minutes instead of 24 hours..
03:12:29 <stepkut> there might be other ways to batch up the events that result in less writes
03:12:37 <tazjin> We'll end up with different types of sessions anyway, these expiring sessions don't work for all applications I think
03:12:47 <stepkut> but.. that one seems easy to implement
03:13:10 <tazjin> Yeah, it is
03:13:26 <tazjin> just the whole crypto part is what I never want to touch
03:13:32 <stepkut> well you can something like, data SessionLife = Forever | Until UTCTime | Inactive DiffTime
03:13:53 <tazjin> That would be the easiest way I suppose
03:14:42 <tazjin> Hmm, slight potential issue with the Forever one. What if a user deletes his cookie and you've got a session in your database for forever that will never be active again? ;P
03:14:58 <stepkut> nah, the thing to do is touch the crypto, claim it is easy and your stuff is totally secure, and then wait for all the people to freak out and show you the bugs :p
03:15:18 <stepkut> what if the user never comes back to the site again ?
03:15:25 <stepkut> forever mean forever!
03:15:51 <tazjin> And then you run a site the size of Twitter and suddenly your harddrives are overflowing with invalid sessions ;P
03:16:03 <stepkut> nah
03:16:27 <stepkut> 9 billion people in the world.. tweets are 140 bytes each.. how much space can that take up ?
03:16:48 <tazjin> do the lookup&query speeds of acid-state begin to get slower with a certain sized data set?
03:16:51 <stepkut> you could fit a lot of sessions on a 1TB disk
03:17:02 <tazjin> yeah :p
03:17:07 <stepkut> as far as acid-state is concerned.. no
03:17:27 <stepkut> the performance of a data-structure is the same whether you are using it in acid-state or not
03:17:44 <tazjin> Well I mean
03:18:04 <tazjin> does it make a difference if you have an IxSet Session with ~20k sessions or 20 million sessions? I'd suppose that it does, but on what scale?
03:18:10 <stepkut> the only thing that is really affects acid-state is the size the arguments you pass to update events
03:18:28 <stepkut> it makes a difference to IxSet
03:18:47 <stepkut> IxSet is basically a balanced tree in the end
03:19:04 <stepkut> so eventually 'n' gets too big for comfort
03:19:21 <tazjin> Do we know how big the comfort zone is?
03:19:51 <stepkut> pretty big I think..
03:19:57 <stepkut> let's say it is a binary tree
03:20:49 <stepkut> if it is 32 levels deep.. you can hold 4 million records?
03:21:29 <stepkut> either you put things in trees.. or you hash them
03:22:30 <stepkut> btrees lookup is O(log n)  as well, so.. you aren't doing fundamentally better with sql
03:22:37 <tazjin> Done some quick calculations, if our app at work ever gets to 100% load (which is unlikely) it would store 1,5 million records
03:23:00 <tazjin> With about 25k being added and deleted every day
03:23:35 <stepkut> nice
03:23:59 <tazjin> If something happens we'll scream and force Amazon to throw more ram at it ;P
03:24:02 <stepkut> the larger issues is probably not lookup time, but space requirements
03:24:18 <tazjin> 32GB should suffice for the beginning
03:24:21 <stepkut> yup
03:24:42 <tazjin> I just noticed that it's almost 5:30, need to get some sleep soon
03:24:47 <stepkut> later!
03:25:11 <stepkut> there are lots of interesting solutions to the space issues -- just no one with a dataset large enough to care yet
03:25:19 <tazjin> Hehe
03:25:20 <stepkut> though, hackage 2 is sort of there
03:25:25 <tazjin> Well, at some point there will be someone
03:25:29 <tazjin> How large is the Hackage DB?
03:25:30 <stepkut> so I do want to get some better tools out soon
03:25:33 <stepkut> too big
03:25:49 <stepkut> on disk it is small.. like. under ten MB
03:26:03 <stepkut> but the data structures they are using are horribly somehow
03:26:07 <stepkut> requiring GBs
03:26:22 <tazjin> Isn't Hackage 2 a complete rewrite?
03:26:40 <stepkut> realistically.. it should take 100MB or something
03:26:42 <stepkut> yes
03:26:55 <stepkut> hackage 1 is a bunch of perl or something awful
03:27:08 <tazjin> > Perl
03:27:08 <lambdabot>   Not in scope: data constructor `Perl'
03:27:15 <tazjin> > Central Haskell package repository
03:27:16 <lambdabot>   Not in scope: data constructor `Central'Not in scope: data constructor `Has...
03:27:20 <tazjin> shut up lambdabot  >.>
03:27:28 <tazjin> Somehow these two don't go well together
03:27:51 <stepkut> ghc used to (and maybe still does?) have an evil mangler
03:28:06 <stepkut> which was a perl script that used regexes to modified the assembly output by gcc
03:28:58 <stepkut> not sure if that is still around.. the idea behind cmm  and llvm is to not have to do that type of silliness :)
03:29:19 <stepkut> ACTION goes to bed too
03:29:41 <tazjin> You know your sleep schedule is wrong when people on the other side of the planet go to bed at the same time as you
03:30:20 <tazjin> Hrrm, at least for one person
03:30:45 <tazjin> anyways, good night - and don't lose any sleep over sessions and related issues :P
03:35:39 <stepkut> night :)
03:35:52 <stepkut> if I lose sleep it will be due to allergies
03:36:14 <stepkut> I have a bad cough that has been keeping me up the last few nights
11:01:13 <bitonic> stepcut: there's a minor formatting issue in the crash course, in "HSX and do syntax
11:01:14 <bitonic> "
13:09:55 <donri> stepcut: we could probably get rid of that IORef in clientsession thanks to our ClientSessionT. i think we'd just use encrypt instead of encryptIO?
14:25:37 <donri> or maybe not.. doesn't seem to expose enough internals
14:37:07 <luite> is it really a worthwile goal? are there situations where you wouldn't want MonadIO where you use ClientSessionT?
14:37:31 <luite> problem is that you need to reseed once in a while, and then you need IO anyway to get new entropy
14:37:40 <donri> just doesn't seem necessary, and why use global mutable state when you can avoid it?
14:40:40 <luite> it might be safer to have the state there than in your monad, since you can't duplicate it with fork or need to worry about lost effects with monad-control
14:40:52 <donri> aye
14:43:24 <luite> that said, I'm a bit unhappy with global IORefs and the necessary NOINLINE tricks, if anything i'd like to have easier safe global state for libraries
17:05:56 <stepcut> donri: I dunno.. ClientSessionT only lasts less than a single request.. how does that help us here?
17:06:17 <donri> true
17:06:36 <stepcut> donri: also, I intend to release darcs version unless you think there are more changes needed
17:07:01 <stepcut> not sure if you saw all my commits from last night
17:07:52 <donri> sounds good
17:08:07 <donri> next we need a happstack-clientsession-hsp...
17:08:16 <stepcut> :-|
17:08:24 <donri> seriously, is there no better solution to these exponential instances? :P
17:08:43 <stepcut> what is the -hsp for ? Just the normal XMLGen instance?
17:08:48 <donri> yea
17:09:49 <stepcut> I usually wrap everything in a newtype anyway, so that doesn't really help
17:10:11 <donri> so how do you do hsp?
17:10:26 <stepcut> niklas and I talked about providing something like HSP.IdentityT in HSX itself, and then providing a derivingXMLGen template haskell function
17:10:37 <stepcut> since the normal deriving mechanism can not handle the type families
17:10:54 <donri> sounds good
17:11:03 <donri> seems silly to have to do this for every newtype http://www.happstack.com/docs/web-routes-hsp-0.22.1/doc/html/web-routes-hsp/src/Web-Routes-XMLGenT.html
17:11:09 <stepcut> yup
17:11:34 <stepcut> I usually just copy a file like that and change all the names to match my top-level newtype
17:11:41 <donri> heh
17:11:49 <stepcut> and then add application specific instances
17:12:38 <donri> what is next on the todo
17:13:01 <stepcut> for client session ? hsx ? or in general ?
17:13:07 <donri> general :)
17:13:45 <donri> i think maybe hsx is the next most pressing matter?
17:13:46 <stepcut> let me look
17:13:52 <stepcut> perhaps
17:16:16 <stepcut> ooo. I can close this bug, http://code.google.com/p/happstack/issues/detail?id=181
17:16:33 <donri> \o/
17:16:57 <stepcut> and this one, http://code.google.com/p/happstack/issues/detail?id=176
17:20:28 <stepcut> so.. for hsx I am waiting for niklas to apply some patches I sent yesterday, and then we can release hsx-qq
17:20:31 <donri> 190/193?
17:21:24 <donri> re hsx i'm most interested in getting rid of overlapping instances and the TH for deriving the instances... i think those two changes would make it much more usable
17:21:26 <stepcut> closed and closed
17:22:52 <stepcut> yeah. I already proposed getting rid of the overlapping instances -- one moment
17:23:02 <donri> yes you showed me :)
17:24:02 <stepcut> ah. Well, after niklas applies the current patches and releases HSX, and I am going to focus on that change next
17:24:10 <donri> \o/
17:24:27 <stepcut> the TH newtype deriving stuff is just waiting on someone to write it
17:24:43 <stepcut> if you want to do it, that'd be great
17:25:05 <donri> the reason i want it in the first place is i don't really grok how to write all those instances in the first place ;)
17:25:23 <stepcut> :)
17:25:36 <stepcut> it's pretty mechanical
17:25:42 <stepcut> hence the reason it can be automated :p
17:28:37 <stepcut> my next tasks are upgrade happstack.com, get the source for happstack.com released, and replace digestive-functors
17:31:06 <donri> good good, i want to hack on the site :)
17:31:13 <stepcut> yup
17:31:18 <stepcut> sorry it is taking so long
17:31:21 <donri> :)
17:34:16 <donri> now that we have transformExp it should be possible to get jmacro to accept xml literals
17:34:36 <stepcut> interesting
17:34:39 <stepcut> that  would be sweet
17:34:54 <stepcut> is that where the problem is ? or what it is haskell-src-exts?
17:34:56 <donri> maybe if it exposes enough internals we can make a new QQ in hsx-jmacro
17:35:21 <stepcut> oh ?
17:35:44 <donri> yea don't want to make jmacro depend on hsx right?
17:36:04 <stepcut> right
17:36:13 <stepcut> because I don't even control jmacro
17:36:22 <stepcut> i just ask nicely for things :)
17:36:43 <donri> and while src-exts understands XmlSyntax, it's just the parsing into an AST... not enough for jmacro to grok the XML
17:37:01 <stepcut> yeah
17:38:05 <donri> i should mention my text port to the author...
18:14:56 <stepcut> fixed
18:15:05 <stepcut> well, fixed the site, and determined what the cause was
18:15:12 <stepcut> haven't fixed the cause yet
18:16:01 <stepcut> the init scripts shutdown the server in such a way that no checkpoint files were written.. and that caused the upgrade to fail
18:17:24 <donri> are you using kill -9?
18:18:12 <stepcut> start-stop-daemon --stop --pidfile /var/run/happstack-dot-com-production --oknodo ${STOPOPTIONS} # --exec /usr/bin/happstack-dot-com -server
18:18:38 <stepcut> STOPOPTIONS="${STOPOPTIONS} --retry 2
18:19:20 <stepcut> I think the answer is that the init script needs a better way to tell the server to gracefully shutdown
18:20:35 <donri> shouldn't SIGTERM do it?
18:20:43 <stepcut> possibly
18:20:54 <stepcut> something that needs to be investigated
18:37:04 <donri> re acid-state recovery, i think functions that don't delete anything but possibly move stuff to temporary locations would be useful
18:38:39 <donri> like, a variant of openLocalState that moves incomplete events and checkpoints and recovers from the latest successful point
18:48:12 <stepcut> maybe
18:50:01 <donri> what happens if acid-state fails to write events during runtime anyway? i mean, not when you're shutting down the app but in the middle of it running
18:59:30 <stepcut> because it is out of disk space or something?
18:59:49 <donri> yea or you chmod/chown the state dir or whatever
19:00:14 <stepcut> i would guess that the update event throws an exception
19:00:30 <donri> what happens to successive events?
19:00:49 <donri> uuh events that are fired after the failing one
19:00:51 <donri> "in succession"
19:00:58 <stepcut> update events? I would assume they all throw exceptions
19:01:27 <donri> because of the first or because the state dir is still not writable? imagine if it becomes writable
19:02:37 <stepcut> then they would probably start working again
19:03:01 <stepcut> how does that make you feel/
19:04:38 <donri> hm no that's probably fine, i was thinking of it incorrectly
19:05:05 <donri> if an update events depends on already written data, it should read the data in the same atomic transaction anyway
19:05:42 <donri> you can't rely on outside state when firing multiple updates in succession
19:07:05 <tazjin> I want to see the acid-state app that runs out of disk space before the memory explodes
19:55:30 <donri> tazjin: quotas? :)
19:56:34 <tazjin> Hmmm
19:58:19 <stepcut> an app that had lots of changes but didn't keep much around could do that
19:58:33 <stepcut> like.. if you used acid-state for 4chan or something
19:58:53 <stepcut> lots of events, but not much retention time
20:11:50 <tazjin> acid-state for 4chan, now that's an interesting thought
20:12:10 <tazjin> also makes me think about how slow Reddit is
20:15:09 <donri> stepcut: chrisdone is doing src-exts -> js http://hpaste.org/67785
20:20:55 <tazjin> stepkut: Nice, now for rewriting Reddit ;-)
20:22:23 <stepkut> donri: so, maybe the next thing to focus on is the clienid stuff for doing serverside sessions?
20:22:51 <stepkut> donri: happstack-authenticate has some code for doing that.. but it doesn't support expiration properly
20:23:14 <stepkut> donri: it sets the right expiration time in the cookie.. but does not on the server side to check it our actually expire sessions
20:23:29 <donri> stepkut: maybe... but i think there are more interesting/pressing matters now that we have a session solution at all?
20:23:45 <stepkut> well. authentication is pretty important :p
20:23:53 <donri> but just doing client ID should be easy
20:24:01 <donri> it's the acid side that's difficult
20:24:26 <stepkut> for happstack-authenticate I need a clientid that expires properly
20:24:34 <stepkut> what do you think is more pressing/interesting?
20:24:39 <stepkut> there are plenty of choices :)
20:25:18 <donri> probably hsx the most ...
20:25:33 <stepkut> which aspect in particular?
20:26:04 <donri> making it more user friendly
20:26:14 <stepkut> yeah
20:26:17 <donri> no overlapping instances, no fixity parse errors
20:26:22 <stepkut> yeah
20:26:31 <donri> fixing the linenos / exact printer
20:26:46 <stepkut> yeah
20:26:53 <tazjin> Will HSX always use an external preprocessor? (Not saying anything against that, just wondering)
20:27:04 <donri> i mean generating html is quite crucial to a web stack, and i am rooting for hsx now :)
20:27:17 <donri> tazjin: there's already a QQ that will be released soon
20:27:21 <stepkut> tazjin: there is a quasi-quote version now, http://src.seereason.com/hsx-qq/
20:27:49 <stepkut> tazjin:  it is not released yet because niklas needs to apply some patches I sent him yesterday and then pull that hsx-qq repo into his account on patch-tag
20:28:15 <tazjin> Hmm, cool
20:28:15 <donri> btw maybe the type classes from hsx should be a separate package, so we can have blaze-like combinators without hsx for example...?
20:28:30 <donri> that's not a pressing matter though :)
20:28:41 <stepkut> donri: niklas proposed that
20:29:17 <tazjin> I think a new forms package would have a rather high priority
20:29:22 <stepkut> donri: first we are trying to get easy issues with the current hsx worked out, and then we are going to refactor a bunch
20:30:03 <stepkut> yeah, the forms stuff and getting the source for happstack.com out are my top priorities at the moment
20:30:10 <stepkut> and keeping niklas moving on hsx
20:30:28 <donri> later we could also do stuff with that like, implement the instances for some xml parser and then we can use hsx' pattern matching with parsed xml etc ... just random ideas
20:30:35 <stepkut> if someone wants to switch HSX to use the exact parser/pretty printer that would be great :) should only take a day or so
20:31:16 <stepkut> we can already make an hsx-html5 combinator library. We do not have to wait to split XMLGen out of the hsx package
20:31:38 <donri> there's a new thread on the ML
20:32:08 <donri> stepkut: exactly... just thinking long-term.
20:32:58 <stepkut> yeah
20:33:31 <donri> but hsx-html5 isn't verily pressing either at the moment
20:33:42 <stepkut> nope
20:34:55 <stepkut> my top 4 priorities for the moment are:
20:35:27 <stepkut> 1. make trhsx work when OverloadedStrings is enabled. The patch in niklas's inbox should resolve that
20:35:51 <stepkut> 2. add hsx QuasiQuoter -- that is done, but requires another patch in niklas's inbox
20:36:36 <stepkut> 3. remove overlapping/undecideable instances from HSX.XMLGenerator... Niklas just needs to be convinced that it will work.. not sure why he isn't
20:37:09 <stepkut> 4. refactor the code so that we do not need to build HJScript/HJavascript or the HSPMonad stuff in HSP
20:38:20 <stepkut> longer term includes, investing if it is worthwhile to switch from String to Text, using the exact parser/pretty printer, fixing HSE to support TH splices, and.. other stuff
20:39:25 <donri> what about TH splices?
20:39:55 <stepkut> you can't do, <p>some $(splice)</p>
20:40:06 <donri> aha
20:40:21 <stepkut> requires a non-trivial fix to HSE
20:43:49 <donri> i wonder if it would be difficult and if it would be useful to support ending any hsx tag with simply </>
20:55:42 <donri> stepkut: there, support for hsx-qq added to vim2hs :)
21:17:22 <stepkut> :)
21:24:15 <tazjin> donri: Which font do you use for vim in the example screenshots on your github page?
21:25:52 <donri> tazjin: ubuntu mono
21:28:01 <donri> colorscheme is hemisu i think but i use neutron at the moment
21:29:05 <donri> vylight is also nice
21:46:06 <tazjin> I can't really switch color schemes because I've optimized them for my transparent terminal windows with Dr. House etc. running in the background
21:46:45 <donri> hah
21:46:53 <donri> get a second monitor :)
21:47:13 <donri> also, not sure if that means you have severe ADHD, or very far from it
21:47:34 <tazjin> I have two iMacs, one of which has an additional monitor connected to it, sitting in front of me
21:47:47 <tazjin> but if I move the show I'm watching to a different monitor I become less productive
21:48:02 <tazjin> because I can't look at two things at the same time ;P
21:48:43 <donri> i'm amazed you can code at all while watching a show
21:48:47 <tazjin> You can still see House ramming a syringe in someone's bladder while defining some data type
21:49:00 <tazjin> It works surprisingly well, you should try it
21:49:23 <donri> i often switch between a show and coding back and forth but can't focus on both at the same time
21:49:36 <luite> can you code better with or without House?
21:50:17 <tazjin> Good question, not sure. It depends on what I'm doing
21:50:34 <tazjin> when I'm writing some front end stuff that I have to look at in a web browser, that doesn't work well while watching a movie/series
21:50:55 <tazjin> with under-the-hood things I'm extremely productive with some entertainment thing going on ;)
22:58:07 <luite> tazjin: hmm, i already get less productive with music on
22:58:37 <luite> meh reinstalling windows 7...
23:01:14 <luite> why so slow
23:13:19 <tazjin> luite: My provider's PPTP server just died, so I guess my last message was lost in the network.
23:13:37 <tazjin> I said that being productive without music isn't possible for me
23:13:56 <luite> oh that's ... interesting
23:15:59 <luite> for some reason i think i'm most productive when either alone in a completely quiet environment, and the second best is with music in some busy environment (train, plane, cafe)