--- Log opened Mon Jul 27 00:00:42 2009
11:43 < stepcut> mae: is there some reason to delay updating happstack to use sendfile 0.4?
15:46 < alexj> sm: just pushed an update of the searchpath homepage.
16:31 < sm> looks good alexj
21:29 < abuiles> Someone around ?
21:31 < mae> stepcut: because there is still problems, i am releasing a new version of sendfile which fixes alot of things
21:31 < mae> namely, short sends, and large file support
21:31 < mae> win32 is the real bitch child
21:31 < mae> i have to wrap TransmitFile in safe
21:32 < mae> because it does some voodoo magic with msvcrt
21:33 < mae> freebsd is also borked
21:33 < mae> because it ignores the current location of the file handle
21:33 < mae> and deals only in absolute offsets
21:34 < mae> so basically i have to overhaul the interface to use an offset again
21:34 < mae> and document that what happens to the handle in terms of mutation is "undefined"
21:34 < mae> good news is, the linux implementation is pretty screaming now, just have to polish a few more bits off
21:45 < abuiles> mae: happstack tutorial is still pointing at the old site, and hasn't been moved to tutorial.happstack.com
21:45 < mae> ahh ok
21:45 < gwern> people've been complaining about it surprisingly often in #haskell
21:45 < abuiles> I just wrote you and email.. I had thought you weren't here
21:46 < gwern> 'whys happstutorial.com down, why's it still down'
21:46 < mae> ah just the link on the tutorials page right
21:46 < mae> hehe
21:48 < mae> ok fixed
21:48 < mae> thanks :)
21:48 < abuiles> do you know guys, with HSX if I send a string containing HTML how can I get the actual html, and no the string chunk..?
21:50 < mae> wow dude
21:51 < mae> after this last patch to sendfile, 3gb is sent without hardly flinching
21:51 < mae> asynchronous i/o is really a beautiful thing
21:54 < stepcut> mae: so, 0.3.1 is better than 0.4?
21:59 < mae> well, the only diff is
21:59 < mae> 0.4 has support for Socket as an output
21:59 < mae> and experimental freebsd support
21:59 < mae> 0.5 will change the interface once again (hopefully the last time), have improved freebsd support, and does much better with larger files (and can handle files >2gb)
22:00 < mae> when sendfile settles down a little, I am going to work with tibbe on the event manager so that we can scale above the 1024 file handle limit currently imposed by the select-based event manager in ghc
22:01 < abuiles> mae: so man, any hint with my problem with hsx ? I've tried converting my string to xml and then rendering but didn't worked
22:02 < mae> as far as happstack goes, I want to trim down util (ie get rid of crypto and other redundant items), alex seems to be renewed and working on -state, and I hope to tweak the http core eventually (when i fix the basic low level stuff)
22:02 < mae> abuiles: I am not the person to be asking about hsx :) I have only dabbled in it
22:02 < mae> have you looked at the example application?
22:03 < abuiles> Yeap. Actually I decided to use HSX because of the example application.
22:04 < mae> stepcut: concurrency dies when you get around 1024 file descriptors (or whatever select limit your os imposes), since the io manager in haskell does not use any of the advanced event-based fd handling which freebsd/linux/windows offers, any net code will suffer from it
22:04 < mae> stepcut: so if we can implement epoll for linux / kqueue for bsd / completion ports for windows, this makes the only arbitrary limit to be basically your ram and your file descriptor limit (linux can be configured to have over a million file descriptors i think)
22:06 < stepcut> mae: is it a bad idea to run a live site that uses sendfile 0.3.1 (and happstack?)
22:06 < stepcut>  
22:06 < mae> stepcut: if your only serving up small static files your probably fine
22:06 < mae> and fileServe before had its share of issues
22:06 < mae> so i'd say your about where you were before :)
22:06 < mae> if not better
22:06 < stepcut> sweet
22:07 < mae> how big are your static files?
22:07 < stepcut> small. Just jpgs and stuff
22:07 < mae> i mean I didn't even know there was a problem until I started testing 3-4mb files, (my tests were using small amounts)
22:07 < mae> i ran test suites with a count of like 100 thousand in quickcheck before without any problem
22:07 < mae> until i went to around 3-4mb
22:08 < mae> so yeah, sendfile 0.5 will be tested up to 3gb
22:08 < mae> it works on linux currently and slowly on windows
22:08 < stepcut> spiffy
22:08 < stepcut> I upgraded our local happstack reposotiry to head today
22:09 < mae> i haven't been able to get the windows version to work reliably without using a "foreign import ccall safe"
22:09 < mae> which means that all io is blocked until transmitFile returns :\
22:09 < stepcut> mae: joy!
22:09 < mae> stepcut: ok, I am still not sure I want the sendfile implementation to stay how it is
22:09 < mae> do you think the constructor should take simply a file with no count?
22:09 < mae> or is the handle / length better
22:10 < mae> we could even put offset in there
22:10 < mae> but i don't know how useful that would realistically be
22:10 < stepcut> mae: no idea, I have not looked at it at all
22:10 < stepcut> mae: step 1 was to become current again :)
22:10 < mae> stepcut: btw, we got a significant performance boost on patch-tag by upgrading from ghc 6.10.1 to 6.10.3
22:11 < stepcut> mae: cool, we just built everything with 6.10.4
22:11 < stepcut> mae: unfortunately, haddock is not being nice... it spits out 3-19MB C files that cause cc1 to take forever to compile (like 30+ minutes)
22:12 < mae> aw man, baby is crying, in any case, look at this: http://patch-tag.com/r/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Types.hs line 100-106, and if you feel brave look at FileServe.hs I dare you.
22:12 < stepcut> though it seems faster on jaunty than hardy
22:12 < stepcut> mae: I'm headed to bed, perhaps tomorrow
22:12 < mae> stepcut: well just look at the first part
22:12 < mae> so it can swim in your head
22:12 < mae> its only 5 lines
22:12 < stepcut> :)
22:12 < mae> and you'll get the jist
22:17 < mae> stepcut: in any case, i'll talk to you when your not sleepy
22:20 < abuiles> mae: hsp.xml cdata function solved my problem :)
22:25 < mae> abuiles: cool :)
22:32 < mae> stepcut: when i was talking about the "sendfile implementation" earlier I meant in the context of happstack, not the library itself
--- Log closed Tue Jul 28 00:00:43 2009