--- Log opened Thu Aug 13 00:00:59 2009
21:11 < Ford_Perfect> Hi I was told that Happs/Happstack is fastest webserver (as in number of responses per second) in Haskell, is there some recent performance test, and how is it compared to Hyena (some other web server mentioned to me in Haskell)?
21:12 < sm> in my tests it was slightly faster than hyena. Not very significant. Hyena is only getting started though
21:13 < Ford_Perfect> and howmany request/second did you get from  Happs? and on what hardware?
21:14 < Ford_Perfect> ( i am planing to serve/cache small 200-300 bytes AJAX requests)
21:33 < sm> Ford_Perfect: I am forgetting. One second.
21:39 < sm> I see about 285 req/s on a macbook for a static file. Just a data point
21:41 < Ford_Perfect> This sounds a little low, as in 2 years old YAWS benchmark could do 3K-4K req/second, Lighthttpd has also 3K+
21:42 < Ford_Perfect> ill try to find benchmark
21:42 < sm> yes, I'd be surprised if this is the best it can do
21:42 < sm> someone else here should know more
21:43 < Ford_Perfect> well you and me are only two awake it seems ;)
21:45 < Ford_Perfect> I mean its strange since Haskell itself has so good performance, something should be very wrong in design of server itself to trail so much (haskel is 1.2-2.0 times slower than C in language shoutout)
21:46 < sm> the WASH paper reported haskell performing on par with apache
21:47 < sm> mightybyte, stepcut, mae: any idea ?
21:48 < Ford_Perfect> do you have link to WASH maybe? i dont think I heard about that
21:48 < Ford_Perfect> i get some stosk exchange on google for wash
21:48 < sm> haskell wash
21:49 < sm> http://www.informatik.uni-freiburg.de/~thiemann/WASH/draft.pdf
21:49 < Ford_Perfect> LOL thanks stupid me ;)
21:49 < Ford_Perfect> forgot to write haskell
21:50 < sm> darn, can't find the speed discussion.
21:50 < sm> anyway. You probably have enough info to go on
21:51 < Ford_Perfect> Ok thanks, I would not like to take any more of your time, ill manage
21:54 < sm> http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=677B6A0DA76372AE1D99E76C5B7A43E7?doi=10.1.1.35.8701&rep=rep1&type=pdf  section 7 is what I was thinking of
21:55 < sm> I imagine YAWS and lighttpd do more in-memory caching than a naive haskell webserver
21:55 < mightybyte> sm: I've never done any performance testing.
21:56 < sm> "fast enough", eh
21:56 < mightybyte> Yep
21:56 < sm> me too
21:56 < mightybyte> No problems for my ~150 uniqs / day.
21:56 < mightybyte> Although I've peaked at ~1000 uniqs in a day.
22:01 < Ford_Perfect> well problem is this is not end user site, it is cache for some other site that has 200K+ requests/sec itself, and whole clusters of servers (PHP)  i am building cache that would primarly be there to spedup response time because it curently takes 100+ms for each ajax request on top of network latency, and 2-4 cache servers would serve ALOT ordinary slow apache/PHP servers so they have to be much faster than them
22:02 < mightybyte> Well, if you're trying to get really good performance I wouldn't serve from files.
22:02 < sm> why not use squid or equivalent ?
22:03 < stepcut> sm: a static file via serveFile ?
22:03 < sm> stepcut: yes
22:03 < stepcut> sm: using the new sendfile code?
22:03 < Ford_Perfect> well requests need to be modified between receiving and sending because alot of users will get same data but embeded session ID in data needs to be replaced
22:04 < sm> Ford here is looking for ballpark figures for how fast current haskell webservers can go
22:04 < stepcut> sm: serveFile is not especially great. Might be more useful to serve static compiled in text?
22:04 < mightybyte> Ford_Perfect: Why not generate it on the fly?
22:05 < Ford_Perfect> and for sendfile, is it working with files only or also memory cache, because im afraid linux box will not be able to withstand 100K file opens per second ;)
22:05 < sm> when I serve a static file with current happstack(+maid), is it reading from disk each time ? I think so
22:05 < mightybyte> Ford_Perfect: Well then don't use a file.
22:06 < Ford_Perfect> I dont actualy need to write anything to disk, there is no data that is "important" as in this is cache server and if there is data loss it will just re-request it
22:07 < Ford_Perfect> i thought that best route would be to hold responses in hash array (for 60 seconds until they stop being valid)
22:08 < mightybyte> Ford_Perfect: I'd start by just generating them all on the fly, and then move to caching if the simple generation approach doesn't work.
22:08 < Ford_Perfect> so request thread would read one of "data nodes" from hash array, replace 1 value with current session ID, and send it off back ...
22:08 < mightybyte> You'll still be able to go quite fast if you're generating from memory and not reading from disk.
22:08 < Ford_Perfect> well i wasnt, SM recomended it
22:09 < sm> not me
22:09  * mightybyte banishes sm from #happs ;)
22:09 < sm> bah
22:09 < Ford_Perfect> ah sorrey it was stepcut
22:09 < Ford_Perfect> <stepcut> sm: using the new sendfile code?
22:09 < mightybyte> :D
22:10 < Ford_Perfect> please dont banish him you know we cant do any damage to him while hes banished ;)
22:10  * sm constrains Ford_Perfect to write it in ruby
22:10 < Ford_Perfect> LOL
22:41 < Ford_Perfect> BTW i found some benchmark with Happs geting 3K requests per second, very nice, it is 2 years old (benchmark)  did Happs/Happstack have any big optimizations since than? link :    http://blog.davber.com/2007/12/10/web-server-performance-shoot-out-simple-pages/
22:47 < sm> Ford_Perfect: nice
22:47 < sm> well, ghc generally makes better code now
22:53 < Ford_Perfect> is it me or are all webservers -  THEORETICALY  (not yelling, no underline here ;) ) much slower than they could be, i mean with 3Ghz and 6GB/sec bandwith they use like 1 milion instructions, and 2MB of memory throughput or more per request thats like few hundred bytes in size, I am sure there are some overheads that mandate this, I would just like to know what they are?
22:59 < gwern> yes
22:59 < gwern> but to get down to the limits require exotic technologies like exokernels
22:59 < gwern> and exotic = bad
23:00 < Ford_Perfect> so you say OS eats all mem bandwith and CPU?
23:01 < stepcut> getting optimal performance out of a single machine is probably not that interesting, because then what? Get 50% optimal performance per machine, but be able to scaled to huge numbers of machines seems better?
23:02 < Ford_Perfect> offcourse scaling is important but thats what you do by not making shards/servers dependant too much on each other
23:02 < Ford_Perfect> its not what is webserver job, its app developer job to do scaling
23:02 < gwern> Ford_Perfect: it's a combination of userland and kernel
23:04 < Ford_Perfect> well gwern I am beliver in 80/20 rule, i am sure some component of stack eats 80% of that, i am just interested (if you know) what component, is it some os component, or context switches, or ...?
23:05 < gwern> no, it's everything
23:06 < gwern> it's inefficies in the kernel's code; it's inefficiencies in copying data around; it's blowing the cache; it's recomputing hashs and alignments all over the place; it's recomputing headers by the server, the OS, etc
23:06 < gwern> assuming there aren't even any high-level problems in the server
23:07 < gwern> it's the OS providing services unnecessary and making assumptions and checks that JIT/runtime code generation could avoid; it's the app using those services, or unavoidably calling things which use them
23:07 < gwern> etc
23:08 < Ford_Perfect> aha so basicly alot of data gets copied around
23:08 < Ford_Perfect> and what about hashes, what hashes are needed?
23:09 < Ford_Perfect> doesnot Linux have option to send part of memory directly to network? (used by file send i think)
23:10 < gwern> zero-copy stuff addresses a few issues
23:10 < gwern> but there are many many more
23:10 < gwern> go read the Synthesis kernel thesis and the exokernel papers just to cover the ones I've mentioned so far
23:11 < Ford_Perfect> as i suspected, i was just hoping its some specific thing that was responsible for majority of it, but thanks gwern now i have somewhat better picture what is hapening ;)
23:11 < Ford_Perfect> im just reading http://en.wikipedia.org/wiki/Exokernel#Networking, ill look into Synthesis  kernel right after that, thanks once more gwern
23:38 < dancor> this is the _cool_ way to underline
23:38 < Ford_Perfect> ah it didnt cross my mind, nice thinking dancor ;)
23:39 < mightybyte> Frequently *'s are bold and _'s are italics
23:40 < Ford_Perfect> * would do ;)
--- Log closed Fri Aug 14 00:00:00 2009