12:59:58 <dark> how should I install happs on gentoo? I don't have both searchpath and happs
13:00:10 <dark> I have cabal though
13:01:40 <dark> I mean, I can't find searchpath at gentoo-haskell overlay
13:03:41 <dark> main = simpleHTTP nullConf {port=5000} , what is this {port=..} syntax?
13:08:29 <Entroacceptor> dark: that's the record syntax
13:08:46 <dark> it's an extension?
13:09:04 <dark> <interactive>:1:0: parse error on input `{'
13:11:06 <Entroacceptor> add a $ after simpleHTTP
13:15:11 <Entroacceptor> or, wait...
13:16:32 <Entroacceptor> you want the handlers
13:18:28 <Entroacceptor> main = simpleHTTP nullConf { port=5000 } $ ok $ toResponse "Hallo"
13:18:30 <Entroacceptor> works for me
13:18:52 <Entroacceptor> it's not an extension, it's the normal haskell record syntax
13:20:12 <Entroacceptor> (and btw., it's called Happstack now)
15:48:42 <stepcut> dark: sounds like you are trying to install happs, which has not been updated in years. Perhaps you want http://www.happstack.com/ ?
15:49:39 <dark> hmmm.. why don't someone update that, then?..
15:49:52 <dark> to redirect people to happstack
15:49:57 <dark> or is happstack a fork?
15:53:39 <stepcut> dunno. None of the happstack developers have access to the happs.org server. Not sure if anyone has tried to get the old happs developers to update it
15:57:41 <stepcut> happstack is not really a fork. The old developers ran out of steam, and so a new development team was put together and the name was changed to reflect a major change in the development model
16:11:36 <Entroacceptor> the last updates from 2008 could give a hint
16:17:58 <dark> but well, this channel is still #happs ...
16:19:57 <stepcut> yes
16:20:25 <stepcut> in theory we could start a #happstack channel, and then get the irc ops to forward #happs to #happstack.. but that is work :)
16:36:18 <dark> I want to generate data using a program, and present the data using another program. The data-presenting program would run forever, and the data-gathering program would run at impredictable intervals. I'm currently using a shell script to generate text-only files, and a small ruby web app to dispaly (using sinatra). I want this to scale. I was thinking about generating the data using a haskell program (to learn haskell, mainly, and because I think has
16:36:18 <dark> kell matches). I decided to use a sql database as the "common format", because it lets one program to update while other is reading, without races
16:36:24 <dark> wow, too big
16:36:41 <dark> but anyway, if I wanted to make the server in haskell too, happstack seems the natural choice
16:37:05 <dark> it seems to discourage sql, for some reason
16:37:45 <dark> can someone discuss this / enlighten me? (I can see the no-sql approach working if the data is produced by the webserver itself)
16:37:53 <stepcut> well, there is nothing in happstack that makes it more difficult to use SQL than it would normally be
16:38:14 <dark> but what would be the "happstack" approach to this class of problems?
16:38:20 <dark> I am using text-only files because I search with grep
16:38:56 <dark> but there is a synchronization problem.. multi-file locking seems like a non-trivial thing to implement in a bare filesystem
16:39:22 <dark> (I avoid it by not implementing everything I thought..)
16:40:42 <stepcut> the files text-files get modified everytime the data-gatherer runs ?
16:40:55 <stepcut> oops
16:41:07 <stepcut> sorry, the text-only files get modified everytime the data-gatherer runs ?
16:42:26 <dark> potentially, yes. the files are re-read each time the page is displayed. (actually, to think about it, I don't avoid the race condition. It could happen, theoretically..)
16:43:10 <stepcut> how much data are you collecting total ?
16:43:12 <dark> like the producer is to change two files; it changes the first; then the data is inconsistent; then the page is displayed
16:43:24 <dark> much data
16:43:34 <stepcut> gigabytes ?
16:44:00 <dark> hmm.. no, but why is it relevant? would you propose something that would not scale to gigabytes?
16:45:20 <stepcut> it depends what you are trying to optimize for.. if you want speed and simplicity, then you can store your dataset in RAM. But that can be costly if you have a lot of data :)
16:46:36 <stepcut> what types of queries are you performing on this data? you said that you are just using grep for the moment ?
16:48:06 <dark> the queries made by the program that is displaying are very simple. it just reads the file and outputs it, mostly
16:48:37 <dark> and, i don't want to store at ram, because at any point of time i am handling with a very small subset of the data. also minimizing the latency of the 'consumer' isn't the concern, but making the 'producer' faster
16:49:25 <dark> (so in fact I could still use ruby and sinatra for displaying)
16:52:24 <stepcut> is the only deficiency of the current scheme the lack of proper 'locking' ?
16:57:01 <stepcut> maybe the producer should write its output to a new set of files, and once it is done it notifiers the viewer, which then switches to the new set ?
16:57:27 <dark> mostly, but also the performance of the 'producer'
16:57:33 <dark> hmm, i know of some renaming tricks
16:57:52 <dark> such as having a directory named foo, write to directory foo.new, then rename foo to foo.old, then foo.new to foo
16:58:01 <dark> but I haven't cared to implement this
16:58:22 <dark> I would have to copy all the data from foo to foo.old first, I think..
16:58:27 <dark> or make hardlinks
16:58:35 <dark> (but I change just a small set..)
16:59:39 <stepcut> seems like writing directly to files is going to be hard to beat in terms of speed.. but maybe it is because I don't understand how those files need to be updated..
17:01:37 <stepcut> for example, if you are just appending data to the end of files or adding new files.. that is going to be pretty fast.. but if you have to read in the old files, find certain sections, and update them, and write out the changes.. then that could be slow
17:03:20 <stepcut> anyway, with out really understanding anything about your data set or needs, here is what you might do in happstack
17:04:18 <stepcut> 1. create some Haskell data types which are nicely suited to storing your data and for which it would be easy to write normal haskell functions to query the data and generate the output you want to display
17:06:28 <stepcut> 2. add some Serialize instances for those types, and write the functions that query the dataset. Now you have most of what you need to store that data in happstack-state.
17:08:29 <stepcut> 3. write some code that allows the server to import new data from the data gatherer. It could be done over http by having the client POST the data as multipart/form-data. Or you could have the server listen on a dedicated socket and receive the data in a specialized binary format.. whatever works for you.
17:08:58 <stepcut> 4. write rest of the code which uses the query functions (from #2) to query the data and generate the HTML to display to the user
17:09:09 <stepcut> but, maybe step 1 isn't sensible for your data ?
17:09:43 <stepcut> in which case, maybe you need a different data storage and query library ?
17:25:32 <dark> sorry, I had to leave
17:26:09 <dark> the performance problem isn't necessarily related to the filesystem writing itself, but maybe due to have use of string processing (sed, etc)
17:26:41 <dark> I haven't did any measurement.. but that would be substituted by haskell data types, plus symbolic processing, yes
17:28:27 <dark> I don't want to code anything special in the server part, it should be dumb and don't know much about anything. reading from a sql database, or from a file, is pretty much standard..
17:47:52 <stepcut> ok.. well if your generator is going to stick things into SQL, and you are fine using sinatra/ruby for the display portion, then I am not sure what use you have for happstack
18:04:41 <dark> but I was thinking. the approach of happstack for persistence (keeping data at ram, writing changes first on the disk), it's adequate for data generated by the web application itself, right?
18:05:39 <stepcut> sure..
18:06:06 <stepcut> though happstack-state is separate from the web stuff.. you can use it in any application
18:07:02 <dark> interesting