00:06:21 <srhb> Using lookFile, what's the correct way to check if I really did receive a file? Sizing up the temporary one?
00:06:35 <srhb> and likeWise for other fields, is matching on "" just the way to go?
00:10:16 <stepkut> hmm
00:10:34 <stepkut> usually
00:11:02 <stepkut> unfortunately, http doesn't do the best thing
00:11:31 <stepkut> you want to know if the user submitted a form with a file field, but did not actually specify a file to upload ?
00:12:00 <srhb> Yes, or did some other weird stuff like specifying a file but it didn't get uploaded (for some reason)
00:12:12 <stepkut> if that is the case.. then I usually check the the uploaded file is null (and also, if the uploaded name is null)
00:12:39 <srhb> Sounds like lookFileMaybe ;)
00:12:53 <stepkut> possibly
00:13:16 <stepkut> I am not against the idea :)
00:14:05 <stepkut> I think when I wrote the file handling combinators, I was under the incorrect impression that the field would be missing entirely if no file was selected (which is how some form elements are handled)
00:14:54 <srhb> But still, I would assume the function I should use should Do The Right Thing whether the user failed to specify or the user's broken browser failed to actually send the file, or anything in between.
00:15:15 <stepkut> yes
00:15:35 <stepkut> broken files is a bit harder/impossible to detect
00:15:35 <srhb> Mind I'm just learning the basics here.
00:15:40 <srhb> Right
00:15:43 <srhb> null files are easier
00:15:53 <srhb>  /possible
00:15:54 <stepkut> if the connect gets broken before the file gets uploaded, then I think the whole connection will be tossed and the code won't even get that far
00:16:01 <srhb> Me too.
00:16:29 <stepkut> if the browser only uploads half the file and then pretends it upload the whole thing.. there is nothing we can do.. unless it used an upload method that encoded the correct file size
00:16:56 <stepkut> if it said it was supposed to upload 1000 bytes, and only uploaded 500.. then some other layer will catch that and return badRequest or something like that
00:55:01 <srhb> What's the right way to go, store a UserId for each Post, and manually make sure the correlation is OK? This is where relational stuff is smart I guess, being able to just refer to an entire User
00:56:51 <stepkut> not sure I understand the question.. something like, data Post = Post { postId :: PostId, postedBy :: UserId, postBody :: Text }, ?
00:57:00 <srhb> Right
00:57:12 <stepkut> that's what I usually do
00:57:23 <srhb> I think I'm basically saying "I can't do Post { postedBy :: User } -- can I?" :P
00:57:32 <srhb> Because I'd just be copying the entire user then
01:01:11 <stepkut> indeed
01:01:23 <srhb> Just making sure my intuition still holds.
01:01:24 <srhb> Thanks.
01:01:27 <stepkut> yeah
01:04:02 <srhb> Why isn't query and update wrapped into the value constructors made by makeAcidic?
01:04:13 <srhb> Seems like you'll never seperate those.
01:05:09 <stepkut> sometimes you use alternative definitions for query and update which do things like pull the AcidState from the monad environment
01:05:46 <srhb> Okay.
01:05:46 <stepkut> also... it can be useful to be able to see where you are actually making database calls..
01:05:51 <srhb> True, true.
01:29:41 <alpounet> donri, after mocking your cabal file generation, i might after all use happstack-yui :]
01:30:22 <stepkut> I don't happstack-yui does that anymore anyway?
01:32:25 <alpounet> oh? haven't checked for a while, you're probably right
01:36:21 <stepkut> i vaguely remember donri saying something that leads me to believe that
09:57:35 <srhb> I'm not sure I understand how migration works. Say I have a record, and I change one of the fields' types. I feel like I will manually have to do something to transition the data already there, but what?
10:05:17 <donri> srhb: typically you copy the old record to new names (but same structure), increment the (new) record's version in its SafeCopy instance and add a migration for the new record from the old
10:05:23 <donri> have you seen http://hackage.haskell.org/packages/archive/safecopy/0.8.1/doc/html/Data-SafeCopy.html ?
10:06:59 <srhb> Ah.. OK, uncolliding record names is where my mind snapped, I think.
10:07:01 <donri> to keep your namespace sane you could strip the copy of record syntax because it's not used by safecopy AFAIK
10:07:16 <donri> yea, safecopy knows nothing about names
10:07:28 <donri> it just picks what you state in the MigrateFrom type family
10:07:38 <srhb> Right.
10:10:00 <srhb> Can I not still ues deriveSafeCopy on my old and new types? Or does that not even help me?
10:10:00 <donri> you can! just use deriveSafeCopy (oldver+1) 'extension ''NewRecord
10:10:00 <donri> i forgot to mention you keed to set the "kind" to "extension" (and keep the first version the "base")
10:10:00 <srhb> Oh, extension..
10:10:41 <donri> s/keed/need/
10:25:27 <srhb> Can I delete my migration afterwards? Or must I keep it for eternity?
11:41:01 <srhb> If anyone could be bothered to scrollback and see if I got an answer to my last question, I'd be thankful. :-)
12:59:17 <srhb> So, what _seems_ to work is to change 'extension to 'base after running the app with the migrations available.
12:59:28 <srhb> But does that only work for certain corner cases?
15:04:54 <donri> srhb: you only need to keep the migration for as long as you want to be able to migrate existing states
15:05:31 <donri> if you only have one deployment and you're the only user of the code, you could remove it after deploying it once
15:06:47 <donri> srhb: btw there's irc logs in the topic
15:06:54 <donri> but no one answered you before
15:15:56 <srhb> Yes, I found the logs, thanks. When you say "after deploying it once" does that mean that once I've run my application with the migration instances, I know that everything is done?
15:16:16 <srhb> And if so, what is the correct way to remove the migration stuff and old data types? It seemed like I had to revert to 'base rather than 'extension to get it to work
15:18:27 <srhb> And am I supposed to revert the version number or leave it?
15:23:20 <srhb> donri: And when you say if I don't want to be able to migrate existing states... Does that mean that it's basically a policy to keep all migrations and previous data types somewhere? Doesn't that get messy fast, or do you segregate it into different files?
15:23:34 <srhb> Sorry for the spamwall, the crash course is not really elucidating this part for me.
15:24:09 <donri> srhb: you probably need to checkpoint, then you can remove the migration
15:25:20 <srhb> Ah, so I explicitly use createCheckpointAndClose
15:25:31 <donri> that's one way yea
15:25:46 <donri> after that the state should be written with the migrated data
15:25:54 <srhb> donri: And then I remove the old datatype, the migration instance and what do I do with the derivesafecopy for the new (current) datatype?
15:26:01 <srhb> Should I revert the version to, say, 0?
15:26:07 <srhb> And what about 'base vs. 'extension?
15:26:20 <donri> i think you just keep the derive as-is, but honestly i don't think i've ever done a migration :D :D
15:26:28 <srhb> Haha! Okay. :-)
15:26:39 <srhb> Where's Lemmih when you need him...
15:27:28 <donri> i'm quite sure you keep the 'version' anyway, not sure about the 'kind'
15:27:49 <donri> not even sure why the kind is needed for safecopy to begin with
15:27:51 <srhb> Basically it didn't compile for me if I kept 'extension, but I'm not sure if I fucked something else up.
15:28:19 <srhb> And this feels like an area where I should tread carefully. :-)
15:30:35 <donri> stepcut should be around soon
15:30:35 <srhb> I'm leaning towards creating a Migrations module right now with every version just in there. It's a bit annoying to have to rewrite all those data types though.
15:30:35 <srhb> Since it's basically just throwing fluff names at every field.
15:30:35 <donri> agreed
15:30:51 <donri> stepcut seems to favor the "tack _V[x] on the end" method, myself i'm thinking with proper use of modules you should be able to keep old versions as-is but just move them
15:31:15 <srhb> AcidTypesV1.hs ... ?
15:31:16 <donri> like, MyApp.Data.SomeRecord.V0.SomeRecord
15:31:21 <srhb> Ah, yes.
15:31:26 <donri> i was thinking per record ;)
15:31:32 <srhb> That's a good idea.
15:31:37 <donri> it's also a bit of work
15:31:40 <srhb> Yes.
15:31:51 <srhb> And cabal-dev gets annoyed when I move things into subdirectories.
15:31:56 <srhb> Well, annoying more like :P
15:32:07 <donri> then in MyApp.Data you import all versions qualified as V0 etc
15:32:13 <srhb> Right.
15:33:01 <donri> you can share 'as'-names for multiple imports if they don't have clashing names
15:33:08 <donri> clashing exports i mean
15:34:11 <donri> i'm not sure you can have a single module per version with multiple types without running in to circular imports
15:34:13 <srhb> So you would still have to name the records differently in each submodule? I'm not sure I follow
15:34:17 <srhb> Hm.
15:35:43 <donri> nah you still call it SomeRecord but you always import qualified as V0 and refer to SomeRecord.V0
15:35:57 <srhb> Ah, right.
15:38:00 <donri> but the definition for SomeRecord might refer to other custom types of yours so it needs to import other records itself
15:38:00 <srhb> Yeah, it can get scary quickly
15:38:00 <srhb> Hm.
15:38:00 <donri> i think my method is more manageable especially if you want to keep migrations (and this way there's little reason not to)
15:38:00 <donri> when you change a record you just copy the latest version module and increment it
15:38:00 <srhb> Yes, I think I will adopt it.
15:38:00 <srhb> Indeed.
15:38:03 <donri> but, i haven't tried either approach in practice :D
15:38:16 <donri> bbl feed kittens
15:51:19 <totimkopf> mew.
16:39:30 <srhb> I'm pondering module hierarchy in cabal. Is the usual process to have Main.hs in the root of the cabal project, or should I move it to root/MyApp/Main.hs ? And then have root/MyApp/HelperModule1/ ... ?
16:40:12 <stepkut> usually it's just Main.hs or src/Main.hs in my experience
16:40:23 <srhb> And then root/Types etc.?
16:40:36 <stepkut> that is what cabal expects.. though can can override it with the Main-is: field
16:40:48 <stepkut> it depends on how you structure your app
16:40:56 <srhb> Yeah, I'm not sure how to structure my app. :P
16:40:58 <stepkut> I try to make as much of my app into a library as sensible
16:41:09 <stepkut> and then make the Main.hs a wrapper that calls into that
16:41:12 <srhb> That's my vague plan as well.
16:41:35 <srhb> So you would avoid having something like root/Types/Foo.hs?
16:42:15 <stepkut> i'm not sure anymore
16:42:19 <srhb> :-)
16:42:28 <stepkut> these days pretty all I write are clckwrks based apps
16:42:44 <stepkut> where there is a mostly stock Main.hs and everything else comes from plugin libraries
16:43:08 <srhb> Ok. :)
16:44:13 <stepkut> for command-line applications, trying to make it into a good library first, and then write a command-line wrapper around it second, is a good idea.. because ultimately, you will want to call your command line app from another application, and you'll wish it was a nice library.. but if you don't start that way, you'll probably never get there
16:44:32 <srhb> Aye.
16:44:41 <stepkut> for a single web application executable.. not sure it really helps much to create an extra layer in the hierarchy
16:44:44 <srhb> Right now Main is just routing and launching the happstack server
16:44:46 <srhb> Indeed.
16:45:00 <srhb> So I think I might just make Types, Forms, Layouts, etc.
16:45:06 <srhb> as subdirectories, I mean
16:45:09 <stepkut> seems fine
16:45:18 <srhb> Thanks for the reassurance :-)
16:45:31 <srhb> By the way, I had a slightly unresolved question about migrations earlier
16:45:55 <stepkut> i think I've tried a lot of different hierarchies.. and in the end it doesn't really make a big impact..
16:46:13 <donri> stepkut: srhb was wondering how you go about removing a migration, do you keep the new version an 'extension' for example?
16:46:14 <stepkut> and if it does, it's not that hard to move things around
16:46:25 <srhb> If I were to delete my migrations after forcing a checkpoint, how would I make my app work again? Should I just change $(deriveSafecopy v 'extended ''Foo) to $(deriveSafecopy v 'base ''Foo) ?
16:46:29 <stepkut> no, you set it to base, but leave the version number the same
16:46:44 <stepkut> 'base' is not the best name for what base really means
16:46:53 <donri> srhb: i keep haskell modules in src/ and executables in bin/ named like the exe
16:46:54 <srhb> I see. That's great to know. :-)
16:47:20 <stepkut> donri: I dislike doing that, because then I have to do :set -isrc to make ghci happy
16:47:46 <donri> needs moar cabal repl
16:48:06 <dcoutts> donri: working on it, working on it...
16:48:14 <donri> :)
16:48:26 <stepkut> i will note, though, that putting the source in src/ is a very common practice
16:48:50 <stepkut> i think happstack does it even (because it was like that when I got it)
16:49:10 <donri> actually cabal will get confused in some situations if you don't use hs-source-dirs
16:49:11 <srhb> Yeah getting cabal-dev to play nice with subdirectories seems to be a pain in general :P
16:49:14 <stepkut> in safecopy, 'base just means 'end-of-the-line
16:49:23 <srhb> stepkut: Ah, okay. That makes sense.
16:49:25 <stepkut> it just marks what the oldest version you support it
16:49:39 <stepkut> when you serialize data, a version number is serialized with that data
16:50:02 <srhb> So altering the version name backwards as a lie is a bad idea. Which doesn't accomplish anything but breaking the app anyway.
16:50:10 <stepkut> when it reads back the data, it checks if the version number in the save data matches the version number in the current code
16:50:11 <donri> why is 'kind' needed though? can't you tell from the MigrateFrom types?
16:50:55 <stepkut> if not, it walks backwards through the Migration instances until it gets to a matching one and then performs all the migrations needed to bring it up to date
16:51:39 <stepkut> donri: dunno, I'd have to look at the code to see
16:51:44 <donri> hm i guess safecopy would have to be a haskell compiler to tell from the instances
16:51:57 <donri> or use the ghc apis
16:53:47 <srhb> Say I have root/Pages/Uploader.hs and I want to import root/Forms/UploadForm.hs .. How does that work when there's no common reference module as the "root" module?
16:54:22 <donri> import Forms.UploadForm
16:54:30 <srhb> Just like that, huh. Nice.
16:54:43 <donri> works if you pass -iroot to ghc, which cabal does for you if you set hs-source-dirs: root
16:54:47 <donri> but call it src, srsrly!
16:56:51 <srhb> so myCabalDir/src/Main.hs,Types,Forms,Blah...
16:56:51 <donri> if you like
16:56:51 <srhb> :-)
16:56:51 <donri> i'd make that myapp/src/MyApp/{Data,Form,...}.hs, myapp/bin/my-app.hs
16:56:51 <donri> but i'm a sucker for order
16:56:51 <stepkut> I do things that way sometimes
16:57:29 <donri> i also like to write my apps as libraries
16:57:37 <stepkut> in other news, I hacked hyperdrive so that you can easily use different parsers
16:57:47 <donri> if you only have an executable it matters less
16:58:04 <stepkut> and hacked up (emphasis on hack) a version that uses the pipes-parse library
16:58:36 <donri> stepkut: cute. does it work by parsing to the same data types? is the parser then simply a pipe in the chain?
17:00:09 <stepkut> yeah
17:00:50 <stepkut> things still don't fit quite right.. need a bit of refactoring and maybe some small type changes
17:01:19 <stepkut> but if you look in here, http://hub.darcs.net/stepcut/hyperdrive/browse/hyperdrive/Serve.hs
17:01:29 <stepkut> you'll see there is now an httpPipe and httpPipe2
17:01:36 <donri> speaking of hierarchies and hyperdrive... hyperdrive/pipes-http-7/Request.hs :D
17:01:53 <stepkut> pipes-http-7 is obsolete
17:02:03 <stepkut> it is in hyperdrive/hyperdrive now
17:02:14 <stepkut> I'll probably move everything else into the attic soon
17:02:54 <stepkut> httpPipe calls the parseRequest from the original parser that just uses normal bytestring manipulation functions
17:03:02 <stepkut> httpPipe2 uses the pipes-parse version
17:03:07 <donri> i was just joking, "call it src? call it root? no call it pipes-http7 and put modules like Request in the toplevel!" :)
17:03:20 <stepkut> though, it really needs the bytestring version of the pipes-parse library, which is not out yet
17:03:43 <stepkut> :)
17:03:58 <stepkut> I usually start with everything at the top, and organize it a bit later
17:04:18 <donri> yea i figured
17:05:30 <donri> i hate typing mkdir -p src/Foo/Bar/Baz oops Baz was the module backspace backspace backspace enter touch src/Foo/Bar/Baz.hs
17:05:36 <donri> so i feel ya'
17:06:00 <donri> i need to automate that in vim somehow
17:14:25 <donri> stepkut: re httpPipe2; ah, cool. how does pipes-parse seem?
17:15:11 <stepkut> seems like a parser
17:15:22 <stepkut> couldn't find a many1 combinator, but perhaps I missed it
17:15:34 <stepkut> has some other interesting ones like 'few'
17:15:42 <stepkut> oh.. many1 == some or something
17:15:51 <donri> :t some
17:15:53 <lambdabot> Alternative f => f a -> f [a]
17:15:56 <donri> :t many
17:15:58 <lambdabot> Alternative f => f a -> f [a]
17:15:59 <donri> gah!
17:16:13 <stepkut> yeah
17:16:19 <stepkut> I just remembered that
17:16:21 <donri> yeah 'some'
17:17:14 <donri> @hackage parsers for extra combinators!
17:17:14 <lambdabot> http://hackage.haskell.org/package/parsers for extra combinators!
17:17:57 <stepkut> now that io-streams it out.. I guess we can figure out what IO mechanism to use
17:18:11 <stepkut> have a shootout of the 4 approaches, lazy IO, strict IO, io-streams, and pipes
17:20:17 <stepkut> and see how they handle concerns like space leaks, generating a streaming response that requires IO, exception handling, etc
17:20:17 <stepkut> ability to abstract away from the actually data source and sinks
17:20:17 <donri> i suspect that some of the problems with lazy IO are of the kind that are hard to simulate with tests and benchmarks
17:20:46 <stepkut> sure.. but we know some issues that we can talk about explicit
17:20:52 <stepkut> there are two in happstack
17:21:19 <stepkut> 1. if you are working on the internals, you have to be careful to preserve the streaming nature of lazy IO and not accidentally read the whole thing into RAM at once
17:21:38 <stepkut> so example, if you are working with the request body.. you shouldn't try to use the 'length' function on it..
17:22:08 <stepkut> that mostly only affects people hacking on the internals, users are usually working with everything through an abstraction layer (like RqData) that takes care of those issues
17:22:37 <stepkut> 2. if you want to generate a streaming response that requires IO.. you need to use unsafeInterleaveIO
17:24:04 <stepkut> for example, if you wanted the response to be an infinite stream that spits out the time every second..
17:24:19 <stepkut> the Respnose body is just a lazy ByteString.. but generating that response requires IO
17:24:36 <stepkut> so you have to resort to usual lazy IO trickier to make things meet
17:25:10 <donri> sounds fun
17:25:24 <stepkut> fortunately, people seldom need to do that either
17:25:36 <stepkut> but it is one of the ugly parts of using lazy IO in happstack
17:25:50 <stepkut> obviously, if those are the only two issues that we know of after all these years.. then lazy IO isn't that bad :)
17:26:27 <stepkut> I'm still not really sure what the advantages of using pipes are
17:26:52 <stepkut> pipes is neat, and all, but I would need to be able to provide some concrete advantages that are gained by using it
17:27:00 <donri> http://stackoverflow.com/questions/5892653/whats-so-bad-about-lazy-i-o
17:27:03 <stepkut> which is what the shootout is about
17:27:07 <donri> sounds good
17:27:37 <donri> empirical tests are always nice, even if "benchmarks be damned" :)
17:28:16 <stepkut> yeah, resource allocation and release is another issue
17:28:26 <stepkut> well, I am not going to do empirical tests really
17:28:55 <stepkut> mostly, I want to create a list of all the issues we care about for web server programming, and then compare how each IO model handles them
17:29:22 <donri> aye
17:30:05 <stepkut> io-streams seems like, perhaps, the other way of doing lazy IO
17:30:31 <donri> i wonder if we have to use IO even with pipes... would be a nice-to-have to be able to test the server purely
17:30:35 <stepkut> lazy IO says, lets take an IO thing and make it into a String so we can use all our normal list functions
17:30:56 <stepkut> and io-streams, seems to go the other way.. let's keep things in IO and lift all the list functions to work with the IO thing
17:31:13 <stepkut> right
17:31:37 <stepkut> hyperdrive tries to abstract over the inner pipes monad so that you could make it be Identity and stay pure for testing purposes
17:31:42 <donri> i vaguely have this dream of a server built up with pipes compisition of many small composable pieces (pipes)
17:31:47 <stepkut> though, in most real apps it will be IO, so you can do things like save files, etc
17:31:54 <donri> i'm not sure how useful or feasible in practice that is
17:32:25 <stepkut> though, it would be neat to make some other monads that are a restricted set of IO operations based on operational monad or something
17:32:26 <stepkut> gotta run
17:33:02 <donri> stepkut: yea but current happstack uses things like MVars in the request IIRC
17:33:19 <donri> i'm wondering if we can get rid of such things without loosing performance
17:34:31 <donri> somewhat relatedly dcoutts i think it was wanted to cache requests or responses, so making them pure might help things like that too
17:38:58 <donri> but i'm thinking at a broader level, like, have a pipe for reading the headers into lines, another pipe to parse the lines into parsed headers ... build up the server compositionally like that and export the pieces for reuse
17:47:10 <donri> meh, s/loosing/losing/ thought i had learned that one by now
19:19:27 <stepcut> donri: the MVar should not be needed anymore. things are a bit more pipeline-ish now. Though too much pipelining is probably not a good thing
19:19:27 <donri> :)
19:19:27 <donri> pipe ALL the things
20:44:43 <srhb> Are there any QueryT-like things that can be made acidic?
20:49:39 <srhb> Ie. will I ever be able to liftIO inside a Query?
20:49:48 <srhb> Or better, an update?
21:31:58 <stepkut> you will never be able to use IO inside a query or update
21:32:16 <stepkut> that would destroy durability and replication
21:34:53 <donri> stepkut: hm even for queries?
21:36:00 <donri> srhb: updates at least, *have* to be replayable and thus pure. but you could put your acid handle in a Reader and write reusable update helpers
21:36:39 <stepkut> I'm not sure. I do not have an example of hand of when it would be bad for queries. But, on the other hand, if your database has not changed, but you get different query results each time.. that seems like trouble..
21:36:46 <donri> newUser = update . CreateUser =<< liftIO makeUUID4
21:42:10 <donri> update event = ask >>= flip Acid.update event
21:42:33 <donri> there's a more general version in the crash course, HasAcidState
21:42:40 <donri> and i think that class is included in happstack-foundation
21:51:25 <stepkut> yeah
21:51:29 <stepkut> and in clckwrks