01:08:23 <levi> You want data types that are open for extension?
01:08:34 <levi> Sorry, old discussion.
14:51:45 <donri> “Remember, the question I'm trying to answer here is "How well does AcidState stack up against other data storage options in Haskell", and at this point the answer looks to be "It destroys them and pisses on the ashes".”
14:51:49 <donri> ^_^
14:52:27 <donri> and that was with ixset! i wonder how the benchmarks would look with say, data-store and future binary-based acid-state
14:53:48 <donri> ( http://langnostic.blogspot.com/2013/03/haskell-benchmarking-third-impact-or.html )
14:54:26 <donri> looks like mongodb beats acid-state on insertion, probably because it completely ignores everything ACID
14:54:30 <donri> "fire and forget"
14:55:08 <stepcut> :)
14:56:22 <stepcut> even faster if he used the block write mode
14:56:47 <donri> what's that?
16:17:10 <donri> also tekmo has plans for a pipes-based serialization library that supposedly is 5x faster for IO than binary and cereal. i wonder if acid-state could use that and benefit
16:19:46 <mm_freak> donri: definitely
16:21:14 <mm_freak> however i suspect that it's largely the fault of how the put monads are implemented and how the instances work
16:21:35 <mm_freak> see boomerang for example…  it's horribly slow, but that's not the fault of the abstraction, but the implementation
16:22:11 <donri> it seems his speed gains come mostly from using Storable and hPutBuf
16:22:25 <mm_freak> yeah, but there is no reason to reinvent the wheel
16:22:36 <mm_freak> blaze-builder does that just fine and has been optimized for a long time
16:23:30 <mm_freak> don't get me wrong, i love pipes…  but it's not a solution for everything =)
16:23:34 <donri> yeah that's what i meant before with "future binary-based acid-state"; the plan is for binary to use the new Builder in bytestring proper, and then for acid-state/safecopy to use that new binary instead of cereal
16:23:43 <donri> are you saying this would be as fast as Storable?
16:24:04 <mm_freak> not sure if the builtin builders are equally fast, but very likely
16:24:22 <donri> bytestring's builder is mostly just a merger with blaze-builder
16:24:53 <mm_freak> then there is nothing wrong with using it
16:25:41 <mm_freak> my suggestion would be to improve boomerang and then make a serialization library based on that =)
16:26:29 <mm_freak> make boomerang faster and incremental
16:28:14 <mm_freak> the reasoning is that boomerang gives you all you need plus some correctness guarantees you wouldn't get otherwise…  just add a few laws and make the serialization something natural (in the categorical sense!)
16:28:42 <donri> it would be interesting to compare Builder+hPut with Storable+hPutBuf. i thought the reason the latter is fast is because it writes directly into the handle
16:29:23 <mm_freak> well, go ahead…  i expect them to almost equally fast
16:29:31 <mm_freak> to be
16:29:35 <donri> yeah if we could enforce the inversion property in the types like with boomerang that'd be nice
16:30:04 <mm_freak> well, you'll have to enforce by law and provide a safe set of predefined combinators
16:30:13 <donri> for safecopy i TH-generated quickcheck tests for every core instance, and actually found a bug in an instance
16:30:52 <mm_freak> as long as you stick with the predefined combinators that should guarantee safety
16:31:00 <donri> yeah
16:31:38 <donri> maybe boomerang should move the constructor to an .Internal module...
16:32:19 <mm_freak> in other words:  start with a better-boomerang, then build better-boomerang-iso based on that, then build better-boomerang-serialize based on that =)
16:32:41 <donri> ...then port safecopy and acid-state :P
16:32:47 <mm_freak> yeah
16:33:35 <mm_freak> for better-boomerang take inspiration from attoparsec (but don't use attoparsec) and blaze-builder (but feel free to use the internal Builder)
16:33:53 <donri> internal?
16:34:02 <mm_freak> of the bytestring library
16:34:28 <donri> it's not internal though
16:34:45 <mm_freak> no attoparsec, because it's unnecessarily slow for this purpose…  you don't need full parsing…  the subset of features you need allows for a much faster abstraction
16:34:45 <mm_freak> yeah, i'm saying "internal" for lack of a better term =)
16:34:47 <mm_freak> "builtin"
16:34:51 <donri> aha
16:36:16 <donri> pipes-parse ;)
16:42:07 <donri> https://github.com/kolmodin/binary/issues/37#issuecomment-24373510 :(
17:33:52 <stepcut> mm_freak: I'll take patches for boomerang :)