16:29:44 <kxra> stepkut: it seems i need to gie a very detaild answer to the question about plugins-ng
16:30:00 <kxra> stepcut: http://lpaste.net/4397565431817175040
16:32:19 <stepcut> not quite sure what you are asking
16:33:59 <stepcut> there are multiple layers of functionality. At the base we need the raw ability to compile and load code into a running application. In the past that was provided by the plugins library itself.
16:34:22 <kxra> "Could you specify a bit more what exactly needs completing, what the technical challenges might be, how they will be addressed, and soforth? Thanks."
16:34:29 <kxra> that is the question carter was referring to
16:34:38 <stepcut> now the GHC API provides that ability -- though in a slightly less convenient form
16:35:12 <stepcut> on top of the base functionality, there are two other general purpose layers of functionality that would be nice
16:35:30 <stepcut> one is a file watching layer -- similar to what the old plugins-auto package does
16:36:47 <stepcut> you start up your application, and have it dynamically recompile and load source from the local directory into the running application. The source files and the recursive depenendencies of the source files are then watched for anything changes. When one of them is modified, the affected modules are transparently recompiled and reloaded into the running application.
16:37:36 <stepcut> There is some proof of concept code in plugins-ng already which does this, but it has a few short comings
16:38:16 <stepcut> 1. it doesn't account for the fact that when your editor does a 'save all' a bunch of files will be changed at almost the same time -- you really only want to recompile once when that happens
16:39:24 <stepcut> 2. if recompilation is already in progress and another one is triggered -- something bad is likely to happen because there is currently no locking. So, there needs to be some well thought out thing that happens if a new recompilation request occurs while one is in progress
16:40:54 <stepcut> 3. there is currently no error handling system in place. I'm not sure what happens if a compilation fails -- but it is probably not good. There needs to be a clean way for compilation errors to be handled and reported
16:42:01 <stepcut> the other nice piece would be a way to fetch packages via cabal, download them, compile them, and load them into a running application
16:42:06 <stepcut> nothing has been done on that yet
16:43:30 <stepcut> it is conceivable that this would really be three packages
16:44:02 <stepcut> 1. a lightweight wrapper around the ghci-api that makes it easy to compile and load a symbol from a source file, or report an error
16:44:40 <stepcut> 2. a library which uses that low-level wrapper and adds support reloading based on file watching
16:45:16 <stepcut> 3. a library that supports fetching, downloading, and loading of packages via cabal
16:45:47 <stepcut> the existing web-plugins library would use #3
16:46:07 <stepcut> there is already a web-plugins-dynamic proof-of-concept package -- but it needs this other stuff to actually be usable
16:47:18 <stepcut> to give this packages some horrible working names, plugins-ng-low-level, plugins-ng-filewatch, and plugins-ng-cabal
16:47:55 <stepcut> one thing to do is to compare plugins-ng-low-level to existing things like plugins and hint
16:49:41 <stepcut> plugins is an ancient library which predates the GHC API. It has to shell out to the ghc executable to compile things, and uses hacks that nobody understands for loading symbols. The current maintainer (me) is willing to apply patches that other people send, but really has no idea what is inside the black box. It is fragile, doesn't know anything about cabal, etc
16:50:11 <kxra> why would #3 not be used by plugins-ng?
16:51:09 <stepcut> now that I have broken the general plugins-ng functionality into three separate packages -- what are you calling plugins-ng ?
16:51:23 <kxra> it sounds like the functionality that is similar to plugins-auto could be separated into a new plugins-auto package
16:51:50 <stepcut> that is what I said...
16:52:04 <kxra> ah sorry, i was calling #1 plugins-ng
16:52:30 <stepcut> i am proposing that 'plugins-ng' is just a name for a family of packages which have the (horrible) names, plugins-ng-low-level, plugins-ng-filewatch, and plugins-ng-cabal
16:52:41 <kxra> ah, okay great!
16:52:43 <kxra> this is perfect
16:52:52 <kxra> sorry for the confusion
16:53:05 <stepcut> the stuff I ultimately want, aka, web-plugins-dynamic, would be built on top of plugins-ng-cabal.
16:53:06 <kxra> i am a little fuzzy as i just woke up
16:53:22 <stepcut> but, the plugins-ng stuff itself is in no way web specific
16:53:31 <stepcut> it could be just as easily used by yi
16:54:09 <stepcut> one good thing to think of is 'what sorts of applications actually want this functionality'
16:54:37 <stepcut> so far I have: web servers and IDEs/text editors
16:55:00 <stepcut> I think that being able to describe some compelling uses of the technology would be good
16:56:21 <stepcut> that is why I brought up clckwrks -- I want to be able to use this library in clckwrks so that end users, with no knowledge of Haskell or hackage can one-click install plugins
16:56:36 <stepcut> but.. there is clearly a lot of stuff that has to go on top of the plugins-cabal package for that to happen ;)
16:56:53 <stepcut> there is a saying.. "three men make a tiger"
16:57:56 <stepcut> We have two uses of the technology -- a extending a CMS via one-click installs, allowing a Haskell text editor, like yi, to extend its own functionality
16:59:23 <stepcut> it could also, perhaps, be used by applications like xmonad, where the configuration file is actually a Haskell source file. xmonad has its own way of recompiling and restarting
16:59:27 <stepcut> also, drye already exists
16:59:39 <stepcut> or something like that
17:01:04 <stepcut> http://hackage.haskell.org/package/dyre
17:03:05 <kxra> so dyre is similar to plugins-ng-filewatch?
17:03:10 <stepcut> no
17:04:00 <stepcut> dyre is similar to plugins-ng-config or something
17:04:05 <kxra> ah, i see
17:04:13 <stepcut> its a different functionality that we have not described
17:05:15 <stepcut> hint uses the GHC API, but seems to be more oriented around its Interpreter monad and programmatically doing things you might do in GHCi
17:05:26 <stepcut> not exactly clear how much it overlaps with plugins
17:05:47 <stepcut> superficially, plugins is about actually compiling code into object code and loading it into a running application
17:06:09 <stepcut> I can not say for sure why hint would not do what I need
17:06:21 <stepcut> it's a question someone might ask though
17:18:13 <kxra> does hint try to do what multiple plugins-ng packages would do? perhaps the advantage to plugins-ng would be it's modular nature
17:22:09 <stepcut> hint definitely does not do what plugins-ng-filewatch or plugins-ng-cabal would do
17:22:25 <stepcut> it does not quite seem to do what plugins-ng-low-level would either
17:23:15 <stepcut> it has a similar API -- but requires that you be in the InterpretT monad transformer and does not give an obvious way to compile to native code rather than interpreted code
18:20:57 <kxra> stepcut: phew, okay I have refactored my proposal with much more detail
18:21:04 <kxra> thanks so mch for explaining that all
18:21:33 <kxra> i guess i'm not sure how to repod to this part of brent's comment "... what the technical challenges might be, how they will be addressed...?"
18:22:08 <kxra> i guess they're asking what will be he hardest part but i really have no idea