This document is in the process of being edited, which (unsurprisingly) loses some information and introduces some mistakes. There is a link to the original raw form at the bottom. * What is the next big change in Pugs? The current execution model -- in which the parse tree is directly evaluated in Haskell -- is going away. It is being replaced by a true multi-phase parse-compile-execute design. * Why make this change? The current model forces many things to be delayed until runtime that would be better done at compile time, including detection of several classes of errors. Some forms of code analysis and transformation are difficult under the current design. Finally, the current Haskell backend may act differently from compiled backends, and we'd like all backends to have consistent behavior. * How will compilation work in the new design? Code will flow through several phases inside Pugs, as follows: FRONT END BACK END | <-- source lang specific --> | <-- runtime specific --> | src file -> Parser -> Compiler -> CodeGen -> Emit -----------> Runtime | | | | | P6->Parrot: Perl 6 Exp PIL PIR-Tree Parrot ->Haskell: Language.TH.Syntax GHC ->Perl 5: Code::Perl Perl 5 ->JavaScript: JS-AST SpiderMonkey ->Perl 6: Exp Pugs Of course, nothing prevents the creation of a frontend for a language other than Perl 6 -- this would allow the creator to reuse the various PIL-to-* backends. * What is PIL? PIL is a kernel language, a core calculus for Perl 6. In practice, PIL is basically Scheme with more calling conventions. It also serves as a single link between all frontends and backends, avoiding the M x N compiler variant problem. * What changes will be required in Pugs? The Haskell evaluator in src/Pugs/Eval.hs needs to be refactored, and the various backends for PIL will need to be written. Most of the existing Pugs code will remain essentially the same. Don't worry that time spent now learning Pugs internals will be wasted; there's no time like the present to get started. * What other options were considered? Perl 5 and Ruby chose to simply augment the parse tree, rather than implement a new intermediate language. However, this makes back end code generation much more difficult, and non-local optimization extremely so. * When will these changes occur? After 6.2.8 is released, refactoring of Eval.hs will begin. The various backends will follow soon afterwards, depending on their interest to Pugs developers. Currently, the interest level is roughly in the order shown in the design overview above. * What will each backend do for us? ** PIR The PIR backend comes first because it forces cleanliness in the PIL design. It would be easy to hide complexity inside the rich Haskell and Perl 5 runtimes, allowing the PIL design to become bloated, crufty, and unfriendly to code generators. PIR forces us to define all operational semantics directly in PIL, so that code generators for all backends can be equally clean. Other runtimes, including Perl 5, are way too "smart" and would allow us to get away with a bad design for PIL without noticing it. Additionally, it will exercise many dormant and unimplemented features in Parrot, so it may one day become the leading runtime for Perl 6. ** Perl 5 Perl 5 is extremely reliable and visible, and brings CPAN to Pugs. It is hoped that the ability to compile Perl 6 to Perl 5, and to reliably use the Perl 5 DBI (and other XS modules), will move Perl 6 from the "slowly gaining more respect" phase to the "hack hack hack" phase on Autrijus' imaginary timeline. ** Haskell Haskell is known to be good for creating specifications that happen to run. The Haskell backend will thus serve as the reference evaluator until (eventually) the Perl 6 backend is completed, completing the bootstrap cycle. Also, it is expected that PIL will have a vastly cleaner implementation in Haskell than in Perl 5. ** JavaScript It's just plain cool, that's why. Oh, and Perl 6 then magically runs inside every vaguely recent web browser. ** Perl 6 In the first pass, this is simply a pretty printer. At the end of Pugs' development, this indicates the completion of bootstrapping; more on this below. * What is required to create a new backend? Aside from code generation, each backend will need a runtime that implements the builtin operations and functions not provided by the Perl 6 prelude. In addition, the new backend will require a native implementation of the Perl 6 object metamodel. * When is a backend considered complete? When it smokes the test suite cleanly. This metric is nicely quantifiable -- it is reasonable to refer to the completeness of a backend by the percentage of the test suite that it passes. * What is the desired final milestone for Pugs? The long term plan is to slowly convert each of the compilation phases and supporting systems -- such as the object metamodel -- from Haskell to Perl 6. This will basically constitute a Perl 6 virtual machine running in Perl 6, and will then replace the Haskell code as the reference implementation. It is possible that everyone will agree to forgo this Perl 6 VM, and use only the Parrot VM, but that decision is still far off. * Would it be possible to run PIR in Perl 5 instead of Parrot? Yes, but it would probably be very slow unless a large portion of Parrot was linked in (libparrot, PMC implementations, etc.) Some PIR opcodes cannot be directly expressed in Perl 5 and would require heavy (slow) emulation otherwise. You are of course welcome to try. * Where did these answers come from? putter interviewed autrijus on #perl6, and dumped a roughly edited version of the interview here. geoffb then cleaned that up, per putter's request. The original interview can be found at: http://colabti.de/irclogger/irclogger_log/perl6?date=2005-07-12,Tue Cleaning up a draft unfortunately often means losing some of the richness of the original form. Here is a link to it: http://rt.openfoundry.org/Foundry/Project/Source/index.html/pugs/checkout/docs/notes/plan?rev=5465 While it has the flavor of an interview, it isn't one really; more a discussion, with several peoples' thoughts mixed in, and then edited. * Will $/ contain a AST for the code in macros? The short answer is no. :) * What about the long answer? Macros get PIL -- in the Pugs world, that is; in the alternate universe you get PAST -- but you never get match objects. However, it is conceivable that you can get match objects attached to PIL, and it may even come with $/ if lwall desires so, but it's unspecced. And you can't return $/ and expect it to work. * But A06 seems to indicate that $/ will contain some sort of AST. "Macros are considered methods on the current parse state object, so they have an invocant. Macros are considered methods on the current parse state object. We treat macros as if they were methods on the parse object returned by the grammar rule, so the first argument is passed as if it were an invocant, and it is always bound to the current parse tree object, known as $0 in Apocalypse 5. Right, and that's written before the understanding that a one-pass parser can't generate AST. :)" A06's view is naive. Its interface may be of use, but all its speculation about internals is to be taken as way off :) (Which is why we need a separate macro synopsis.) Back when A06 was written, the dominant view was that there's no separate compilation and P6 AST will just be like P5 -- a parse tree annotated with some type information. That view is now shown as naive, and I don't think any of @Larry is still holding onto it. From what I gather at the hackathon, the consensus is now separate compilation with a separate AST structure. (This is also because pmichaud and I both observed that we can't share a parse tree structure with python, but there's a good chance that we can share something at the PIL level with Python.) (And macros are better done at that post-compilation level, not the naive pre-compilation parse level.) "Are better" is an understatement; "only sane way" is more like it :) * So if I'm going to get PIL how would I get to it? You likely get an object in the style of ext/Perl6-Compiler/ Perl-Compiler anyway. the deal is that $/ will not be a regular match object tree; it will be PIL annotated with match objects, or match objects anotated with PIL so maybe you can indeed return $/ and expect it to work, but the structure of $/ is likely to be not a regular Match object. pmichaud mentioned something about rebinding $/ inside rules and make it return something else entirely. * Does that mean that some macros are impossible to be portable across Perl 6 implimentations? If the macro synopsis is written with a certain interface of syntax nodes then that becomes normative and all implementations are supposed to honour it.