What's New in Qi II

This page is about the future. It deals with the near future and the far future. I use the words 'near' and 'far' with a time frame relative to that of computing where 'near' means 6 months and 'far' means 3-5 years. The near future is dominated by Qi II and you can read about this right below. The far future is discussed near the bottom of the page. Finally, if you get that far, there's a piece on how you can help which includes a comment box. And no, help does not necessarily mean money or even writing code!

Qi II, targeted for August 2008, is a complete rebuild of Qi and the first since Qi was released in 2005. Let's remind ourselves how far we've come in three years. From being the personal tool of one person in 2005, Qi has been downloaded to over 1000 desktops in 2007 alone. The now unmoderated members group on Qilang is approaching triple digit membership and is significantly more active than F# - the Microsoft approved cousin to O'Caml. Qi now runs under four implementations of Common Lisp and has acquired a personalised EMACS version tailored to Qi syntax.

Qi II will occupy only 6K LOC in Common Lisp making it pound per punch one of the most powerful systems for its size around. It is smaller than Qi I, but is actually more powerful. Qi II will run Qi I code with no changes. In fact I would expect that most users would not notice the difference at all. That's intentional.

But there are several significant differences. This is the first significant change to the language standard and probably the last such. For that reason the documentation is published as a hard copy fixing the standard. However there are some potent changes which make Qi II more powerful than Qi I.

	In	Out
minor changes	m-syntax and s-syntax for Prolog 4 speed compiler setting polyadic let, @p, append, and, or, +, *, /.	hard wired abstract datatypes hard wired structures theory declarations macroexpand
major changes	improved programmable syntax in place of Lisp macros on demand lazy evaluation rule closures
library stuff as code samples	structures, classes, abstract datatypes

Inbuilt abstract datatypes and structures are thrown out. This seems drastic but it is not. Actually both these things are easily definable using sequent calculus and chapter 12 shows you how to do it and gives you the code. So this stuff is now in a library. The theory declaration used in part III of FPQi is totally superceded by the more powerful and elegant rule closures and is dropped. macroexpand is dropped in favour of a more powerful syntax manipulating approach.

Some minor stuff; Qi Prolog includes both M and S syntax. The latter is for people who wnat to machine generate Qi Prolog. (+ 1 2 3), (append [1 2 3] [4 5 6] [7 8 9]) etc. are all legal.

The Qi II Merlin compiler is based on the successful Turbo-E version. Merlin produces superfast code from 4 speed settings, with the highest generating code which not only factorises overlapping patterns and inserts type declarations, but (unlike Turbo-E) uses type information to substitute user tests with faster type secure equivalents. This compiler is named after the famous Merlin engine that powered the first Spitfires. At the highest setting you will be able to blow most native CL programmers away.

More important is an improved programmable syntax that allows you to organise large numbers of sugar functions into of just having macroexpand. How about a case statement? You want to dispense with nested ifs and have a nice case statement in Qi II.

(define cases
[cases] -> [error "case failure~%"]
[cases true B | _] -> B
[cases A B | Cases] -> [if A B (cases [cases | Cases])]
X -> X)

Then (sugar in cases 0) declares cases as a sugaring function and places it in pole position i.e. to be applied before any other sugar function.

This is useful because although you could organise a macro in CL to do this it would have the disadvantage that it would not typecheck. So you'd have to write a piece of sequent calculus to give the type theory of 'cases'. Here you don't have to do this because Qi II treats 'cases' as a syntactic fiction - that is to say, a figure of speech for a nested conditional. Unlike Qi I, your definition of Qi is programmable at a very deep level because the syntax altering action isn't in one function in which your changes will overwrite anothers. Qi II doesn't try to make a cup of coffee to everybody's taste - thats impossible. Instead it gives everybody a coffee machine.

In Qi II, sequent rules can be placed inside an operator which converts them into a closure. Hence the user can build rule closures which behave like ordinary closures except that their functionality is determined by the sequent rule embedded in them.

Here is the rule of modus ponens using rule closures.

(define modus-ponens
{wff --> problem --> problem}
   A P -> ((rule [A => B]; A;
                     ____________
                      B;) P)

Rule closures enabled a great shortening of the Qi II source (the theory declaration is dropped in Qi II) and the final 100 pages of FPQi are written around them. This is the big change. If you're into expert systems or automated reasoning, this is a lot of power at your elbow! I will be illustrating the power of rule closures with a new series of study aids.

Qi II contains library code for type secure classes with multiple inheritance and default values. No, the class system is not based on CLOS. Why? My dislike of certain aspects of CLOS (see why). Qi II classes have .....

1. Simple class manipulation functions; the Qi II classes package contains only 6 system functions.
2. A commitment to a pure declarative model for classes. Qi II classes are closures.
3. Qi II class instances are lists and can be searched just like regular lists.
4. Type checking is used to handle potential inheritance clashes.
5. A small open source program that can be ported to any Lisp-like language because it is not CLOS reliant.

Since Qi II classes represent superset of the functionality of structures, structures are dropped from Qi II. The programs are short and you can hack them.

Functional Programming in Qi II is the new hardcopy guide to Qi II. It includes 2 entirely new chapters and the 75 pages are extensively rewritten and improved to incorporate the innovations introduced by rule abstractions. New chapters include material on abstract programming, including classes, and the compilation techniques used to drive both Merlin and the sequent calculus compiler. All the innovations listed above are covered in FPQi II. The source code for Qi II is annotated function by function with respect to FPQi II page numbers. FPQi II also includes new introductory appendices dealing with Qi Prolog and Qi YACC. Altogether over 150 pages have been either revised or introduced for the first time.

Qi II will appear under the MIT licence enabling you to develop closed source applications without bother.

So what's the far future? Well, the far future is to implement the L21 project and this means a type secure Qi programming environment including graphics, threads and web programming all built on a layer of Common Lisp. We expect to import a lot of excellent existing work done in CL libraries to help us reach that goal and when we're finished you will have a turn-key open source system under MIT license that you can just download and run. You'll be able to go to Amazon and buy a book on it and expect to use the system out of the box without worrying about CL platform differences like how to talk to the OS etc. The environment will offer all the resources of modern FPL with Qi II running at the top level providing type secure access to all these facilities and CL at hands reach if you need to dive down that deep.

Generally request for help equate to requests for money. I'm certainly not adverse to that. If you want to donate then send me a check using the address on the donations page (I much prefer this to PayPal, its safer). But you can help for nothing by simply registering your support. How do you do this? Well you can do this in several ways.

You can send in your endorsement. You can use the form below to register your support. It asks for your professional details simply because they are useful in building a case for a grant. "I think this is great" coming from Head of Research at Google carries a lot more weight than it does from an average person. This is not snobbery; this is how grants actually work! I'm not going to put your details online unless you give your permission. But any professional qualifications that you have will help build the case.

By sending these comments in, you are giving permission for me to use them in a case for support. If you don't want to use the form, you can just mail your support using my email address at the bottom of this page, but follow the same format. You can mail your ideas on grant sources too.

Comments: I've read Mark Tarver's plans and I think he's got an amazing idea. To build a portable layer on top of Common Lisp which provides full access to the underlying Lisp while providing the innovations of modern functional programming and a full set of libraries for modern applications would be a major contribution to computing. Specifically, it would be a major contribution to those of us who are building complex business applications in the real world. Too many of the ideas in question remain in a kind of academic ghetto and those of us who work on applications don't get to use them. Tarver is bringing these into a production environment. In fact, he's bringing them into the Common Lisp production environment(s) which are rock-solid, and apparently doing it in a way that enhances rather than obstructs the computing power of that classic, mature platform. This is exactly the right way to go about it. There are few people who could pull this off, so I think this project is highly deserving of full funding. The benefits to people like me would exceed the cost by orders of magnitude.

Comments: Qi is a truly monumental creation, not only incorporating unusual new developments but including documentation making them useful and understandable to more than just their creator. The project is fabulous, and I heartily encourage everyone involved to continue. Many thanks!

Aemon Cannon

Comments: I'm excited for the future of Qi. Many argue that Common Lisp has stagnated. Qi keeps what's great about Common Lisp (a lot) and finds a way forward.

Mark Tarvers Qi manual remains the best introduction to FPL and CL that I have encountered thus far in my recent exp. learning both. I am super excited to see what the new QiII improvements bring, esp. the production environment. Providing a common portable high level interface to both FPL and CL will bring a real boost in interest for programmers seeking new tools best suited for emerging semantic web and nlp applications. Mr. Tarver maintains a clear, focused, goal oriented approach to the development of Qi that bolsters the early adopters willingness to engage and commit to learning the ins and outs of this innovative language.

Marcus Eskilsson

Comments: This is very nice indeed! Haskell has been a bit of pain for me (lack of macros, no live image to modify, etc.) and this just brings CL what is badly needed! It'd be very much interesting if a good concurrency model of some sort was added to this baby. Keep up the good work!

Andrew Garman

Comments: I just started reading your book. It's clear & concise. Qi looks more like what I've been looking for. The path here was a bit winding: CL to F# to Haskell to OCAML to Qi. I second that adding concurrency support would be extremely useful, especially paired with reactive programming.

Name:
Email:
Organisation:
Qualifications:
Post my comments but omit all details except my name.
Comments.