This is my wishlist for Delphi when the new company running it is planning the next version. My requests are not large things (except maybe the last one), and are in order of priority.

Summary:

• Language construct: return
• . operators on basic types
• Include all source code (midas, dbExpress)
• Ruby

Return

Readable concise code is something we all try to acheive when coding. I really miss the ‘return’ statement from C, because it makes code more concise. I’m bored of writing:

if blah then
begin
  Result := -1;
  exit;
end;

I want to be able to write just

if blah then Return(-1);

Shorter, and the intention of the statement is much more obvious. I would imagine it would be fairly easy to add to Delphi as a pseudo function (like ‘continue’ and ‘break’).

Dot Opeators

I’ve been doing a fair amount of Ruby recently, as well as some C#. Having the dot operator available on all basic types is lovely, and makes code much more readable. I’d much prefer

address.length

over

length(address)

Less visual clutter, much more OO. Even if behind the scenes is just syntactic sugar to call the length procedure, that would be a huge win for the readability of Object Pascal in these Object Orientated times.

Source Code

At Concept First we currently use the following third party libraries:

• Devexpress: Quantim Grid, Quantum Bars
• TChart charting component
• CoreLabs dbExpress driver for SQL Server

We will only buy a component if it comes with full source code. Why ?

1. Fixing bugs when we find them
2. Adding new features
3. Tracking their changes under source control

We’ve made changes to all of the components we use, and I have all their source code under Subversion, so I can see what they have changed and see what possible impact it could have on my code.

One of the great strengths of Delphi has always been that the VCL comes with full sources. The two exceptions are the TClientDataset (midaslib.pas) and the dbExpress drivers. Its very annoying not to be able to debug into their code. I know Midas is in C++, but I have the full BDS with C++ support so that is not an issue. I use TClientDataset all over the place, and there are a few easy extensions that could make life much easier.

Ruby

As mentioned before, i’ve been playing around with Ruby and Ruby on Rails recently, and the first thing I had to do with is knock up a quick syntax highlighting editor using the SynEdit control. Having Ruby (and in an ideal world Javascript) in Delphi, with code complete, would be great, something I would be happy to pay for). Web development seems to be the way of the future, and having a nice IDE supporting Ruby could be a big win for Delphi / Devco.

Summary

I don’t know if my use of Delphi is similar to the majority of other users, but my priorities from it are:

• Fast to develop in
• Looks nice (generally easy thanks to DevExpress)
• Source code so I don’t get held up by bugs
• Reliable and fast IDE
• Small self contained EXEs, without frameworks to install.

I use delphi for Win32 client applications. I don’t use it for web applications. I now use either Ruby on Rails ( my personal choice ) or Cold Fusion ( trusted in Enterprise environments ).

I’m not interested in:

• Delphi.net
• 64 Bit support
• Mobile development (be nice if Delphi supported, but such a small % of our work I’d be happy enough to use Visual Studio).

Anyway if I could have the requested changes made by the end of next week, that would be great DevCo ;-)

Pretty Please ?

This post follows up my previous one about RTTI in Delphi, inspired by Hallvard’s 2 posts here and here, and covers some advanced RTTI features in Delphi that I haven’t seen mentioned anywhere else.

$METHODINFO

While playing around with Websnap in Delphi, trying to extend some of the objects available for scripting, I came across the compiler directive METHODINFO.

The online documentation says:

The $METHODINFO switch directive is only effective when runtime type information (RTTI) has been turned on with the {$TYPEINFO ON} switch. In the {$TYPEINFO ON} state, the $METHODINFO directive controls the generation of more detailed method descriptors in the RTTI for methods in an interface. Though {$TYPEINFO ON} will cause some RTTI to be generated for published methods, the level of information is limited. The $METHODINFO directive generates much more detailed (and much larger) RTTI for methods, which describes how the parameters of the method should be passed on the stack and/or in registers. There is seldom, if ever, any need for an application to directly use the $METHODINFO compiler switch. The method information adds considerable size to the executable file, and is not recommended for general use.

My previous article showed this isn’t completely accurate, detailed RTTI is available for any Interface which has $TYPEINFO or $M around it. $METHODINFO seems to affect classes, in particular it will store detailed RTTI information for not only Published methods, but also Public ones.

Doing a search for this compiler directive in the delphi win32 source code gives us only 1 instance in WebSnapObjs.pas.

{$METHODINFO ON}
TScriptableObject = class(TObjectDispatch)
private
  FLookupList: TStringList;
  FLookupValues: TInterfaceList;
protected
  FPreferChild: Boolean;
  function DispatchOfName(const AName: string): IDispatch; virtual;
  function FindObject(const AName: string): TObject; virtual;
public
  constructor Create;
  destructor Destroy; override;
  class function DispatchOfObject(const AObject: TObject): IDispatch;
  function GetIDsOfNames(const IID: TGUID; Names: Pointer;
    NameCount: Integer; LocaleID: Integer; DispIDs: Pointer): HRESULT;
    override;
  function Invoke(DispID: Integer; const IID: TGUID; LocaleID: Integer;
    Flags: Word; var Params; VarResult: Pointer; ExcepInfo: Pointer;
    ArgErr: Pointer): HRESULT; override;
end;
{$METHODINFO OFF}

Websnap

Websnap is the poor cousin in the web framework world for delphi. Its never had much support, and seems now to be overshadow by ASP.net and Intraweb. I personally quite like it, although I code my own templates in VBScript or JavaScript rather than use any of the Design Time webpage design stuff.

Under the hood, websnap uses the ActiveScript engines provided in Windows. ActiveScript is a scripting host that can support many different COM based scripting languages, and Windows comes with VBScript and JScript (which is basically JavaScript). Other ActiveScript lanaguages are avaialbe including Python and Perl.

The original ASP by Microsoft uses the ActiveScripting engine to do its work. The asp template is turned into a vBScript or JScript program containing the HTML to output as well as the logic of the page. This is fed into the ActiveScripting engine and compiled ready for running. The ActiveScripting engine then has ‘objects’ added to it so the program can do useful work. The most obvious one is the Response object, but there are others like the Session object, etc. The program is then run and the page rendered.

Websnap pages, at least those using a TPageProducer, use this same process to produce HTML pages. The problem for the Delphi deisgners was how to link arbitary Delphi objects up to the ActiveScripting engine, which uses late bound IDispatch COM for communication. The IDispatch interface, one of the main underpining of the COM framework in Windows, uses a single call, Invoke for all method calls. This is where $METHODINFO comes it, the rich method RTTI is provided to allow a single procedure entry, Invoke, to call arbitary Delphi methods.

The VBScript or Javascript script running in the scripting of the websnap page needs to talk to Delphi objects (e.g. Page, Session), and it uses this Rich RTTI to acheive this. You can see the websnap objects that are exposed to the script, have a look in WebSnapObjs.pas, where TResponseObj, TProducerObj, etc.

The unit ObjAuto contains the code and header for retrieving the RTTI information using the following function:

function GetMethodInfo(Instance: TObject; const MethodName: ShortString): PMethodInfoHeader;

In turn, the base class of TScriptableObject (marked with $METHODINFO) uses the RTTI to find methods, and call them, at run time.

ObjAuto.pas

This contains the code to search for a method’s RTTI. Looking at GetMethodInfo, you can see it uses the system.pas vmtMethodTable offset to get hold the method table for the class. It then uses a search to find the correct entry. It also contains the code that allows an arbitary call to an object supporting RTTI to jump to the correct routine:

function ObjectInvoke(Instance: TObject; MethodHeader: PMethodInfoHeader;
  const ParamIndexes: array of Integer; const Params: array of Variant): Variant;

As you can see you just pass it parameters and variants, and it packages them into the correct types and does the call. The source code to this call shows all the complexity of packaging up all the parameters according different conventions, etc. This is ultimately how VBScript objects call methods on Delphi objects inside Websnap.

DetailedRTTI.pas

While playing with the metadata, I coded a few helper classes to aid exploration. You can download the code if you want to have somewhere to start. Just calling .RTTIMethodsAsString() on any object to get a list of its methods and their parameters. Its a bit rough and ready but you’re welcome to use it for whatever.

Summary

This article, and the previous one have shown that rich metadata for methods is available in Delphi, with supporting routines for accessing it. Interface metadata allows the VCL to support SOAP, multiple methods multiplexed to a single call. The rich class metadata allows the VCL to support a single function automatically being routed to other methods, allowing Websnap to expose objects to COM IDispatch automatically.

Comments (originally on blogger.com)

Hallvard Vassbotn said...

Great posts, David!

I reference them here

Reading an article and its follow up by Hallvard about RTTI inspired me to put together a couple of posts about two related areas of RTTI in Delphi. In particular one of the comments on Hallvard’s blog about using this RTTI to call objects in some late bound fashion. This post and the next cover some of the advanced RTTI that I haven’t seen covered in other places. This post covers some of the possibilities for Interface metadata, and the next one will contain details about richer class RTTI for methods.

Interface Metadata

Delphi actually has richer metadata support for methods in an Interface that in a normal class. It looks like this was added to support the SOAP features of the VCL. I’m not sure which version of Delphi it was added so your mileage may vary if your not using 2006.

IInvokable

To use SOAP, you use a WDSL file to specify the method calls, parameters, etc. If you import a WSDL in Delphi, you will notice that all Interfaces in the generated file will be derived from IInvokable. A quick peak in the System unit will show that IInvokable is:

{$M+}
  IInvokable = interface(IInterface)
  end;
{$M-}

I.e. just a standard interface, but with RTTI metadata compiled in.

Looking at the help in BDS 2006 for {$TYPEINFO ON} mentions this:

Note: The IInvokable interface defined in the System unit is declared in the {$M+} state, so any interface derived from IInvokable will have RTTI generated. The routines in the IntfInfo unit can be used to retrieved the RTTI.

IntfInfo.pas

The main procedure of interest in IntIfnfo is:

procedure GetIntfMetaData(Info: PTypeInfo; var IntfMD: TIntfMetaData; IncludeAllAncMethods: Boolean = False);

This will give us a series of records describing the methods on the interface and the parameters needed for these interfaces, as well as the unit it was defined within, the ancestor Interface and the interface’s GUID. All the names are available. both function / procedures and the names of their parameters. Calling this procedure with an interface not having RTTI will raise an exception, calling it with a class’s typeinfo will just cause an a/v :-)

When doing SOAP calls, the developer just uses the defined interface like a normal interface. Behind the scenes, Delphi packages up the parameters and sends them via a SOAP envelope to the remote server. How Delphi does this shows us some of the potential of this RTTI in Delphi, and respect for the Voodoo that is TRIO.

RIO.pas

Located in the soap folder of Delphi’s source code, RIO.pas contains the class TRIO. TRIO is an object that represents a remote object, presumably it stands for Remote Interfaced Object.

When an application casts a TRIO descendant to a registered invokable interface, it dynamically generates an in-memory method table, providing an implementation to that invokable interface.

Looking at the source for TRIO, I’ve come to the conclusion that:

MyRioObject as IMyInvokableInterface

Will cause the TRio object to

1. Get the meta data for IMyInvokableInterface (from a registry InvRegistry object defined in InvokeRegistry.pas)
2. Allocate memory for a vtable for the interface
3. Allocate memory for ‘stub’ routines, marks it as containing executable code
4. Writes machine code stubs that takes the parameters and packages them up, then calls TRIO.Generic

This is a very crude representation I knocked up in Visio:

When you then make a call on the ‘generated’ interface, Delphi calls the vtable, the vtables holds the address of the generated machine code. The generated machine code pushes the parameters then calls the Generic function. This packages up the parameters, and then uses a SOAP call to call the remote service. The return is then packaged up and returned in a similar way, back through the generated stub. If you are interested in how the actual machine code is generated (taking into account the 5 different calling conventions, etc.) take a look at TRIO.GenVTable function.

I don’t know which of the Delphi team wrote this code, but its very very impressive.

Anwyay I hope this has given you a feel for some of the advanced metadata available with Interfaces. The RIO approach would allow you to write Interface proxies of any Interface with metadata, for security, logging and indeed other forms of RPC remoting. Let me know if anybody suceeds in such a thing !

My followup article on class RTTI.