April 18, 2013

Automatic Reference Counting for Delphi

The coming Delphi for iOS compiler will have a brand new memory management model for Delphi, ARC.

The coming Delphi for iOS compiler will have a brand new memory management model for Delphi, ARC. This was mentioned in my previous blog post (has been 2 hectic weeks, sorry for not blogging for so long), and by Olaf Monien in some blog posts (including http://www.monien.net/delphi-xe-4-is-arc-for-everyone/ and http://www.monien.net/arc-what-if-i-call-free-will-i-get-tared-and-feathered/).

Here I don't want to and have no room for an in-depth discussion of ARC, just want to share some information. A detailed description will become available soon in an in-depth technical white paper I wrote and Embarcadero is going to release as the product ships. So this is just a summary.

1. Automatic Reference Counting is an alternative to Garbage Collection, in which a developers has more control and some extra responsibilities. It has been adopted, among others, by Apple for iOS development. You can read about ARC in Clang at http://clang.llvm.org/docs/AutomaticReferenceCounting.html.

2. In a simple scenario, you create an object, assign it to a local variable, and as the variable goes out of scope the object is destroyed. No need to write a Free call, and not need for a finally block to protect it.

3. In a more complex scenario, all standing references to an object are counted, when the count reaches 0, the object is destroyed.

4. If this looks like reference counting for interfaces in Delphi, this is indeed the case. Similar idea. In fact, you can now mix interfaces and object references, given the implementation is shared (while in the past this would easily cause memory problems)

5. You can have a reference that plays on different rules and doesn't increment the reference count. This is called a weak reference and marked with the [weak] attribute in the source code. This is used, for example, for a child object to refer back to the parent object, avoiding a reference cycle (which will break the reference count, as it never goes to zero).

5. You can force the execution of the destructor for an object, before it is destroyed, by calling the new special purpose DisposeOf function.

6. For backwards compatibility, DisposeOf calls Free on Win32/Mac. For forward compatibility, Free doesn't call the destructor, but sets the reference to nil on iOS, as described in Olaf's second blog post linked above. You can even keep FreeAndNil calls around (if you really, really want to; but don't tell Nick I suggested this).

7. As Olaf mentions (in the first post), this is for the iOS compiler and Simulator only, not for the Win/Mac compilers. It will be likely used for Android. How/if/when to make ARC available on the desktop platforms is under evaluation.

8. This is a big change, requires looking to existing code with care. There is a lot of backwards compatibility mixed in (like keeping your Free calls). But there are many scenarios requiring special coding (weak references, calls to DisposeOf, or just a change in the class architecture).

9. Once grasped, ARC really makes coding in Delphi easier... and more modern. And it is still a deterministic memory management style.

There is way more, in technical terms, to automatic reference counting in the upcoming Delphi for iOS compiler, Embarcadero's first Delphi compiler for ARM. This blog post is more of a teaser, than a technical introduction. More will follow. Stay tuned (and I promise to get back to blogging more).

- Written in Scotts Valley, California -




Automatic Reference Counting for Delphi 

"A detailed description will become available soon in 
an in-depth technical white paper I wrote and 
Embarcadero is going to release as the product ships."

This sentence makes it seem like you are not employed 
by Embarcadero anymore.
Comment by jed on April 18, 06:36

Automatic Reference Counting for Delphi 

 I understand that ARC is not available for desktop (AKA 
Windows). What about the attribute [weak] - Will that 
for ARC usage only?
Comment by Nicholas Ring on April 18, 06:49

Automatic Reference Counting for Delphi 

"For backwards compatibility, DisposeOf calls Free"

And where is "DisposeOf" in erlier versions? I think you 
should remove the expression "For backwards 
compatibility" here as it is wrong (or am I 
misunderstanding something here?)
Comment by Andreas Hausladen [] on April 18, 10:05

Automatic Reference Counting for Delphi 

Will there be something like IDisposable as in the
Delphi.net world?
Comment by Roman on April 18, 10:08

Automatic Reference Counting for Delphi 

What about "zeroing" weak references, which are part 
of the ARC model, and need a specific process - in 
some cases, plain weak references are not enough, as 
stated by Apple techs, and related in my blog article 
linked above.

Without zeroing weak interfaces, the implementation 
seems broken. Could you make a statement/feedback 
about this?

The fact that it is global to the target platform is a 
bit disapointing to me. Not only it will not work with 
Win/Mac compilers, but also it will be somewhat 
breaking. Even if you put a void "try..finally Free" 
in your code, with a no-operative Free method in case 
of ARC, it will be a source of confusion.

A dedicated class type may have allowed clear 
distinction between the "classic" model expecting 
manual Free of instances, and the new ARC model with 
its own style of coding.

I like very much the FreePascal approach, with its 
syntax modes: you can have several syntax and even 
class models in the same application. It is used for 
instance to mix objective pascal and classic pascal 
classes in the same project. Clean and easy.

But, adopting the ARC model will make the OS X 
integration much easier than the current interface-
based marshalling of the Objective C APIs within 
Delphi. A lot of plumbing which will be leveraged at 
compiler level. Very nice! Do you plan to use it for 
Mac also (perhaps with a LLVM-based 32/64 OSX backend, 
since LLVM features this memory model from its 
internals, AFAIK)?

IMHO ARC is a much better memory model than a Garbage 
Collector, for the future of Delphi. Such a 
deterministic model does make sense. Otherwise, just 
switch to Java, C# or any script language.
Comment by Arnaud Bouchez [http://blog.synopse.info/post/2012/10/06/Delphi-XE3-is-preparing-reference-counting-for-class-instances] on April 18, 11:52

Automatic Reference Counting for Delphi 

"As Olaf mentions (in the first post), this is for the
iOS compiler and Simulator only, not for the Win/Mac

This will introduce code fragmentation between mobile
and desktop. Library written on iOS will most likely
cause memory leak in desktop.

Is this Emba... has not enough confident to put ARC on
the bigger platform? Can Emba... introduces ARC as an
option to the desktop?
Comment by ahmoy on April 18, 13:00

Automatic Reference Counting for Delphi 

"but don't tell Nick I suggested this"

Hehe.  ;-) 
Comment by Nick Hodges [http://www.nickhodges.com] on April 18, 14:02

Automatic Reference Counting for Delphi 

Some comments, more info is coming...

jed, could have written "we" are going to release, rather than 
"Embarcadero", felt the latter was more clear. Maybe this is just my view 
of the English language.

Nicholas, you can use [weak] on dektop, it will be ignored.  

Andreas, you are right I was not correct. It is not for "backwards" 
compatibility, it is for Win/Mac compatibility.

Roman, no IDisposable coming. DisposeOf in Delphi is conceptually very 
different from implementing iDispose in a GC environment like .NET. 
DisposeOf forces the execution of the destructor, putting the object in a 
"zombie" state. You can query an object for that state.

Arnoud, I personally disagree on offering multiple models within a single 
language. Gives programmers a lot of control, sure, but makes it very 
difficult for most programmers to understand what to do and when. But I 
understand your point. In any case, the fact you can keep using "old" 
syntax (like try-finally-Free code) on ARC makes the transition and 
coexistence smoother in many scenarios.

ahmoy, there are way to write code so that it compiles and works in both 
scenarios. Our RTL is written that way, and external code has been 
ported in a rather short time frame.
Comment by Marco Cantu [http://www.marcocantu.com] on April 18, 15:05

Automatic Reference Counting for Delphi 

Awesome! And you wrote this while in CA. I still 
think you should make it your permanent location.
Comment by ObjectMethodology.com [http://www.objectmethodology.com] on April 18, 15:13

Automatic Reference Counting for Delphi 

 What about Linux?  Xe4 sounds great, but can I build 
Linux apps with it?
Comment by Tony [http://www.lightningadmin.com] on April 18, 15:22

More modern? Sure, as modern as COM... 

I still remember when people was babbling about GC 
being superior to refcount, not refcount is the 
fashion again due to iOS, please let me manage my 
memory... I know how to dispose of my memory far 
better than any compiler. Any language designed to 
protect the developer from himself is not 
a "development" tool, is just a tool to cobble some 
code together and hope it will work as intended.
Comment by Luigi D. Sandon on April 18, 15:30

Automatic Reference Counting for Delphi 

If I am writing software for both iOS and Windows, isn't 
it going to be necessary to do it old way (i.e. try 
finally blocks)?  Otherwise, the Windows code will never 
free the objects when they go out of scope. 
Comment by Taylor Jacobson on April 18, 16:27

Automatic Reference Counting for Delphi 

>I know how to dispose of my memory far 
>better than any compiler. 

There's a lot of things where any human can still
perform better than any computer. Computers still
perform those tasks, though, because the enormous
savings in human time/effort more than make up for the
small performance difference. 

There's a saying going around nowadays "Developer time
is more expensive than CPU time" in terms of
optimization that's probably applicable in this
situation here. It's a lot cheaper to upgrade a CPU
(or in this case add a little memory) than to pay for
days or weeks of more developer effort to achieve
marginal efficiency gains.

Time you're not spending doing something the computer
can do - worry about managing memory - is time you can
spend working on the actual problem your program is
attempting to solve. That was the whole original point
of Delphi - it allowed one to concentrate on the
actual programming problem and not spend hours and
hours fighting with the Win32 library managing every
single event being passed to your program. Delphi
abstracted that all away and you no longer needed to
deal with routine events. Reference counting also
abstracts away a lot of "accounting" work and lets you
spend more time on solving your problem and increases
your development speed. Why worry about something
that'll be handled just fine by the computer at least
99/100 times? Think of all the wasted time Delphi
developers now spend managing memory. It goes against
the whole concept of RAD. 

>Any language designed to 
>protect the developer from himself

Like static typing? :-) Reference counting isn't about
protection so much as it is about handling routine
matters the developer shouldn't need to worry about,
just like the VCL and FireMonkey do now for Delphi. 

> is not 
>a "development" tool, is just a tool to cobble some 
>code together and hope it will work as intended. 

Now that's a hard statement to swallow because you've
just said that most of the mainstream computer
languages used today aren't real development tools
while (pre-XE4) Delphi and a few other (mostly very
old) languages are. It also seems contradictory -
first it was claimed that you're protected from
yourself and now you say that you just have to hope? 

I say relax, open your mind to new things and give it
a chance. Almost the entire (non-Pascal, non-C) world
is getting along fine with reference counting (e.g.
Python) and/or garbage collection (e.g. C#). You've
already got it for strings and dynamic arrays, no
thanks to Wirth. Speaking of which, he implemented
garbage collection in his subsequent language Oberon,
so it can't be all bad. :-) 

Languages or frameworks that employ rc/gc are taught
by and developed by some of the most prominent
computer scientists in the world and the most
prestigious universities. I doubt Delphi for iOS using
reference counting will be the downfall of software
development. If anything, there will probably be more
complaints about wanting it for the desktop than
getting rid of it on iOS. Any tool that wants to
survive, including Delphi, needs to embrace the state
of art (if not occasionally manage to exceed it) or be
eclipsed. Embarcadero should be commended for trying
to face the future with a remaining user base that
often seems more inclined than most to stay in the past. 
Comment by Joseph G. Mitzen on April 20, 04:01

Reference Cycles 

>You can have a reference that plays on different 
>and doesn't increment the reference count. This is 
>called a weak reference and marked with the [weak] 
>attribute in the source code. This is used, for 
>example, for a child object to refer back to the 
>parent object, avoiding a reference cycle (which
>will break the reference count, as it never goes to 

If I read this correctly, reference cycles have to be
avoided manually in code and there is no cycle
detector built in? 

"...a cycle detector... works to detect reference
cycles. This allows applications to not worry about
creating direct or indirect circular references; these
are the weakness of garbage collection implemented
using only reference counting. Reference cycles
consist of objects which contain (possibly indirect)
references to themselves, so that each object in the
cycle has a reference count which is non-zero. Typical
reference counting implementations are not able to
reclaim the memory belonging to any objects in a
reference cycle, or referenced from the objects in the
cycle, even though there are no further references to
the cycle itself....The cycle detector is able to
detect garbage cycles and can reclaim them so long as
there are no finalizers implemented...."

Comment by Joseph G. Mitzen on April 20, 05:36

Automatic Reference Counting for Delphi 

 Joseph, I aeree with tour commenta. There is a cycle-detection 
functions you can call on an object.
Comment by Marco Cantù [http://www.marcocantu.com] on April 20, 06:53

Automatic Reference Counting for Delphi 

i'm very happy to see that delphi go more modern than 
ever, but still we are looking for Bidi support, any 
informations about that?
Comment by ahmed [http://www.alfouadgc.com/] on April 20, 08:03

Automatic Reference Counting for Delphi 

I am very positive on the ARC approach. But I do see a
problem having this enhancement only for the mobile area.

A basic problem is the existence of all the different
compilers. The optimal approach would be (not sure if
it works):

a) all compilers based on the same LLVM environment
supporting ARC for all environments (lets say XE5)

b) still allow to compile the "old style units" by
turning off ARC. This would give developers and 3rd
party tools time to switch to the new ARC syntax

c) for a short time keep the existing DCC32/DCC64
compilers parallel but pushing the developers to the
LLVM-win32/64 (keep them for XE5 but remove them for XE6)

d) Support ARC for all packages delivered with Delphi
(not forget on VCLs as this is the GUI surface we earn
95% of the money with!)

e) The same approach would help a on C++ (what I see
are there are still a lot of problems with C++64 and a
common LLVM environment should help for sure).

I think we are at a status were Delphi has to move
into the "modern programming language space". This
move has to be in steps were the developers can
upgrade within a certain amount of time (e.g. 2
Versions). As well EMBT should focus the resources and
cannot spend efforts into n parallel branches of

best regards Günther

Comment by Günther Schoch [http://www.gs-soft.com] on April 20, 10:32


 marco, you did not answer my question about Zeroing weak 

Are they available or only plain weak pointers?

This may be a big issue, which has been identified as such by Apple, 
Comment by Arnaud Bouchez [http://synopse.info] on April 21, 19:48

Automatic Reference Counting for Delphi 

Use new key word to implements ARC. 
  TSomeClass = IntelClass(SomeOthers)

Do not chang "Class" behaver, please.

Please hold a place for old style programers.

Otherwise, those people who likes c/c++ style maybe
abandon Delphi.
Comment by ChinaStone on April 24, 08:45

Automatic Reference Counting for Delphi 

It does not make any sense to have to different version of compilers and 
language for different part of applications. Have you think about the case 
where you have server  running on Windows and desktop and mobile 
client. It doesn't make things better then writing that in different 
languages.  So what is the reason to use Delphi for iOS?  Business apps 
need server side other developers will not pay for Delphi because they 
can get XCode for free. What, Im trying to say is that unless you make 
Delphi consistent across all platforms you going to end like with Kylix.
Comment by Tomasz on May 1, 21:22

Automatic Reference Counting for Delphi 

Dont introduce anything unless it is supported by all 
the platforms, please.
Comment by Thomas on August 8, 13:27

Post Your Comment

Click here for posting your feedback to this blog.

There are currently 0 pending (unapproved) messages.