RFC: reference counted Throwable

Post by Andrei Alexandrescu via Digitalmars-d
Whenever a reference to a Throwable is copied about, passed to
functions, the compiler inserts appropriately calls to e.g. incRef and
decRef. (Compiler may assume they cancel each other for optimization
purposes.) Implementation of these is up to the runtime library.

Are you suggesting we implement ARC?

--
/Jacob Carlborg

Andrei Alexandrescu via Digitalmars-d

2014-09-19 22:14:08 UTC

Are you suggesting we implement ARC?

Yes. -- Andrei

via Digitalmars-d

2014-09-20 08:20:45 UTC

On Friday, 19 September 2014 at 22:14:08 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via Digitalmars-d
Whenever a reference to a Throwable is copied about, passed to
functions, the compiler inserts appropriately calls to e.g.
incRef and
decRef. (Compiler may assume they cancel each other for
optimization
purposes.) Implementation of these is up to the runtime
library.

Are you suggesting we implement ARC?

Yes. -- Andrei

I don't think ARC is needed.

library RC + borrowing + uniqueness/moving = WIN

Dicebot via Digitalmars-d

2014-09-20 08:32:54 UTC

Post by via Digitalmars-d
On Friday, 19 September 2014 at 22:14:08 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
Whenever a reference to a Throwable is copied about, passed
to
functions, the compiler inserts appropriately calls to e.g.
incRef and
decRef. (Compiler may assume they cancel each other for
optimization
purposes.) Implementation of these is up to the runtime
library.

Are you suggesting we implement ARC?

Yes. -- Andrei

I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

You can't do polymorphic entity RC (like exceptions) without at
least some help from compiler because language currently does not
provide tools for lifetime control of classes. At least _some_
form of ARC is necessary.

via Digitalmars-d

2014-09-20 09:05:23 UTC

On Saturday, 20 September 2014 at 08:20:47 UTC, Marc SchÃŒtz

Post by via Digitalmars-d
I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

You can't do polymorphic entity RC (like exceptions) without at
least some help from compiler because language currently does
not provide tools for lifetime control of classes. At least
_some_ form of ARC is necessary.

I think we can, using templated alias this. We're not there yet,
but it's probably feasable, seeing that Igor Stepanov already
implemented multiple alias this [1]. With that and maybe a little
opDispatch magic, it should be possible to making wrappers that
are implicitly convertible to wrappers of parents of their inner
types.

Granted, for exceptions there's more needed: There needs to be
support for throwing and catching these wrappers, and for
catching wrappers of derived types, too. But note that
throw/catch itself doesn't involve copying, it's a moving
operation, which might make it easier.

[1] https://github.com/D-Programming-Language/dmd/pull/3998

via Digitalmars-d

2014-09-20 09:16:59 UTC

Post by Dicebot via Digitalmars-d

Post by via Digitalmars-d
I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

You can't do polymorphic entity RC (like exceptions) without
at least some help from compiler because language currently
does not provide tools for lifetime control of classes. At
least _some_ form of ARC is necessary.

I think we can, using templated alias this. We're not there
yet, but it's probably feasable, seeing that Igor Stepanov
already implemented multiple alias this [1]. With that and
maybe a little opDispatch magic, it should be possible to
making wrappers that are implicitly convertible to wrappers of
parents of their inner types.
Granted, for exceptions there's more needed: There needs to be
support for throwing and catching these wrappers, and for
catching wrappers of derived types, too. But note that
throw/catch itself doesn't involve copying, it's a moving
operation, which might make it easier.
[1] https://github.com/D-Programming-Language/dmd/pull/3998

Thinking further about it, we don't need templated `alias this`,
because we can just have one `alias this` that returns
`scope!this(T)` (this needs to be there anyway to be safe). It
would automatically behave like a normal class with regard to
polymorphy.

What's still needed then is a little magic to actually be able to
throw and catch the wrapper.

Dicebot via Digitalmars-d

2014-09-20 09:19:17 UTC

Post by Dicebot via Digitalmars-d

Post by via Digitalmars-d
I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

You can't do polymorphic entity RC (like exceptions) without
at least some help from compiler because language currently
does not provide tools for lifetime control of classes. At
least _some_ form of ARC is necessary.

I think we can, using templated alias this. We're not there
yet, but it's probably feasable, seeing that Igor Stepanov
already implemented multiple alias this [1]. With that and
maybe a little opDispatch magic, it should be possible to
making wrappers that are implicitly convertible to wrappers of
parents of their inner types.
Granted, for exceptions there's more needed: There needs to be
support for throwing and catching these wrappers, and for
catching wrappers of derived types, too. But note that
throw/catch itself doesn't involve copying, it's a moving
operation, which might make it easier.
[1] https://github.com/D-Programming-Language/dmd/pull/3998

Yeah but implicitly convertible to what? If you convert it to
`Throwable` your reference counting facilities are circumvented
resulting in dangling exception reference at point where
`Throwable` gets caught.

Special casing catching such wrappers to still preserve original
ref-counted type while pretending to be Throwable at call site
sounds like a terrible hack, much worse than any sort of ARC
complexity.

via Digitalmars-d

2014-09-20 11:50:25 UTC

On Saturday, 20 September 2014 at 09:05:24 UTC, Marc SchÃŒtz

Post by Dicebot via Digitalmars-d

Post by via Digitalmars-d
I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

You can't do polymorphic entity RC (like exceptions) without
at least some help from compiler because language currently
does not provide tools for lifetime control of classes. At
least _some_ form of ARC is necessary.

I think we can, using templated alias this. We're not there
yet, but it's probably feasable, seeing that Igor Stepanov
already implemented multiple alias this [1]. With that and
maybe a little opDispatch magic, it should be possible to
making wrappers that are implicitly convertible to wrappers of
parents of their inner types.
Granted, for exceptions there's more needed: There needs to be
support for throwing and catching these wrappers, and for
catching wrappers of derived types, too. But note that
throw/catch itself doesn't involve copying, it's a moving
operation, which might make it easier.
[1] https://github.com/D-Programming-Language/dmd/pull/3998

As I said, throw/catch is at its core a moving operation. For
classes this isn't important (they are references), but for an RC
wrapper it would be, so we could specify that.

Move-constructing from an lvalue in the context of D can be seen
as a three-step process:

1) create a temporary, initialize with T.init
2) bit-swap the variable with the temporary
3) destroy the variable

It can be seen that the value that is to be moved (the RC
wrapper) must be non-const (only at the head, it may still be
tail-const).

Now, any generic RC type that wants to be implicitly convertible
to its payload type must do this via borrowing in order to be
safe (see my proposal at [1]). Using const-borrowing, we can
guarantee that the wrapper will not be thrown (or moved, in
general) as long as borrowed references to its payload exist:

struct RC(T) {
// ...
T _payload;
scope!(const this) borrow() {
return _payload;
}
alias borrow this;
}

This already solves avoiding dangling references to the
exception: No references can be left behind when the exception is
thrown, and the wrapper will not be destroyed, but moved, thus
not releasing the exception's memory.

The second part is really "just" some way to transport the
wrapper via the exception mechanism, including support for
catching wrappers of derived exception types.

Special casing catching such wrappers to still preserve
original ref-counted type while pretending to be Throwable at
call site sounds like a terrible hack, much worse than any sort
of ARC complexity.

IMO it would only a hack _if_ they were indeed special cased. I'm
sure we can find a more generic mechanism that would allow this
to be implemented cleanly. Note that the other requirements I
described above (borrowing, move semantics) are also things that
just happen to be usable here: they are desirable in general,
exceptions are just one possible application.

[1] http://wiki.dlang.org/User:Schuetzm/scope

via Digitalmars-d

2014-09-20 12:33:37 UTC

Post by Dicebot via Digitalmars-d
Yeah but implicitly convertible to what? If you convert it to
`Throwable` your reference counting facilities are
circumvented resulting in dangling exception reference at
point where `Throwable` gets caught.

As I said, throw/catch is at its core a moving operation. For
classes this isn't important (they are references), but for an
RC wrapper it would be, so we could specify that.
Move-constructing from an lvalue in the context of D can be
1) create a temporary, initialize with T.init
2) bit-swap the variable with the temporary
3) destroy the variable
It can be seen that the value that is to be moved (the RC
wrapper) must be non-const (only at the head, it may still be
tail-const).
Now, any generic RC type that wants to be implicitly
convertible to its payload type must do this via borrowing in
order to be safe (see my proposal at [1]). Using
const-borrowing, we can guarantee that the wrapper will not be
thrown (or moved, in general) as long as borrowed references to
struct RC(T) {
// ...
T _payload;
scope!(const this) borrow() {
return _payload;
}
alias borrow this;
}
This already solves avoiding dangling references to the
exception: No references can be left behind when the exception
is thrown, and the wrapper will not be destroyed, but moved,
thus not releasing the exception's memory.
The second part is really "just" some way to transport the
wrapper via the exception mechanism, including support for
catching wrappers of derived exception types.

Post by Dicebot via Digitalmars-d
Special casing catching such wrappers to still preserve
original ref-counted type while pretending to be Throwable at
call site sounds like a terrible hack, much worse than any
sort of ARC complexity.

IMO it would only a hack _if_ they were indeed special cased.
I'm sure we can find a more generic mechanism that would allow
this to be implemented cleanly. Note that the other
requirements I described above (borrowing, move semantics) are
also things that just happen to be usable here: they are
desirable in general, exceptions are just one possible
application.
[1] http://wiki.dlang.org/User:Schuetzm/scope

The mechanism might simply be to allow anything to be thrown that
a) supports moving, e.g. provides a method `release()` that
returns a unique expression (this requires moving and uniqueness
to be specified first, which is a good idea anyway), and
b) is implicitly convertible to Throwable.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:27:34 UTC

Are you suggesting we implement ARC?

Yes. -- Andrei

I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

s/WIN/Rust/

deadalnix via Digitalmars-d

2014-09-21 00:30:44 UTC

On Saturday, 20 September 2014 at 16:27:33 UTC, Andrei

Post by via Digitalmars-d
I don't think ARC is needed.
library RC + borrowing + uniqueness/moving = WIN

s/WIN/Rust/

Well Rust does some things well. We are going in the same
direction anyway (type qualifier, uniqueness) expect we do it in
an ad hoc manner that is guaranteed to yield a C++ish result.

bearophile via Digitalmars-d

2014-09-20 12:17:54 UTC

Post by Jacob Carlborg via Digitalmars-d
Are you suggesting we implement ARC?

Yes. -- Andrei

I think it's better first to design & implement a first version
of memory ownership management, and then later add a reference
counting scheme.

Bye,
bearophile

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:45:13 UTC

Post by Jacob Carlborg via Digitalmars-d
Are you suggesting we implement ARC?

Yes. -- Andrei

I think it's better first to design & implement a first version of
memory ownership management, and then later add a reference counting
scheme.

That's not quite informative. -- Andrei

Vladimir Panteleev via Digitalmars-d

2014-09-20 00:27:38 UTC

On Friday, 19 September 2014 at 15:32:38 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via Digitalmars-d
First, there must be some compiler flag -nogc or something,
which triggers the RC exceptions. All modules of an application
must be compiled with this flag if it is to work (such that one
module can throw an exception caught by the other). Of course a
lot of refinement needs to be added here (what happens if one
tries to link modules built with and without -nogc, allowing
people to detect the flag programmatically by using
version(nogc) etc).

Won't using -nogc preclude using this in Phobos/Druntime?

How about instead of a new switch, change the name mangling of
anything that inherits from RCObject? E.g. 'C' is used for
classes, 'R' could be used for reference-counting classes. Then
an ordinary version block can be used to switch between the two.

Walter Bright via Digitalmars-d

2014-09-20 01:18:37 UTC

As discussed, having exception objects being GC-allocated is clearly a large
liability that we need to address. They prevent otherwise careful functions from
litter here and there.
The Throwable hierarchy is somewhat separate from everything else, which makes
it a great starting point for investigating an automated reference count
approach. Here's what I'm thinking.
First, there must be some compiler flag -nogc or something, which triggers the
RC exceptions. All modules of an application must be compiled with this flag if
it is to work (such that one module can throw an exception caught by the other).
Of course a lot of refinement needs to be added here (what happens if one tries
to link modules built with and without -nogc, allowing people to detect the flag
programmatically by using version(nogc) etc).

Having a compiler switch to change the behavior of every module in incompatible
ways would be a disastrous balkanization. It has to be done in such a way that
the ARC and the existing exceptions can coexist.

If -nogc is passed, the compiler severs the inheritance relationship between
Throwable and Object, making it impossible to convert a Throwable to an Object.
From then henceforth, Throwable and Object form a two-rooted forest. (In all
likelihood we'll later add an RCObject root that Throwable inherits.)
Whenever a reference to a Throwable is copied about, passed to functions, the
compiler inserts appropriately calls to e.g. incRef and decRef. (Compiler may
assume they cancel each other for optimization purposes.) Implementation of
these is up to the runtime library. Null checking may be left to either the
compiler or the library (in the latter case, the functions must be nonmember).
However it seems the compiler may have an advantage because it can elide some of
these checks.
Once we get this going, we should accumulate good experience that we can later
apply to generalizing this approach to more objects. Also, if things go well we
may as well define _always_ (whether GC or not) Throwable to be reference
counted; seems like a good fit for all programs.
Please chime in with thoughts.
Andrei

Andrei Alexandrescu via Digitalmars-d

2014-09-20 01:48:43 UTC

Post by Walter Bright via Digitalmars-d
Having a compiler switch to change the behavior of every module in
incompatible ways would be a disastrous balkanization. It has to be done
in such a way that the ARC and the existing exceptions can coexist.

Could you please elaborate why the disaster? -- Andrei

Walter Bright via Digitalmars-d

2014-09-20 02:20:17 UTC

Could you please elaborate why the disaster? -- Andrei

1. Every library for D will have to come in two versions. One which works, and
the other is likely untested. This will divide the D community in half.

2. RC and GC engender different styles of programming. The idea that one can
successfully switch between them with merely a compiler switch is a fantasy. You
cannot write non-trivial programs that function well and pay no attention to how
memory management is done.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 04:43:01 UTC

Could you please elaborate why the disaster? -- Andrei

1. Every library for D will have to come in two versions. One which
works, and the other is likely untested. This will divide the D
community in half.

I understand where you're coming from. And you're right that we're
looking at an important change - as big as -m64 vs. -m32 or porting to a
new platform. It's big. But it's needed, soon. And it would be, it seems
to me, a mistake to approach this big change as a small change that we
can sneak in.

This judgment - that RC vs. GC would balkanize the community - has
become a prejudice that is worth revisiting. Just trotting it out again
won't do.

Post by Walter Bright via Digitalmars-d
2. RC and GC engender different styles of programming. The idea that one
can successfully switch between them with merely a compiler switch is a
fantasy. You cannot write non-trivial programs that function well and
pay no attention to how memory management is done.

I think we're at a point where we owe it to ourselves to consider
realizing the fantasy.

Andrei

Paulo Pinto via Digitalmars-d

2014-09-20 06:28:13 UTC

Could you please elaborate why the disaster? -- Andrei

1. Every library for D will have to come in two versions. One which
works, and the other is likely untested. This will divide the D
community in half.

I understand where you're coming from. And you're right that we're
looking at an important change - as big as -m64 vs. -m32 or porting to a
new platform. It's big. But it's needed, soon. And it would be, it seems
to me, a mistake to approach this big change as a small change that we
can sneak in.
This judgment - that RC vs. GC would balkanize the community - has
become a prejudice that is worth revisiting. Just trotting it out again
won't do.

I think we're at a point where we owe it to ourselves to consider
realizing the fantasy.
Andrei

This is one of the reasons why the Objective-C GC failed.

Mixing Frameworks compiled with both modes never worked properly.

--
Paulo

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 07:35:46 UTC

Post by Paulo Pinto via Digitalmars-d
This is one of the reasons why the Objective-C GC failed.
Mixing Frameworks compiled with both modes never worked
properly.

Yeah, mixing is bad unless you template eveything. But in pure
nogc it would work.

Except, I think most pure nogc projects want to eliminate
rtti/exceptions too...

Dicebot via Digitalmars-d

2014-09-20 07:44:25 UTC

On Saturday, 20 September 2014 at 07:35:47 UTC, Ola Fosheim

Post by Ola Fosheim Grostad via Digitalmars-d
Except, I think most pure nogc projects want to eliminate
rtti/exceptions too...

It should be possible to do on top of plain @nogc Phobos by
simply hooking into druntime _d_throw function and providing some
own domain-specific handler instead. Of course it doesn't help if
exceptions are used to indicate casual data processing signals
but it is something else to be fixed in Phobos :)

Olivier Pisano via Digitalmars-d

2014-09-20 07:46:31 UTC

Post by Paulo Pinto via Digitalmars-d
This is one of the reasons why the Objective-C GC failed.
Mixing Frameworks compiled with both modes never worked
properly.
--
Paulo

Yes this was a huge failure to take into account.

Linking code where Throwable inherits from Object with code where
Throwable inherits from RCObject... :(

If making the GC completely optional is a must, then error
handling shouldn't rely on it at all, no? What about completely
switching exception handling to RC ? Would it have an impact on
memory safety since exeption handling mecanism is somehow
"magical code generated by the compiler" ?

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:07:20 UTC

Post by Olivier Pisano via Digitalmars-d
If making the GC completely optional is a must, then error
handling shouldn't rely on it at all, no? What about completely
switching exception handling to RC ? Would it have an impact on
memory safety since exeption handling mecanism is somehow
"magical code generated by the compiler" ?

The more I think of it the more sensible this is. Exceptions are
unlikely to create cycles, not copied extensively, and are generally
short lived. So an RC scheme backed by malloc/free seems to be most
appropriate.

There would be breakage, though: Throwable would not be convertible to
Object. I wonder what the impact in the real world that would cause.

Andrei

Mike Parker via Digitalmars-d

2014-09-20 07:48:48 UTC

Could you please elaborate why the disaster? -- Andrei

1. Every library for D will have to come in two versions. One which
works, and the other is likely untested. This will divide the D
community in half.

I understand where you're coming from. And you're right that we're
looking at an important change - as big as -m64 vs. -m32 or porting to a
new platform. It's big. But it's needed, soon. And it would be, it seems
to me, a mistake to approach this big change as a small change that we
can sneak in.
This judgment - that RC vs. GC would balkanize the community - has
become a prejudice that is worth revisiting. Just trotting it out again
won't do.

Unless I'm missing something, Walter is referring specifically to the
-nogc switch (or any such switch) and not to the concept of RC vs GC in
general. And on that I am in strong agreement, FWIW.

Recall back during the D1/D2, Phobos/Tango period, library maintainers
had to either choose which D version and library to support (D1/Phobos,
D1/Tango, D2/Phobos) or come up with a means of supporting all of them.
Wrapper modules and mixins were a workable (though terribly annoying)
solution for some cases, but not all. It was a nasty mess.

I don't know yet what the ultimate impact of a -nogc switch would be on
library maintenance, but I have a strong suspicion that simple wrapper
modules and mixins wouldn't be enough to maintain compatibility. If
there is even the slightest possibility that library maintainers will
find themselves in that same situation of having to choose a or b
because maintaining both is too troublesome to bother, then all the
implications need to be hashed out beforehand.

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:15:45 UTC

Post by Mike Parker via Digitalmars-d
I don't know yet what the ultimate impact of a -nogc switch would be on
library maintenance, but I have a strong suspicion that simple wrapper
modules and mixins wouldn't be enough to maintain compatibility. If
there is even the slightest possibility that library maintainers will
find themselves in that same situation of having to choose a or b
because maintaining both is too troublesome to bother, then all the
implications need to be hashed out beforehand.

We need to explore that. A possibility is to support coexistence and
then have the option to use a tool statically pinpoint the uses of GC.
-- Andrei

Adam D. Ruppe via Digitalmars-d

2014-09-20 16:31:47 UTC

On Saturday, 20 September 2014 at 16:15:45 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
We need to explore that. A possibility is to support
coexistence and then have the option to use a tool statically
pinpoint the uses of GC. -- Andrei

What, *exactly*, does "uses of GC" mean? In other words, what
specifically makes GC.malloc evil that must be avoided at any
cost while C malloc (+ GC.addRange most likely) is an acceptable
replacement?

A few things that come to mind are:

1) Obviously, GC.malloc can trigger a collection. But this can be
easily disabled.

2) The GC lock? I don't know how malloc handles this though.

3) Is GC.free substantially different than C's free?

4) Programmers don't typically explicitly free GC memory... but
we could.

5) Bookkeeping overhead? I know malloc has some too though, is
this really a dealbreaker?

6) Public relations.

...that's all I can think of. What am I missing? Which one of
these is actually causing the problem that we're supposed to be
fixing here?

Andrei Alexandrescu via Digitalmars-d

2014-09-20 17:17:06 UTC

Post by Adam D. Ruppe via Digitalmars-d

Post by Andrei Alexandrescu via Digitalmars-d
We need to explore that. A possibility is to support coexistence and
then have the option to use a tool statically pinpoint the uses of GC.
-- Andrei

What, *exactly*, does "uses of GC" mean? In other words, what
specifically makes GC.malloc evil that must be avoided at any cost while
C malloc (+ GC.addRange most likely) is an acceptable replacement?
1) Obviously, GC.malloc can trigger a collection. But this can be easily
disabled.
2) The GC lock? I don't know how malloc handles this though.
3) Is GC.free substantially different than C's free?
4) Programmers don't typically explicitly free GC memory... but we could.
5) Bookkeeping overhead? I know malloc has some too though, is this
really a dealbreaker?
6) Public relations.
....that's all I can think of. What am I missing? Which one of these is
actually causing the problem that we're supposed to be fixing here?

The only problem is that GC.malloc doesn't need to be paired by a call
to GC.free, whereas malloc must.

Andrei

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:26:05 UTC

Agreed. -- Andrei

Vladimir Panteleev via Digitalmars-d

2014-09-20 02:04:21 UTC

On Saturday, 20 September 2014 at 01:19:05 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Having a compiler switch to change the behavior of every module
in incompatible ways would be a disastrous balkanization.

Don't -version, -debug etc. have this problem anyway?

Anyway, if you change the name mangling, then you'll get link
errors instead of mysterious crashes.

Post by Walter Bright via Digitalmars-d
It has to be done in such a way that the ARC and the existing
exceptions can coexist.

How?

Walter Bright via Digitalmars-d

2014-09-20 02:26:22 UTC

Post by Vladimir Panteleev via Digitalmars-d

Post by Walter Bright via Digitalmars-d
Having a compiler switch to change the behavior of every module in
incompatible ways would be a disastrous balkanization.

Don't -version, -debug etc. have this problem anyway?

I submit that it is a poor practice to write code that way.

Post by Vladimir Panteleev via Digitalmars-d
Anyway, if you change the name mangling, then you'll get link errors instead of
mysterious crashes.

Post by Walter Bright via Digitalmars-d
It has to be done in such a way that the ARC and the existing exceptions can
coexist.

How?

Good question. It's a challenge. But it has to be done, or D will divide in half
and both halves will fail.

We've had this discussion numerous times before - "throw the magic compiler
switch" and D becomes an ARC system and everything is wonderful. It cannot work.
ARC and GC are not equivalent.

Daniel N via Digitalmars-d

2014-09-20 03:13:26 UTC

On Saturday, 20 September 2014 at 02:26:49 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Good question. It's a challenge. But it has to be done, or D
will divide in half and both halves will fail.
We've had this discussion numerous times before - "throw the
magic compiler switch" and D becomes an ARC system and
everything is wonderful. It cannot work. ARC and GC are not
equivalent.

I basically agree with Walter on this one, no switch please, it's
maintenance nightmare for library devs.

My proposal would be to permanently use ARC for Throwable, no
flags.

What does the GC bring to exceptions that makes it sufficiently
invaluable to warrant two parallel implementations? It can't be
about performance, since _thrown_ exceptions are already in the
slow path... Backwards compatibility?

Walter Bright via Digitalmars-d

2014-09-20 03:25:33 UTC

What does the GC bring to exceptions that makes it sufficiently invaluable to
warrant two parallel implementations? It can't be about performance, since
_thrown_ exceptions are already in the slow path... Backwards compatibility?

Backwards compatibility.

Daniel N via Digitalmars-d

2014-09-20 03:45:16 UTC

On Saturday, 20 September 2014 at 03:25:39 UTC, Walter Bright

Post by Daniel N via Digitalmars-d
What does the GC bring to exceptions that makes it
sufficiently invaluable to
warrant two parallel implementations? It can't be about
performance, since
_thrown_ exceptions are already in the slow path... Backwards
compatibility?

Backwards compatibility.

Thanks!

As much as I applaud our efforts for Backwards compatibility, it
only affects code which already has been written, whereas a
switch would affect "all" code that will be written, imho the
burden both on library and app devs would be less if we simply
redefined Throwable.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 04:54:28 UTC

As much as I applaud our efforts for Backwards compatibility, it only
affects code which already has been written, whereas a switch would
affect "all" code that will be written, imho the burden both on library
and app devs would be less if we simply redefined Throwable.

We need -nogc in the same way we need -m32 and -m64. -- Andrei

Andrei Alexandrescu via Digitalmars-d

2014-09-20 04:48:32 UTC

Post by Daniel N via Digitalmars-d

Post by Walter Bright via Digitalmars-d
Good question. It's a challenge. But it has to be done, or D will
divide in half and both halves will fail.
We've had this discussion numerous times before - "throw the magic
compiler switch" and D becomes an ARC system and everything is
wonderful. It cannot work. ARC and GC are not equivalent.

I basically agree with Walter on this one, no switch please, it's
maintenance nightmare for library devs.

Why is it a maintenance nightmare?

Post by Daniel N via Digitalmars-d
My proposal would be to permanently use ARC for Throwable, no flags.

How about other objects? Throwable is but the first step, and a good one
to inform larger designs. We can't force RC on all objects on all
applications.

Post by Daniel N via Digitalmars-d
What does the GC bring to exceptions that makes it sufficiently
invaluable to warrant two parallel implementations? It can't be about
performance, since _thrown_ exceptions are already in the slow path...
Backwards compatibility?

Conversion from Throwable to Object.

Andrei

Daniel N via Digitalmars-d

2014-09-20 11:55:31 UTC

On Saturday, 20 September 2014 at 04:48:33 UTC, Andrei

Post by Daniel N via Digitalmars-d
I basically agree with Walter on this one, no switch please,
it's
maintenance nightmare for library devs.

Why is it a maintenance nightmare?

Sorry, let's defer this, I might have overstated the negative
side-effects before carefully considering methods to mitigate the
impact.

Post by Daniel N via Digitalmars-d
My proposal would be to permanently use ARC for Throwable, no
flags.

How about other objects? Throwable is but the first step, and a
good one to inform larger designs. We can't force RC on all
objects on all applications.

I was viewing everything from a totally different perspective:
Normally what prevents an API to be nogc is exceptions, even
_emplace_ falls victim of this, in my line of reasoning I was
counting on the ripple effect to get us quite far, the remaining
issues could be fixed by refining the API to make it possible to
avoid allocations if desired.

However, since you had a different goal in mind, your solution
starts making more sense to me, it's also very easy to flip a
switch and benchmark individual applications, maybe people from
both camps will be surprised at the results.

Post by Daniel N via Digitalmars-d
What does the GC bring to exceptions that makes it sufficiently
invaluable to warrant two parallel implementations? It can't
be about
performance, since _thrown_ exceptions are already in the slow path...
Backwards compatibility?

Conversion from Throwable to Object.
Andrei

The conversion could allocate on the GC, which would be
functionally backwards compatible and if the performance
degradation is a concern the code could be updated to avoid the
superfluous allocation by staying in the RCObject hierarchy.

Daniel

Andrei Alexandrescu via Digitalmars-d

2014-09-20 04:47:00 UTC

Post by Vladimir Panteleev via Digitalmars-d

Post by Walter Bright via Digitalmars-d
Having a compiler switch to change the behavior of every module in
incompatible ways would be a disastrous balkanization.

Don't -version, -debug etc. have this problem anyway?

I submit that it is a poor practice to write code that way.

Post by Vladimir Panteleev via Digitalmars-d
Anyway, if you change the name mangling, then you'll get link errors instead of
mysterious crashes.

Post by Walter Bright via Digitalmars-d
It has to be done in such a way that the ARC and the existing exceptions can
coexist.

How?

Good question. It's a challenge. But it has to be done, or D will divide
in half and both halves will fail.

I don't think there will be division, and I don't think the parts will
fail. I see it as enrichment.

We've said for years that D is usable without a GC. The time has come to
make that into reality. To the extent it's automatic, we'll make it
automatic. To the extent we'll require adjustment in use, we'll ask
users to adjust use. The worst thing about that is to enter the design
with a prejudice tied to our ankles.

Post by Walter Bright via Digitalmars-d
We've had this discussion numerous times before - "throw the magic
compiler switch" and D becomes an ARC system and everything is
wonderful. It cannot work. ARC and GC are not equivalent.

I think past discussions have been inconclusive at best and must be
reopened.

Andrei

Walter Bright via Digitalmars-d

2014-09-20 05:24:05 UTC

I think past discussions have been inconclusive at best and must be reopened.

For starters,

1. ARC is slow and generates bloated code.

2. To deal with (1), existing ARC systems allow escapes from it. This has severe
implications for memory safety. Rust's entire type system appears to be designed
around trying to deal with this, and AFAIK they're the only ones who have tried.
C++ shared_ptr and ObjectiveC's ARC are not memory safe. They are not. Not not
not, and they don't even try. :-)

3. ARC objects require an embedded count. This means they cannot be mixed with
non-ARC objects. This is fundamentally DIFFERENT from how GC behaves, and we
cannot pretend or wish this away or add a compiler switch to make it go away.

None of these points were addressed in previous discussions beyond the
supposition of some nonexistent compiler technology that would make them go away.

D can have ref counted objects, but it will not work with a compiler switch to
switch them back and forth. They'll have to coexist peacefully with GC objects.

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 05:32:58 UTC

On Saturday, 20 September 2014 at 05:24:11 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
D can have ref counted objects, but it will not work with a
compiler switch to switch them back and forth. They'll have to
coexist peacefully with GC objects.

If you don't expose the refcount, write libraries as if you have
only RC, and don't expose destructors then you probably can. Just
turn inc/dec into dummies.

Walter Bright via Digitalmars-d

2014-09-20 06:01:13 UTC

Post by Walter Bright via Digitalmars-d
D can have ref counted objects, but it will not work with a compiler switch to
switch them back and forth. They'll have to coexist peacefully with GC objects.

If you don't expose the refcount, write libraries as if you have only RC, and
don't expose destructors then you probably can. Just turn inc/dec into dummies.

This doesn't address any of the 3 concerns.

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 07:17:46 UTC

On Saturday, 20 September 2014 at 06:01:19 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
This doesn't address any of the 3 concerns.

1. RC efficiency is architecture dependent. E.g. TSX is coming
even if there is a bug in lower end CPUs. Suggest making
performance oriented prototypes on different architectures before
concluding.

2. As long as you have a RC pointer to an obj on the stack you
can switch to regular references. No magic involved for the easy
case, just semantic analysis.

3. True, but you can keep the refcount at a negative offset for
new-based allocations. Besides it only affects those who do @nogc
and they should know what they are doing.

Walter Bright via Digitalmars-d

2014-09-20 08:39:35 UTC

Post by Walter Bright via Digitalmars-d
This doesn't address any of the 3 concerns.

1. RC efficiency is architecture dependent.

Please show me the efficient assembler for the x86.

Post by Ola Fosheim Grostad via Digitalmars-d
E.g. TSX is coming even if there is
a bug in lower end CPUs. Suggest making performance oriented prototypes on
different architectures before concluding.

I'd love to see it.

Post by Ola Fosheim Grostad via Digitalmars-d
2. As long as you have a RC pointer to an obj on the stack you can switch to
regular references. No magic involved for the easy case, just semantic analysis.

As soon as you pass a reference to a function, that all goes out the window.
There's a reason why Rust has invested so much effort in the notion of a
"borrowed" pointer.

Post by Ola Fosheim Grostad via Digitalmars-d
3. True, but you can keep the refcount at a negative offset for new-based
what they are doing.

If this is so simple, why doesn't everyone do it?

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 10:00:41 UTC

On Saturday, 20 September 2014 at 08:39:41 UTC, Walter Bright

Post by Ola Fosheim Grostad via Digitalmars-d
E.g. TSX is coming even if there is
a bug in lower end CPUs. Suggest making performance oriented
prototypes on
different architectures before concluding.

I'd love to see it.

A 128 bit CAS instruction is at about 19-25 cycles, but a
transaction on the other hand can by using xbegin/xend cover both
refcounting, locking and rollback of multiple objects so you need
cooperation from code gen. Basically all changes between
xbegin/xend are kept in cache and written to memory upon success.
On failure you have a slower fallback.

I don't know what that leads to in amortized cost reduction, but
30-70% might be possible if it is done right.

Post by Walter Bright via Digitalmars-d
As soon as you pass a reference to a function, that all goes
out the window. There's a reason why Rust has invested so much
effort in the notion of a "borrowed" pointer.

The pure @nogc crowd care less about safety, but you should be
able to track this using dataflow?

Post by Ola Fosheim Grostad via Digitalmars-d
3. True, but you can keep the refcount at a negative offset
for new-based
they should know
what they are doing.

If this is so simple, why doesn't everyone do it?

Never said performance and thread safe RC was easy. It is
probably difficult to get below 10 cycles for inc/dec pairs even
with excellent code gen...? And probably closer to 40 cycles for
regular code gen. Just guessing.

Walter Bright via Digitalmars-d

2014-09-20 18:17:50 UTC

I'd love to see it.

A 128 bit CAS instruction is at about 19-25 cycles, but a transaction on the
other hand can by using xbegin/xend cover both refcounting, locking and rollback
of multiple objects so you need cooperation from code gen. Basically all changes
between xbegin/xend are kept in cache and written to memory upon success. On
failure you have a slower fallback.
I don't know what that leads to in amortized cost reduction, but 30-70% might be
possible if it is done right.

Please show me the inc/dec optimized x86 code.

Post by Walter Bright via Digitalmars-d
As soon as you pass a reference to a function, that all goes out the window.
There's a reason why Rust has invested so much effort in the notion of a
"borrowed" pointer.

this using dataflow?

Suppose I pass a pointer to:

void foo(T* p);

How do I do dataflow?

Post by Ola Fosheim Grostad via Digitalmars-d
3. True, but you can keep the refcount at a negative offset for new-based
what they are doing.

If this is so simple, why doesn't everyone do it?

Never said performance and thread safe RC was easy. It is probably difficult to
get below 10 cycles for inc/dec pairs even with excellent code gen...? And
probably closer to 40 cycles for regular code gen. Just guessing.

The question was about mixing RC'd and non-RC'd objects.

via Digitalmars-d

2014-09-20 18:26:55 UTC

On Saturday, 20 September 2014 at 18:18:05 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Please show me the inc/dec optimized x86 code.

There's no optimization of the inc/dec, you use regular inc/dec
within a larger xbegin/xend block.

You amortize the cost over all the lock-prefixed instructions you
would otherwise have executed in that block. There is no locking,
there is a transaction instead. Only if the transaction fails do
you execute the locking fallback code.

Post by Walter Bright via Digitalmars-d
void foo(T* p);
How do I do dataflow?

You mean a separate compilation unit that the compiler don't have
access to? You don't, obviously.

Post by Walter Bright via Digitalmars-d
The question was about mixing RC'd and non-RC'd objects.

Well, if you are talking about negative offset for ref counts,
then people do it. E.g. there are examples of systems that
provide C-compatible strings with length at offset -4 and ref
count at -8.

If you need to distinguish you can either use address-space info
or keep a negative int in the non-RC'd object.

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 18:52:58 UTC

On Saturday, 20 September 2014 at 18:26:56 UTC, Ola Fosheim

Post by via Digitalmars-d
On Saturday, 20 September 2014 at 18:18:05 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Please show me the inc/dec optimized x86 code.

There's no optimization of the inc/dec, you use regular inc/dec
within a larger xbegin/xend block.

Actually you dont need to inc/dec if you do both in the
transaction. You only need to read the refcount. If somebody else
writes to the ref count the transaction fails.

So, you cannot succeed a transaction while somebody else decrease
the ref count.

Sounds right?

Walter Bright via Digitalmars-d

2014-09-20 22:07:37 UTC

On 9/20/2014 11:26 AM, "Ola Fosheim GrÃžstad"

Post by Walter Bright via Digitalmars-d
Please show me the inc/dec optimized x86 code.

There's no optimization of the inc/dec, you use regular inc/dec within a larger
xbegin/xend block.
You amortize the cost over all the lock-prefixed instructions you would
otherwise have executed in that block. There is no locking, there is a
transaction instead. Only if the transaction fails do you execute the locking
fallback code.

I strongly suggest taking a look at C++ shared_ptr<T>, compile a simple example,
and examine the generated code. It is NOT AT ALL as simple as emitting an
inc/dec pair. I.e. what happens if an exception is thrown between the inc and
the dec?

If RC was as costless and simple as you are suggesting, everyone would be doing
it and would have for decades.

Post by Walter Bright via Digitalmars-d
void foo(T* p);
How do I do dataflow?

You mean a separate compilation unit that the compiler don't have access to? You
don't, obviously.

Right. "dataflow" is not the answer.

Post by Walter Bright via Digitalmars-d
The question was about mixing RC'd and non-RC'd objects.

Well, if you are talking about negative offset for ref counts, then people do
it. E.g. there are examples of systems that provide C-compatible strings with
length at offset -4 and ref count at -8.

Now embed that object as a field in some other object.

If you need to distinguish you can either use address-space info or keep a
negative int in the non-RC'd object.

Now add that runtime distinguishing logic to the inc/dec code.

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 22:30:44 UTC

On Saturday, 20 September 2014 at 22:07:45 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
On 9/20/2014 11:26 AM, "Ola Fosheim GrÃžstad"

Post by via Digitalmars-d
On Saturday, 20 September 2014 at 18:18:05 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Please show me the inc/dec optimized x86 code.

There's no optimization of the inc/dec, you use regular
inc/dec within a larger
xbegin/xend block.
You amortize the cost over all the lock-prefixed instructions
you would
otherwise have executed in that block. There is no locking,
there is a
transaction instead. Only if the transaction fails do you
execute the locking
fallback code.

C++ provides weak references. If you throw issue xabort.

Nobody have said it is free, and TSX is new tech. So what you can
do, who knows?

Post by Walter Bright via Digitalmars-d
If RC was as costless and simple as you are suggesting,
everyone would be doing it and would have for decades.

Transactions are not free.

Post by Walter Bright via Digitalmars-d
Now embed that object as a field in some other object.

Uhm, I referred to new-allocated objects.

Post by Walter Bright via Digitalmars-d
Now add that runtime distinguishing logic to the inc/dec code.

Test N flag?

Andrei Alexandrescu via Digitalmars-d

2014-09-20 05:46:43 UTC

I think past discussions have been inconclusive at best and must be reopened.

For starters,
1. ARC is slow and generates bloated code.

Agreed. It has the cost distributed a different way.

Post by Walter Bright via Digitalmars-d
2. To deal with (1), existing ARC systems allow escapes from it. This
has severe implications for memory safety. Rust's entire type system
appears to be designed around trying to deal with this, and AFAIK
they're the only ones who have tried. C++ shared_ptr and ObjectiveC's
ARC are not memory safe. They are not. Not not not, and they don't even
try. :-)

That's fine. You can't have everything.

Post by Walter Bright via Digitalmars-d
3. ARC objects require an embedded count. This means they cannot be
mixed with non-ARC objects. This is fundamentally DIFFERENT from how GC
behaves, and we cannot pretend or wish this away or add a compiler
switch to make it go away.

In nogc mode, everything is ARC so I'm not getting this.

Post by Walter Bright via Digitalmars-d
None of these points were addressed in previous discussions beyond the
supposition of some nonexistent compiler technology that would make them go away.

That was indeed a crappy argument, and not advanced by me. I'm not
advocating for that.

Post by Walter Bright via Digitalmars-d
D can have ref counted objects, but it will not work with a compiler
switch to switch them back and forth. They'll have to coexist peacefully
with GC objects.

We need to figure out a design. All I'm saying is we must not bring
prejudice to the table. And a very basic point is: there will be a way
to COMPLETELY disable the GC. That is a must.

Andrei

Walter Bright via Digitalmars-d

2014-09-20 06:15:59 UTC

That's fine. You can't have everything.

You're willing to dispense with memory safety?

In nogc mode, everything is ARC so I'm not getting this.

Not even ARC languages try to make everything ARC - because it is BLOATED and
SLOW. Not C++, not ObjectiveC, not Rust.

Post by Walter Bright via Digitalmars-d
D can have ref counted objects, but it will not work with a compiler
switch to switch them back and forth. They'll have to coexist peacefully
with GC objects.

We need to figure out a design. All I'm saying is we must not bring prejudice to
the table. And a very basic point is: there will be a way to COMPLETELY disable
the GC. That is a must.

It doesn't have to be disabled. Just don't call it. The GC is not a bridge troll
that is going to pop up and extract a toll even if you aren't allocating with it.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 15:59:44 UTC

That's fine. You can't have everything.

You're willing to dispense with memory safety?

No.

In nogc mode, everything is ARC so I'm not getting this.

Not even ARC languages try to make everything ARC - because it is
BLOATED and SLOW. Not C++, not ObjectiveC, not Rust.

Agreed.

Post by Walter Bright via Digitalmars-d
D can have ref counted objects, but it will not work with a compiler
switch to switch them back and forth. They'll have to coexist peacefully
with GC objects.

We need to figure out a design. All I'm saying is we must not bring prejudice to
the table. And a very basic point is: there will be a way to
COMPLETELY disable
the GC. That is a must.

It doesn't have to be disabled. Just don't call it. The GC is not a
bridge troll that is going to pop up and extract a toll even if you
aren't allocating with it.

Well obviously there's a hierarchy of desirability:

1. Using the GC inadvertently causes a compile-time error.

2. Using the GC inadvertently causes a link-time error.

3. Using the GC inadvertently causes a run-time error.

4. Using the GC inadvertently goes undetected.

I disagree that (4) is good for everyone.

Andrei

Walter Bright via Digitalmars-d

2014-09-20 18:19:01 UTC

Post by Andrei Alexandrescu via Digitalmars-d
4. Using the GC inadvertently goes undetected.
I disagree that (4) is good for everyone.

We've already implemented @nogc.

via Digitalmars-d

2014-09-20 08:16:39 UTC

On Saturday, 20 September 2014 at 05:46:44 UTC, Andrei

Post by Walter Bright via Digitalmars-d
D can have ref counted objects, but it will not work with a
compiler
switch to switch them back and forth. They'll have to coexist
peacefully
with GC objects.

We need to figure out a design. All I'm saying is we must not
bring prejudice to the table. And a very basic point is: there
will be a way to COMPLETELY disable the GC. That is a must.

The ref counted objects still need to be considered as roots for
the GC. This limits how much "completely" can be achieved.

Dicebot via Digitalmars-d

2014-09-20 07:25:00 UTC

On Saturday, 20 September 2014 at 02:26:49 UTC, Walter Bright

Post by Vladimir Panteleev via Digitalmars-d
On Saturday, 20 September 2014 at 01:19:05 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Having a compiler switch to change the behavior of every
module in
incompatible ways would be a disastrous balkanization.

Don't -version, -debug etc. have this problem anyway?

I submit that it is a poor practice to write code that way.

Post by Vladimir Panteleev via Digitalmars-d
Anyway, if you change the name mangling, then you'll get link
errors instead of
mysterious crashes.

Post by Walter Bright via Digitalmars-d
It has to be done in such a way that the ARC and the existing exceptions can
coexist.

How?

Good question. It's a challenge. But it has to be done, or D
will divide in half and both halves will fail.
We've had this discussion numerous times before - "throw the
magic compiler switch" and D becomes an ARC system and
everything is wonderful. It cannot work. ARC and GC are not
equivalent.

I am fully with Walter here. Only way to do it in acceptable way
is to make both reference counted exceptions and current ones
co-exist. Anything else means that we will need to distribute 4
builds of Phobos - debug-nogc, debug-gc, release-nogc,
release-gc. Guess you can smell exponential explosion here as
well as I do.

ketmar via Digitalmars-d

2014-09-20 03:30:30 UTC

On Fri, 19 Sep 2014 08:32:38 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d at puremagic.com>

Post by Andrei Alexandrescu via Digitalmars-d
First, there must be some compiler flag -nogc or something, which

please, no compiler switches! it's partly ok for "-release", it's not
very good for "-version" and it's awful to add another one which
completely alters the way something works. as much as i want to make
throwables nogc, it's better to change nothing than to add such switch.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140920/cf50fc7f/attachment.sig>

Andrei Alexandrescu via Digitalmars-d

2014-09-20 04:53:32 UTC

Post by ketmar via Digitalmars-d
On Fri, 19 Sep 2014 08:32:38 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d at puremagic.com>

Post by Andrei Alexandrescu via Digitalmars-d
First, there must be some compiler flag -nogc or something, which

Changing nothing is not an option. Ideas? -- Andrei

ketmar via Digitalmars-d

2014-09-20 05:25:55 UTC

On Fri, 19 Sep 2014 21:53:32 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d at puremagic.com>

Post by Andrei Alexandrescu via Digitalmars-d
Changing nothing is not an option. Ideas? -- Andrei

let compiler do it's ARC magic on Throwable and allow programmer to
plug in his own allocator. and drop out GC mechanics for Throwables
entirely (except scanning Throwable instance for GC-managed pointers).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140920/978e8b55/attachment.sig>

Dicebot via Digitalmars-d

2014-09-20 07:29:51 UTC

On Saturday, 20 September 2014 at 04:53:32 UTC, Andrei

Post by ketmar via Digitalmars-d
On Fri, 19 Sep 2014 08:32:38 -0700
Andrei Alexandrescu via Digitalmars-d
<digitalmars-d at puremagic.com>

Post by Andrei Alexandrescu via Digitalmars-d
First, there must be some compiler flag -nogc or something,
which

Changing nothing is not an option. Ideas? -- Andrei

I have been thinking about something like `ARC` interface and
`ARCException : Throwable, ARC` with all the automatic ref
inc/dec calls you have mentioned added for any class that
implements `ARC` interface.

Dicebot via Digitalmars-d

2014-09-20 07:33:06 UTC

Post by Dicebot via Digitalmars-d
I have been thinking about something like `ARC` interface and
`ARCException : Throwable, ARC` with all the automatic ref
inc/dec calls you have mentioned added for any class that
implements `ARC` interface.

..and yes, I'd expect it to be the default one used by Phobos
(possibly with some backing pool allocator). It isn't critical
for GC-enabled applications to use GC-based exceptions but
contrary is true.

Walter Bright via Digitalmars-d

2014-09-20 05:42:11 UTC

Consider:

https://github.com/D-Programming-Language/phobos/blob/master/std/c/windows/com.d#L218

This is D's support for COM objects. They have AddRef() and Release(), i.e. they
are reference counted. They do not inherit from Object. The refcounting is
currently manual.

D could allow an additional root (not a replacement) for thrown Exceptions. This
root would be based on ComObject.

As a bonus, COM objects were designed for C++, and C++ implements them well,
i.e. they would be a compatible object structure.

catch statements could be changed to catch ComThrowable, which would not catch
Throwables, and catches for Throwables would not catch ComThrowables.

Dmitry Olshansky via Digitalmars-d

2014-09-20 08:06:52 UTC

Post by Andrei Alexandrescu via Digitalmars-d
As discussed, having exception objects being GC-allocated is clearly a
large liability that we need to address. They prevent otherwise careful
be okay with a little litter here and there.
The Throwable hierarchy is somewhat separate from everything else, which
makes it a great starting point for investigating an automated reference
count approach. Here's what I'm thinking.
First, there must be some compiler flag -nogc or something, which
triggers the RC exceptions. All modules of an application must be
compiled with this flag if it is to work (such that one module can throw
an exception caught by the other). Of course a lot of refinement needs
to be added here (what happens if one tries to link modules built with
and without -nogc, allowing people to detect the flag programmatically
by using version(nogc) etc).
If -nogc is passed, the compiler severs the inheritance relationship
between Throwable and Object, making it impossible to convert a
Throwable to an Object. From then henceforth, Throwable and Object form
a two-rooted forest. (In all likelihood we'll later add an RCObject root
that Throwable inherits.)
Whenever a reference to a Throwable is copied about, passed to
functions, the compiler inserts appropriately calls to e.g. incRef and
decRef. (Compiler may assume they cancel each other for optimization
purposes.) Implementation of these is up to the runtime library. Null
checking may be left to either the compiler or the library (in the
latter case, the functions must be nonmember). However it seems the
compiler may have an advantage because it can elide some of these checks.
Once we get this going, we should accumulate good experience that we can
later apply to generalizing this approach to more objects. Also, if
things go well we may as well define _always_ (whether GC or not)
Throwable to be reference counted; seems like a good fit for all programs.

All good, it wasn't a good idea to have GC-ed exceptions in the first
place (It's a truly rare thing to have cycles of exceptions). It doesn'
t matter GC or @nogc - this must go.

Speaking of RC-ed Throwable - why adding a switch? Just make a
transition in 2 steps - all exceptions inherit from RCObject (but stay
essentially GC-ed), then hack compiler to perform ARC on RCObject
descendants (maybe even with GC.malloc/GC.free for starters, i.e. the
same heap). Then we can mess with custom allocation of these.

I can understand a transitory switch, say -exception-ongc for the
minority of folks that indeed DO rely on exceptions being GC and not
counted.

--
Dmitry Olshansky

Uranuz via Digitalmars-d

2014-09-20 08:42:50 UTC

I'm quite a noobie in memory models but from position of user of
D language I have some ideas about syntax of switching between
memory models. I think that having everywhere declarations using
wrapper structs (like RefCounted) is not very user-friendly. But
still we need some way to say to compiler how we want to do
memory management. I have read tutorial about Rust language. It
looks not very clear for me, but I like the idea that type of
memory management is included in variable declaration.

Intead of wrapper struct like scoped!A or RefCounted!A we could
declare variables with annotations (attributes in D) to say what
memory model we want to use. Compiler should understand this
annotation and create some code for memory management. I thing
it's possible to declare some interface (maybe struct with
compile-time duck typing) to support some user-defined memory
models or modifications of basic (implemented in a language)
memory models.

Why I am saying about annotations? Because we can annotate some
function, class or just block at the module scope that should use
some sort of memory management and compiler will create
corresponding code. In that case we don't need to put wrapper
struct around all variables that use ref counting. We just
annotate some function or class declaration as ref-counted and
that's it!

It's just a concept of what I like to see in the language design
of D in future)) Of course there are a lot of problems in a
practice.

Also I think that we shouldn't impose some way of memory model to
programmer and let him choose his approach but syntax should be
simple and clear as much as it could be. Also some defaults
should be for most common cases and for *novice* users of D.

Waiting for critic or thoughts!!!))

Uranuz via Digitalmars-d

2014-09-20 08:46:30 UTC

Also it's interesting waht is the correspondence between memory
management, memory model and allocators? I know that
std.allocator is in development. I guess that it should be
considered in complex.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:32:44 UTC

Post by Uranuz via Digitalmars-d
Also it's interesting waht is the correspondence between memory
management, memory model and allocators? I know that std.allocator is in
development. I guess that it should be considered in complex.

1. Memory management: when are calls to allocation and deallocation
primitives made?

2. Allocators: what are the allocation and deallocation primitives, and
how are they implemented?

3. Memory model: how is memory effected by programs? (of interest for
compiler implementers and thread safety)

There is occasional confusion among the three. I myself was in the woods
for a while thinking that allocators should do memory management.

Andrei

via Digitalmars-d

2014-09-20 12:50:12 UTC

Post by Uranuz via Digitalmars-d
I'm quite a noobie in memory models but from position of user
of D language I have some ideas about syntax of switching
between memory models. I think that having everywhere
declarations using wrapper structs (like RefCounted) is not
very user-friendly. But still we need some way to say to
compiler how we want to do memory management. I have read
tutorial about Rust language. It looks not very clear for me,
but I like the idea that type of memory management is included
in variable declaration.
Intead of wrapper struct like scoped!A or RefCounted!A we could
declare variables with annotations (attributes in D) to say
what memory model we want to use. Compiler should understand
this annotation and create some code for memory management. I
thing it's possible to declare some interface (maybe struct
with compile-time duck typing) to support some user-defined
memory models or modifications of basic (implemented in a
language) memory models.
Why I am saying about annotations? Because we can annotate some
function, class or just block at the module scope that should
use some sort of memory management and compiler will create
corresponding code. In that case we don't need to put wrapper
struct around all variables that use ref counting. We just
annotate some function or class declaration as ref-counted and
that's it!
It's just a concept of what I like to see in the language
design of D in future)) Of course there are a lot of problems
in a practice.
Also I think that we shouldn't impose some way of memory model
to programmer and let him choose his approach but syntax should
be simple and clear as much as it could be. Also some defaults
should be for most common cases and for *novice* users of D.
Waiting for critic or thoughts!!!))

You are right that there is an intricate relationship between the
various types of memory (better: ownership) management,
allocators, ref-counting, uniqueness, moving etc, as you write in
your other post. They interact in very specific ways. For this
reason I don't think it is feasible or desirable to set the type
of memory management "from the outside", the types in question
need to know how their innards work. In the same vein, if your
code uses a specific memory management strategy, it has to be
written in a way conforming to it. Walter has already stated
that: you cannot simply slap an attribute onto your code saying
"please do reference counting", and expect it to work, especially
not efficiently.

I think we can get a lot further if we work out said
relationships, and create good general purpose wrapper types that
implement the different strategies accordingly. I believe,
borrowing is crucial to this, which is why I made a proposal
about it [1], which also deals with the other related topics.

You do have a point about a simple, accessible syntax being
important. Still, I don't see a big difference between
@annotations and RC!T wrappers in this regard (except that the
former could apply to entire sections of code, which is not a
good idea IMO). And I think with auto/scope/const type deduction,
code is quite pleasant to read.

[1] http://wiki.dlang.org/User:Schuetzm/scope

ponce via Digitalmars-d

2014-09-20 09:05:13 UTC

On Friday, 19 September 2014 at 15:32:38 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

via Digitalmars-d

2014-09-20 09:09:37 UTC

Post by Vladimir Panteleev via Digitalmars-d
On Friday, 19 September 2014 at 15:32:38 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

But how would you chain them? And they have to implement
Throwable's interface.

ponce via Digitalmars-d

2014-09-20 15:28:10 UTC

Post by Vladimir Panteleev via Digitalmars-d
On Friday, 19 September 2014 at 15:32:38 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

But how would you chain them?

Owning pointer to next exception.

Post by via Digitalmars-d
And they have to implement Throwable's interface.

One of their member would. Speculative design:

struct ValueException
{
Unique!ValueException next;
Unique!Throwable content;
}

Language would change to throw structs and catch them (without
matching).
At the end of a catch bloc, its destructor is called.
This breaks polymorphic catch so it need sugar unfortunately.

via Digitalmars-d

2014-09-20 16:28:25 UTC

On Saturday, 20 September 2014 at 09:09:38 UTC, Marc SchÃŒtz

Post by Vladimir Panteleev via Digitalmars-d
On Friday, 19 September 2014 at 15:32:38 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

But how would you chain them?

Owning pointer to next exception.

Post by via Digitalmars-d
And they have to implement Throwable's interface.

struct ValueException
{
Unique!ValueException next;
Unique!Throwable content;
}
Language would change to throw structs and catch them (without
matching).
At the end of a catch bloc, its destructor is called.
This breaks polymorphic catch so it need sugar unfortunately.

Ah, now I see, you just want a wrapper that contains a Throwable.
I thought you wanted to throw _any_ value types, like in C++.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:41:21 UTC

Post by ponce via Digitalmars-d

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

My knee-jerk reaction is we shouldn't. If considered, that would need to
be well argued. We've derived good benefits from using polymorphic
exceptions. -- Andrei

ponce via Digitalmars-d

2014-09-20 18:17:35 UTC

On Saturday, 20 September 2014 at 16:41:20 UTC, Andrei

Post by Vladimir Panteleev via Digitalmars-d
On Friday, 19 September 2014 at 15:32:38 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

My knee-jerk reaction is we shouldn't. If considered, that
would need to be well argued. We've derived good benefits from
using polymorphic exceptions. -- Andrei

What do you think of the solution briefly outlined here:
http://forum.dlang.org/post/iqenumncmczqvprwutmg at forum.dlang.org

Andrei Alexandrescu via Digitalmars-d

2014-09-20 18:48:56 UTC

Post by ponce via Digitalmars-d

Post by ponce via Digitalmars-d

Post by Andrei Alexandrescu via Digitalmars-d
Please chime in with thoughts.

Coudln't we throw value types instead? (like in C++)

My knee-jerk reaction is we shouldn't. If considered, that would need
to be well argued. We've derived good benefits from using polymorphic
exceptions. -- Andrei

http://forum.dlang.org/post/iqenumncmczqvprwutmg at forum.dlang.org

The short answer is I wouldn't advise pursuing it. -- Andrei

Jacob Carlborg via Digitalmars-d

2014-09-20 12:27:23 UTC

Assuming this would eventually be implemented for regular classes, it
would be nice if it could be made compatible with Objective-C ARC [1].

[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

--
/Jacob Carlborg

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 13:40:50 UTC

On Saturday, 20 September 2014 at 12:27:23 UTC, Jacob Carlborg

Post by Jacob Carlborg via Digitalmars-d
Assuming this would eventually be implemented for regular
classes, it would be nice if it could be made compatible with
Objective-C ARC [1].
[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

Isn't Objective-C ARC known to be terribly efficient?

Paulo Pinto via Digitalmars-d

2014-09-20 14:33:43 UTC

Post by Jacob Carlborg via Digitalmars-d
Assuming this would eventually be implemented for regular classes, it
would be nice if it could be made compatible with Objective-C ARC [1].
[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

Isn't Objective-C ARC known to be terribly efficient?

It requires compiler support, though.

- Place retain/release at each place expected by Cocoa programming
conventions, as if manually written

- Do a second pass with data-flow analysis to remove retain/release
pairs that don't escape current scope.

If you have an Apple account search for:

"Transitioning to ARC Release Notes", ARC
"Memory Management Programming Guide for Core Foundation" - Cocoa
conventions and allocators
"WWDC 2012: Adopting Automatic Reference Counting" - Overral description
"WWDC 2012: What's New in LLVM" - Some slides about ARC Optimizer

--
Paulo

via Digitalmars-d

2014-09-20 14:42:12 UTC

Post by Paulo Pinto via Digitalmars-d

Post by Jacob Carlborg via Digitalmars-d
Assuming this would eventually be implemented for regular
classes, it
would be nice if it could be made compatible with Objective-C ARC [1].
[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

Isn't Objective-C ARC known to be terribly efficient?

I'm really sorry, but that was a very confusing typo on my part.

I meant "INefficient"! :-)

Post by Paulo Pinto via Digitalmars-d
It requires compiler support, though.

:-) Yes, but I was more talking about the representation in
memory based on reading the source code that can be found in the
repository. I read it the last time ARC was discussedâŠ

But, I am not 100% sure what code Apple actually ships.

Jacob Carlborg via Digitalmars-d

2014-09-20 15:08:56 UTC

Post by Paulo Pinto via Digitalmars-d
It requires compiler support, though.

The first thing I asked in this thread was "Are you suggesting we
implement ARC?" and the answer was "Yes" [1]. So it looks like Andrei
already wants to implement ARC. My definition of ARC is that the
compiler inserts the calls to retain/release (or whatever you call them).

[1]
http://forum.dlang.org/thread/lvhiam$1pno$1 at digitalmars.com#post-lvi0ve:2429il:241:40digitalmars.com

--
/Jacob Carlborg

Paulo Pinto via Digitalmars-d

2014-09-20 16:53:01 UTC

Post by Paulo Pinto via Digitalmars-d
It requires compiler support, though.

The first thing I asked in this thread was "Are you suggesting we
implement ARC?" and the answer was "Yes" [1]. So it looks like Andrei
already wants to implement ARC. My definition of ARC is that the
compiler inserts the calls to retain/release (or whatever you call them).
[1]
http://forum.dlang.org/thread/lvhiam$1pno$1 at digitalmars.com#post-lvi0ve:2429il:241:40digitalmars.com

I would say ARC == RC. I never saw a distinction in literature between
both, before Apple used the term.

With all these discussions I have been digging into old papers, and due
to that I am starting to change my opinion.

As it seems, RC with GC for cycle collection was more common than I
thought of.

In the end, what matters is having automatic memory management, be it
via GC, RC or compiler dataflow analysis.

From an outsider point of view, I just think that whatever the final
outcome, it should be in a way that avoids the runtime episode from
repeating itself.

--
Paulo

Ola Fosheim Grostad via Digitalmars-d

2014-09-20 17:19:49 UTC

Post by Paulo Pinto via Digitalmars-d
As it seems, RC with GC for cycle collection was more common
than I thought of.

I once read a paper that suggested optimizing GC by having RC on
all objects and stop inc/dec ref counting when it reached 5 or
something. I guess the idea was to get rid of short lived objects
to make the GC run less often while cutting RC overhead.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:56:33 UTC

Post by Paulo Pinto via Digitalmars-d
It requires compiler support, though.

The first thing I asked in this thread was "Are you suggesting we
implement ARC?" and the answer was "Yes" [1]. So it looks like Andrei
already wants to implement ARC. My definition of ARC is that the
compiler inserts the calls to retain/release (or whatever you call them).
[1]
http://forum.dlang.org/thread/lvhiam$1pno$1 at digitalmars.com#post-lvi0ve:2429il:241:40digitalmars.com

Please don't take me in a court of law. But yes, I am talking about the
compiler inserting calls to increment and decrement reference counts. --
Andrei

Paulo Pinto via Digitalmars-d

2014-09-20 14:33:43 UTC

Post by Jacob Carlborg via Digitalmars-d
Assuming this would eventually be implemented for regular classes, it
would be nice if it could be made compatible with Objective-C ARC [1].
[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

Isn't Objective-C ARC known to be terribly efficient?

Walter Bright via Digitalmars-d

2014-09-20 18:19:49 UTC

Assuming this would eventually be implemented for regular classes, it would be
nice if it could be made compatible with Objective-C ARC [1].
[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

There was a long thread about that a year back or so. It wasn't looking good.

Walter Bright via Digitalmars-d

2014-09-20 18:23:01 UTC

Assuming this would eventually be implemented for regular classes, it would be
nice if it could be made compatible with Objective-C ARC [1].
[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

http://www.digitalmars.com/d/archives/digitalmars/D/draft_proposal_for_ref_counting_in_D_211885.html

Adam D. Ruppe via Digitalmars-d

2014-09-20 14:31:34 UTC

How often do you store an exception reference anyway that escapes
a catch block? I think all this talk is overkill to solve a
non-problem in 99% of practice.

Correct me if I'm wrong, but aren't *all* exceptions in a
particular thread generally unreferenced at the end of a catch()
block, unless the programmer explicitly escaped that reference?

If so, we don't need refcounting! Here's my solution:

1) Throwables are allocated in a separate thread-local memory
pool than most other objects. The @nogc version just aborts the
program if this pool ever runs out of memory. The gc version can
do a collection cycle and grow the size if necessary. (Though if
you need a lot of Throwable objects alive at once, I question wtf
is up with your code...)

2) class Throwable adds a method, gcClone, which copies it to the
regular GC heap for those cases when you do want to store it or
pass it between threads or whatever.

3) A function, @system void clearExceptions() is added. You may
manually call this when you are done handling exceptions in this
thread.

4) Might also have @safe void prepareExceptions() which
pre-allocates the pool. This could also be done automatically or
on demand or whatever.

Advantages:

* Throwables are still GC managed as far as most code is
concerned. The only time you need caution is if you call the new
@system function to free the pool... and even then, you just make
sure you don't escape those references before you do.

* If you want to avoid the GC, just manually clear your
exceptions from time to time.

* Allocating/deallocating from this specialized memory pool is
prolly faster than any other scheme anyway.

* No language changes required, this is all library stuff.

Disadvantages:

* There's a few manual steps to get all the benefit. (We could
insert a call to clearExceptions at the end of catch blocks
automatically, but without a static check to ensure the objects
actually haven't escaped, this would break memory safety. But a
manual call isn't that big of a deal)

* If you do it wrong, you'll be annoyed.

* ???

I really think this is a win. I've done a proof of concept before
by hacking _d_newclass to use the pool based on the TypeInfo
passed in, gives a speedup in all the simple cases I tried. But
since that hack still uses new, it wouldn't pass the @nogc test.
However, a library function like emplace could be @nogc and do
the same thing. Since @nogc is an addition to the function,
you'll be modifying it anyway, so changing "throw new" to
whatever the new lib function is called can be done at the same
time.

It is the catch code that have to worry about freeing the
pool.... and even then, only if you want to avoid the GC. Normal
code can just let the pool be collected when it is collected.

I think I'll write a little module we can play with.

Dicebot via Digitalmars-d

2014-09-20 14:33:20 UTC

On Saturday, 20 September 2014 at 14:31:36 UTC, Adam D. Ruppe

Post by Adam D. Ruppe via Digitalmars-d
How often do you store an exception reference anyway that
escapes a catch block? I think all this talk is overkill to
solve a non-problem in 99% of practice.

Pretty much any time you do fibers + async I/O : to emulate
blocking API one needs to catch and store exceptions from I/O
routines so that later those can be re-thrown from resumed fiber
context.

Adam D. Ruppe via Digitalmars-d

2014-09-20 15:05:37 UTC

Post by Dicebot via Digitalmars-d
Pretty much any time you do fibers + async I/O : to emulate
blocking API one needs to catch and store exceptions from I/O
routines so that later those can be re-thrown from resumed
fiber context.

Ah, indeed, and that could have a great many of them alive at
once. Blargh, it'd need a real allocator to handle freeing them
out of order.

Nevertheless though, I still think the lifetime management there
is simple enough for the user-programmer that freeing it manually
isn't a big hassle and not worth making major changes to the
language over.

Dicebot via Digitalmars-d

2014-09-20 15:12:44 UTC

Post by Adam D. Ruppe via Digitalmars-d

On Saturday, 20 September 2014 at 15:05:38 UTC, Adam D. Ruppe

Ah, indeed, and that could have a great many of them alive at
once. Blargh, it'd need a real allocator to handle freeing them
out of order.
Nevertheless though, I still think the lifetime management
there is simple enough for the user-programmer that freeing it
manually isn't a big hassle and not worth making major changes
to the language over.

I don't follow. Lifetime is totally unknown there and completely
up to actual application logic - from the point of view of the
caller of something like `readFile()` it just a normal allocated
exception that doesn't need any explicit management. Requiring
any kind of manual handling would be a huge breaking change.

Adam D. Ruppe via Digitalmars-d

2014-09-20 16:11:36 UTC

Requiring any kind of manual handling would be a huge breaking
change.

I'm not talking about *requiring* it; I want it to be a garbage
collected object that 1) allocating it never triggers a
collection cycle (so it is a @nogc allocation) and 2) you're
allowed to explicitly free it if you want to, but if you don't,
the gc will get around to it eventually.

We have #2 already: delete (though I'd like it to not be
something which is allegedly deprecated). #1 is fairly easy too:
just offer a new @nogc allocation function which creates garbage
but does not garbage collect.

This could be a new function or the new keyword could be modified
to suppress collection when allocating Throwable or whatever.

Maybe I'm missing something though.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:55:09 UTC

Nevertheless though, I still think the lifetime management there is
simple enough for the user-programmer that freeing it manually isn't a
big hassle and not worth making major changes to the language over.

The language will change in major ways, and this is the warmup before
that. -- Andrei

Adam D. Ruppe via Digitalmars-d

2014-09-20 17:50:20 UTC

On Saturday, 20 September 2014 at 16:55:08 UTC, Andrei

Post by Andrei Alexandrescu via Digitalmars-d
The language will change in major ways, and this is the warmup
before that. -- Andrei

Ugh. I'm very concerned about the direction this is going,
replacing a system that works with something that offers little
benefit and a lot of risk.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 18:44:57 UTC

Post by Andrei Alexandrescu via Digitalmars-d
The language will change in major ways, and this is the warmup before
that. -- Andrei

Ugh. I'm very concerned about the direction this is going, replacing a
system that works with something that offers little benefit and a lot of
risk.

Fasten your seatbelt, it's gonna be a bumpy ride! :o)

Andrei

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:50:08 UTC

How often do you store an exception reference anyway that escapes a
catch block? I think all this talk is overkill to solve a non-problem
in 99% of practice.

Pretty much any time you do fibers + async I/O : to emulate blocking API
one needs to catch and store exceptions from I/O routines so that later
those can be re-thrown from resumed fiber context.

Interesting. Are those stored as Object or Throwable/Exception? -- Andrei

Dicebot via Digitalmars-d

2014-09-20 14:33:20 UTC

On Saturday, 20 September 2014 at 14:31:36 UTC, Adam D. Ruppe

Post by Adam D. Ruppe via Digitalmars-d
How often do you store an exception reference anyway that
escapes a catch block? I think all this talk is overkill to
solve a non-problem in 99% of practice.

Pretty much any time you do fibers + async I/O : to emulate
blocking API one needs to catch and store exceptions from I/O
routines so that later those can be re-thrown from resumed fiber
context.

Andrei Alexandrescu via Digitalmars-d

2014-09-20 16:48:46 UTC

[snip]

Thanks. I personally favor RC based on malloc to this. It'll be safer
and simpler to use. -- Andrei

Jakob Ovrum via Digitalmars-d

2014-09-20 17:01:35 UTC

On Saturday, 20 September 2014 at 14:31:36 UTC, Adam D. Ruppe

Post by Adam D. Ruppe via Digitalmars-d
How often do you store an exception reference anyway that
escapes a catch block? I think all this talk is overkill to
solve a non-problem in 99% of practice.
Correct me if I'm wrong, but aren't *all* exceptions in a
particular thread generally unreferenced at the end of a
catch() block, unless the programmer explicitly escaped that
reference?

I proposed something very similar in this thread[1], except I
proposed a solution that is fully backwards-compatible except the
rare case when exceptions are escaped, in which case it causes a
compile-time error. Might be worth a look.

I don't think it's acceptable that code using Phobos is suddenly
leaking until they add the new manual cleanup call.

[1]
http://forum.dlang.org/post/stlslhjndgugecvmbowd at forum.dlang.org

Adam D. Ruppe via Digitalmars-d

2014-09-20 14:31:34 UTC

Oren Tirosh via Digitalmars-d

2014-09-20 19:48:07 UTC

Hi everyone. Unlurking to make my first comment here.

Here is an idea for making RC and GC coexist peacefully. I think
this technique may be used to make the Throwable transition to RC
while keeping full backward compatibility.

A Throwable object would have a reference counter to track the
number of RC references to it. It would also have a flag
indicating whether there are any outstanding GC references to the
object as well. This flag may be implemented by setting the MSB
of the reference counter.

RC references would be a different type from GC references. An RC
reference may be converted to a GC reference by setting the MSB,
indicating that there may be one or more uncounted references to
the object and that it should not be destroyed until successfully
reclaimed by the GC. This RC to GC conversion may happen when a
reference to a Throwable leaves @nogc code and enters GC land.
When a GC operation can find no more references to the Throwable
it would clear the counter MSB. If the number of RC references is
zero at this point the object would be immediately destroyed.

A crucial part of making this work is to ensure that RC
references are *not* seen by GC scanning. This may be done by
mangling the pointer value in some way. Alternatively, if
mark-and-sweep is replaced with count-and-sweep the number of
counted RC references may be deducted.

One way to mangle the pointer is to decrement the value to a
lower memory address. Incrementing the pointer won't work because
the D garbage collector must handle inner references for slices
etc. Any dereferencing of the RC reference would compensate for
this offset by incrementing the field offset. An obvious piece of
information to put in the location reserved by decrementing the
pointer would the the reference counter itself. Conversion of RC
reference to GC would require incrementing the pointer and
setting the counter MSB. Converting GC reference to RC is done by
just decrementing the pointer.

I'm sure there are many things I'm missing here, but could
something along these line be made to work?

deadalnix via Digitalmars-d

2014-09-21 00:45:48 UTC

Post by Oren Tirosh via Digitalmars-d
Hi everyone. Unlurking to make my first comment here.
Here is an idea for making RC and GC coexist peacefully. I
think this technique may be used to make the Throwable
transition to RC while keeping full backward compatibility.
A Throwable object would have a reference counter to track the
number of RC references to it. It would also have a flag
indicating whether there are any outstanding GC references to
the object as well. This flag may be implemented by setting the
MSB of the reference counter.

I was thinking about this recently (and also, Andrei's talk at
cppcon provided some interesting data).

Most reference count are small. I think it make sense for us to
propose a ref count system, that allocate on the GC heap, and
that do not free when the counter saturate.

It would help to:
- get intrusive refcount without putting too much crap into
objects (it is all about cache line).
- get a way to escape RC object to the GC heap (by saturating
the refcount).

The GC is still there, but do kick in randomly, it simply act as
a safety net.

Also given the type qualifier in D, the whole synchronization
thing is embeded in the type, so we can be safe and fast on that
one. All of this can be done as library.

deadalnix via Digitalmars-d

2014-09-21 00:36:03 UTC