Lazy eval

[Frank Benoit]

I think the lazy eval is a great feature, but in this form it has also
great drawbacks.

The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

There is no possibility to choose between

func( char[] a ) vs. func( char[] delegate() dg )

func( funcRetInt() ); vs. func( &funcRetInt );

It would be really important for me to have readable code. I want to
look at the code, and want to have an impression what will happen.

What really would help, is if the the delegate is marked in some way as
such. In the last releases of DMD the {} syntax was there. With it you
needed the return statement. Perhaps we can choose the {} syntax with an
optional return statement....


{ "abc" } => char[] delegate()
{ return "abc"; } => char[] delegate()

func( "abc" ) calls func( char[] a )
func({ "abc" }) calls func( char[] delegate() dg )
func({ return "abc"; }) calls func( char[] delegate() dg )

With that syntax one can immidiatly see "this is a delegate, if it is
called or not or more than once depends on func()" and the more typing
of {} is not too much.

Frank

[Frank Benoit]

One more argument against the lazy-eval in the current form:

If you have no chance to figure out what the code does, without looking
at docs or function signature....
you will end up in writing 3 times more lines. Making each statements
separated, to be sure to get them evaluated exactly once, without
looking all that stuff up.

[Pragma]

Post by Frank Benoit
If you have no chance to figure out what the code does, without looking
at docs or function signature....
you will end up in writing 3 times more lines. Making each statements
separated, to be sure to get them evaluated exactly once, without
looking all that stuff up.

You're right that the lazy eval, in it's fullest form, is a very
stealthy (and arguably confusing) modification to the language. I have
some similar concerns myself.

However, Walter makes use of a slightly more obvious idiom in his
discussion of this feature:

void log(char[] delegate() dg)
{
if (logging)
fwritefln(logfile, dg());
}

void foo(int i)
{
log( { return "Entering foo() with i set to " ~ toString(i); });
}

IMO, this is probably the best way to go as it is perfectly obvious what
is going on. I like to think of it's counterpart (completely implicit
conversion of expressions to delegates) as something a little more apt
for generic programming - like being able to swap out a delegate for
something else entirely.

[Walter Bright]

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But there
also isn't a clue whether the arguments are in, out, or inout. There
also is no syntactic clue what the function *does*. One must look at the
function interface and documentation to use it successfully anyway.

It's going to take some caution to use this capability in a productive way.

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );

That's right.

Post by Frank Benoit
It would be really important for me to have readable code. I want to
look at the code, and want to have an impression what will happen.

I hear you, but I'll argue that the onus is on the function designer to
have a name for the function that clues the user in to what it does. I
know my example code has functions named "foo" a lot, but such a generic
meaningless name would be unacceptable for production code.

Post by Frank Benoit
What really would help, is if the the delegate is marked in some way as
such. In the last releases of DMD the {} syntax was there. With it you
needed the return statement. Perhaps we can choose the {} syntax with an
optional return statement....
{ "abc" } => char[] delegate()
{ return "abc"; } => char[] delegate()
func( "abc" ) calls func( char[] a )
func({ "abc" }) calls func( char[] delegate() dg )
func({ return "abc"; }) calls func( char[] delegate() dg )
With that syntax one can immidiatly see "this is a delegate, if it is
called or not or more than once depends on func()" and the more typing
of {} is not too much.

I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

[Frank Benoit]

Post by Walter Bright
I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

I use the { return ...; } very much. And I like it very much. And I
fully agree that the return..; is annoying, because if often use it for
a search criteria or something like that. The syntax { singlestatement }
would be pretty cool. But leaving the braces completely out it terrible.
And {} is really not too much to type.

Imagine the terrible bugs that will occur with that. I think one of the
biggest advantages of D vs. C++ is the readability. That is so
important. Really. Please don't destroy that.

[kris]

Post by Frank Benoit

Post by Walter Bright
I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

I use the { return ...; } very much. And I like it very much. And I
fully agree that the return..; is annoying, because if often use it for
a search criteria or something like that. The syntax { singlestatement }
would be pretty cool. But leaving the braces completely out it terrible.
And {} is really not too much to type.
Imagine the terrible bugs that will occur with that. I think one of the
biggest advantages of D vs. C++ is the readability. That is so
important. Really. Please don't destroy that.

I think the new sugar works really well where the code in question is
*designed* around the notion of callbacks. The cues are already solidly
in place at that point, so it may be less of an issue there? I'm
thinking specifically of the logging examples, and some fairly advanced
distributed processing paradigms.

However, in typical or 'mixed' D programming, the lack of cues in this
latest sugar will surely become problematic. Walter suggests the author
of the callee should use an appropriate name, such that overloading
would (presumeably) not be an issue and the appropriate cues would be
retained. One has to wonder if that is a good idea or not.

If it were possible to make the '{' '}' optional, for example, then
another option would be for the coder to "make the call" as to how
readable/explicit the code actually is. This is typical the case anyway,
since any coder can write obfuscated code regardless of the language :)

The question is then "how much different is it to add or omit the
'return' keyword?" ... personally, I think it makes a big difference; so
does Walter, apparently. Thus, if we had a choice of writing these three
options:


# int i;
# somefunk (++i);
#
# or
#
# somefunk ({++i});
#
# or
#
# somefunk (return {++i;});

I think the later should be dropped, and the former two supported :)

[Tom S]

Post by kris

Post by Frank Benoit

Post by Walter Bright
I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

I use the { return ...; } very much. And I like it very much. And I
fully agree that the return..; is annoying, because if often use it for
a search criteria or something like that. The syntax { singlestatement }
would be pretty cool. But leaving the braces completely out it terrible.
And {} is really not too much to type.
Imagine the terrible bugs that will occur with that. I think one of the
biggest advantages of D vs. C++ is the readability. That is so
important. Really. Please don't destroy that.

I think the new sugar works really well where the code in question is
*designed* around the notion of callbacks. The cues are already solidly
in place at that point, so it may be less of an issue there? I'm
thinking specifically of the logging examples, and some fairly advanced
distributed processing paradigms.
However, in typical or 'mixed' D programming, the lack of cues in this
latest sugar will surely become problematic. Walter suggests the author
of the callee should use an appropriate name, such that overloading
would (presumeably) not be an issue and the appropriate cues would be
retained. One has to wonder if that is a good idea or not.
If it were possible to make the '{' '}' optional, for example, then
another option would be for the coder to "make the call" as to how
readable/explicit the code actually is. This is typical the case anyway,
since any coder can write obfuscated code regardless of the language :)
The question is then "how much different is it to add or omit the
'return' keyword?" ... personally, I think it makes a big difference; so
does Walter, apparently. Thus, if we had a choice of writing these three
# int i;
# somefunk (++i);
#
# or
#
# somefunk ({++i});
#
# or
#
# somefunk (return {++i;});
I think the later should be dropped, and the former two supported :)

You have a typo in the 3rd sample, it should read:
# somefunk ({return ++i;});

... which clearly shows that the 3rd option isn't the preferred one ;)

--
Tomasz Stachowiak

[Sean Kelly]

Post by Tom S

Post by kris

Post by Frank Benoit

Post by Walter Bright
I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

I use the { return ...; } very much. And I like it very much. And I
fully agree that the return..; is annoying, because if often use it for
a search criteria or something like that. The syntax { singlestatement }
would be pretty cool. But leaving the braces completely out it terrible.
And {} is really not too much to type.
Imagine the terrible bugs that will occur with that. I think one of the
biggest advantages of D vs. C++ is the readability. That is so
important. Really. Please don't destroy that.

I think the new sugar works really well where the code in question is
*designed* around the notion of callbacks. The cues are already
solidly in place at that point, so it may be less of an issue there?
I'm thinking specifically of the logging examples, and some fairly
advanced distributed processing paradigms.
However, in typical or 'mixed' D programming, the lack of cues in this
latest sugar will surely become problematic. Walter suggests the
author of the callee should use an appropriate name, such that
overloading would (presumeably) not be an issue and the appropriate
cues would be retained. One has to wonder if that is a good idea or not.
If it were possible to make the '{' '}' optional, for example, then
another option would be for the coder to "make the call" as to how
readable/explicit the code actually is. This is typical the case
anyway, since any coder can write obfuscated code regardless of the
language :)
The question is then "how much different is it to add or omit the
'return' keyword?" ... personally, I think it makes a big difference;
so does Walter, apparently. Thus, if we had a choice of writing these
# int i;
# somefunk (++i);
#
# or
#
# somefunk ({++i});
#
# or
#
# somefunk (return {++i;});
I think the later should be dropped, and the former two supported :)

# somefunk ({return ++i;});
... which clearly shows that the 3rd option isn't the preferred one ;)

I'm not sure I'd want to do away with option 3 unless option 2 supported
multiple statements and a void return in a way that made sense.


Sean

[kris]

Post by Sean Kelly

Post by Tom S

Post by kris

Post by Frank Benoit

Post by Walter Bright
I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

I use the { return ...; } very much. And I like it very much. And I
fully agree that the return..; is annoying, because if often use it for
a search criteria or something like that. The syntax {
singlestatement }
would be pretty cool. But leaving the braces completely out it terrible.
And {} is really not too much to type.
Imagine the terrible bugs that will occur with that. I think one of the
biggest advantages of D vs. C++ is the readability. That is so
important. Really. Please don't destroy that.

I think the new sugar works really well where the code in question is
*designed* around the notion of callbacks. The cues are already
solidly in place at that point, so it may be less of an issue there?
I'm thinking specifically of the logging examples, and some fairly
advanced distributed processing paradigms.
However, in typical or 'mixed' D programming, the lack of cues in
this latest sugar will surely become problematic. Walter suggests the
author of the callee should use an appropriate name, such that
overloading would (presumeably) not be an issue and the appropriate
cues would be retained. One has to wonder if that is a good idea or not.
If it were possible to make the '{' '}' optional, for example, then
another option would be for the coder to "make the call" as to how
readable/explicit the code actually is. This is typical the case
anyway, since any coder can write obfuscated code regardless of the
language :)
The question is then "how much different is it to add or omit the
'return' keyword?" ... personally, I think it makes a big difference;
so does Walter, apparently. Thus, if we had a choice of writing these
# int i;
# somefunk (++i);
#
# or
#
# somefunk ({++i});
#
# or
#
# somefunk (return {++i;});
I think the later should be dropped, and the former two supported :)

# somefunk ({return ++i;});
... which clearly shows that the 3rd option isn't the preferred one ;)

I'm not sure I'd want to do away with option 3 unless option 2 supported
multiple statements and a void return in a way that made sense.
Sean

True; and some have pointed out potential problems with #2.

It's worth noting that C# uses the '=>' operator for the equivalent
sugar: IIRC the equivalent example in C# would be

# somefunk (i => ++i);

so, perhaps, just perhaps, it's worth introducing an operator (*gasp*)
specifically to make this completely unambiguous? For example:

# somefunk (@++i);

where '@' (or whatever) would be a low-priority operator ?

[Frank Benoit]

Post by kris
If it were possible to make the '{' '}' optional, for example, then
another option would be for the coder to "make the call" as to how
readable/explicit the code actually is. This is typical the case anyway,
since any coder can write obfuscated code regardless of the language :)

Writing it without {} will always be obfuscating, if the reader doesn't
know what will happen from the function names. And a good function name
does perhaps not imply a delegate call to another person reading the code.

The {} can make that clear, and they leave the user the choice to call
the overloaded function without lazy eval.

[Derek Parnell]

Post by Walter Bright

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But there
also isn't a clue whether the arguments are in, out, or inout. There
also is no syntactic clue what the function *does*. One must look at the
function interface and documentation to use it successfully anyway.
It's going to take some caution to use this capability in a productive way.

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )

Would it possible to use ...

func ( cast(char[]) "abc" );

to force the compiler to chose 'func( char[] a)' instead of the delgated
version?

[kris]

Post by Derek Parnell

Post by Walter Bright

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But there
also isn't a clue whether the arguments are in, out, or inout. There
also is no syntactic clue what the function *does*. One must look at the
function interface and documentation to use it successfully anyway.
It's going to take some caution to use this capability in a productive way.

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )

Would it possible to use ...
func ( cast(char[]) "abc" );
to force the compiler to chose 'func( char[] a)' instead of the delgated
version?

arghhh!!! Please ... cast() is only for exceptional circumstances :(

[nobody]

Post by kris

Post by Derek Parnell

Post by Walter Bright

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But
there also isn't a clue whether the arguments are in, out, or inout.
There also is no syntactic clue what the function *does*. One must
look at the function interface and documentation to use it
successfully anyway.
It's going to take some caution to use this capability in a
productive way.

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )

Would it possible to use ...
func ( cast(char[]) "abc" );
to force the compiler to chose 'func( char[] a)' instead of the delgated
version?

arghhh!!! Please ... cast() is only for exceptional circumstances :(

cast( :-) )
( arghhh!!! Please ... cast() is only for exceptional circumstances :( );

[Derek Parnell]

Post by kris

Post by Derek Parnell

Post by Walter Bright

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But there
also isn't a clue whether the arguments are in, out, or inout. There
also is no syntactic clue what the function *does*. One must look at the
function interface and documentation to use it successfully anyway.
It's going to take some caution to use this capability in a productive way.

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )

Would it possible to use ...
func ( cast(char[]) "abc" );
to force the compiler to chose 'func( char[] a)' instead of the delgated
version?

arghhh!!! Please ... cast() is only for exceptional circumstances :(

And this situation is not exceptional? Okay, than how about a keyword ...

func ( forget_the_delegate_and_use_the_other_type_instead "abc" );

;-)

[Lutger]

Post by Derek Parnell

Post by Walter Bright

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But there
also isn't a clue whether the arguments are in, out, or inout. There
also is no syntactic clue what the function *does*. One must look at the
function interface and documentation to use it successfully anyway.
It's going to take some caution to use this capability in a productive way.

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )

Would it possible to use ...
func ( cast(char[]) "abc" );
to force the compiler to chose 'func( char[] a)' instead of the delgated
version?

As for ambiguity between overloads, doesn't it make more sense to change
the overloading rules so that 'func( char[] a)' takes precedence, if
this can be done?
I'd think that it would make sense as it is similar to how implicit
conversions for integral types work. The {} syntax would then 'force'
the second form to be taken, but this is no cast.
Then again, I could be very wrong.

[Oskar Linde]

Post by Walter Bright

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

It's true there is no clue from the user's side which it is. But there
also isn't a clue whether the arguments are in, out, or inout.

A feature request that pops up now and then is to at least allow using
the in/out/inout keywords at call site, just to help document the code.
That has analogs with the current discussion, but with delegates, we
already have the option of explicitly using a {return...;} delegate
where it makes sense.

Possibly, we might see coding guidelines one day, saying that one should
write:

func(/*lazy*/ i++);

where the laziness has a semantic difference. I think it is a good idea
to document your code as such, but the fraction of functions using lazy
evaluation will probably be low, and the number of cases where the
laziness of the function argument will have side-effects will be even lower.

Post by Walter Bright

Post by Frank Benoit
func( "abc" ) calls func( char[] a )
func({ "abc" }) calls func( char[] delegate() dg )
func({ return "abc"; }) calls func( char[] delegate() dg )
With that syntax one can immidiatly see "this is a delegate, if it is
called or not or more than once depends on func()" and the more typing
of {} is not too much.

I agree with you that replacing exp with { return exp; } is clear and
not much additional typing. But I've discovered over and over to my
surprise that the additional typing causes people to not realize that D
has that capability. The extra typing simply kills it.

Yes. I think the additional typing makes the feature seem more complex.
In our search for solutions, we favor the most simple ones. By reducing
the apparent complexity of a feature, it is much more likely to be used.

/Oskar

[Tom S]

While I partly agree with you, Frank, I don't want to dismiss this new
feature too soon. So let me disagree a bit ;)

Post by Frank Benoit
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );

That is a serious problem as it breaks existing code. But if these
functions were written with lazy evaluation in mind, there would be no
sense in having such two overloads. The coder would ensure that the
expression is evaluated just once.
In order to overcome the problem with code having both func(Type) and
func(Type delegate()) overloads, but not really meaning them to be used
with lazy expression evaluation, the coder would have to make it
straight that lazy evaluation is preferred in the remaining cases.

One possible method to accomplish it is by adding a new storage
specifier, 'lazy' to the bunch of 'in', 'inout' and 'out'.

The consequences of such an approach would be:


----
void foo(lazy int x) {
static assert(is(typeof(x) : int delegate()));
writefln(x());
}

void foo(int x) {
writefln(x);
}
---
Error: foo(lazy int) conflicts with foo(int)



----
void foo(lazy int x) {
static assert(is(typeof(x) : int delegate()));
writefln(x());
}

void bar(int delegate() x) {
foo(x);
}

void bar(int x) {
writefln(x);
}


foo({return 5;}); // foo(lazy int) called
foo(5 + 5); // foo(lazy int) called, expression evaluated lazily
bar(5); // bar(int) called
foo(5 + 5); // bar(int) called, expression evaluated at call-time
bar({return 5;}); // bar(int delegate()) called
----


Which really means that lazy args and delegates could be converted
between each other without explicit casts, but in order to make a
function accept lazily-evaluated expressions, the programmer would have
to declare his/her intent clearly.

Post by Frank Benoit
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?

Just as Walter stated, that's the same case as with 'in' vs 'inout' vs
'out' arguments. You don't know what will happen in either case


--
Tomasz Stachowiak

[Frank Benoit]

Post by Tom S
Just as Walter stated, that's the same case as with 'in' vs 'inout' vs
'out' arguments. You don't know what will happen in either case

If an argument is in/out/inout it is clear from the content and context.
If it is out, you call the function to get something.

lazy-eval syntax
When I code, I often guess what the method is called and what arguments
it has.
For example i have a container with a add( element ) method. Now this
method is implemented with lazy-eval, if the container is already full,
the argument is not evaluated.
Or the get( index ) method does not eval the arg if the container is
empty. And every time I have not only to remember the method name and
arguments, now I have to know details of the implementation.
Great, that was the silly thing they wanted to take away from me with
this OOP ;)

Solution: Force the user the make {} around a delegate argument.

Now I get a compiler error when writing container.get(i++); Then I know,
this is a delegate, take care with the increment, writing
container.get({i}); i++;

[Tom S]

Post by Frank Benoit

Post by Tom S
Just as Walter stated, that's the same case as with 'in' vs 'inout' vs
'out' arguments. You don't know what will happen in either case

If an argument is in/out/inout it is clear from the content and context.
If it is out, you call the function to get something.
lazy-eval syntax
When I code, I often guess what the method is called and what arguments
it has.
For example i have a container with a add( element ) method. Now this
method is implemented with lazy-eval, if the container is already full,
the argument is not evaluated.
Or the get( index ) method does not eval the arg if the container is
empty. And every time I have not only to remember the method name and
arguments, now I have to know details of the implementation.

Nah, just that the arg might be left not-evaluated... I wouldn't use it
with functions like 'get' or 'set' but with stuff for which it makes
sense, just like with 'assert'.

I'm just trying to defend a point I don't really believe... You're
probably right, the implicit approach brings more traps than advantages :(


--
Tomasz Stachowiak

[Frank Benoit]

Post by Tom S
I'm just trying to defend a point I don't really believe... You're
probably right, the implicit approach brings more traps than advantages :(

Yes, and that is my only point.
delegate arguments will become a often used feature. Well, I like to use
them since 0.163.
But without making them explicit, they will make D to hell. I see me
sitting at night, staring at the same code, and after hours of debugging
"NOOOO, its a delegate!!!"

[Walter Bright]

Post by Frank Benoit
For example i have a container with a add( element ) method. Now this
method is implemented with lazy-eval, if the container is already full,
the argument is not evaluated.
Or the get( index ) method does not eval the arg if the container is
empty. And every time I have not only to remember the method name and
arguments, now I have to know details of the implementation.
Great, that was the silly thing they wanted to take away from me with
this OOP ;)
Solution: Force the user the make {} around a delegate argument.
Now I get a compiler error when writing container.get(i++); Then I know,
this is a delegate, take care with the increment, writing
container.get({i}); i++;

While I understand your concern, I don't think the examples illustrate
it. If a get() is done for an element that doesn't exist, the usual way
to deal with it is throw an exception, and have some complex scheme to
try and undo the side effects from evaluating the argument. With lazy
evaluation, no need to undo the side effects, as only if the get() is
guaranteed to succeed will the argument be evaluated.

The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

[kris]

Post by Walter Bright

Post by Frank Benoit
For example i have a container with a add( element ) method. Now this
method is implemented with lazy-eval, if the container is already full,
the argument is not evaluated.
Or the get( index ) method does not eval the arg if the container is
empty. And every time I have not only to remember the method name and
arguments, now I have to know details of the implementation.
Great, that was the silly thing they wanted to take away from me with
this OOP ;)
Solution: Force the user the make {} around a delegate argument.
Now I get a compiler error when writing container.get(i++); Then I know,
this is a delegate, take care with the increment, writing
container.get({i}); i++;

While I understand your concern, I don't think the examples illustrate
it. If a get() is done for an element that doesn't exist, the usual way
to deal with it is throw an exception, and have some complex scheme to
try and undo the side effects from evaluating the argument. With lazy
evaluation, no need to undo the side effects, as only if the get() is
guaranteed to succeed will the argument be evaluated.
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Can you at least make it optional? Or, can you come up with an operator
or something that is low-impact? Making this unambiguous instead would
surely be a win-win?

[Frank Benoit]

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

I am a programmer. I like them. I like to understand my and foreign
code. I hate C++ for that reason. All the time i have to look in hundred
files and have thousands of informations to care about only to get a
clue of what is going on.

Implicit or explicit, this is really an important step.
A decision between readability in big projects and nice looking example
code.

http://www.digitalmars.com/d/overview.html
Who D is For
Teams who write apps with a million lines of code in it.

Look at the example with explicit delegates:

void foo()
{
int v = 2;
cond
({
scase(v == 1, {writefln("it is 1")}),
scase(v == 2, {writefln("it is 2")}),
scase(v == 3, {writefln("it is 3")}),
scase(true, {writefln("it is the default")})
});
}

Why should one not like it? It looks sexy ;D

[Derek Parnell]

Post by Walter Bright

Post by Frank Benoit
For example i have a container with a add( element ) method. Now this
method is implemented with lazy-eval, if the container is already full,
the argument is not evaluated.
Or the get( index ) method does not eval the arg if the container is
empty. And every time I have not only to remember the method name and
arguments, now I have to know details of the implementation.
Great, that was the silly thing they wanted to take away from me with
this OOP ;)
Solution: Force the user the make {} around a delegate argument.
Now I get a compiler error when writing container.get(i++); Then I know,
this is a delegate, take care with the increment, writing
container.get({i}); i++;

While I understand your concern, I don't think the examples illustrate
it. If a get() is done for an element that doesn't exist, the usual way
to deal with it is throw an exception, and have some complex scheme to
try and undo the side effects from evaluating the argument. With lazy
evaluation, no need to undo the side effects, as only if the get() is
guaranteed to succeed will the argument be evaluated.
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

[Walter Bright]

Post by Derek Parnell

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

Did you use it before 0.165? Did anyone? Everyone I'd show the { }
syntax to reacted with zzzzzz. I show them the one without, and all of a
sudden they are excited about it and can think of all kinds of uses.

C++ programmers have been trying to do this for over a decade - first
with preprocessor macros, and now with expression templates.

Perhaps what is so off-putting of this feature, as opposed to out,
inout, const, implicit conversions, and other effects that require one
to look at the interface, is it's very unusual (even unique?) for a
C-like language.

[Derek Parnell]

Post by Walter Bright

Post by Derek Parnell

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

Did you use it before 0.165?

Yes. However I couldn't release it in Build until GDC caught up to the
ability, so I took it out again.

Post by Walter Bright
Did anyone? Everyone I'd show the { }
syntax to reacted with zzzzzz.

I didn't.

Post by Walter Bright
I show them the one without, and all of a
sudden they are excited about it and can think of all kinds of uses.

And so am I, but now I *also* wary of the pitfalls that this implicit
conversion to delegates open up.

Post by Walter Bright
C++ programmers have been trying to do this for over a decade - first
with preprocessor macros, and now with expression templates.

And do I give a toss about C++ programming foibles ;-)

Post by Walter Bright
Perhaps what is so off-putting of this feature, as opposed to out,
inout, const, implicit conversions, and other effects that require one
to look at the interface, is it's very unusual (even unique?) for a
C-like language.

Not sure about that. I see the problem with existing calls to functions
suddenly behaving differently with no apparent reason. With this new
ability, a library writer can change the interface and my program will
still compile, but may now behave in unexpected ways.


Let's assume the original API said

void SomeFunc(char[] x);

so I code ...

void xyzzy(inout x)
{
char[] r = std.string.format("%d", x);
x++;
}
. . .
int y = 7;
SomeFunc( std.string.format("The number is " ~ xyzzy(y)) );


Then the library writer changes this to

void SomeFunc(char[] delegate() x);

My code still compiles but there is now no guarantee that my xyzzy()
function will be called. The API function may chose to not call it for some
reason valid to itself. But I'm left scratching my head trying to work out
why my variable 'y' is sometimes being updated and other times not.

[Walter Bright]

Post by Derek Parnell

Post by Walter Bright

Post by Derek Parnell

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

Did you use it before 0.165?

Yes. However I couldn't release it in Build until GDC caught up to the
ability, so I took it out again.

Post by Walter Bright
Did anyone? Everyone I'd show the { }
syntax to reacted with zzzzzz.

I didn't.

Ok.

Post by Derek Parnell

Post by Walter Bright
I show them the one without, and all of a
sudden they are excited about it and can think of all kinds of uses.

And so am I, but now I *also* wary of the pitfalls that this implicit
conversion to delegates open up.

It's good to be wary of unusual new features. There are usually
unanticipated problems with them.

Post by Derek Parnell

Post by Walter Bright
C++ programmers have been trying to do this for over a decade - first
with preprocessor macros, and now with expression templates.

And do I give a toss about C++ programming foibles ;-)

LOL! But expression templates, horrible hack that they are, are often
listed as the reason C++ is so powerful and therefore why should one
change? Expression templates are how C++ does "domain specific
languages" and David Abraham explains them to packed conference rooms.
So I believe there is a serious demand for them, but not the way C++
does it, as probably only 5 people on the planet are able to create a
DNS using them.

With the lazy evaluation thing, though, writing DNSs becomes simple and
straightforward, which may (just may) catapult D forward like defmac
rescued Lisp from oblivion.

Post by Derek Parnell
Not sure about that. I see the problem with existing calls to functions
suddenly behaving differently with no apparent reason. With this new
ability, a library writer can change the interface and my program will
still compile, but may now behave in unexpected ways.

He can do that anyway - I've already enumerated several ways. But
there's been an awful lot of threads here that boil down to adding more
information to the function declaration so that it's easier for the user
to see what is happening. (And it's a little hard to hide a delegate
declaration!)

I should also point out that:
void foo(int x);
and:
void foo(int delegate() x);
will have different name mangling. So if you change the library, and
don't recompile the user code, it won't link.

Post by Derek Parnell
Let's assume the original API said
void SomeFunc(char[] x);
so I code ...
void xyzzy(inout x)
{
char[] r = std.string.format("%d", x);
x++;
}
. . .
int y = 7;
SomeFunc( std.string.format("The number is " ~ xyzzy(y)) );
Then the library writer changes this to
void SomeFunc(char[] delegate() x);
My code still compiles but there is now no guarantee that my xyzzy()
function will be called. The API function may chose to not call it for some
reason valid to itself. But I'm left scratching my head trying to work out
why my variable 'y' is sometimes being updated and other times not.

Suppose there's an API function:

void anotherFunc(int x);

which I call:

int y = 3;
anotherFunc(y);
writefln(y);

then the library writer changes the API to:

void anotherFunc(inout int x);

and internally increments x. I'm left wondering why 4 is now being
printed instead of 3.

Changing an API and thereby breaking (obviously or subtly) the user code
is as old as programming ("DLL hell" is a term probably older than many
programmers!). The only answer I can think of is, if you must change the
API of a function, and you have legacy users of it, give the changed one
a new name and tag the old name with 'deprecated'.

[Derek Parnell]

On Mon, 21 Aug 2006 20:54:36 -0700, Walter Bright wrote:


By the way, are you also saying that given two functions ...

foo(T x);
and:
foo(T delegate() x);

that the delegate signature will *always* be called rather than the
non-delegate signature when calling with a 'T' argument?

Post by Derek Parnell
Let's assume the original API said

I guess my point is that this new feature increases the number of ways we
can inadvertently stuff things up. So is the benefit going to outweigh the
cost? Obviously its too early to know the answer to that yet.

[Walter Bright]

Post by Derek Parnell
By the way, are you also saying that given two functions ...
foo(T x);
foo(T delegate() x);
that the delegate signature will *always* be called rather than the
non-delegate signature when calling with a 'T' argument?

No. It'll be an ambiguity error.

Post by Derek Parnell

Post by Derek Parnell
Let's assume the original API said

I guess my point is that this new feature increases the number of ways we
can inadvertently stuff things up. So is the benefit going to outweigh the
cost? Obviously its too early to know the answer to that yet.

I agree.

[Sean Kelly]

Post by Walter Bright

Post by Derek Parnell

Post by Walter Bright
C++ programmers have been trying to do this for over a decade - first
with preprocessor macros, and now with expression templates.

And do I give a toss about C++ programming foibles ;-)

LOL! But expression templates, horrible hack that they are, are often
listed as the reason C++ is so powerful and therefore why should one
change? Expression templates are how C++ does "domain specific
languages" and David Abraham explains them to packed conference rooms.
So I believe there is a serious demand for them, but not the way C++
does it, as probably only 5 people on the planet are able to create a
DNS using them.

In my limited experience, the attendees of those conferences are what
I'd consider "typical" C++ programmers--they tend to use a fairly
limited subset of the language's features and their code probably looks
like what was popular maybe 15 years ago. Templates are a fairly new
thing to many of them, and many are even just getting their feet wet
with the STL. I would guess that Dave's presentation on expression
templates draws a crowd for a few simple reasons: people have heard of
Dave because he's written a book and is the front man for Boost in many
respects, and because expression templates have a high "gee whiz"
factor. I think that people simply interested in learning how to write
expression templates would go out and buy a copy of "C++ Templates: The
Complete Guide" by D. Vandevoorde and N. Josuttis and probably not
bother with the lecture at all.

Post by Walter Bright
With the lazy evaluation thing, though, writing DNSs becomes simple and
straightforward, which may (just may) catapult D forward like defmac
rescued Lisp from oblivion.

It was simple and straightforward using the previous syntax as well.
Though I'll grant that the "return" bit took up a bit of horizontal
screen real estate that some might not find appealing.

Post by Walter Bright
void anotherFunc(int x);
int y = 3;
anotherFunc(y);
writefln(y);
void anotherFunc(inout int x);
and internally increments x. I'm left wondering why 4 is now being
printed instead of 3.

But that would be a blatant change in behavior for the API function,
while converting to a delegate parameter is arguably not a change in
behavior, just in the way the value is being transferred. It's probably
also worth noting that in a language supporting const features,
switching an in param to inout would probably result in compile errors
in at least a few uses of the function.

That aside, it may be worth noting that this new delegate syntax has
some small potential for faking const behavior in D, though it would
likely make a mess of code attempting this.


Sean

[BCS]

Post by Walter Bright
Changing an API and thereby breaking (obviously or subtly) the user code
is as old as programming ("DLL hell" is a term probably older than many
programmers!). The only answer I can think of is, if you must change the
API of a function, and you have legacy users of it, give the changed one
a new name and tag the old name with 'deprecated'.

The problem isn't that changing things, breaks stuff. It it that going
from (T) to (T delegate()) is almost always valid, but the programmer
can *never* assume that it is.

Thinking of it that way, changing the API from one to the other is
*never* safe. This means that using a T delegate() as a parameter might
as well be considered an unsafe practice. If you started with just T,
your locked in for all time. Starting with T delegate() is almost as bad.

Having a feature that is nearly impossible to use safely may well be
worse than not having the feature.

[Walter Bright]

Post by BCS
Thinking of it that way, changing the API from one to the other is
*never* safe. This means that using a T delegate() as a parameter might
as well be considered an unsafe practice. If you started with just T,
your locked in for all time. Starting with T delegate() is almost as bad.
Having a feature that is nearly impossible to use safely may well be
worse than not having the feature.

Changing the API is always problematic, including with traditional
features like implicit conversions, inout, etc. As I posted before, the
pragmatic way to change an API is to create a new name with the new
interface, and deprecate the old one. This is not a problem that is new
or unique to delegate conversions. It's as old as programming.

As for delegate conversions being nearly impossible to use safely, I
don't understand that at all. I would agree that it is a probably unique
feature for C like languages, and hence people will perhaps make
mistakes with it until it becomes familiar, expected, and passe.

[John Reimer]

On Mon, 21 Aug 2006 19:41:24 -0700, Walter Bright

Post by Walter Bright

Post by Derek Parnell

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

Did you use it before 0.165? Did anyone? Everyone I'd show the { }
syntax to reacted with zzzzzz. I show them the one without, and all of a
sudden they are excited about it and can think of all kinds of uses.

Zzzz? I really don't think that was the response. I believe most people
here were quite excited about the { } form when it came out (especially
after experimenting with it for a time). They just may not have expressed
it in your hearing (ie. in this newsgroup where you are so accustomed to
feedback). Drop in the #D IRC channel on freenode sometime. You might
see more discussion about the language then you imagine. I've run across
many people that were overjoyed with {} feature, including long time D
users here.

Silence doesn't mean that there's no support for a feature. In the
context of D, silence really is a good sign. As you know, most people
here are quite outspoken about any little feature added that might not be
consistant with the D language. :)

-JJR

[kris]

Post by Walter Bright

Post by Derek Parnell

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

Did you use it before 0.165? Did anyone? Everyone I'd show the { }
syntax to reacted with zzzzzz. I show them the one without, and all of a
sudden they are excited about it and can think of all kinds of uses.

I did, and continue to do so, along with a number of others I know
working on delegate-oriented systems.

What was unweidly about the prior syntax was simply the return and the
'superfluous' semicolons. The braces don't get in the way at all, and in
fact, make it quite clear exactly what is going on.

As it stands now, blatant ambiguities have been introduced. And for what?

Post by Walter Bright
C++ programmers have been trying to do this for over a decade - first
with preprocessor macros, and now with expression templates.

Eh. Callbacks have been around for decades. Syntax to support that is
good or bad depending upon language you choose.

Post by Walter Bright
Perhaps what is so off-putting of this feature, as opposed to out,
inout, const, implicit conversions, and other effects that require one
to look at the interface, is it's very unusual (even unique?) for a
C-like language.

Perhaps, but that doesn't mean the syntax needs to to be ambiguous. Does
it? We all know that perfectly well.

[Walter Bright]

Post by kris

Post by Walter Bright
C++ programmers have been trying to do this for over a decade - first
with preprocessor macros, and now with expression templates.

Eh. Callbacks have been around for decades. Syntax to support that is
good or bad depending upon language you choose.

I wasn't thinking of callbacks. Take a look at the common practice:

#define log_printf if (logging) printf e;

and the endless variations on it, all trying to do lazy evaluation of e.
Expression templates are just the latest method.

[Chris Miller]

On Mon, 21 Aug 2006 22:41:24 -0400, Walter Bright

Post by Walter Bright

Post by Derek Parnell

Post by Walter Bright
The problem with requiring the { } around the argument is that
programmers just don't like it. I don't think I can make them like it.

Huh? You asked them all? You didn't ask me and I like it.

Did you use it before 0.165? Did anyone? Everyone I'd show the { }
syntax to reacted with zzzzzz. I show them the one without, and all of a
sudden they are excited about it and can think of all kinds of uses.

I very much like the { } syntax, it was just surprising to me.

The first piece of code I attempted to compile after installing DMD v0.165
choked on this new no-brace feature.

tabs.d(120): function dfl.application.Application.run called with argument
types:
(MainForm)
matches both:
dfl.application.Application.run(void delegate())
and:
dfl.application.Application.run(Form)

MainForm is derived from Form; it doesn't exactly match with Form so
implicit matching rules kick in. Somehow MainForm turned into a delegate
returning void, which even seems like 2 levels of implicit matching, when
Form is obviously the best match.

The previously noted foo(bar++) doesn't say how many times bar is
incremented, if at all. Perhaps there could be a compromise in syntax:
require { } but if there's just one expression in it, it can become the
return value and type: foo({bar++}); I'm not sure if a semicolon after
the expression would be needed or cause problems.


Back on function parameter matching rules, not having more than 2 levels
of matching rules (exact / implicit across the board) sucks in many cases.
It very much hinders the programmer when I don't think it even needs the
complexity of C++'s matching rules to be improved. As long as there is a
well-defined set of rules, it should be fine to have a few levels of
matching rules.

Examples:
* A derived class should match moreso with its base class.
* When one argument doesn't match exactly, the rest of the arguments
shouldn't automatically kick into implicit mode too.

I'm sure there are more; I've had to have a bunch of workarounds along the
way and forgot about them. If interested, I'll try harder to find them.

[Walter Bright]

Post by Chris Miller
I very much like the { } syntax, it was just surprising to me.
The first piece of code I attempted to compile after installing DMD
v0.165 choked on this new no-brace feature.
tabs.d(120): function dfl.application.Application.run called with
(MainForm)
dfl.application.Application.run(void delegate())
dfl.application.Application.run(Form)
MainForm is derived from Form; it doesn't exactly match with Form so
implicit matching rules kick in. Somehow MainForm turned into a delegate
returning void, which even seems like 2 levels of implicit matching,
when Form is obviously the best match.

A cast of a value to:
void delegate()
is a match with conversions, as is a conversion of a derived class to a
base class. Hence the ambiguity error.

Post by Chris Miller
The previously noted foo(bar++) doesn't say how many times bar is
incremented, if at all.

It's once if foo's parameter is not a delegate. If it is a delegate, it
is defined by the person who implemented foo().

Post by Chris Miller
require { } but if there's just one expression in it, it can become the
return value and type: foo({bar++}); I'm not sure if a semicolon after
the expression would be needed or cause problems.
Back on function parameter matching rules, not having more than 2 levels
of matching rules (exact / implicit across the board) sucks in many
cases. It very much hinders the programmer when I don't think it even
needs the complexity of C++'s matching rules to be improved. As long as
there is a well-defined set of rules, it should be fine to have a few
levels of matching rules.
* A derived class should match moreso with its base class.
* When one argument doesn't match exactly, the rest of the arguments
shouldn't automatically kick into implicit mode too.
I'm sure there are more; I've had to have a bunch of workarounds along
the way and forgot about them. If interested, I'll try harder to find them.

I admit there's a strong temptation to add more levels. But I don't
think it's so clear which matches are obviously "better", and the more
levels there are the more interactions there are, especially when
multiple arguments are in play. With the current 2 level system, it can
be frustrating, but it may in the end be beneficial by forcing one to be
clear about which function will get called.

[Sai]

Other people already proposed the { expr } syntax, I liked it too, can't
we think of it like this ...


We usually write compound statement as:

{
statement1;
statement2;
return expr;
}

We can ignore ';' if there is only statement, so following is valid:

{
statement
}

{
return expr
}

If there is only a return statement, and the return type of the delegate
matches the expression, we can make the 'return' keyword optional:

{
expr
}


well ... what do you all say ?
(hope Walter likes this)
Sai

[kris]

Post by Sai
Other people already proposed the { expr } syntax, I liked it too, can't
we think of it like this ...
{
statement1;
statement2;
return expr;
}
{
statement
}
{
return expr
}
If there is only a return statement, and the return type of the delegate
{
expr
}
well ... what do you all say ?
(hope Walter likes this)
Sai

Sure is better than deliberately introducing ambiguity into the language.

[kris]

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );
It would be really important for me to have readable code. I want to
look at the code, and want to have an impression what will happen.
What really would help, is if the the delegate is marked in some way as
such. In the last releases of DMD the {} syntax was there. With it you
needed the return statement. Perhaps we can choose the {} syntax with an
optional return statement....
{ "abc" } => char[] delegate()
{ return "abc"; } => char[] delegate()
func( "abc" ) calls func( char[] a )
func({ "abc" }) calls func( char[] delegate() dg )
func({ return "abc"; }) calls func( char[] delegate() dg )
With that syntax one can immidiatly see "this is a delegate, if it is
called or not or more than once depends on func()" and the more typing
of {} is not too much.

So, we have some existing code for logging. Very efficient, and highly
flexible. It follows the design patterns popularized by Log4J, with
Appenders, Layouts, etc

The D version of a logger has five 'levels' of logging:

log.trace (char[])
log.info (char[])
log.warn (char[])
log.error (char[])
log.fatal (char[])

and the intent (with dmd.164) was to add these flavours, specifically to
address the point Walter makes about unnecessary message construction:

log.trace (char[] delegate() dg)
log.info (char[] delegate() dg)
log.warn (char[] delegate() dg)
log.error (char[] delegate() dg)
log.fatal (char[] delegate() dg)

The support code was installed fairly recently, and the above
delegate-wrappers were on the todo list.

However, dmd.165 will presumeably not compile this code? Won't there be
an overload ambiguity? If so, it illustrates a serious shortcoming in
the new syntax

[Walter Bright]

Post by kris
So, we have some existing code for logging. Very efficient, and highly
flexible. It follows the design patterns popularized by Log4J, with
Appenders, Layouts, etc
log.trace (char[])
log.info (char[])
log.warn (char[])
log.error (char[])
log.fatal (char[])
and the intent (with dmd.164) was to add these flavours, specifically to
log.trace (char[] delegate() dg)
log.info (char[] delegate() dg)
log.warn (char[] delegate() dg)
log.error (char[] delegate() dg)
log.fatal (char[] delegate() dg)
The support code was installed fairly recently, and the above
delegate-wrappers were on the todo list.
However, dmd.165 will presumeably not compile this code?

Correct.

Post by kris
Won't there be an overload ambiguity?

Yes.

Post by kris
If so, it illustrates a serious shortcoming in the new syntax

Just get rid of the (char[]) versions. You could argue "what about the
efficiency?"

1) Passing a delegate is exactly the same number of instructions as
passing a char[], i.e., it is two values being passed.

2) Actually calling the dg() will, of course, cost more instructions
than just referencing a []. This is mitigated by, presumably, logging
being normally off, and being overshadowed by the rest of the actual
logging cost.

3) It is possible that the delegate can be inlined, thus eliminating any
extra overhead.

[kris]

Post by Walter Bright

Post by kris
So, we have some existing code for logging. Very efficient, and highly
flexible. It follows the design patterns popularized by Log4J, with
Appenders, Layouts, etc
log.trace (char[])
log.info (char[])
log.warn (char[])
log.error (char[])
log.fatal (char[])
and the intent (with dmd.164) was to add these flavours, specifically
log.trace (char[] delegate() dg)
log.info (char[] delegate() dg)
log.warn (char[] delegate() dg)
log.error (char[] delegate() dg)
log.fatal (char[] delegate() dg)
The support code was installed fairly recently, and the above
delegate-wrappers were on the todo list.
However, dmd.165 will presumeably not compile this code?

Correct.

Post by kris
Won't there be an overload ambiguity?

Yes.

Post by kris
If so, it illustrates a serious shortcoming in the new syntax

Just get rid of the (char[]) versions. You could argue "what about the
efficiency?"
1) Passing a delegate is exactly the same number of instructions as
passing a char[], i.e., it is two values being passed.
2) Actually calling the dg() will, of course, cost more instructions
than just referencing a []. This is mitigated by, presumably, logging
being normally off, and being overshadowed by the rest of the actual
logging cost.
3) It is possible that the delegate can be inlined, thus eliminating any
extra overhead.

lol! I can just imagine 1001 log statements inlined within the log code ;D

There's a fairly significant overhead in making a callback. Just as
much, or more, than handling a significant proportion of log-messages
themselves (I find a significant number to be static text; not
contructed dynamically). In addition, if the compiler can't prove these
logging delegate do not escape, each hosting function may well have
their frames allocated on the heap ... we've been discussing that just
recently.

All this because you insist programmers can't be bothered to add a tiny
bit of syntax (even optionally) to clarify the intent? This is madness!

Utter madness. Please restore some sanity here. Why don't you conduct a
poll asking exactly which 'programmers' won't use the {} delimeters to
unambiguously declare a delegate?

[Walter Bright]

Post by kris

Post by Walter Bright
Just get rid of the (char[]) versions. You could argue "what about the
efficiency?"
1) Passing a delegate is exactly the same number of instructions as
passing a char[], i.e., it is two values being passed.
2) Actually calling the dg() will, of course, cost more instructions
than just referencing a []. This is mitigated by, presumably, logging
being normally off, and being overshadowed by the rest of the actual
logging cost.
3) It is possible that the delegate can be inlined, thus eliminating
any extra overhead.

lol! I can just imagine 1001 log statements inlined within the log code ;D
There's a fairly significant overhead in making a callback. Just as
much, or more, than handling a significant proportion of log-messages
themselves (I find a significant number to be static text; not
contructed dynamically). In addition, if the compiler can't prove these
logging delegate do not escape, each hosting function may well have
their frames allocated on the heap ... we've been discussing that just
recently.

I've been thinking a lot about the escape problem. I'm pretty sure that:
char[] delegate() { return "foo"; }
can be detected and so I can assure you it won't cause the enclosing
function's variables to be allocated on the heap.

Post by kris
All this because you insist programmers can't be bothered to add a tiny
bit of syntax (even optionally) to clarify the intent? This is madness!
Utter madness. Please restore some sanity here.

Calling it 'madness' is blowing things way out of proportion. (And you
still can use the { } syntax like before.)

Post by kris
Why don't you conduct a
poll asking exactly which 'programmers' won't use the {} delimeters to
unambiguously declare a delegate?

I've struggled to get people to accept the {} version ever since D
adopted anonymous delegates. Haven't made much headway in getting such
used or in it having any sort of significant impact. How many have made
a "dotimes(n, exp)" function or any sort of syntax extension using it?
None that I've seen.

It's sort of like the itunes. Apple didn't invent anything new. They
just removed a couple annoying button pushes, and voila, suddenly the
handheld music player gained traction.

I hesitate to use argumentum ad verecundiam because it's a logical
fallacy, so you can take the following for what it's worth. Andrei and I
certainly have our differences of opinion. But when I disagree with him,
I'd better have done my homework, or I'll get cut to pieces. He thinks
(and he obviously convinced me) that removing the { } makes all the
difference.

I want to give it a fair shot. It has potential to be a big win for D,
and the consequences of it being a mistake are small. How many here have
experience with defmac in Common Lisp, or something equivalent? I sure
don't, but people who use CL successfully say it's huge.

[kris]

Post by Walter Bright

Post by kris

Post by Walter Bright
Just get rid of the (char[]) versions. You could argue "what about
the efficiency?"
1) Passing a delegate is exactly the same number of instructions as
passing a char[], i.e., it is two values being passed.
2) Actually calling the dg() will, of course, cost more instructions
than just referencing a []. This is mitigated by, presumably, logging
being normally off, and being overshadowed by the rest of the actual
logging cost.
3) It is possible that the delegate can be inlined, thus eliminating
any extra overhead.

lol! I can just imagine 1001 log statements inlined within the log code ;D
There's a fairly significant overhead in making a callback. Just as
much, or more, than handling a significant proportion of log-messages
themselves (I find a significant number to be static text; not
contructed dynamically). In addition, if the compiler can't prove
these logging delegate do not escape, each hosting function may well
have their frames allocated on the heap ... we've been discussing that
just recently.

char[] delegate() { return "foo"; }
can be detected and so I can assure you it won't cause the enclosing
function's variables to be allocated on the heap.

Post by kris
All this because you insist programmers can't be bothered to add a
tiny bit of syntax (even optionally) to clarify the intent? This is
madness!
Utter madness. Please restore some sanity here.

Calling it 'madness' is blowing things way out of proportion. (And you
still can use the { } syntax like before.)

Asserting "programmers won't type the {}" as the reason for deliberate
introduction of ambiguous syntax is throwing caution to wind; in the
extreme. Any for what? There's not even any evidence to uphold that
assertion. You didn't even bother with a poll of the people who actually
use delegates ...

To me, that's madness. Marketing is clearly in the driving seat, and you
apparently won't even consider other options.

Post by Walter Bright

Post by kris
Why don't you conduct a poll asking exactly which 'programmers' won't
use the {} delimeters to unambiguously declare a delegate?

I've struggled to get people to accept the {} version ever since D
adopted anonymous delegates. Haven't made much headway in getting such
used or in it having any sort of significant impact. How many have made
a "dotimes(n, exp)" function or any sort of syntax extension using it?
None that I've seen.

How do you know this? Where exactly does your information come from, and
how is it measured?

Post by Walter Bright
It's sort of like the itunes. Apple didn't invent anything new. They
just removed a couple annoying button pushes, and voila, suddenly the
handheld music player gained traction.
I hesitate to use argumentum ad verecundiam because it's a logical
fallacy, so you can take the following for what it's worth. Andrei and I
certainly have our differences of opinion. But when I disagree with him,
I'd better have done my homework, or I'll get cut to pieces. He thinks
(and he obviously convinced me) that removing the { } makes all the
difference.

Perhaps Andrei doesn't write much D, or perhaps he's just wrong? Either
way, you've deliberately introduced a serious syntactic ambiguity.
That's going to cause notable detractment, and for good reason.

Post by Walter Bright
I want to give it a fair shot. It has potential to be a big win for D,
and the consequences of it being a mistake are small. How many here have
experience with defmac in Common Lisp, or something equivalent? I sure
don't, but people who use CL successfully say it's huge.

It is, but you don't lose anything in usability by typing the curly
parens (or some other operator/indicator). Plus the ambiguity would not
exist.

Some say Paris Hilton is "famous for being famous" -- perhaps one of the
most empty accomplishments one could possibly achieve. What we
apparently have here is ambiguity for the sake of ambiguity. Notice the
similarity?

This is *not* breaking new ground, Walter -- it's only going to break
code. You can resolve that simply and easily with a minor adjustment.

[kris]

Post by Walter Bright
char[] delegate() { return "foo"; }
can be detected and so I can assure you it won't cause the enclosing
function's variables to be allocated on the heap.

And what about those cases where the expression is somewhat more
complex? Will you revert all this ambiguity when a bogus heap-frame is
demonstrated? One where the intent was never to create a delegate, but
simply to evaluate an argument expr instead? Or will you insist that a
smarter detector is the solution?

One has to suspect that there are far more productive ways to take risks
with a language than this one:

# somefunk (++i);
#
# vs
#
# somefunk ({++i});

the latter is not only unambiguous, it even /looks/ like a delegate. The
former? Who knows what it does anymore

[Walter Bright]

Post by kris

Post by Walter Bright
char[] delegate() { return "foo"; }
can be detected and so I can assure you it won't cause the enclosing
function's variables to be allocated on the heap.

And what about those cases where the expression is somewhat more
complex? Will you revert all this ambiguity when a bogus heap-frame is
demonstrated? One where the intent was never to create a delegate, but
simply to evaluate an argument expr instead? Or will you insist that a
smarter detector is the solution?

It's straightforward at one level - if a delegate does not reference any
variables on the enclosing functions stack frame, there's no reason to
allocate it on the heap. Something very similar is done now: variables
referenced by nested functions are marked so they are not allocated into
registers.

Post by kris
One has to suspect that there are far more productive ways to take risks
# somefunk (++i);
#
# vs
#
# somefunk ({++i});
the latter is not only unambiguous, it even /looks/ like a delegate. The
former? Who knows what it does anymore

Without knowledge of what somefunc() does by looking at the declaration
for it and the documentation, there's no way to know what it does
anyway. If one does look at the function declaration, it's pretty
obvious if it has a delegate parameter.

I expect function writers to use this in a way that makes sense, not in
a way to pull the rug out from under client code. At some level, you
have to trust them, with or without this feature. For example, doesn't
it make sense that dotimes(int n, void delegate() exp) evaluates exp n
times?

[Sean Kelly]

Post by Walter Bright
I've struggled to get people to accept the {} version ever since D
adopted anonymous delegates. Haven't made much headway in getting such
used or in it having any sort of significant impact. How many have made
a "dotimes(n, exp)" function or any sort of syntax extension using it?
None that I've seen.

I've been working on a predicate-oriented algorithm module on and off
for the past few weeks. A few looping constructs have been left out
because foreach seems a preferable solution in D, but the rest is coming
along nicely. About the only problem I had was inlining delegates
passed to the functions, so the default operations are passed as structs
with opCall defined instead.

Post by Walter Bright
I hesitate to use argumentum ad verecundiam because it's a logical
fallacy, so you can take the following for what it's worth. Andrei and I
certainly have our differences of opinion. But when I disagree with him,
I'd better have done my homework, or I'll get cut to pieces. He thinks
(and he obviously convinced me) that removing the { } makes all the
difference.

I'll admit that doing so makes a difference to me, as it allows for
tricks that aren't possible with explicit delegate signifiers. But I'm
undecided as to whether this is a "good thing" or not--I plan to give it
some time before I decide. I do think it's unfortunate that Kris'
logging code will be unable to accept both string and delegate
parameters however. And I'd like to note that all the tricks I'm
thinking of for the new syntax tend to involve refactoring existing code
(which is potentially dangerous, given the issue with side-effects)
rather than writing new code. With new code I don't see much benefit to
omitting all evidence that a parameter will be converted to a delegate,
as it's something the user must be aware of to write correct code.


Sean

[Walter Bright]

Post by Sean Kelly
I'll admit that doing so makes a difference to me, as it allows for
tricks that aren't possible with explicit delegate signifiers. But I'm
undecided as to whether this is a "good thing" or not--I plan to give it
some time before I decide.

Good - I don't think any of us are prescient enough to see all the
ramifications of this. Nobody predicted where C++ templates wound up
(the good and the bad).

Post by Sean Kelly
I do think it's unfortunate that Kris'
logging code will be unable to accept both string and delegate
parameters however.

True, but I don't think it's a serious deficiency. At worst, Kris can
define two sets of them with different names, one set with values and
the other with delegate parameters.

Post by Sean Kelly
And I'd like to note that all the tricks I'm
thinking of for the new syntax tend to involve refactoring existing code
(which is potentially dangerous, given the issue with side-effects)
rather than writing new code. With new code I don't see much benefit to
omitting all evidence that a parameter will be converted to a delegate,
as it's something the user must be aware of to write correct code.

I can't stress this enough - one must *already* be aware of the
declaration to write correct code. Implicit conversions can cause
trouble, as well as in, out, inout, const, and all the other storage
classes discussed.

Previous discussions here frequently revolved around the need to put
information in the function declaration so people have some guarantees
about what the function does - const is a prime example. Another is the
suggestion to move the contracts from the implementation to the declaration.

There is no way to know what:

somefunc(++i);

does without looking at the declaration for somefunc() - even without
lazy evaluation. Heck, you can't even know what ++i does without looking
at the type of i. It might be a class instance with a bizarre operator
overload.

[Sean Kelly]

Post by Walter Bright

Post by Sean Kelly
I'll admit that doing so makes a difference to me, as it allows for
tricks that aren't possible with explicit delegate signifiers. But
I'm undecided as to whether this is a "good thing" or not--I plan to
give it some time before I decide.

Good - I don't think any of us are prescient enough to see all the
ramifications of this. Nobody predicted where C++ templates wound up
(the good and the bad).

C++ templates are well-intentioned and really quite powerful, but
they're a horrible hack, even when used as originally intended. The
need for "template" as a means to distinguish dependent type names is a
perfect example of something that should be completely unnecessary.

Post by Walter Bright

Post by Sean Kelly
And I'd like to note that all the tricks I'm thinking of for the new
syntax tend to involve refactoring existing code (which is potentially
dangerous, given the issue with side-effects) rather than writing new
code. With new code I don't see much benefit to omitting all evidence
that a parameter will be converted to a delegate, as it's something
the user must be aware of to write correct code.

In light of my opening paragraph, I really shoudn't have said "all."

Post by Walter Bright
I can't stress this enough - one must *already* be aware of the
declaration to write correct code. Implicit conversions can cause
trouble, as well as in, out, inout, const, and all the other storage
classes discussed.

True enough. But I consider this a somewhat different class of
unpredictable behavior, if only because it isn't something present in
this way in any other C-like language (C# 3.0 will have lambda functions
similar to this with a leading => at the call point).

Post by Walter Bright
Previous discussions here frequently revolved around the need to put
information in the function declaration so people have some guarantees
about what the function does - const is a prime example. Another is the
suggestion to move the contracts from the implementation to the declaration.
somefunc(++i);
does without looking at the declaration for somefunc() - even without
lazy evaluation.

But the possibilities are rather limited, either i will be mutated or it
will not. I suppose what worries me isn't that I need to look at the
declaration to determine what will happen with implicit delegate
conversion, but that even looking at the declaration won't tell me what
may happen.

By the same token, I actually like C's method of pass-by-reference
because call-side code inspection reveals which parameters may be
modified. I think this is why I like the suggestion to retain curly
braces and perhaps derive the return value automatically--it makes the
contract between caller and callee explicit.

But this is a new feature and I do want to spend more time with it
before making a final decision. If nothing else, I agree that time and
experience will reduce the chance of mistakes demonstrated recently, as
this becomes "just another tool" we have available.


Sean

[kris]

Post by Sean Kelly

Post by Walter Bright

Post by Sean Kelly
I'll admit that doing so makes a difference to me, as it allows for
tricks that aren't possible with explicit delegate signifiers. But
I'm undecided as to whether this is a "good thing" or not--I plan to
give it some time before I decide.

Good - I don't think any of us are prescient enough to see all the
ramifications of this. Nobody predicted where C++ templates wound up
(the good and the bad).

C++ templates are well-intentioned and really quite powerful, but
they're a horrible hack, even when used as originally intended. The
need for "template" as a means to distinguish dependent type names is a
perfect example of something that should be completely unnecessary.

Post by Walter Bright

Post by Sean Kelly
And I'd like to note that all the tricks I'm thinking of for the new
syntax tend to involve refactoring existing code (which is
potentially dangerous, given the issue with side-effects) rather than
writing new code. With new code I don't see much benefit to omitting
all evidence that a parameter will be converted to a delegate, as
it's something the user must be aware of to write correct code.

In light of my opening paragraph, I really shoudn't have said "all."

Post by Walter Bright
I can't stress this enough - one must *already* be aware of the
declaration to write correct code. Implicit conversions can cause
trouble, as well as in, out, inout, const, and all the other storage
classes discussed.

True enough. But I consider this a somewhat different class of
unpredictable behavior, if only because it isn't something present in
this way in any other C-like language (C# 3.0 will have lambda functions
similar to this with a leading => at the call point).

Post by Walter Bright
Previous discussions here frequently revolved around the need to put
information in the function declaration so people have some guarantees
about what the function does - const is a prime example. Another is
the suggestion to move the contracts from the implementation to the
declaration.
somefunc(++i);
does without looking at the declaration for somefunc() - even without
lazy evaluation.

But the possibilities are rather limited, either i will be mutated or it
will not. I suppose what worries me isn't that I need to look at the
declaration to determine what will happen with implicit delegate
conversion, but that even looking at the declaration won't tell me what
may happen.
By the same token, I actually like C's method of pass-by-reference
because call-side code inspection reveals which parameters may be
modified. I think this is why I like the suggestion to retain curly
braces and perhaps derive the return value automatically--it makes the
contract between caller and callee explicit.
But this is a new feature and I do want to spend more time with it
before making a final decision. If nothing else, I agree that time and
experience will reduce the chance of mistakes demonstrated recently, as
this becomes "just another tool" we have available.
Sean

Tom's suggestion is to introduce a different style of delegate for this
lambda expression thingy. That would resolve the difficulties here
(assuming delegates can be auto-converted to the lambda expr; per Tom's
suggestion).

But for delegate itself, it sure would be nice to have:

# somefunk ({++i});

as a shorthand for

# somefunk ({return ++i;});

[Oskar Linde]

Post by Sean Kelly

Post by Walter Bright
I've struggled to get people to accept the {} version ever since D
adopted anonymous delegates. Haven't made much headway in getting such
used or in it having any sort of significant impact. How many have
made a "dotimes(n, exp)" function or any sort of syntax extension
using it? None that I've seen.

I've been working on a predicate-oriented algorithm module on and off
for the past few weeks. A few looping constructs have been left out
because foreach seems a preferable solution in D, but the rest is coming
along nicely.

Interesting. I have been doing some of this myself too, but my available
time has been limited. I have also been looking at the concept of
iterators. I agree that foreach is the preferable solution, but in some
cases, only supporting foreach is too limited. For example, only using
foreach, you can not iterate over two collections simultaneously.

It would be great if we could come up with a standardized Iterator
concept for D that supported both foreach style iteration (opApply) and
some kind of single step iteration (maybe Java-style next/hasNext).

Here is a sample of how it could look (but this particular sample seems
to give a segfault on DMD 0.165 for some reason):

template opApplyMixin() {
int opApply(int delegate(inout ValueType v) dg) {
ValueType *t;
while(hasNext()) {
t = next();
if (auto status = dg(*t))
return status;
}
return 0;
}
}

/// Simple array iterator wrapper
struct ArrayIterator(T) {
T[] arr;
alias T ValueType;

T* next() { T* r = arr.ptr; arr = arr[1..$]; return r; }
bool hasNext() { return arr.length > 0; }
mixin opApplyMixin;
}


I have been looking at implementing many of my functional algorithms as
array views/iterators. For example:

foreach(word; read!(char)("file.txt").splitIterate(&isSpace)) {
writefln("%s",word);
}

where splitIterate is implemented just as the regular split, but instead
of allocating and returning char[][] array, it is implemented as an
iterator returning the next char[] slice for each iteration without
allocating any extra data.

read could probably be implemented in a smart way, reading the file a
certain chunk at a time. Never using more than a constant amount of memory.

Other functions as iterators I have been implementing are:

split(array,predicate|substring|element)
select(array,predicate|element)
reverse(array)
range(start,stop,next)
generate(init,next)
map(array,func)
indexmap(array,indexarray|nextindexfunc)
and so on...

Post by Sean Kelly
About the only problem I had was inlining delegates
passed to the functions, so the default operations are passed as structs
with opCall defined instead.

I've encountered this too. Delegates passed to functions are often
called in tight inner loops. So far, I've never seen DMD doing a good
job at inlining those cases. Structs with opCall is an excellent idea,
that I will steal right away. :) It would be interesting to know if we
in the future can count on const delegates being inlined just as
efficiently.

/Oskar

[Sean Kelly]

Post by Oskar Linde

Post by Sean Kelly

Post by Walter Bright
I've struggled to get people to accept the {} version ever since D
adopted anonymous delegates. Haven't made much headway in getting
such used or in it having any sort of significant impact. How many
have made a "dotimes(n, exp)" function or any sort of syntax
extension using it? None that I've seen.

I've been working on a predicate-oriented algorithm module on and off
for the past few weeks. A few looping constructs have been left out
because foreach seems a preferable solution in D, but the rest is
coming along nicely.

Interesting. I have been doing some of this myself too, but my available
time has been limited. I have also been looking at the concept of
iterators. I agree that foreach is the preferable solution, but in some
cases, only supporting foreach is too limited. For example, only using
foreach, you can not iterate over two collections simultaneously.
It would be great if we could come up with a standardized Iterator
concept for D that supported both foreach style iteration (opApply) and
some kind of single step iteration (maybe Java-style next/hasNext).

I agree. And given D's somewhat limited overloading mechanism, there
may simply be no point in trying to support pointers as iterators (ie.
to use traits templates). This suggests that some sort of
self-contained design may be best, but I haven't thought about the issue
enough to comment on the details.

Post by Oskar Linde
Here is a sample of how it could look (but this particular sample seems
template opApplyMixin() {
int opApply(int delegate(inout ValueType v) dg) {
ValueType *t;
while(hasNext()) {
t = next();
if (auto status = dg(*t))
return status;
}
return 0;
}
}
/// Simple array iterator wrapper
struct ArrayIterator(T) {
T[] arr;
alias T ValueType;
T* next() { T* r = arr.ptr; arr = arr[1..$]; return r; }
bool hasNext() { return arr.length > 0; }
mixin opApplyMixin;
}

So this would cover unidirectional iterators (forward and reverse) but
not bidirectional iterators? Perhaps we need a hasPrev as well? And
then there's random access iterators...

Post by Oskar Linde
I have been looking at implementing many of my functional algorithms as
foreach(word; read!(char)("file.txt").splitIterate(&isSpace)) {
writefln("%s",word);
}
where splitIterate is implemented just as the regular split, but instead
of allocating and returning char[][] array, it is implemented as an
iterator returning the next char[] slice for each iteration without
allocating any extra data.

I think Matthew played with this idea a bit in DTL, but I don't think he
got quite this far. It's a great idea however. I can't recall if he
had a name for it, but "view" seems to fit.

Post by Oskar Linde
read could probably be implemented in a smart way, reading the file a
certain chunk at a time. Never using more than a constant amount of memory.
split(array,predicate|substring|element)
select(array,predicate|element)
reverse(array)
range(start,stop,next)
generate(init,next)
map(array,func)
indexmap(array,indexarray|nextindexfunc)
and so on...

That's a solid proof of concept :-)


Sean

[Chris Nicholson-Sauls]

Post by Sean Kelly

Post by Oskar Linde

Post by Sean Kelly

Post by Walter Bright
I've struggled to get people to accept the {} version ever since D
adopted anonymous delegates. Haven't made much headway in getting
such used or in it having any sort of significant impact. How many
have made a "dotimes(n, exp)" function or any sort of syntax
extension using it? None that I've seen.

I've been working on a predicate-oriented algorithm module on and off
for the past few weeks. A few looping constructs have been left out
because foreach seems a preferable solution in D, but the rest is
coming along nicely.

Interesting. I have been doing some of this myself too, but my
available time has been limited. I have also been looking at the
concept of iterators. I agree that foreach is the preferable solution,
but in some cases, only supporting foreach is too limited. For
example, only using foreach, you can not iterate over two collections
simultaneously.
It would be great if we could come up with a standardized Iterator
concept for D that supported both foreach style iteration (opApply)
and some kind of single step iteration (maybe Java-style next/hasNext).

I agree. And given D's somewhat limited overloading mechanism, there
may simply be no point in trying to support pointers as iterators (ie.
to use traits templates). This suggests that some sort of
self-contained design may be best, but I haven't thought about the issue
enough to comment on the details.

Post by Oskar Linde
Here is a sample of how it could look (but this particular sample
template opApplyMixin() {
int opApply(int delegate(inout ValueType v) dg) {
ValueType *t;
while(hasNext()) {
t = next();
if (auto status = dg(*t))
return status;
}
return 0;
}
}
/// Simple array iterator wrapper
struct ArrayIterator(T) {
T[] arr;
alias T ValueType;
T* next() { T* r = arr.ptr; arr = arr[1..$]; return r; }
bool hasNext() { return arr.length > 0; }
mixin opApplyMixin;
}

So this would cover unidirectional iterators (forward and reverse) but
not bidirectional iterators? Perhaps we need a hasPrev as well? And
then there's random access iterators...

Post by Oskar Linde
I have been looking at implementing many of my functional algorithms
foreach(word; read!(char)("file.txt").splitIterate(&isSpace)) {
writefln("%s",word);
}
where splitIterate is implemented just as the regular split, but
instead of allocating and returning char[][] array, it is implemented
as an iterator returning the next char[] slice for each iteration
without allocating any extra data.

I think Matthew played with this idea a bit in DTL, but I don't think he
got quite this far. It's a great idea however. I can't recall if he
had a name for it, but "view" seems to fit.

Post by Oskar Linde
read could probably be implemented in a smart way, reading the file a
certain chunk at a time. Never using more than a constant amount of memory.
split(array,predicate|substring|element)
select(array,predicate|element)
reverse(array)
range(start,stop,next)
generate(init,next)
map(array,func)
indexmap(array,indexarray|nextindexfunc)
and so on...

That's a solid proof of concept :-)
Sean

I still think a good move would be to allow delegates as foreach aggregates. Then a class
could expose "iterators" in the form of methods, and algorithms could be represented as
member delegates.

-- Chris Nicholson-Sauls

[kris]

Post by Walter Bright

Post by kris
So, we have some existing code for logging. Very efficient, and highly
flexible. It follows the design patterns popularized by Log4J, with
Appenders, Layouts, etc
log.trace (char[])
log.info (char[])
log.warn (char[])
log.error (char[])
log.fatal (char[])
and the intent (with dmd.164) was to add these flavours, specifically
log.trace (char[] delegate() dg)
log.info (char[] delegate() dg)
log.warn (char[] delegate() dg)
log.error (char[] delegate() dg)
log.fatal (char[] delegate() dg)
The support code was installed fairly recently, and the above
delegate-wrappers were on the todo list.
However, dmd.165 will presumeably not compile this code?

Correct.

Post by kris
Won't there be an overload ambiguity?

Yes.

Post by kris
If so, it illustrates a serious shortcoming in the new syntax

Just get rid of the (char[]) versions.

Can't do that; the char[] version have to stay.

[Oskar Linde]

Post by kris

Post by Walter Bright

Post by kris
So, we have some existing code for logging. Very efficient, and
highly flexible. It follows the design patterns popularized by Log4J,
with Appenders, Layouts, etc
log.trace (char[])
log.info (char[])
log.warn (char[])
log.error (char[])
log.fatal (char[])
and the intent (with dmd.164) was to add these flavours, specifically
to address the point Walter makes about unnecessary message
log.trace (char[] delegate() dg)
log.info (char[] delegate() dg)
log.warn (char[] delegate() dg)
log.error (char[] delegate() dg)
log.fatal (char[] delegate() dg)
The support code was installed fairly recently, and the above
delegate-wrappers were on the todo list.
However, dmd.165 will presumeably not compile this code?

Correct.

Post by kris
Won't there be an overload ambiguity?

Yes.

Post by kris
If so, it illustrates a serious shortcoming in the new syntax

Just get rid of the (char[]) versions.

Can't do that; the char[] version have to stay.

This may not be a very clean solution, but you could make them
templates, using ifti. (If that only worked for member functions...)

trace(T)(T str) {...}

Then

trace("string");

will not be called lazily, while

trace({return "string";});

will.

/Oskar

[Walter Bright]

Post by kris
Can't do that; the char[] version have to stay.

Why?

[kris]

Post by kris
Can't do that; the char[] version have to stay.

Why?

Because my Boss says so

[Paolo Invernizzi]

Post by kris

Post by kris
Can't do that; the char[] version have to stay.

Why?

Because my Boss says so

kris, really, is it a joke?

If not, and I'm supposing you are using the mango logging package in
some big D application, can you post us the performance difference you
have noticed using the char[] version versus the delegate version?

---
Paolo Invernizzi

[John Reimer]

On Wed, 23 Aug 2006 00:33:08 -0700, Paolo Invernizzi

Post by Paolo Invernizzi

Post by kris

Post by kris
Can't do that; the char[] version have to stay.

Why?

Because my Boss says so

kris, really, is it a joke?
If not, and I'm supposing you are using the mango logging package in
some big D application, can you post us the performance difference you
have noticed using the char[] version versus the delegate version?
---
Paolo Invernizzi

I think he was being sarcastic.

-JJR

[kris]

Post by John Reimer
On Wed, 23 Aug 2006 00:33:08 -0700, Paolo Invernizzi

Post by Paolo Invernizzi

Post by kris

Post by kris
Can't do that; the char[] version have to stay.

Why?

Because my Boss says so

kris, really, is it a joke?
If not, and I'm supposing you are using the mango logging package in
some big D application, can you post us the performance difference
you have noticed using the char[] version versus the delegate version?
---
Paolo Invernizzi

I think he was being sarcastic.
-JJR

No sarcasm required. The recent development of deliberately introducing
ambiguity into the language warrants serious concern. It's something
that should, and does, trouble most of us quite deeply (judging by the
responses).

As for performance differences, I suspect you're missing the bigger
picture, Paolo? I'll try to draw one: you may have noticed my concern
about delegates and heap-based frames? If so, you'll probably have seen
a reponse from Walter noting that specific types of expr can probably be
identified as not escaping the hosting function? There's a gaping hole
in that marketing-style claim, and you should treat it with a heathly
dose of suspicion.

For example; I'd like to use delegates as a nice clean shortcut for
logging. Instead of writing

# if (log.enabled (log.Trace))
# log.trace (sprint ("my log message %d, %s", i, s));

I'd instead like to write this

# log.trace ({return sprint ("my log message %d, %s", i, s);});


Makes sense, right? We don't want to invoke the cost of the sprint()
function unless tracing is enabled. Note that these are guaranteed to be
synchronous callbacks. The delegates are not "saved for later" like a
gui delegate might be. Thus, the frame of the hosting function should be
stack-based.

However, because the callback references local variables, there's no
guarantee the host-frame will not be allocated on the heap instead. This
is the issue we'd been trying to address in another thread. Namely, if
there's going to be ambiguity over whether a delegate is invoked
syncrhonously or asynchronously, we probably need a way to tell the
compiler what's what (rather than let it make the wrong choice).

In simplistic terms, if I have a function like so (note that there's no
delegate intended here):

# real somefunk (real x, real y)
# {
# if (x is real.nan || y is real.nan)
# log.error (sprint ("invalid args %s, %s", x, y));
#
# // do something with x & y
# }

It's quite possible that the frame for this will be heap-allocated, even
though the logger invocation is not intending to use delegates at all.
The damn compiler might go and turn the char[] expr into a sneaky
delegate, and then decide it needs a heap-frame instead of a stack one.
That, my friend, is a serious performance issue (heap allocation each
and every time somefunk() is called), and it's all hidden behind a plush
blue-velvet curtain.


Supposing we actually *want* to use a delegate in this situation? e.g.

# real somefunk (real x, real y)
# {
# if (x is real.nan || y is real.nan)
# log.error ({return sprint ("invalid args %s, %s", x, y);});
#
# // do something with x & y
# }

We still need some manner in which to tell the compiler that the
callback is synchronous, rather than asynchronous, and that it should
not be trying to allocate a heap-frame for this function. Again, this is
what the other thread was discussing in depth. One of the ideas
put-forward there was to somehow mark the hosting function as being a
synchronous-host; something like this

# scope real somefunk (real x, real y);

Another opinion was to mark the /usage/ of the delegate itself, the
logger in this case, with something akin to the following

# void error (scope char[] delegate());


Each of these have their relative pros and cons. What *is* evident, is
that the compiler cannot be trusted to make the appropriate decision. In
contrast, it's quite likely to make a highly conservative, and very
poor, decision. Implied behaviour is often entirely the wrong thing to
do and, without a means for manual control, will quickly become a most
serious problem.

I hope you can see where the recent ambiguities lead to? You can easily
wind up with unexpected (and unwanted) delegates, which may well lead to
unexpected (and unwanted) heap-based frames. To resolve, the ambiguities
must be removed. Additionally, some *dependable* mechanism should be put
in place to manage (or override) how & when heap-based frames are
allocated.

[Paolo Invernizzi]

Post by kris

Post by John Reimer
On Wed, 23 Aug 2006 00:33:08 -0700, Paolo Invernizzi
can you post us the performance difference
you have noticed using the char[] version versus the delegate version?

As for performance differences, I suspect you're missing the bigger
picture, Paolo?

I've followed the discussion.

While I agree that ambiguities must be avoided if possible, and that
compiler-made decision are sometimes frustrating (but we all are happy
about class-all-virtual-methods!), I was just curious of the performance
impact of such a change...

---
Paolo Invernizzi

[kris]

Post by Paolo Invernizzi

Post by kris

Post by John Reimer
On Wed, 23 Aug 2006 00:33:08 -0700, Paolo Invernizzi
can you post us the performance difference you have noticed using
the char[] version versus the delegate version?

As for performance differences, I suspect you're missing the bigger
picture, Paolo?

I've followed the discussion.
While I agree that ambiguities must be avoided if possible, and that
compiler-made decision are sometimes frustrating (but we all are happy
about class-all-virtual-methods!), I was just curious of the performance
impact of such a change...

That's simple ~ time a trivial function by calling it several million
times. Now add a malloc(40) or thereabouts to the same function and time
it again. There's your performance hit. Note that multithreaded apps
will run afoul of mutex-contention also.

As for virtual methods; you can choose which are virtual and which are
not. Seriously though, virtual methods do not allocate from the heap
when they are called. Imagine if they did ...

[kris]

Post by kris
Can't do that; the char[] version have to stay.

Why?

Further, I had to remove the very useful delegate() overload that was
added using dmd164, since dmd165 just had a fit about ambiguous
arguments. So much for backward compatability :/

I don't know where you get your claim about people not willing to use
"{}" but, judging by the response around here, it has no truth behind it
whatsoever. Perhaps you'll honour us with some background on that?

I remember when you asked folks here if it would be ok to change the
cast() syntax way back when, to remove that minor ambiguity. Now you're
using a suspect and pithy "{}" assertion to deliberately break what is
otherwise an unambiguous syntax. And you don't ask or care whether
anyone would actually mind the ensuing chaos. Whatever you've been
smoking, Walter, I'd like to try some of it ~ please pass it around

Cheers;

[Sean Kelly]

Post by Walter Bright
3) It is possible that the delegate can be inlined, thus eliminating any
extra overhead.

Really? Are you saying that if the receiving function is short enough
the function and its delegate use may be inlined in the calling code? I
can't imagine that it would simply be inlined in the receiving function
and duplicates of that wold be generated. Or at least, DMD doesn't
appear to do that now.


Sean

[Walter Bright]

Post by Sean Kelly

Post by Walter Bright
3) It is possible that the delegate can be inlined, thus eliminating
any extra overhead.

Really? Are you saying that if the receiving function is short enough
the function and its delegate use may be inlined in the calling code? I
can't imagine that it would simply be inlined in the receiving function
and duplicates of that wold be generated. Or at least, DMD doesn't
appear to do that now.

It doesn't do it now, but it is possible to do. What an advanced
compiler can do is duplicate the function into two, one gets the value
arguments that have no computation, the other gets the side effect
arguments and ones that involve computation as delegates.

[BCS]

Post by Walter Bright

Post by Sean Kelly

Post by Walter Bright
3) It is possible that the delegate can be inlined, thus eliminating
any extra overhead.

Really? Are you saying that if the receiving function is short enough
the function and its delegate use may be inlined in the calling code?
I can't imagine that it would simply be inlined in the receiving
function and duplicates of that wold be generated. Or at least, DMD
doesn't appear to do that now.

It doesn't do it now, but it is possible to do. What an advanced
compiler can do is duplicate the function into two, one gets the value
arguments that have no computation, the other gets the side effect
arguments and ones that involve computation as delegates.

a.k.a

foo(char[] delegate())

gets converted by the compiler into

foo(char[] delegate())
foo(char[])

???

That would be REALLY cool. But I'm not sure I trust the compiler that much.

[Unknown W. Brackets]

Well, if I saw:

auto new_p = coalesce(p, last_p, new P())

I would not assume that new P() would be evaluated, unless p and last_p
were null.

The same goes for a lot of macro function usage seen with the C
preprocessor. People are already dealing with this problem. A lot.

I like this new feature, and would hate having to type the curlies. I
feel confident I can correctly document my functions and name them such
that no one would be confused.

I won't, however, be using this feature with other function. Or half,
or even 15%. I'll use it when it makes sense.

In my opinion it is a good feature. And it makes D stand out.

-[Unknown]

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );
It would be really important for me to have readable code. I want to
look at the code, and want to have an impression what will happen.
What really would help, is if the the delegate is marked in some way as
such. In the last releases of DMD the {} syntax was there. With it you
needed the return statement. Perhaps we can choose the {} syntax with an
optional return statement....
{ "abc" } => char[] delegate()
{ return "abc"; } => char[] delegate()
func( "abc" ) calls func( char[] a )
func({ "abc" }) calls func( char[] delegate() dg )
func({ return "abc"; }) calls func( char[] delegate() dg )
With that syntax one can immidiatly see "this is a delegate, if it is
called or not or more than once depends on func()" and the more typing
of {} is not too much.
Frank

[BCS]

Post by Unknown W. Brackets
auto new_p = coalesce(p, last_p, new P())
I would not assume that new P() would be evaluated, unless p and last_p
were null.
The same goes for a lot of macro function usage seen with the C
preprocessor. People are already dealing with this problem. A lot.
I like this new feature, and would hate having to type the curlies. I
feel confident I can correctly document my functions and name them such
that no one would be confused.

I feel confident I can document all sorts of bazaar things in my own
code. I also feel confident that if they are allowed/promoted then other
people will use them and *NOT* document them.

As to others using your/my code, you can put a page and a half of
documentation in warning about side effects and it wont do one wit of
good if the other guy doesn't read it. And there is no way to ensure
that it will get read.

I can just see the buzz now: "Did you hear about the bonehead over in
XYZ department that messed up the billing program? He didn't read all of
the comments in the new library and ended up costing the cooperation $n
million dollars because of this "cool" feature in the D programming
language. I'll never use a D lib in /my/ work."

[Frank Benoit]

Post by BCS
I feel confident I can document all sorts of bazaar things in my own
code. I also feel confident that if they are allowed/promoted then other
people will use them and *NOT* document them.
As to others using your/my code, you can put a page and a half of
documentation in warning about side effects and it wont do one wit of
good if the other guy doesn't read it. And there is no way to ensure
that it will get read.
I can just see the buzz now: "Did you hear about the bonehead over in
XYZ department that messed up the billing program? He didn't read all of
the comments in the new library and ended up costing the cooperation $n
million dollars because of this "cool" feature in the D programming
language. I'll never use a D lib in /my/ work."

Well said.

Go for non-implicit delegates.

[Walter Bright]

Post by BCS
I feel confident I can document all sorts of bazaar things in my own
code. I also feel confident that if they are allowed/promoted then other
people will use them and *NOT* document them.
As to others using your/my code, you can put a page and a half of
documentation in warning about side effects and it wont do one wit of
good if the other guy doesn't read it. And there is no way to ensure
that it will get read.
I can just see the buzz now: "Did you hear about the bonehead over in
XYZ department that messed up the billing program? He didn't read all of
the comments in the new library and ended up costing the cooperation $n
million dollars because of this "cool" feature in the D programming
language. I'll never use a D lib in /my/ work."

I agree with you that comments are always wrong, out of date, or
missing. With the delegate feature, one doesn't need to document it, as
it is there in the declaration of the function.

The fact that the delegate parameter exists is a pretty good sign that
the function isn't intended to evaluate the argument exactly once, so it
is worth trying to figure out what the function is doing before calling
it with parameters that have side effects.

Sometimes, corporations have coding standards that ban certain
constructs - like using templates in C++. If a particular feature does
turn out to cause grief, it can be so banned.

[Stewart Gordon]

Post by Frank Benoit
I think the lazy eval is a great feature, but in this form it has also
great drawbacks.
The code isn't that much readable as it was before. You don't know what
will happen. Will that expression be evaluated or not? Or will it be
evaluated more than once?
There is no possibility to choose between
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );

If that's the case, then something's wrong. If funcRetInt returns a
char[], despite the name, then funcRetInt() is of type char[].
Therefore it should match this first.

Post by Frank Benoit
It would be really important for me to have readable code. I want to
look at the code, and want to have an impression what will happen.
What really would help, is if the the delegate is marked in some way as
such. In the last releases of DMD the {} syntax was there. With it you
needed the return statement. Perhaps we can choose the {} syntax with an
optional return statement....
{ "abc" } => char[] delegate()

<snip>

That syntax looks confusingly like an array initialiser....

Stewart.

[Frank Benoit]

Post by Stewart Gordon

Post by Frank Benoit
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );

If that's the case, then something's wrong. If funcRetInt returns a
char[], despite the name, then funcRetInt() is of type char[]. Therefore
it should match this first.

This is the first example, showing a ambiguity
func( char[] a ) vs. func( char[] delegate() dg )

This is a second example, which has no relation to the first.
func( funcRetInt() ); vs. func( &funcRetInt );

And the "vs." is not a object reference. Sorry.

Post by Stewart Gordon

Post by Frank Benoit
{ "abc" } => char[] delegate()

That syntax looks confusingly like an array initialiser....

the '=>' was not a syntax proposal
"{ "abc" }" /is of type/ "char[] delegate()"

[Stewart Gordon]

Post by Frank Benoit

Post by Stewart Gordon

Post by Frank Benoit
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );

If that's the case, then something's wrong. If funcRetInt returns a
char[], despite the name, then funcRetInt() is of type char[]. Therefore
it should match this first.

This is the first example, showing a ambiguity
func( char[] a ) vs. func( char[] delegate() dg )

If one overload matches exactly, it goes without saying that that's the
one that's called. As such, the programmer would do

func("qwert");

to call the first, and

func({ return "qwert"; });

to call the second. Just like in any other case.

Post by Frank Benoit
This is a second example, which has no relation to the first.
func( funcRetInt() ); vs. func( &funcRetInt );

In that case, I don't get what your problem is.

Post by Frank Benoit
And the "vs." is not a object reference. Sorry.

Pardon?

Post by Frank Benoit

Post by Stewart Gordon

Post by Frank Benoit
{ "abc" } => char[] delegate()

That syntax looks confusingly like an array initialiser....

the '=>' was not a syntax proposal
"{ "abc" }" /is of type/ "char[] delegate()"

The '=>' doesn't look anything like an array initialiser either. Unless
there's some other kind of array initialiser that I haven't discovered
yet....

Wasn't it obvious that I was talking about the bit to the left of those
two characters?

[Chris Nicholson-Sauls]

Post by Stewart Gordon

Post by Frank Benoit

Post by Stewart Gordon

Post by Frank Benoit
func( char[] a ) vs. func( char[] delegate() dg )
func( funcRetInt() ); vs. func( &funcRetInt );

If that's the case, then something's wrong. If funcRetInt returns a
char[], despite the name, then funcRetInt() is of type char[]. Therefore
it should match this first.

This is the first example, showing a ambiguity
func( char[] a ) vs. func( char[] delegate() dg )

If one overload matches exactly, it goes without saying that that's the
one that's called. As such, the programmer would do
func("qwert");
to call the first, and
func({ return "qwert"; });
to call the second. Just like in any other case.

That's what I would have thought as well, but its already been said that these cases are
considered ambiguous... For example, the following code:

# module lazy0 ;
#
# import std .stdio ;
#
# void foo (char[] txt) {
# writefln(`%s`c, txt);
# }
#
# void foo (char[] delegate () expr) {
# writefln(`%s`c, expr());
# }
#
# void main () {
# foo(`Alpha`c);
# }

Produces this error:

# lazy0.d(14): function lazy0.foo called with argument types:
# (char[5])
# matches both:
# lazy0.foo(char[])
# and:
# lazy0.foo(char[] delegate())

Post by Stewart Gordon

Post by Frank Benoit
This is a second example, which has no relation to the first.
func( funcRetInt() ); vs. func( &funcRetInt );

In that case, I don't get what your problem is.

Post by Frank Benoit
And the "vs." is not a object reference. Sorry.

Pardon?

Post by Frank Benoit

Post by Stewart Gordon

Post by Frank Benoit
{ "abc" } => char[] delegate()

That syntax looks confusingly like an array initialiser....

the '=>' was not a syntax proposal
"{ "abc" }" /is of type/ "char[] delegate()"

The '=>' doesn't look anything like an array initialiser either. Unless
there's some other kind of array initialiser that I haven't discovered
yet....
Wasn't it obvious that I was talking about the bit to the left of those
two characters?

The only thing I can think of is the PHP format:
# $foo = array(
# 'Alpha' => 3 ,
# 'Beta' => 6
# );

As far as I know, the D way is to use a colon (:) for this sort of thing. Although the
bit on the left doesn't look like an array initialiser either, in terms of D, as they use
brackets rather than braces.
# const int[] foo = [3, 6, 9] ;

Static structure literals do use braces, however, so there's still some ambiguity.

# struct S { int foo; float bar; }
#
# static S MyS = {42, 3.14} ;

-- Chris Nicholson-Sauls

[Stewart Gordon]

<snip>

Post by Chris Nicholson-Sauls

Post by Stewart Gordon

Post by Frank Benoit
This is the first example, showing a ambiguity
func( char[] a ) vs. func( char[] delegate() dg )

If one overload matches exactly, it goes without saying that that's
the one that's called. As such, the programmer would do
func("qwert");
to call the first, and
func({ return "qwert"; });
to call the second. Just like in any other case.

That's what I would have thought as well, but its already been said that

Said by whom, where exactly? Moreover:

http://www.digitalmars.com/d/function.html

"In D, function overloading is simple. It matches exactly, it matches
with implicit conversions, or it does not match. If there is more than
one match, it is an error."

The irony is that this seems to be a case of what this sentence says,
rather than what it was actually supposed to mean.

<snip>

Post by Chris Nicholson-Sauls

Post by Stewart Gordon
The '=>' doesn't look anything like an array initialiser either.
Unless there's some other kind of array initialiser that I haven't
discovered yet....
Wasn't it obvious that I was talking about the bit to the left of
those two characters?

# $foo = array(
# 'Alpha' => 3 ,
# 'Beta' => 6
# );

Well I don't speak PHP, so no wonder I didn't think of that.

Post by Chris Nicholson-Sauls
As far as I know, the D way is to use a colon (:) for this sort of
thing. Although the bit on the left doesn't look like an array
initialiser either, in terms of D, as they use brackets rather than braces.
# const int[] foo = [3, 6, 9] ;

<snip>

Of course! I'm normally more fluent in D than to confuse these
notations. So a struct initialiser is what it looks confusingly like.

Stewart.

[Chris Nicholson-Sauls]

Post by Stewart Gordon
<snip>

Post by Chris Nicholson-Sauls

Post by Stewart Gordon

Post by Frank Benoit
This is the first example, showing a ambiguity
func( char[] a ) vs. func( char[] delegate() dg )

If one overload matches exactly, it goes without saying that that's
the one that's called. As such, the programmer would do
func("qwert");
to call the first, and
func({ return "qwert"; });
to call the second. Just like in any other case.

That's what I would have thought as well, but its already been said
that these cases are considered ambiguous... For example, the

http://www.digitalmars.com/d/function.html
"In D, function overloading is simple. It matches exactly, it matches
with implicit conversions, or it does not match. If there is more than
one match, it is an error."
The irony is that this seems to be a case of what this sentence says,
rather than what it was actually supposed to mean.
<snip>

I know I've read it, don't recall exactly by whom or suchlike. However, it speaks for
itself, try it with DMD 0.165 and behold the error. Mind you I'm in agreement that it
shouldn't be that way: thus I feel the lazy evaluation feature, while nifty and full of
Trendy Cooltasticism (TM), is essentially b0rked in DMD165. I do look forward to the next
version and the 'lazy' parameter class.

-- Chris Nicholson-Sauls