Program logic bugs vs input/environmental errors

[Walter Bright via Digitalmars-d]

This issue comes up over and over, in various guises. I feel like Yosemite Sam here:

In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.

When I say "They are NOT for debugging programs", I mean they are NOT for
debugging programs.

assert()s and contracts are for debugging programs.

dmd test.d

test.d(15) Error: missing } thrown from dmd/src/parse.c(283)

?

See:

https://issues.dlang.org/show_bug.cgi?id=13543

As for the programmer wanting to know where the message "missing }" came from,

grep -r dmd/src/*.c "missing }"

works nicely. I do that sort of thing all the time. It really isn't a problem.

[bearophile via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
As for the programmer wanting to know where the message
"missing }" came from,
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really
isn't a problem.

grep is not useful for the purposes explained in issue 13543
because the file name is often inside a string variable, that is
initialized elsewhere or generated in some way. So the exception
is useful to know where's the instruction in user code that has
tried the failed I/O action, as I've explained in that issue.

Bye,
bearophile

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
As for the programmer wanting to know where the message "missing }" came from,
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really isn't a problem.

grep is not useful for the purposes explained in issue 13543 because the file
name is often inside a string variable, that is initialized elsewhere or
generated in some way. So the exception is useful to know where's the
instruction in user code that has tried the failed I/O action, as I've explained
in that issue.

Even if that is what you wanted, you won't get that from FileException, as it
will only show file/lines emanating from calls inside std.file, not from higher
level callers.

Besides, take a bit of care when formulating a string for exceptions, and you
won't have any trouble grepping for it. This isn't rocket science.

Presenting internal debugging data to users for input/environmental errors is
just bad programming practice. We shouldn't be enshrining it in Phobos and
presenting it as a professional way to code.

[H. S. Teoh via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Walter Bright via Digitalmars-d
As for the programmer wanting to know where the message "missing }" came from,
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really isn't a problem.

grep is not useful for the purposes explained in issue 13543 because
the file name is often inside a string variable, that is initialized
elsewhere or generated in some way. So the exception is useful to
know where's the instruction in user code that has tried the failed
I/O action, as I've explained in that issue.

Even if that is what you wanted, you won't get that from
FileException, as it will only show file/lines emanating from calls
inside std.file, not from higher level callers.
Besides, take a bit of care when formulating a string for exceptions,
and you won't have any trouble grepping for it. This isn't rocket
science.
Presenting internal debugging data to users for input/environmental
errors is just bad programming practice. We shouldn't be enshrining it
in Phobos and presenting it as a professional way to code.

My take on this, is that uncaught exceptions are a program bug. Any
messages displayed to the user ought to come from a catch block that not
only prints the exception message (*without* things like line numbers
and stack traces, btw), but also provides context (e.g., "Error in
configuration file section 'abc': illegal field value" instead of just
"illegal field value" with no context of where it might have been
triggered).

Uncaught exceptions (which ideally should only be Errors, not
Exceptions) are a program bug that ought to be fixed. In the case that
somehow one managed to elude your catch blocks, the full debug infodump
(source file, line number, stack trace) is useful for users to hand back
to you in a bug report, so that you can track down the problem. The user
should not be expected to understand the infodump from an uncaught
exception, whereas a message printed from a catch block ought to be
user-understandable (like "can't open 'myphoto.jpg': file not found",
not "internal error on line 12345" which makes no sense to a user).


T

[bearophile via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
Even if that is what you wanted, you won't get that from
FileException, as it will only show file/lines emanating from
calls inside std.file, not from higher level callers.

Can't we use the template arguments like __LINE__ to offer the
line of code in the IO function in user code?

Post by Walter Bright via Digitalmars-d
Besides, take a bit of care when formulating a string for
exceptions, and you won't have any trouble grepping for it.
This isn't rocket science.

The exception is generated inside library code, not in user code.
There's nothing to grep in the user code. The D script often
looks like this:

void main() {
import std.stdio;
auto file_name = "some_file;
// some code here
auto file_name1 = file_name ~ "1.txt";
auto f1 = File(file_name1);
// some code here
auto file_name2 = file_name ~ "2.txt";
auto f2 = File(file_name2, "w");
// some code here
}

Post by Walter Bright via Digitalmars-d
Presenting internal debugging data to users for
input/environmental errors is just bad programming practice.7
We shouldn't be enshrining it in Phobos and presenting it as a
professional way to code.

The file/line of code is not meant to replace serious practices
for dealing with I/O failures in serious D programs.

Bye,
bearophile

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
Even if that is what you wanted, you won't get that from FileException, as it
will only show file/lines emanating from calls inside std.file, not from
higher level callers.

Can't we use the template arguments like __LINE__ to offer the line of code in
the IO function in user code?

We could, but again, WHOA CAMEL!

Also, one should be careful in using __LINE__ in a template, as it will be
forced to generate a new instantiation with every use.

Post by Walter Bright via Digitalmars-d
Besides, take a bit of care when formulating a string for exceptions, and you
won't have any trouble grepping for it. This isn't rocket science.

The exception is generated inside library code, not in user code. There's
void main() {
import std.stdio;
auto file_name = "some_file;
// some code here
auto file_name1 = file_name ~ "1.txt";
auto f1 = File(file_name1);
// some code here
auto file_name2 = file_name ~ "2.txt";
auto f2 = File(file_name2, "w");
// some code here
}

Come on. What is hard about "Cannot open file `xxxx` in mode `xxxx`", then you
grep for "Cannot open file" ?

grep "Cannot open file" *.d

stdio.d: text("Cannot open file `", name, "' in mode `",
stdio.d:// new StdioException("Cannot open file `"~fName~"' for reading"));
stdio.d:// new StdioException("Cannot open file `"~fName~"' for reading"));

Besides, File doesn't throw a FileException and doesn't add caller's __FILE__
and __LINE__.

If you're seriously proposing adding __FILE__ and __LINE__ to every function
call, then D is the wrong language. This would have disastrous bloat and
performance problems. You mentioned Python and Ruby, both of which are well
known for miserable performance.

Building the equivalent of a symbolic debugger into RELEASED D code is just not
acceptable.

Post by Walter Bright via Digitalmars-d
Presenting internal debugging data to users for input/environmental errors is
just bad programming practice.7 We shouldn't be enshrining it in Phobos and
presenting it as a professional way to code.

The file/line of code is not meant to replace serious practices for dealing with
I/O failures in serious D programs.

That's exactly what you're doing with it.

[Walter Bright via Digitalmars-d]

Post by H. S. Teoh via Digitalmars-d
My take on this, is that uncaught exceptions are a program bug.

Not to me. Custom messages would be better, but the exception message should be
serviceable.

Post by H. S. Teoh via Digitalmars-d
Uncaught exceptions (which ideally should only be Errors, not
Exceptions) are a program bug that ought to be fixed. In the case that
somehow one managed to elude your catch blocks, the full debug infodump
(source file, line number, stack trace) is useful for users to hand back
to you in a bug report, so that you can track down the problem. The user
should not be expected to understand the infodump from an uncaught
exception, whereas a message printed from a catch block ought to be
user-understandable (like "can't open 'myphoto.jpg': file not found",
not "internal error on line 12345" which makes no sense to a user).

Whoa, Camel! You're again thinking of Exceptions as a debugging tool.

[Timon Gehr via Digitalmars-d]

Post by H. S. Teoh via Digitalmars-d
My take on this, is that uncaught exceptions are a program bug.

Not to me. ...

It is not worth fixing if a program terminates with a stack trace?

[Walter Bright via Digitalmars-d]

Post by Timon Gehr via Digitalmars-d

Post by H. S. Teoh via Digitalmars-d
My take on this, is that uncaught exceptions are a program bug.

Not to me. ...

It is not worth fixing if a program terminates with a stack trace?

I never was in favor of adding the stack trace output, either, for the same
reason. Exceptions are not programming bugs.

For Errors, a case can be made for them.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
Whoa, Camel! You're again thinking of Exceptions as a debugging tool.

They can be. What if an API you're using throws an exception you
didn't expect, and therefore don't handle? This might be
considered a logic error if the exception is recoverable and you
don't intend the program to abort from that operation.

[Sean Kelly via Digitalmars-d]

On Sunday, 28 September 2014 at 00:40:26 UTC, Walter Bright

Post by Walter Bright via Digitalmars-d
Whoa, Camel! You're again thinking of Exceptions as a
debugging tool.

They can be. What if an API you're using throws an exception
you didn't expect, and therefore don't handle? This might be
considered a logic error if the exception is recoverable and
you don't intend the program to abort from that operation.

Also, I think the idea that a program is created and shipped to
an end user is overly simplistic. In the server/cloud
programming world, when an error occurs, the client who submitted
the request will get a response appropriate for them and the
system will also generate log information intended for people
working on the system. So things like stack traces and assertion
failure information is useful even for production software. Same
with any critical system, as I'm sure you're aware. The systems
are designed to handle failures in specific ways, but they also
have to leave a breadcrumb trail so the underlying problem can be
diagnosed and fixed. Internal testing is never perfect, and
achieving a high coverage percentage is nearly impossible if the
system wasn't designed from the ground up to be testable in such
a way (mock frameworks and such).

[Walter Bright via Digitalmars-d]

Also, I think the idea that a program is created and shipped to an end user is
overly simplistic. In the server/cloud programming world, when an error occurs,
the client who submitted the request will get a response appropriate for them
and the system will also generate log information intended for people working on
the system. So things like stack traces and assertion failure information is
useful even for production software. Same with any critical system, as I'm sure
you're aware. The systems are designed to handle failures in specific ways, but
they also have to leave a breadcrumb trail so the underlying problem can be
diagnosed and fixed. Internal testing is never perfect, and achieving a high
coverage percentage is nearly impossible if the system wasn't designed from the
ground up to be testable in such a way (mock frameworks and such).

Then use assert(). That's just what it's for.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Sean Kelly via Digitalmars-d
Also, I think the idea that a program is created and shipped
to an end user is overly simplistic. In the server/cloud
programming world, when an error occurs, the client who
submitted the request will get a response appropriate for them
and the system will also generate log information intended for
people working on the system. So things like stack traces and
assertion failure information is useful even for production
software. Same with any critical system, as I'm sure you're
aware. The systems are designed to handle failures in
specific ways, but they also have to leave a breadcrumb trail
so the underlying problem can be diagnosed and fixed.
Internal testing is never perfect, and achieving a high
coverage percentage is nearly impossible if the system wasn't
designed from the ground up to be testable in such a way (mock
frameworks and such).

Then use assert(). That's just what it's for.

What if I don't want to be forced to abort the program in the
event of such an error?

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
Then use assert(). That's just what it's for.

What if I don't want to be forced to abort the program in the event of such an
error?

Then we are back to the discussion about can a program continue after a logic
error is uncovered, or not.

In any program, the programmer must decide if an error is a bug or not, before
shipping it. Trying to avoid making this decision leads to confusion and using
the wrong techniques to deal with it.

A program bug is, by definition, unknown and unanticipated. The idea that one
can "recover" from it is fundamentally wrong. Of course, in D one can try and
recover from them anyway, but you're on your own trying that, just as you're on
your own when casting integers to pointers.

On the other hand, input/environmental errors must be anticipated and can often
be recovered from. But presenting debug traces to the users for these implies at
the very least a sloppily engineered product, in my not so humble opinion :-)

[Joseph Rushton Wakeling via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Walter Bright via Digitalmars-d
Then use assert(). That's just what it's for.

What if I don't want to be forced to abort the program in the event of such an
error?

Then we are back to the discussion about can a program continue after a logic
error is uncovered, or not.
In any program, the programmer must decide if an error is a bug or not, before
shipping it. Trying to avoid making this decision leads to confusion and using
the wrong techniques to deal with it.
A program bug is, by definition, unknown and unanticipated. The idea that one
can "recover" from it is fundamentally wrong. Of course, in D one can try and
recover from them anyway, but you're on your own trying that, just as you're on
your own when casting integers to pointers.

Allowing for your "you can try ..." remarks, I still feel this doesn't really
cover the practical realities of how some applications need to behave.

Put it this way: suppose we're writing the software for a telephone exchange,
which is handling thousands of simultaneous calls. If an Error is thrown inside
the part of the code handling one single call, is it correct to bring down
everyone else's call too?

I appreciate that you might tell me "You need to find a different means of error
handling that can distinguish errors that are recoverable", but the bottom line
is, in such a scenario it's not possible to completely rule out an Error being
thrown (an obvious cause would be an assert that gets triggered because the
programmer forgot to put a corresponding enforce() statement at a higher level
in the code).

However, it's clearly very desirable in this use-case for the application to
keep going if at all possible and for any problem, even an Error, to be
contained in its local context if we can do so. (By "local context", in
practice this probably means a thread or fiber or some other similar programming
construct.)

Sean's touched on this in the current thread with his reference to Erlang, and I
remember that he and Dicebot brought the issue up in an earlier discussion on
the Error vs. Exception question, but I don't recall that discussion having any
firm conclusion, and I think it's important to address; we can't simply take "An
Error is unrecoverable" as a point of principle for every application.

(Related note: If I recall right, an Error or uncaught Exception thrown within a
thread or fiber will not actually bring the application down, only cause that
thread/fiber to hang, without printing any indication of anything going wrong.
So on a purely practical basis, it can be essential for the top-level code of a
thread or fiber to have a catch {} block for both Errors and Exceptions, just in
order to be able to report what has happened effectively.)

[Walter Bright via Digitalmars-d]

Post by Joseph Rushton Wakeling via Digitalmars-d
However, it's clearly very desirable in this use-case for the application to
keep going if at all possible and for any problem, even an Error, to be
contained in its local context if we can do so. (By "local context", in
practice this probably means a thread or fiber or some other similar programming
construct.)

If the program has entered an unknown state, its behavior from then on cannot be
predictable. There's nothing I or D can do about that. D cannot officially
endorse such a practice, though D being a systems programming language it will
let you do what you want.

I would not even consider such a practice for a program that is in charge of
anything that could result in injury, death, property damage, security breaches,
etc.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Joseph Rushton Wakeling via Digitalmars-d
However, it's clearly very desirable in this use-case for the
application to
keep going if at all possible and for any problem, even an
Error, to be
contained in its local context if we can do so. (By "local
context", in
practice this probably means a thread or fiber or some other
similar programming
construct.)

If the program has entered an unknown state, its behavior from
then on cannot be predictable. There's nothing I or D can do
about that. D cannot officially endorse such a practice, though
D being a systems programming language it will let you do what
you want.
I would not even consider such a practice for a program that is
in charge of anything that could result in injury, death,
property damage, security breaches, etc.

Well... suppose you design a system with redundancy such that an
error in a specific process isn't enough to bring down the
system. Say it's a quorum method or whatever. In the instance
that a process goes crazy, I would argue that the system is in an
undefined state but a state that it's designed specifically to
handle, even if that state can't be explicitly defined at design
time. Now if enough things go wrong at once the whole system
will still fail, but it's about building systems with the
expectation that errors will occur. They may even be logic
errors--I think it's kind of irrelevant at that point.

Even a network of communicating processes, one getting in a bad
state can theoretically poison the entire system and you're often
not in a position to simply shut down the whole thing and wait
for a repairman. And simply rebooting the system if it's a bad
sensor that's causing the problem just means a pause before
another failure cascade. I think any modern program designed to
run continuously (increasingly the typical case) must be designed
with some degree of resiliency or self-healing in mind. And that
means planning for and limiting the scope of undefined behavior.

[Walter Bright via Digitalmars-d]

Well... suppose you design a system with redundancy such that an error in a
specific process isn't enough to bring down the system. Say it's a quorum
method or whatever. In the instance that a process goes crazy, I would argue
that the system is in an undefined state but a state that it's designed
specifically to handle, even if that state can't be explicitly defined at design
time. Now if enough things go wrong at once the whole system will still fail,
but it's about building systems with the expectation that errors will occur.
They may even be logic errors--I think it's kind of irrelevant at that point.
Even a network of communicating processes, one getting in a bad state can
theoretically poison the entire system and you're often not in a position to
simply shut down the whole thing and wait for a repairman. And simply rebooting
the system if it's a bad sensor that's causing the problem just means a pause
before another failure cascade. I think any modern program designed to run
continuously (increasingly the typical case) must be designed with some degree
of resiliency or self-healing in mind. And that means planning for and limiting
the scope of undefined behavior.

I've said that processes are different, because the scope of the effects is
limited by the hardware.

If a system with threads that share memory cannot be restarted, there are
serious problems with the design of it, because a crash and the necessary
restart are going to happen sooner or later, probably sooner.

I don't believe that the way to get 6 sigma reliability is by ignoring errors
and hoping. Airplane software is most certainly not done that way.

I recall Toyota got into trouble with their computer controlled cars because of
their idea of handling of inevitable bugs and errors. It was one process that
controlled everything. When something unexpected went wrong, it kept right on
operating, any unknown and unintended consequences be damned.

The way to get reliable systems is to design to accommodate errors, not pretend
they didn't happen, or hope that nothing else got affected, etc. In critical
software systems, that means shut down and restart the offending system, or
engage the backup.

There's no other way that works.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I've said that processes are different, because the scope of
the effects is limited by the hardware.
If a system with threads that share memory cannot be restarted,
there are serious problems with the design of it, because a
crash and the necessary restart are going to happen sooner or
later, probably sooner.

Right. But if the condition that caused the restart persists,
the process can end up in a cascading restart scenario. Simply
restarting on error isn't necessarily enough.

Post by Walter Bright via Digitalmars-d
I don't believe that the way to get 6 sigma reliability is by
ignoring errors and hoping. Airplane software is most certainly
not done that way.

I believe I was arguing the opposite. More to the point, I think
it's necessary to expect undefined behavior to occur and to plan
for it. I think we're on the same page here and just
miscommunicating.

Post by Walter Bright via Digitalmars-d
I recall Toyota got into trouble with their computer controlled
cars because of their idea of handling of inevitable bugs and
errors. It was one process that controlled everything. When
something unexpected went wrong, it kept right on operating,
any unknown and unintended consequences be damned.
The way to get reliable systems is to design to accommodate
errors, not pretend they didn't happen, or hope that nothing
else got affected, etc. In critical software systems, that
means shut down and restart the offending system, or engage the
backup.

My point was that it's often more complicated than that. There
have been papers written on self-repairing systems, for example,
and ways to design systems that are inherently durable when it
comes to even internal errors. I think what I'm trying to say is
that simply aborting on error is too brittle in some cases,
because it only deals with one vector--memory corruption that is
unlikely to reoccur. But I've watched always-on systems fall
apart from some unexpected ongoing situation, and simply
restarting doesn't actually help.

[Walter Bright via Digitalmars-d]

Right. But if the condition that caused the restart persists, the process can
end up in a cascading restart scenario. Simply restarting on error isn't
necessarily enough.

When it isn't enough, use the "engage the backup" technique.

Post by Walter Bright via Digitalmars-d
I don't believe that the way to get 6 sigma reliability is by ignoring errors
and hoping. Airplane software is most certainly not done that way.

I believe I was arguing the opposite. More to the point, I think it's necessary
to expect undefined behavior to occur and to plan for it. I think we're on the
same page here and just miscommunicating.

Assuming that the program bug couldn't have affected other threads is relying on
hope. Bugs happen when the program went into an unknown and unanticipated state.
You cannot know, until after you debug it, what other damage the fault caused,
or what other damage caused the detected fault.

My point was that it's often more complicated than that. There have been papers
written on self-repairing systems, for example, and ways to design systems that
are inherently durable when it comes to even internal errors.

I confess much skepticism about such things when it comes to software. I do know
how reliable avionics software is done, and that stuff does work even in the
face of all kinds of bugs, damage, and errors. I'll be betting my life on that
tomorrow :-)

Would you bet your life on software that had random divide by 0 bugs in it that
were just ignored in the hope that they weren't serious? Keep in mind that
software is rather unique in that a single bit error in a billion bytes can
render the software utterly demented.

Remember the Apollo 11 lunar landing, when the descent computer software started
showing self-detected faults? Armstrong turned it off and landed manually. He
wasn't going to bet his ass that the faults could be ignored. You and I
wouldn't, either.

I think what I'm
trying to say is that simply aborting on error is too brittle in some cases,
because it only deals with one vector--memory corruption that is unlikely to
reoccur. But I've watched always-on systems fall apart from some unexpected
ongoing situation, and simply restarting doesn't actually help.

In such a situation, ignoring the error seems hardly likely to do any better.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I confess much skepticism about such things when it comes to
software. I do know how reliable avionics software is done, and
that stuff does work even in the face of all kinds of bugs,
damage, and errors. I'll be betting my life on that tomorrow :-)
Would you bet your life on software that had random divide by 0
bugs in it that were just ignored in the hope that they weren't
serious? Keep in mind that software is rather unique in that a
single bit error in a billion bytes can render the software
utterly demented.

I'm not saying the errors should be ignored, but rather that
there are other approaches to handling errors besides (or in
addition to) terminating the process. For me, the single most
important thing is detecting errors as soon as possible so
corrective action can be taken before things go too far south (so
hooray for contracts!). From there, the proper response depends
on the error detected and the type of system I'm working on.
Like with persistent stateful systems, even if a restart occurs
can you assume that the persisted state is valid? With a mesh of
communicating systems, if one node goes insane, what impact might
it have on other nodes in the network? I think the definition of
what constitutes an interdependent system is application defined.

And yes, I know all about tiny bugs creating insane problems.
With event-based asynchronous programming, the most common
serious bugs I encounter memory corruption problems from dangling
pointers, and the only way to find and fix these is by analyzing
gigabytes worth of log files to try and unpack what happened
after the fact. Spending a day looking at the collateral damage
from what ultimately turns out to be a backwards conditional
expression in an error handler somewhere gives a pretty healthy
respect for the brittleness of memory unsafe code. This is one
area where having a GC is an enormous win.

Post by Walter Bright via Digitalmars-d
Remember the Apollo 11 lunar landing, when the descent computer
software started showing self-detected faults? Armstrong turned
it off and landed manually. He wasn't going to bet his ass that
the faults could be ignored. You and I wouldn't, either.

And this is great if there's a human available to take over. But
what if this were a space probe?

Post by Walter Bright via Digitalmars-d

I think what I'm trying to say is that simply aborting on
error is too brittle in some cases, because it only deals with
one vector--memory corruption that is unlikely to reoccur.
But I've watched always-on systems fall apart from some
unexpected ongoing situation, and simply restarting doesn't
actually help.

In such a situation, ignoring the error seems hardly likely to
do any better.

Again, not ignoring, but rather that a restart may not be the
appropriate response to the problem. Or it may be a part of the
appropriate response, but other things need to happen as well.

[Paolo Invernizzi via Digitalmars-d]

On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright
Right. But if the condition that caused the restart persists,
the process can end up in a cascading restart scenario. Simply
restarting on error isn't necessarily enough.

This can be mitigated: a cascade reboot would occur if the
problem affects the reboot sequence itself.

---
/Paolo

[Sean Kelly via Digitalmars-d]

On Monday, 29 September 2014 at 07:52:33 UTC, Paolo Invernizzi

Post by Paolo Invernizzi via Digitalmars-d

On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright
Right. But if the condition that caused the restart persists,
the process can end up in a cascading restart scenario.
Simply restarting on error isn't necessarily enough.

This can be mitigated: a cascade reboot would occur if the
problem affects the reboot sequence itself.

Or if an ongoing situation causes the problem to rapidly reoccur.
Look at most MMO game launches for example. Production load
hits and some process falls over in a weird way, which increases
load because everyone goes nuts trying to log back in, and when
the system comes back up it immediately falls over again. Rinse,
repeat.

[Paolo Invernizzi via Digitalmars-d]

Post by Sean Kelly via Digitalmars-d
On Monday, 29 September 2014 at 07:52:33 UTC, Paolo Invernizzi

Post by Paolo Invernizzi via Digitalmars-d

On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright
Right. But if the condition that caused the restart
persists, the process can end up in a cascading restart
scenario. Simply restarting on error isn't necessarily
enough.

This can be mitigated: a cascade reboot would occur if the
problem affects the reboot sequence itself.

Or if an ongoing situation causes the problem to rapidly
reoccur.
Look at most MMO game launches for example. Production load
hits and some process falls over in a weird way, which increases
load because everyone goes nuts trying to log back in, and when
the system comes back up it immediately falls over again.
Rinse,
repeat.

Is it not better to throttle down the connection volumes before
it reach processes not being able to handle an overload in in a
correct way?
---
/Paolo

[Sean Kelly via Digitalmars-d]

On Monday, 29 September 2014 at 21:49:44 UTC, Paolo Invernizzi

Post by Paolo Invernizzi via Digitalmars-d
Is it not better to throttle down the connection volumes before
it reach processes not being able to handle an overload in in a
correct way?

Well, it many cases it isn't the pure load numbers that are the
problem but the interaction of the behavior of actual real users.
Like it isn't overly difficult to simulate a large volume of
traffic on a system, but it's extremely difficult to simulate
realistic behavior patters. I think in many cases the problem
ends up being unexpected interactions of different behaviors that
cause problems rather than simply high volume.

[Bruno Medeiros via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I recall Toyota got into trouble with their computer controlled cars
because of their idea of handling of inevitable bugs and errors. It
was one process that controlled everything. When something unexpected
went wrong, it kept right on operating, any unknown and unintended
consequences be damned.
The way to get reliable systems is to design to accommodate errors,
not pretend they didn't happen, or hope that nothing else got
affected, etc. In critical software systems, that means shut down and
restart the offending system, or engage the backup.

My point was that it's often more complicated than that. There have
been papers written on self-repairing systems, for example, and ways to
design systems that are inherently durable when it comes to even
internal errors. I think what I'm trying to say is that simply aborting
on error is too brittle in some cases, because it only deals with one
vector--memory corruption that is unlikely to reoccur. But I've watched
always-on systems fall apart from some unexpected ongoing situation, and
simply restarting doesn't actually help.

Sean, I fully agree with the points you have been making so far.
But if Walter is fixated on thinking that all the practical uses of D
will be critical systems, or simple (ie, single-use, non-interactive)
command-line applications, it will be hard for him to comprehend the
whole point that "simply aborting on error is too brittle in some cases".

PS: Walter, what browser to you use?

[bearophile via Digitalmars-d]

Post by Bruno Medeiros via Digitalmars-d
But if Walter is fixated on thinking that all the practical
uses of D will be critical systems, or simple (ie, single-use,
non-interactive) command-line applications,

There's still some of way to go for D design to make it well fit
for high integrity systems (some people even use a restricted
subset of C for such purposes, but it's a bad language for it).

Bye,
bearophile

[Joseph Rushton Wakeling via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Also, I think the idea that a program is created and shipped to an end user is
overly simplistic. In the server/cloud programming world, when an error occurs,
the client who submitted the request will get a response appropriate for them
and the system will also generate log information intended for people working on
the system. So things like stack traces and assertion failure information is
useful even for production software. Same with any critical system, as I'm sure
you're aware. The systems are designed to handle failures in specific ways, but
they also have to leave a breadcrumb trail so the underlying problem can be
diagnosed and fixed. Internal testing is never perfect, and achieving a high
coverage percentage is nearly impossible if the system wasn't designed from the
ground up to be testable in such a way (mock frameworks and such).

Then use assert(). That's just what it's for.

I don't follow this point. How can this approach work with programs that are
built with the -release switch?

Moreover, Sean's points here are absolutely on the money -- there are cases
where the "users" of a program may indeed want to see traces even for
anticipated errors. And even if you design a nice structure of throwing and
catching exceptions so that the simple error message _always_ gives good enough
context to understand what went wrong, you still have the other issue that Sean
raised -- of an exception accidentally escaping its intended scope, because you
forgot to handle it -- when a trace may be extremely useful.

Put it another way -- I think you make a good case that stack traces for
exceptions should be turned off by default (possibly just in -release mode?),
but if that happens I think there's also a good case for a build flag that
ensures stack traces _are_ shown for Exceptions as well as Errors.

[ketmar via Digitalmars-d]

On Mon, 29 Sep 2014 01:18:02 +0200
Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d at puremagic.com>

Post by Joseph Rushton Wakeling via Digitalmars-d

Post by Walter Bright via Digitalmars-d
Then use assert(). That's just what it's for.

I don't follow this point. How can this approach work with programs
that are built with the -release switch?

don't use "-release" switch. the whole concept of "release version" is
broken by design. ship what you debugged, not what you think you
debugged.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/73a9dac7/attachment.sig>

[Walter Bright via Digitalmars-d]

Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with programs that are
built with the -release switch?

All -release does is not generate code for assert()s. To leave the asserts in,
do not use -release. If you still want the asserts to be in even with -release,

if (condition) assert(0);

Post by Joseph Rushton Wakeling via Digitalmars-d
Moreover, Sean's points here are absolutely on the money -- there are cases
where the "users" of a program may indeed want to see traces even for
anticipated errors.
And even if you design a nice structure of throwing and
catching exceptions so that the simple error message _always_ gives good enough
context to understand what went wrong, you still have the other issue that Sean
raised -- of an exception accidentally escaping its intended scope, because you
forgot to handle it -- when a trace may be extremely useful.
Put it another way -- I think you make a good case that stack traces for
exceptions should be turned off by default (possibly just in -release mode?),
but if that happens I think there's also a good case for a build flag that
ensures stack traces _are_ shown for Exceptions as well as Errors.

The -g switch should take care of that. It's what I use when I need a stack
trace, as there are many ways a program can fail (not just Errors).

[Timon Gehr via Digitalmars-d]

Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with programs that are
built with the -release switch?

All -release does is not generate code for assert()s. ...

(Euphemism for undefined behaviour.)

[Timon Gehr via Digitalmars-d]

Post by Timon Gehr via Digitalmars-d

Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with programs that are
built with the -release switch?

All -release does is not generate code for assert()s. ...

(Euphemism for undefined behaviour.)

Also, -release additionally removes contracts, in particular invariant
calls, and enables version(assert).

[Johannes Pfau via Digitalmars-d]

Am Sun, 28 Sep 2014 17:47:56 -0700

Post by Walter Bright via Digitalmars-d

Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with
programs that are built with the -release switch?

All -release does is not generate code for assert()s. To leave the
asserts in, do not use -release. If you still want the asserts to be
in even with -release,
if (condition) assert(0);

Right now, but some time ago there was a huge debate whether it should
be valid for the compiler to optimize based on asserts.

I wonder if these 'use asserts for stack traces' and 'an assert is
always supposed to pass, so it's valid to assume the condition holds
(in release)' notions can go together. I guess it might at least lead
to programs that are unusable when compiled with -release.

[Daniel N via Digitalmars-d]

Post by Johannes Pfau via Digitalmars-d
Right now, but some time ago there was a huge debate whether it should
be valid for the compiler to optimize based on asserts.
I wonder if these 'use asserts for stack traces' and 'an assert is
always supposed to pass, so it's valid to assume the condition
holds
(in release)' notions can go together. I guess it might at
least lead
to programs that are unusable when compiled with -release.

For this reason I think it makes more sense to use abort() if you
plan to use "-release".

[ketmar via Digitalmars-d]

On Mon, 29 Sep 2014 10:18:27 +0200

Post by Johannes Pfau via Digitalmars-d
I wonder if these 'use asserts for stack traces' and 'an assert is
always supposed to pass, so it's valid to assume the condition holds
(in release)' notions can go together. I guess it might at least lead
to programs that are unusable when compiled with -release.

compiler never optimizes away `assert(0)`, AFAIR. `assert(0)` is a
special thing, it means `abort()`.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/108a2330/attachment.sig>

[Joseph Rushton Wakeling via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with
programs that are
built with the -release switch?

All -release does is not generate code for assert()s. To leave
the asserts in, do not use -release. If you still want the
asserts to be in even with -release,
if (condition) assert(0);

The reason I queried your approach here is because I feel you're
conflating two things:

* the _definition_ of an Exception vs. an Error, on which we
100%
agree: the former as an anticipated possibility which a
program
is committed to try and handle, the latter a failure which
is
fundamentally wrong and should not happen under any
conditions.

* the way in which a program should report these different
kinds
of error.

You seem to be advocating that, by definition, Exceptions and
Errors should be reported differently (one without, and one with,
a trace). I don't at all object to that as a sensible default,
but I think that the ultimate decision on how Exceptions and
Errors report themselves should be in the hands of the program
developer, depending on the use-case of the application.

[via Digitalmars-d]

On Monday, 29 September 2014 at 15:39:08 UTC, Joseph Rushton

with, a trace). I don't at all object to that as a sensible
default, but I think that the ultimate decision on how
Exceptions and Errors report themselves should be in the hands
of the program developer, depending on the use-case of the
application.

That's a very sensible argument. From a pragmatic point of view I
am usually not interested in the whole stack trace.

I am primarily interested in where it was thrown, where it was
turned into an error and where it was logged + "function call
input" at that point.

Maybe a trade off can be found that is less costly than building
a full stack trace.

[Walter Bright via Digitalmars-d]

Post by Sean Kelly via Digitalmars-d

Post by Walter Bright via Digitalmars-d
Whoa, Camel! You're again thinking of Exceptions as a debugging tool.

They can be.

Of course they can be. But it's inappropriate to use them that way, and we
should not be eschewing such in the library.

Post by Sean Kelly via Digitalmars-d
What if an API you're using throws an exception you didn't expect,
and therefore don't handle?

Then the app user sees the error message. This is one of the cool things about D
- I can write small apps with NO error handling logic in it, and I still get
appropriate and friendly messages when things go wrong like missing files.

That is, until recently, when I get a bunch of debug stack traces and internal
file/line messages, which are of no use at all to an app user and look awful.

Post by Sean Kelly via Digitalmars-d
This might be considered a logic error if the
exception is recoverable and you don't intend the program to abort from that
operation.

Adding file/line to all exceptions implies that they are all bugs, and
encourages them to be thought of as bugs and debugging tools, when they are NOT.
Exceptions are for:

1. enabling recovery from input/environmental errors
2. reporting input/environmental errors to the app user
3. making input/environmental errors not ignorable by default

They are not for detecting logic errors. Assert is designed for that.

[H. S. Teoh via Digitalmars-d]

[...]

Post by Walter Bright via Digitalmars-d

Post by Sean Kelly via Digitalmars-d
What if an API you're using throws an exception you didn't expect,
and therefore don't handle?

Then the app user sees the error message. This is one of the cool
things about D - I can write small apps with NO error handling logic
in it, and I still get appropriate and friendly messages when things
go wrong like missing files.
That is, until recently, when I get a bunch of debug stack traces and
internal file/line messages, which are of no use at all to an app user
and look awful.

It looks even more awful when the person who wrote the library code is
Russian, and the user speaks English, and when an uncaught exception
terminates the program, you get a completely incomprehensible message in
a language you don't know. Not much different from a line number and
filename that has no meaning for a user.

That's why I said, an uncaught exception is a BUG. The only place where
user-readable messages can be output is in a catch block where you
actually have the chance to localize the error string. But if no catch
block catches it, then by definition it's a bug, and you might as while
print some useful info with it that your users can send back to you,
rather than unhelpful bug reports of the form "the program crashed with
error message 'internal error'". Good luck finding where in the code
that is. (And no, grepping does not work -- the string 'internal error'
could have come from a system call or C library error code translated by
a generic code-to-message function, which could've been called from
*anywhere*.)

Post by Walter Bright via Digitalmars-d

Post by Sean Kelly via Digitalmars-d
This might be considered a logic error if the exception is
recoverable and you don't intend the program to abort from that
operation.

Adding file/line to all exceptions implies that they are all bugs, and
encourages them to be thought of as bugs and debugging tools, when
1. enabling recovery from input/environmental errors
2. reporting input/environmental errors to the app user
3. making input/environmental errors not ignorable by default
They are not for detecting logic errors. Assert is designed for that.

I do not condone adding file/line to exception *messages*. Catch blocks
can print / translate those messages, which can be made user-friendly,
but if the program failed to catch an exception, you're already screwed
anyway so why not provide more info rather than less?

Unless, of course, you're suggesting that we put this around every
main() function:

void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}


T

[Sean Kelly via Digitalmars-d]

On Sunday, 28 September 2014 at 20:58:20 UTC, H. S. Teoh via

Post by H. S. Teoh via Digitalmars-d
That's why I said, an uncaught exception is a BUG. The only
place where
user-readable messages can be output is in a catch block where
you
actually have the chance to localize the error string. But if
no catch
block catches it, then by definition it's a bug, and you might
as while
print some useful info with it that your users can send back to
you,
rather than unhelpful bug reports of the form "the program
crashed with
error message 'internal error'".

Pretty much every system should generate a localized error
message for the user and a detailed log of the problem for the
programmer. Ideally, the user message will indicate how to
provide the detailed information to the developer so the problem
can be fixed.

The one case where uncaught exceptions aren't really a bug is
with programs that aren't being used outside the group that
developed them. In these cases, the default behavior is pretty
much exactly what's desired--a message, file/line info, and a
stack trace. Which is why it's there. The vast bulk of today's
shipping code doesn't run from the command line anyway, so the
default exception handler should be practically irrelevant.

[Cliff via Digitalmars-d]

On Sunday, 28 September 2014 at 20:58:20 UTC, H. S. Teoh via

Post by H. S. Teoh via Digitalmars-d
I do not condone adding file/line to exception *messages*.
Catch blocks
can print / translate those messages, which can be made
user-friendly,
but if the program failed to catch an exception, you're already
screwed
anyway so why not provide more info rather than less?
Unless, of course, you're suggesting that we put this around
every
void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}

In our production C# code, we had a few practices which might be
applicable here:

1. main() definitely had a top-level try/catch handler to produce
useful output messages. Because throwing an uncaught exception
out to the user *is* a bug, we naturally want to not just toss
out a stack trace but information on what to do with it should a
user encounter it. Even better if there is additional runtime
information which can be provided for a bug report.

2. We also registered a top-level unhandled exception handler on
the AppDomain (equivalent to a process in .NET, except that
multiple AppDomains may exist within a single OS process), which
allows the catching to exceptions which would otherwise escape
background threads. Depending on the nature of the application,
these could be logged to some repository to which the user could
be directed. It's hard to strictly automate this because exactly
what you can do with an exception which escapes a thread will be
application dependent. In our case, these exceptions were
considered bugs, were considered to be unrecoverable and resulted
in a program abort with a user message indicating where to find
the relevant log outputs and how to contact us.

3. For some cases, throwing an exception would also trigger an
application dump suitable for post-mortem debugging from the
point the exception was about to be thrown. This functionality
is, of course, OS-specific, but helped us on more than a few
occasions by eliminating the need to try to pre-determine which
information was important and which was not so the exception
could be usefully populated.

I'm not a fan of eliminating the stack from exceptions. While
exceptions should not be used to catch logic errors, an uncaught
exception is itself a logic error (that is, one has omitted some
required conditions in their code) and thus the context of the
error needs to be made available somehow.

[Walter Bright via Digitalmars-d]

Post by H. S. Teoh via Digitalmars-d
It looks even more awful when the person who wrote the library code is
Russian, and the user speaks English, and when an uncaught exception
terminates the program, you get a completely incomprehensible message in
a language you don't know. Not much different from a line number and
filename that has no meaning for a user.

I cannot buy into the logic that since Russian error messages are
incomprehensible to me, that therefore incomprehensible messages are ok.

Post by H. S. Teoh via Digitalmars-d
That's why I said, an uncaught exception is a BUG.

It's a valid opinion, but is not the way D is designed to work.

Post by H. S. Teoh via Digitalmars-d
The only place where
user-readable messages can be output is in a catch block where you
actually have the chance to localize the error string. But if no catch
block catches it, then by definition it's a bug, and you might as while
print some useful info with it that your users can send back to you,
rather than unhelpful bug reports of the form "the program crashed with
error message 'internal error'".

If anyone is writing code that throws an Exception with "internal error", then
they are MISUSING exceptions to throw on logic bugs. I've been arguing this all
along.

Post by H. S. Teoh via Digitalmars-d
if the program failed to catch an exception, you're already screwed
anyway

This is simply not true. One can write utilities with no caught exceptions at
all, and yet have the program emit user friendly messages about "disk full" and
stuff like that.

Post by H. S. Teoh via Digitalmars-d
so why not provide more info rather than less?

Because having an internal stack dump presented to the app user for when he,
say, puts in invalid command line arguments, is quite inappropriate.

Post by H. S. Teoh via Digitalmars-d
Unless, of course, you're suggesting that we put this around every
void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}

I'm not suggesting that Exceptions are to be thrown on programmer screwups - I
suggest the OPPOSITE.

[Timon Gehr via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
...

Post by H. S. Teoh via Digitalmars-d
Unless, of course, you're suggesting that we put this around every
void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}

I'm not suggesting that Exceptions are to be thrown on programmer
screwups - I suggest the OPPOSITE.

He does not suggest that Exceptions are to be thrown on programmer
screw-ups, but rather that the thrown exception itself is the screw-up,
with a possibly complex cause.

It is not:

if(screwedUp()) throw Exception("");


It is rather:

void foo(int x){
if(!test(x)) throw Exception(""); // this may be an expected code
path for some callers
}

void bar(){
// ...
int y=screwUp();
foo(y); // yet it is unexpected here
}

[via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
If anyone is writing code that throws an Exception with
"internal error", then they are MISUSING exceptions to throw on
logic bugs. I've been arguing this all along.

Nothing wrong with it. Quite common and useful for a non-critical
web service to log the exception, then re-throw something like
"internal error", catch the internal error at the root and
returning the appropriate 5xx HTTP response, then keep going.

You are arguing as if it is impossible to know whether the logic
error is local to the handler, or not, with a reasonable
probability. "Division by zero" is usually not a big deal, but it
is a logic error. No need to shut down the service.

Post by Walter Bright via Digitalmars-d
I'm not suggesting that Exceptions are to be thrown on
programmer screwups - I suggest the OPPOSITE.

It is impossible to verify what the source is. It might be a bug
in a boolean expression leading to a throw when the system is ok.

assert()s should also not be left in production code. They are
not for catching runtime errors, but for testing at the expense
of performance.

Uncaught exceptions should be re-thrown higher up in the call
chain to a different error level based on the possible impact on
the system. Getting an unexpected mismatch exception in a
form-validator is not a big deal. Getting out-of-bounds error in
main storage is a big deal. Whether it is a big deal can only be
decided at the higher level.

It is no doubt useful to be able to obtain a stack trace so that
you can log it when an exception turns out to fall into the "big
deal" category and therefore should be re-thrown as a critical
failture. The deciding factor should be performance.

[Walter Bright via Digitalmars-d]

On 9/28/2014 9:31 PM, "Ola Fosheim GrÞstad"

Nothing wrong with it. Quite common and useful for a non-critical web service to
log the exception, then re-throw something like "internal error", catch the
internal error at the root and returning the appropriate 5xx HTTP response, then
keep going.

Lots of bad practices are commonplace.

You are arguing as if it is impossible to know whether the logic error is local
to the handler, or not, with a reasonable probability.

You're claiming to know that a program in an unknown and unanticipated state is
really in a known state. It isn't.

assert()s should also not be left in production code. They are not for catching
runtime errors, but for testing at the expense of performance.

Are you really suggesting that asserts should be replaced by thrown exceptions?
I suspect we have little common ground here.

Uncaught exceptions should be re-thrown higher up in the call chain to a
different error level based on the possible impact on the system. Getting an
unexpected mismatch exception in a form-validator is not a big deal. Getting
out-of-bounds error in main storage is a big deal. Whether it is a big deal can
only be decided at the higher level.

A vast assumption here that you know in advance what bugs you're going to have
and what causes them.

It is no doubt useful to be able to obtain a stack trace so that you can log it
when an exception turns out to fall into the "big deal" category and therefore
should be re-thrown as a critical failture. The deciding factor should be
performance.

You're using exceptions as a program bug reporting mechanism. Whoa camel, indeed!

[via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
Lots of bad practices are commonplace.

This is not an argument, it is a postulate.

Post by Walter Bright via Digitalmars-d

Post by via Digitalmars-d
You are arguing as if it is impossible to know whether the
logic error is local
to the handler, or not, with a reasonable probability.

You're claiming to know that a program in an unknown and
unanticipated state is really in a known state. It isn't.

It does not have to be known, it is sufficient that it is
isolated or that it is improbable to be global or that it is of
low impact to long term integrity.

Post by Walter Bright via Digitalmars-d
Are you really suggesting that asserts should be replaced by
thrown exceptions? I suspect we have little common ground here.

No, regular asserts should not be caught except for mailing the
error log to the developer. They are for testing only.

Pre/postconditions between subsystems are on a different level
though. They should not be conflated with regular asserts.

Post by Walter Bright via Digitalmars-d
A vast assumption here that you know in advance what bugs
you're going to have and what causes them.

I know in advance that a "divison-by-zero" error is of limited
scope with high probability or that an error in a strictly pure
validator is of low impact with high probability. I also know
that any sign of a flaw in a transaction engine is a critical
error that warrants a shutdown.

We know in advance that all programs above low complexity will
contain bugs, most of them innocent and they are not a good
excuse of shutting down the entire service for many services.

If you have memory safety, reasonable isolation and well tested
global data-structures it is most desirable to keep the system
running if it is incapable of corrupting a critical database.

Post by Walter Bright via Digitalmars-d
You're using exceptions as a program bug reporting mechanism.

Uncaught exceptions are bugs and should be logged as such. If a
form validator throws an unexpected exception then it is a bug.
It makes the validation questionable, but does not affect the
rest of the system. It is a non-critical bug that needs attention.

Post by Walter Bright via Digitalmars-d
Whoa camel, indeed!

By your line of reasoning no software should ever be shipped,
without a formal proof, because they most certainly will be buggy
and contain unspecified undetected state.

Keep in mind that a working program, in the real world, is a
program that provides reasonable output for reasonable input.
Total correctness is a pipe dream, it is not within reach for
most real programs. Not even with formal proofs.

[Johannes Pfau via Digitalmars-d]

Am Sun, 28 Sep 2014 15:59:45 -0700

Post by Walter Bright via Digitalmars-d

Post by H. S. Teoh via Digitalmars-d
if the program failed to catch an exception, you're already screwed
anyway

This is simply not true. One can write utilities with no caught
exceptions at all, and yet have the program emit user friendly
messages about "disk full" and stuff like that.

You're always thinking of simple console apps but this is the only place
where the default 'print exception to console' strategy works.

In a daemon which logs to syslog or in a GUI application or a game an
uncaught 'disk full exception' would go completely unnoticed and that's
definitely a bug.

[Walter Bright via Digitalmars-d]

Post by Johannes Pfau via Digitalmars-d
In a daemon which logs to syslog or in a GUI application or a game an
uncaught 'disk full exception' would go completely unnoticed and that's
definitely a bug.

Failure to respond properly to an input/environmental error is a bug. But the
input/environmental error is not a bug. If it was, then the program should
assert on the error, not throw.

[Bruno Medeiros via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Johannes Pfau via Digitalmars-d
In a daemon which logs to syslog or in a GUI application or a game an
uncaught 'disk full exception' would go completely unnoticed and that's
definitely a bug.

Failure to respond properly to an input/environmental error is a bug.
But the input/environmental error is not a bug. If it was, then the
program should assert on the error, not throw.

I agree. And isn't that exactly what Teoh said then:
"That's why I said, an uncaught exception is a BUG. "

I think people should be more careful with the term "uncaught exception"
because it's not very precise.

[Steven Schveighoffer via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.

Library code often cannot make that choice. The issue with exceptions
vs. errors is that often you don't know where the input comes from.

e.g.:

auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception

How does File know what it's target file name came from?

-Steve

[Walter Bright via Digitalmars-d]

Post by Steven Schveighoffer via Digitalmars-d

Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.

Library code often cannot make that choice.
The issue with exceptions vs. errors
is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?

If the app is concerned about invalid filenames as bugs, you should scrub the
filenames first. I.e. the interface is defined improperly if the code confuses a
programming bug with input errors.

[Steven Schveighoffer via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d

Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.

Library code often cannot make that choice.
The issue with exceptions vs. errors
is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?

If the app is concerned about invalid filenames as bugs, you should
scrub the filenames first. I.e. the interface is defined improperly if
the code confuses a programming bug with input errors.

OK, so if you want to avoid improperly having errors/exceptions, don't
put bugs into your code.

A simple plan!

-Steve

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d

Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.

Library code often cannot make that choice.
The issue with exceptions vs. errors
is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?

If the app is concerned about invalid filenames as bugs, you should
scrub the filenames first. I.e. the interface is defined improperly if
the code confuses a programming bug with input errors.

OK, so if you want to avoid improperly having errors/exceptions, don't put bugs
into your code.
A simple plan!

Validating user input is not the same thing as removing all the logic bugs from
the program.

[Steven Schveighoffer via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Walter Bright via Digitalmars-d
If the app is concerned about invalid filenames as bugs, you should
scrub the filenames first. I.e. the interface is defined improperly if
the code confuses a programming bug with input errors.

OK, so if you want to avoid improperly having errors/exceptions, don't put bugs
into your code.
A simple plan!

Validating user input is not the same thing as removing all the logic
bugs from the program.

What if it's not user input?

-Steve

[Jacob Carlborg via Digitalmars-d]

Post by Steven Schveighoffer via Digitalmars-d
Library code often cannot make that choice. The issue with exceptions
vs. errors is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?

Both of theses should throw an exception. Most stuff related to file
operations should throw an exception, not an error.

[Steven Schveighoffer via Digitalmars-d]

Post by Jacob Carlborg via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d
Library code often cannot make that choice. The issue with exceptions
vs. errors is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?

Both of theses should throw an exception. Most stuff related to file
operations should throw an exception, not an error.

That makes no sense. The opening of the file is subject to issues with
the filesystem, which means they may be environmental or user errors,
not programming errors. But that doesn't mean the opening of the file
failed because the file doesn't exist, it could be an error in how you
construct the file name.

What about:

File f;
f.open(null);

Is that an environmental error or User error?

-Steve

[Walter Bright via Digitalmars-d]

Post by Steven Schveighoffer via Digitalmars-d
File f;
f.open(null);
Is that an environmental error or User error?

Passing invalid arguments to a function is a programming bug.

[Steven Schveighoffer via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d
File f;
f.open(null);
Is that an environmental error or User error?

Passing invalid arguments to a function is a programming bug.

That throws an exception. My point exactly.

-Steve

[Jacob Carlborg via Digitalmars-d]

Post by Steven Schveighoffer via Digitalmars-d
That makes no sense. The opening of the file is subject to issues with
the filesystem, which means they may be environmental or user errors,
not programming errors. But that doesn't mean the opening of the file
failed because the file doesn't exist, it could be an error in how you
construct the file name.
File f;
f.open(null);
Is that an environmental error or User error?

That depends on what "open" expect from its argument. In this particular
case, in D, "null" is the same as the empty string. I don't see why that
technically shouldn't be allowed.

Of course, you can specify that "open" expects a string argument with
the length that is greater then 0, in that case it's a bug by the
programmer that uses the "open" function.

[Steven Schveighoffer via Digitalmars-d]

Post by Jacob Carlborg via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d
That makes no sense. The opening of the file is subject to issues with
the filesystem, which means they may be environmental or user errors,
not programming errors. But that doesn't mean the opening of the file
failed because the file doesn't exist, it could be an error in how you
construct the file name.
File f;
f.open(null);
Is that an environmental error or User error?

That depends on what "open" expect from its argument. In this particular
case, in D, "null" is the same as the empty string. I don't see why that
technically shouldn't be allowed.

My entire point is, it doesn't matter what is expected or what is
treated as "correct." what matters is where the input CAME from. To a
library function, it has no idea. There is no extra type info saying
"this parameter comes from user input."

Post by Jacob Carlborg via Digitalmars-d
Of course, you can specify that "open" expects a string argument with
the length that is greater then 0, in that case it's a bug by the
programmer that uses the "open" function.

Is it? I can think of cases where it's programmer error, and cases where
it's user error.

There are also better example functions, but I'm focusing on File
because that's what this thread is about.

The largest issue I see with this whole scheme is that exceptions can be
turned into errors, but not the reverse. Once an error is thrown, it's
pretty much game over. So defensive coding would suggest when you don't
know the answer, throw an exception, and something higher up would say
"Oh, that is really a program error, rethrowing"

But expecting developers to do this at EVERY CALL is really impossible.

My expectation is that an exception is really an error unless caught. It
would be nice to postpone generating the stack trace unless the
exception is caught outside the main function. I don't know enough about
how exceptions work to know if this is possible or not.

-Steve

[Jeremy Powers via Digitalmars-d]

On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d <

My entire point is, it doesn't matter what is expected or what is treated
as "correct." what matters is where the input CAME from. To a library
function, it has no idea. There is no extra type info saying "this
parameter comes from user input."
From the method's view, parameters passed in are user input. Full stop.

One thing that seems to be talked around a bit here is the
separation/encapsulation of things. It is perfectly fine, and expected,
for a library method to throw exceptions on bad input to the method - even
if this input turns out to be a programming bug elsewhere. From the
standpoint of the method, it does not know (and does not care) where the
thing ultimately came from - all it knows is that it is input here, and it
is wrong.

If you call a method with bad input, and fail to catch the resulting
exception, then _that_ is a bug, not the method throwing. It may be
perfectly recoverable to ignore/retry/whatever, or it may be a symptom of
something that should abort the program. But the method throwing does not
know or care.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/a79b902b/attachment.html>

[Jeremy Powers via Digitalmars-d]

On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d <

Post by Steven Schveighoffer via Digitalmars-d
The largest issue I see with this whole scheme is that exceptions can be
turned into errors, but not the reverse. Once an error is thrown, it's
pretty much game over. So defensive coding would suggest when you don't
know the answer, throw an exception, and something higher up would say "Oh,
that is really a program error, rethrowing"
But expecting developers to do this at EVERY CALL is really impossible.

And this is an argument for checked exceptions - being able to explicitly
state 'these are known fatal cases for this component, you should deal with
them appropriately' when defining a method. Cuts down the catch/check to
just the common cases, and makes such cases explicit to the caller.
Anything not a checked exception falls into the 'error, abort!' path.
(Memory corruption etc. being abort scenarios)

If I really needed to write a robust program in D right now, I would
(attempt) to wrap every call in a try/catch, and check if the thrown
exception was of a handleable type. But knowing which types for which
methods would lead me to basically hacking up some documentation-enforced
checked exceptions, and being entirely unmaintainable.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/a67801ce/attachment.html>

[Steven Schveighoffer via Digitalmars-d]

Post by Jeremy Powers via Digitalmars-d
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d
My entire point is, it doesn't matter what is expected or what is
treated as "correct." what matters is where the input CAME from. To
a library function, it has no idea. There is no extra type info
saying "this parameter comes from user input."
From the method's view, parameters passed in are user input. Full stop.

This is missing the point of an exception. An uncaught exception is an
error which crashes the program. If you catch the exception, you can
handle it, but if you don't expect it, then it's a bug. Any uncaught
exceptions are BY DEFINITION programming errors.

Post by Jeremy Powers via Digitalmars-d
One thing that seems to be talked around a bit here is the
separation/encapsulation of things. It is perfectly fine, and expected,
for a library method to throw exceptions on bad input to the method -
even if this input turns out to be a programming bug elsewhere. From
the standpoint of the method, it does not know (and does not care) where
the thing ultimately came from - all it knows is that it is input here,
and it is wrong.

What is being discussed here is removing the stack trace and printout
when an exception is thrown.

Imagine this error message:

myprompt> ./mycoolprogram
(1 hour later)
FileException: Error opening file xgghfsnbuer
myprompt>

Now what? xgghfsnbuer may not even be in the code anywhere. There may be
no hints at all as to what caused it to happen. You don't even know
which line of code YOU wrote that was causing the issue! You have to
examine every File open, and every call that may have opened a file, and
see if possibly that file name was passed into it.

Whereas if you get a trace, you can at least see where the exception
occurred, and start from there.

Now, RELYING on this printout to be your interface to the user, that is
incorrect design, I will agree. But one cannot possibly be expected to
handle every possible exception at every possible call so one can throw
an error in the cases where it actually is an error. D doesn't even
require listing the exceptions that may be thrown on the API (and no,
I'm not suggesting it should).

Post by Jeremy Powers via Digitalmars-d
If you call a method with bad input, and fail to catch the resulting
exception, then _that_ is a bug, not the method throwing. It may be
perfectly recoverable to ignore/retry/whatever, or it may be a symptom
of something that should abort the program. But the method throwing
does not know or care.

Sure, but it doesn't happen. Just like people do not check return values
from syscalls.

The benefit of the exception printing is at least you get a trace of
where things went wrong when you didn't expect them to. Ignoring a
call's return value doesn't give any notion something is wrong until
much later.

-Steve

[Jeremy Powers via Digitalmars-d]

On Mon, Sep 29, 2014 at 11:58 AM, Steven Schveighoffer via Digitalmars-d <

Post by Steven Schveighoffer via Digitalmars-d

Post by Jeremy Powers via Digitalmars-d
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d
My entire point is, it doesn't matter what is expected or what is
treated as "correct." what matters is where the input CAME from. To
a library function, it has no idea. There is no extra type info
saying "this parameter comes from user input."
From the method's view, parameters passed in are user input. Full stop.

This is missing the point of an exception. An uncaught exception is an
error which crashes the program. If you catch the exception, you can handle
it, but if you don't expect it, then it's a bug. Any uncaught exceptions
are BY DEFINITION programming errors.

I agree (except the bit about missing the point). The point I wanted to
make was that encapsulation means what is a fatal error to one part of a
program may be easily handled by the containing part. Just because an
exception is thrown somewhere does not mean the program is broken - it is
the failure to handle the exception (explicit or inadvertent) that
indicates an error.


What is being discussed here is removing the stack trace and printout when

Post by Steven Schveighoffer via Digitalmars-d
an exception is thrown.
....

Sure, but it doesn't happen. Just like people do not check return values

Post by Steven Schveighoffer via Digitalmars-d
from syscalls.
The benefit of the exception printing is at least you get a trace of where
things went wrong when you didn't expect them to. Ignoring a call's return
value doesn't give any notion something is wrong until much later.
-Steve

I absolutely do not want a removal of stack trace information. If an
uncaught exception bubbles up and terminates the program, this is a bug and
I sure as hell want to know as much about it as possible. If having such
information presented to the end user is unacceptable, then wrap and spew
something better.

Ignoring an error return value is like ignoring an exception - bad news,
and indicative of a broken program.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/c385d0b3/attachment.html>

[Bruno Medeiros via Digitalmars-d]

Any uncaught exceptions are BY DEFINITION programming errors.

Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.

[Steven Schveighoffer via Digitalmars-d]

Post by Bruno Medeiros via Digitalmars-d

Any uncaught exceptions are BY DEFINITION programming errors.

Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.

No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.

-Steve

[Bruno Medeiros via Digitalmars-d]

Post by Steven Schveighoffer via Digitalmars-d

Post by Bruno Medeiros via Digitalmars-d

Any uncaught exceptions are BY DEFINITION programming errors.

Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.

No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.
-Steve

Well, at the very least it's bad UI design for sure (textual UI is still
UI).
But it's only a *bug* if it's not the behavior the programmer intended.

[Steven Schveighoffer via Digitalmars-d]

Post by Bruno Medeiros via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d

Post by Bruno Medeiros via Digitalmars-d

Any uncaught exceptions are BY DEFINITION programming errors.

Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.

No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.
-Steve

Well, at the very least it's bad UI design for sure (textual UI is still
UI).
But it's only a *bug* if it's not the behavior the programmer intended.

Sure, one could also halt a program by reading a null pointer on
purpose. This is a grey area that I think reasonable people can
correctly call a bug if they so wish, despite the intentions of the
developer.

-Steve

[Andrej Mitrovic via Digitalmars-d]

On 10/1/14, Steven Schveighoffer via Digitalmars-d

Post by Steven Schveighoffer via Digitalmars-d
No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.

So when they file a bug report are you going to also ask them to run
the debugger so they capture the stack trace and file that to you?
Come on.

[Sean Kelly via Digitalmars-d]

On Monday, 29 September 2014 at 18:59:59 UTC, Jeremy Powers via

Post by Jeremy Powers via Digitalmars-d
And this is an argument for checked exceptions - being able to
explicitly state 'these are known fatal cases for this
component, you should deal with them appropriately' when
defining a method. Cuts down the catch/check to just the
common cases, and makes such cases explicit to the caller.
Anything not a checked exception falls into the 'error, abort!'
path. (Memory corruption etc. being abort scenarios)

Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.

[Jeremy Powers via Digitalmars-d]

On Mon, Sep 29, 2014 at 12:28 PM, Sean Kelly via Digitalmars-d <

Post by Sean Kelly via Digitalmars-d
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.

I've heard this before, but have not seen a reasonable argument as to why
they are a failure. Last time this was discussed a link to a blog was
provided, with lots of discussion there - which as far as I could tell
boiled down to 'catching exceptions is ugly, and people just do the wrong
thing anyway which is ugly when you have checked exceptions.'

I am unlucky enough to write Java all day, and from my standpoint checked
exceptions are a huge win. There are certain edges which can catch you,
but they are immensely useful in developing robust programs. Basically
checked exceptions -> recoverable problems, unchecked ->
unrecoverable/programming errors (like asserts or memory errors).

Note I am not advocating adding checked exceptions to D (though I would
like it). Point is to acknowledge that there are different kinds of
exceptions, and an exception for one part of the code may not be a problem
for the bit that invokes it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/8c59e583/attachment-0001.html>

[Steven Schveighoffer via Digitalmars-d]

Post by Jeremy Powers via Digitalmars-d
On Mon, Sep 29, 2014 at 12:28 PM, Sean Kelly via Digitalmars-d
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.
I've heard this before, but have not seen a reasonable argument as to
why they are a failure. Last time this was discussed a link to a blog
was provided, with lots of discussion there - which as far as I could
tell boiled down to 'catching exceptions is ugly, and people just do the
wrong thing anyway which is ugly when you have checked exceptions.'
I am unlucky enough to write Java all day, and from my standpoint
checked exceptions are a huge win. There are certain edges which can
catch you, but they are immensely useful in developing robust programs.
Basically checked exceptions -> recoverable problems, unchecked ->
unrecoverable/programming errors (like asserts or memory errors).

Well, the failure comes from the effort to effect a certain behavior.

Sun was looking to make programmers more diligent about handling errors.
However, humans are lazy worthless creatures. What ends up happening is,
the compiler complains they aren't handling an exception. They can't see
any reason why the exception would occur, so they simply catch and
ignore it to shut the compiler up.

In 90% of cases, they are right -- the exception will not occur. But
because they have been "trained" to simply discard exceptions, it ends
up defeating the purpose for the 10% of the time that they are wrong.

If you have been able to resist that temptation and handle every
exception, then I think you are in the minority. But I have no evidence
to back this up, it's just a belief.

Post by Jeremy Powers via Digitalmars-d
Note I am not advocating adding checked exceptions to D (though I would
like it). Point is to acknowledge that there are different kinds of
exceptions, and an exception for one part of the code may not be a
problem for the bit that invokes it.

I think this is appropriate for a lint tool for those out there like
yourself who want that information. But requiring checked exceptions is
I think a futile attempt to outlaw natural human behavior.

-Steve

[Jeremy Powers via Digitalmars-d]

On Tue, Sep 30, 2014 at 5:43 AM, Steven Schveighoffer via Digitalmars-d <

Post by Steven Schveighoffer via Digitalmars-d
...
Well, the failure comes from the effort to effect a certain behavior.
Sun was looking to make programmers more diligent about handling errors.
However, humans are lazy worthless creatures. What ends up happening is,
the compiler complains they aren't handling an exception. They can't see
any reason why the exception would occur, so they simply catch and ignore
it to shut the compiler up.
In 90% of cases, they are right -- the exception will not occur. But
because they have been "trained" to simply discard exceptions, it ends up
defeating the purpose for the 10% of the time that they are wrong.

That's the argument, but it doesn't seem valid to me. Without checked
exceptions, you will always be ignoring exceptions. With checked
exceptions, you have to explicitly ignore (some) exceptions, and when you
do it is immediately obvious in the code. You go from everyone ignoring
exceptions all the time, to some people ignoring them - and being able to
easily notice and call out such.

Anyone 'trained' to ignore checked exceptions are simply shooting
themselves in the foot - same as if there were no checked exceptions, but
with more verbosity. This is not a failure of checked exceptions, but a
failure of people to use a language feature properly. (Which, yeah, meta is
a failure of the feature... not going to go there)

Post by Steven Schveighoffer via Digitalmars-d
If you have been able to resist that temptation and handle every
exception, then I think you are in the minority. But I have no evidence to
back this up, it's just a belief.

In my world of professional java, ignoring exceptions is an immediate,
obvious indicator of bad code. You will be called on it, and chastised
appropriately. So from my standpoint, Sun was successful in making
programmers more diligent about handling errors.

Post by Steven Schveighoffer via Digitalmars-d
Note I am not advocating adding checked exceptions to D (though I would

Post by Jeremy Powers via Digitalmars-d
like it). Point is to acknowledge that there are different kinds of
exceptions, and an exception for one part of the code may not be a
problem for the bit that invokes it.

I think this is appropriate for a lint tool for those out there like
yourself who want that information. But requiring checked exceptions is I
think a futile attempt to outlaw natural human behavior.

Perhaps I shouldn't have mentioned checked exceptions at all, seem to be
distracting from what I wanted to say. The important bit I wanted to bring
to the discussion is that not all exceptions are the same, and different
sections of code have their own ideas of what is a breaking problem. A
module/library/component/whatever treats any input into itself as its
input, and thus appropriately throws exceptions on bad input. But code
using that whatever may be perfectly fine handling exceptions coming from
there.

Exceptions need to be appropriate to the given abstraction, and dealt with
by the user of that abstraction.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140930/5271274d/attachment.html>

[Bruno Medeiros via Digitalmars-d]

Post by Sean Kelly via Digitalmars-d
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.

That is the conventional theory, the established wisdom.
But the more I become experienced with Java, over the years, I've become
convinced otherwise.

What has failed is not the concept of checked exceptions per se, but
mostly, the failure of Java programmers to use checked exceptions
effectively, and properly design their code around this paradigm.

Like Jeremy mentioned, if one puts catch blocks right around the
function that throws an exception, and just swallow/forget it there
without doing anything else, then it's totally the programmers fault for
being lazy.

If one is annoyed that often, adding a throws clause in a function will
require adding the same throws clause function to several other
functions, well, that is editing work you have to accept for the sake of
more correctness. But also one should understand there are ways to
mitigate this editing work:

First point is that in a lot of code, is better to have a function throw
just one generic (but checked) exception, that can wrap any other
specific errors/exceptions. If you are doing an operation that can throw
File-Not-Found, Invalid-Path, No-Permissions, IO-Exception, etc., then
often all of these will be handled in the same user-reporting code, so
they could be wrapped under a single exception that would be used in the
throws clause. And so the whole function call chain doesn't need to be
modified every time a new exception is added or removed.

If you're thinking that means adding a "throws Exception" to such
functions in Java, then no. Because this will catch RuntimeExceptions
too (the unchecked exceptions of Java), and these you often want to
handle elsewhere than where you handle the checked exceptions. In this
regard, Java does have a design fault, IMO, which is that there is no
common superclass for checked Exceptions. (there is only for unchecked
exceptions)

The second point, is that even adding (or modifying) the throws clause
of function signatures cause be made much easier with an IDE, and in
particular Eclipse JDT helps a lot. If you have an error in the editor
about a checked exception that is not caught or thrown, you can just
press Ctrl-1 to automatically add either a throws clause, or a
surrounding try-catch block.

[ketmar via Digitalmars-d]

On Sat, 27 Sep 2014 16:15:40 -0700

After all, what would you think of a compiler that spewed out

dmd test.d

test.d(15) Error: missing } thrown from dmd/src/parse.c(283)
?

"wow, that's cool! one more pleasing feature!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140928/e12d80ff/attachment.sig>

[luka8088 via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
http://youtu.be/hBhlQgvHmQ0
In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
assert()s and contracts are for debugging programs.

dmd test.d

test.d(15) Error: missing } thrown from dmd/src/parse.c(283)
?
https://issues.dlang.org/show_bug.cgi?id=13543
As for the programmer wanting to know where the message "missing }" came from,
li
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really isn't a problem.

We has this issue at work (we are working with php). We outputted a
stack trace for both exceptions and asserts but the lines that should be
addressed are not always so obvious.

I found a solution and it works great for us. All library code is marked
appropriately so when stack is outputted it is shadows out (with gray
color) all the lines in library code and point out first non-library
line from the top of the stack. In 95% of the time it is the line that
the programmer should look into. Other 5% is the time when it shows the
line where programmer is forwarding a call to the library but turns out
to be ok as it turns out to be much more comprehensible than the entire
stack. One note worth mentioning is that juniors have much easier time
understanding which lines concern them, and from that I can only
conclude that such approach is more intuitive.

Marking is done on namespace level so it can be easily disabled for
entire namespace.

I think outputting a stack trace for asserts is a must because of that
5%. And for exceptions I agree completely with your arguments and I
think that there is no need for stack.

From my experience this has been a good approach and I think is worth
considering.

[bearophile via Digitalmars-d]

All library code is marked appropriately so when stack is
outputted it is shadows out (with gray color) all the lines in
library code and point out first non-library line from the top
of the stack. In 95% of the time it is the line that the
programmer should look into. Other 5% is the time when it shows
the line where programmer is forwarding a call to the library
but turns out to be ok as it turns out to be much more
comprehensible than the entire stack. One note worth mentioning
is that juniors have much easier time understanding which lines
concern them, and from that I can only conclude that such
approach is more intuitive.

This looks like a little enhancement request, to colourize
differently the parts of the stack trace of the library
code/runtime from the user code.

And for exceptions I agree completely with your arguments and I
think that there is no need for stack.

I think Walter is not suggesting to remove the stack trace for
exceptions.

Bye,
bearophile

[Walter Bright via Digitalmars-d]

And for exceptions I agree completely with your arguments and I think that
there is no need for stack.

I think Walter is not suggesting to remove the stack trace for exceptions.

I suggest removal of stack trace for exceptions, but leaving them in for asserts.

Asserts are a deliberately designed debugging tool. Exceptions are not.

[Andrei Alexandrescu via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

And for exceptions I agree completely with your arguments and I think that
there is no need for stack.

I think Walter is not suggesting to remove the stack trace for exceptions.

I suggest removal of stack trace for exceptions, but leaving them in for asserts.
Asserts are a deliberately designed debugging tool. Exceptions are not.

I'm fine with that philosophy, with the note that it's customary
nowadays to inflict things like the stack trace on the user. -- Andrei

[bearophile via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving
them in for asserts.

I suggest to keep stack trace for both cases, and improve it with
colors :-) Another possibility is to keep the stack trace for
exceptions in nonrelease mode only.

Post by Walter Bright via Digitalmars-d
Asserts are a deliberately designed debugging tool. Exceptions
are not.

Exceptions are often used to help debugging... We have even
allowed exceptions inside D contracts (but I don't know why).

Bye,
bearophile

[Walter Bright via Digitalmars-d]

Post by bearophile via Digitalmars-d
Exceptions are often used to help debugging...

http://youtu.be/hBhlQgvHmQ0

[luka8088 via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

Post by bearophile via Digitalmars-d
Exceptions are often used to help debugging...

http://youtu.be/hBhlQgvHmQ0

Example exception messages:

Unable to connect to database
Invalid argument count
Invalid network package format

All this messages do not require a stack trace as they do not require
code fixes, they indicate an issue outside the program itself. If stack
trace is required then assert should have been used instead.

Or to better put it: can anyone give an example of exception that would
require stack trace?

[Jacob Carlborg via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving them in for asserts.

If you don't like the stack track, just wrap the "main" function in a
try-catch block, catch all exceptions and print the error message.

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving them in for asserts.

If you don't like the stack track, just wrap the "main" function in a try-catch
block, catch all exceptions and print the error message.

That's what the runtime that calls main() is supposed to do.

[Marco Leise via Digitalmars-d]

Am Sun, 28 Sep 2014 13:14:43 -0700

Post by Walter Bright via Digitalmars-d

Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving them in for asserts.

If you don't like the stack track, just wrap the "main" function in a try-catch
block, catch all exceptions and print the error message.

That's what the runtime that calls main() is supposed to do.

Guys, a druntime flag could settle matters in 10 minutes.
But this topic is clearly about the right school of thought.

I use contracts to check for logical errors, like when an
argument must not be null or a value less than the length of
some data structure.
I use exceptions to check for invalid input and the return
values of external libraries. External libraries can be
anything from my own code in the same project to OpenGL from
vendor XY. They could error out on valid input (if we leave
out of memory aside for now), because of bugs or incorrect
assumptions of the implementation.
If that happens and all I get is:
"Library XY Exception: code 0x13533939 (Invalid argument)."
I'm at a loss, where the library might have had a hickup.
Did some function internally handle a uint as an int and
wrapped around?
Maybe with std.logger we will see single line messages on the
terminal and multi-line exception traces in the logs (which by
default print to stderr as well). And then this discussion can
be resolved.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

And for exceptions I agree completely with your arguments and I think that
there is no need for stack.

I think Walter is not suggesting to remove the stack trace for exceptions.

I suggest removal of stack trace for exceptions, but leaving
them in for asserts.
Asserts are a deliberately designed debugging tool. Exceptions
are not.

Fair. So we generate traces for Errors but not Exceptions.

[Ary Borenszweig via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
http://youtu.be/hBhlQgvHmQ0
In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
assert()s and contracts are for debugging programs.

For me, assert is useless.

We are developing a language using LLVM as our backend. If you give LLVM
something it doesn't like, you get something this:

~~~
Assertion failed: (S1->getType() == S2->getType() && "Cannot create
binary operator with two operands of differing type!"), function Create,
file Instructions.cpp, line 1850.

Abort trap: 6
~~~

That is what the user gets when there is a bug in the compiler, at least
when we are generating invalid LLVM code. And that's one of the good
paths, if you compiled LLVM with assertions, because otherwise I guess
it's undefined behaviour.

What I'd like to do, as a compiler, is to catch those errors and tell
the user: "You've found a bug in the app, could you please report it in
this URL? Thank you.". We can't: the assert is there and we can't change it.

Now, this is when you interface with C++/C code. But inside our language
code we always use exceptions so that programmers can choose what to do
in case of an error. With assert you loose that possibility.

Raising an exception is costly, but that should happen in exceptional
cases. Installing an exception handler is cost-free, so I don't see why
there is a need for a less powerful construct like assert.

[Xiao Xie via Digitalmars-d]

On Sunday, 28 September 2014 at 15:10:26 UTC, Ary Borenszweig

Post by Ary Borenszweig via Digitalmars-d
What I'd like to do, as a compiler, is to catch those errors
and tell the user: "You've found a bug in the app, could you
please report it in this URL? Thank you.". We can't: the assert
is there and we can't change it.

Why is SIGABRT handler not working for your usecase? print and
exit?

[Walter Bright via Digitalmars-d]

Post by Ary Borenszweig via Digitalmars-d
For me, assert is useless.
We are developing a language using LLVM as our backend. If you give LLVM
~~~
Assertion failed: (S1->getType() == S2->getType() && "Cannot create binary
operator with two operands of differing type!"), function Create, file
Instructions.cpp, line 1850.
Abort trap: 6
~~~
That is what the user gets when there is a bug in the compiler, at least when we
are generating invalid LLVM code. And that's one of the good paths, if you
compiled LLVM with assertions, because otherwise I guess it's undefined behaviour.
"You've found a bug in the app, could you please report it in this URL? Thank
you.". We can't: the assert is there and we can't change it.

You can hook D's assert and do what you want with it.

Post by Ary Borenszweig via Digitalmars-d
Now, this is when you interface with C++/C code. But inside our language code we
always use exceptions so that programmers can choose what to do in case of an
error. With assert you loose that possibility.

If you want to use Exceptions for debugging in your code, I won't try and stop
you. But using them for debugging in official Phobos I strongly object to.

Post by Ary Borenszweig via Digitalmars-d
Installing an exception handler is cost-free,

Take a look at the assembler dump from std.file.copy() that I posted in the
other thread.

Post by Ary Borenszweig via Digitalmars-d
so I don't see why there is a need
for a less powerful construct like assert.

Exceptions are meant for RECOVERABLE errors. If you're using them instead of
assert for logic bugs, you're looking at undefined behavior. Logic bugs are not
recoverable.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
You can hook D's assert and do what you want with it.

With the caveat that you must finish by either exiting the app or
throwing an exception, since the compiler doesn't generate a
stack frame that can be returned from.

Post by Walter Bright via Digitalmars-d
Exceptions are meant for RECOVERABLE errors. If you're using
them instead of assert for logic bugs, you're looking at
undefined behavior. Logic bugs are not recoverable.

In a multithreaded program, does this mean that the thread must
be terminated or the entire process? In a multi-user system,
does this mean the transaction or the entire process? The scope
of a logic bug can be known to be quite limited. Remember my
earlier point about Erlang, where a "process" there is actually
just a logical thread in the VM.

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
Exceptions are meant for RECOVERABLE errors. If you're using them instead of
assert for logic bugs, you're looking at undefined behavior. Logic bugs are
not recoverable.

In a multithreaded program, does this mean that the thread must be terminated or
the entire process? In a multi-user system, does this mean the transaction or
the entire process? The scope of a logic bug can be known to be quite limited.
Remember my earlier point about Erlang, where a "process" there is actually just
a logical thread in the VM.

This has been asked many times before.

If the threads share memory, the only robust choice is to terminate all the
threads and the application.

If the thread is in another process, where the memory is not shared, then
terminating and possibly restarting that process is quite acceptable.

The scope of a logic bug can be known to be quite limited.

If you know about the bug, then you'd have fixed it already instead of inserting
recovery code for unknown problems. I can't really accept that one has "unknown
bugs of known scope".

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
If the threads share memory, the only robust choice is to
terminate all the threads and the application.
If the thread is in another process, where the memory is not
shared, then terminating and possibly restarting that process
is quite acceptable.

The scope of a logic bug can be known to be quite limited.

If you know about the bug, then you'd have fixed it already
instead of inserting recovery code for unknown problems. I
can't really accept that one has "unknown bugs of known scope".

Well, say you're using SafeD or some other system where you know
that memory corruption is not possible (pure functional
programming, for example). In this case, if you know what data a
particular execution flow touches, you know the scope of the
potential damage. And if the data touched is all either shared
but read-only or generated during the processing of the request,
you can be reasonably certain that nothing outside the scope of
the transaction has been adversely affected at all.

[Dmitry Olshansky via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
If the threads share memory, the only robust choice is to terminate
all the threads and the application.
If the thread is in another process, where the memory is not shared,
then terminating and possibly restarting that process is quite
acceptable.

The scope of a logic bug can be known to be quite limited.

If you know about the bug, then you'd have fixed it already instead of
inserting recovery code for unknown problems. I can't really accept
that one has "unknown bugs of known scope".

Well, say you're using SafeD or some other system where you know that
memory corruption is not possible (pure functional programming, for
example).
In this case, if you know what data a particular execution
flow touches, you know the scope of the potential damage. And if the
data touched is all either shared but read-only or generated during the
processing of the request, you can be reasonably certain that nothing
outside the scope of the transaction has been adversely affected at all.

not possible / highly unlikely (i.e. bug in VM or said system)

But otherwise agreed, dropping the whole process is not always a good
idea or it easily becomes a DoS attack vector in a public service.

[Sean Kelly via Digitalmars-d]

On Sunday, 28 September 2014 at 21:16:51 UTC, Dmitry Olshansky

Post by Dmitry Olshansky via Digitalmars-d
But otherwise agreed, dropping the whole process is not always
a good idea or it easily becomes a DoS attack vector in a
public service.

What I really want to work towards is the Erlang model where an
app is a web of communicating processes (though Erlang processes
are effectively equivalent to D objects). Then, killing a
process on an error is absolutely correct. It doesn't affect the
resilience of the system. But if these processes are actually
threads or fibers with memory protection, things get a lot more
complicated. I really need to spend some time investigating how
modern Linux systems handle tons of processes running on them and
try to find a happy medium.

[Dmitry Olshansky via Digitalmars-d]

Post by Dmitry Olshansky via Digitalmars-d
But otherwise agreed, dropping the whole process is not always a good
idea or it easily becomes a DoS attack vector in a public service.

What I really want to work towards is the Erlang model where an app is a
web of communicating processes (though Erlang processes are effectively
equivalent to D objects). Then, killing a process on an error is
absolutely correct. It doesn't affect the resilience of the system.
But if these processes are actually threads or fibers with memory
protection, things get a lot more complicated.

One thing I really appreciated about JVM is exactly the memory safety
with ability to handle this pretty much in the same way Erlang does.

I really need to spend
some time investigating how modern Linux systems handle tons of
processes running on them and try to find a happy medium.

Keep us posted.

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d

The scope of a logic bug can be known to be quite limited.

If you know about the bug, then you'd have fixed it already instead of
inserting recovery code for unknown problems. I can't really accept that one
has "unknown bugs of known scope".

Well, say you're using SafeD or some other system where you know that memory
corruption is not possible (pure functional programming, for example). In this
case, if you know what data a particular execution flow touches, you know the
scope of the potential damage. And if the data touched is all either shared but
read-only or generated during the processing of the request, you can be
reasonably certain that nothing outside the scope of the transaction has been
adversely affected at all.

You may know the error is not a memory corrupting one, but that doesn't mean
there aren't non-corrupting changes to the shared memory that would result in
additional unexpected failures. Also, the logic bug may be the result of an
@system part of the code going wrong. You do not know, because YOU DO NOT KNOW
the cause the error. And if you knew the cause, you wouldn't need a stack trace
to debug it anyway.

I.e. despite being 'safe' it does not imply the program is in a predictable or
anticipated state.

I can't get behind the notion of "reasonably certain". I certainly would not use
such techniques in any code that needs to be robust, and we should not be using
such cowboy techniques in Phobos nor officially advocate their use.

[Sean Kelly via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I can't get behind the notion of "reasonably certain". I
certainly would not use such techniques in any code that needs
to be robust, and we should not be using such cowboy techniques
in Phobos nor officially advocate their use.

I think it's a fair stance not to advocate this approach. But as
it is I spend a good portion of my time diagnosing bugs in
production systems based entirely on archived log data, and
analyzing the potential impact on the system to determine the
importance of a hot fix. The industry seems to be moving towards
lowering the barrier between engineering and production code
(look at what Netflix has done for example), and some of this
comes from an isolation model akin to the Erlang approach, but
the typical case is still that hot fixing code is incredibly
expensive and so you don't want to do it if it isn't necessary.
For me, the correct approach may simply be to eschew assert() in
favor of enforce() in some cases. But the direction I want to be
headed is the one you're encouraging. I simply don't know if
it's practical from a performance perspective. This is still
developing territory.

[Walter Bright via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I can't get behind the notion of "reasonably certain". I certainly would not
use such techniques in any code that needs to be robust, and we should not be
using such cowboy techniques in Phobos nor officially advocate their use.

I think it's a fair stance not to advocate this approach. But as it is I spend
a good portion of my time diagnosing bugs in production systems based entirely
on archived log data, and analyzing the potential impact on the system to
determine the importance of a hot fix. The industry seems to be moving towards
lowering the barrier between engineering and production code (look at what
Netflix has done for example), and some of this comes from an isolation model
akin to the Erlang approach, but the typical case is still that hot fixing code
is incredibly expensive and so you don't want to do it if it isn't necessary.
For me, the correct approach may simply be to eschew assert() in favor of
enforce() in some cases. But the direction I want to be headed is the one
you're encouraging. I simply don't know if it's practical from a performance
perspective. This is still developing territory.

You've clearly got a tough job to do, and I understand you're doing the best you
can with it. I know I'm hardcore and uncompromising on this issue, but that's
where I came from (the aviation industry).

I know what works (airplanes are incredibly safe) and what doesn't work
(Toyota's approach was in the news not too long ago). Deepwater Horizon and
Fukushima are also prime examples of not dealing properly with modest failures
that cascaded into disaster.

[Bruno Medeiros via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
I can't get behind the notion of "reasonably certain". I certainly would
not use such techniques in any code that needs to be robust,
I know I'm hardcore and uncompromising on this issue, but that's where I
came from (the aviation industry).

Walter, you do understand that not all software has to be robust - in
the critical systems sense - to be quality software? And that in fact,
the majority of software is not critical systems software?...

I was under the impression that D was meant to be a general purpose
language, not a language just for critical systems. Yet, on language
design issues, you keep making a series or arguments and points that
apply *only* to critical systems software.

[Bruno Medeiros via Digitalmars-d]

Post by Walter Bright via Digitalmars-d
http://youtu.be/hBhlQgvHmQ0
In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.

This is incorrect.

Yes, the primary purpose of Exceptions is not for debugging, but to
report exceptional state to the user (or some other component of the
system).

But they also have a purpose for debugging, particularly the stack
traces of exceptions. Take what you said:

"Failure to respond properly to an input/environmental error is a bug.
But the input/environmental error is not a bug. If it was, then the
program should assert on the error, not throw. "

So, some component/function Foo detects an environmental error, and
throws and Exception, accordingly. Foo is not responsible for handling
these errors, but some other component is.

Component/function Bar is the one that should handle such an error (for
example, it should display a dialog to the user, and continue the
application). But due to a bug, it doesn't do so, and the Exception goes
all the way through main().

The programmer notices this happening, and clearly recognizes it's a bug
(but doesn't know where the bug is, doesn't know that it's Bar that
should be handling it). Now, what is best, to just have the Exception
message (something like "File not found") displayed to the programmer -
or even an end-user that could report a bug -, or to have the stack
trace of the Exception so that the programmer can more easily look at
which function should be handling it?