Discussion:
Program logic bugs vs input/environmental errors
(too old to reply)
Walter Bright via Digitalmars-d
2014-09-27 23:15:40 UTC
Permalink
This issue comes up over and over, in various guises. I feel like Yosemite Sam here:

https://www.youtube.com/watch?v=hBhlQgvHmQ0

In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.

When I say "They are NOT for debugging programs", I mean they are NOT for
debugging programs.

assert()s and contracts are for debugging programs.
dmd test.d
test.d(15) Error: missing } thrown from dmd/src/parse.c(283)

?

See:

https://issues.dlang.org/show_bug.cgi?id=13543

As for the programmer wanting to know where the message "missing }" came from,

grep -r dmd/src/*.c "missing }"

works nicely. I do that sort of thing all the time. It really isn't a problem.
bearophile via Digitalmars-d
2014-09-27 23:33:50 UTC
Permalink
Post by Walter Bright via Digitalmars-d
As for the programmer wanting to know where the message
"missing }" came from,
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really
isn't a problem.
grep is not useful for the purposes explained in issue 13543
because the file name is often inside a string variable, that is
initialized elsewhere or generated in some way. So the exception
is useful to know where's the instruction in user code that has
tried the failed I/O action, as I've explained in that issue.

Bye,
bearophile
Walter Bright via Digitalmars-d
2014-09-27 23:42:18 UTC
Permalink
Post by Walter Bright via Digitalmars-d
As for the programmer wanting to know where the message "missing }" came from,
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really isn't a problem.
grep is not useful for the purposes explained in issue 13543 because the file
name is often inside a string variable, that is initialized elsewhere or
generated in some way. So the exception is useful to know where's the
instruction in user code that has tried the failed I/O action, as I've explained
in that issue.
Even if that is what you wanted, you won't get that from FileException, as it
will only show file/lines emanating from calls inside std.file, not from higher
level callers.

Besides, take a bit of care when formulating a string for exceptions, and you
won't have any trouble grepping for it. This isn't rocket science.

Presenting internal debugging data to users for input/environmental errors is
just bad programming practice. We shouldn't be enshrining it in Phobos and
presenting it as a professional way to code.
H. S. Teoh via Digitalmars-d
2014-09-27 23:55:53 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Walter Bright via Digitalmars-d
As for the programmer wanting to know where the message "missing }" came from,
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really isn't a problem.
grep is not useful for the purposes explained in issue 13543 because
the file name is often inside a string variable, that is initialized
elsewhere or generated in some way. So the exception is useful to
know where's the instruction in user code that has tried the failed
I/O action, as I've explained in that issue.
Even if that is what you wanted, you won't get that from
FileException, as it will only show file/lines emanating from calls
inside std.file, not from higher level callers.
Besides, take a bit of care when formulating a string for exceptions,
and you won't have any trouble grepping for it. This isn't rocket
science.
Presenting internal debugging data to users for input/environmental
errors is just bad programming practice. We shouldn't be enshrining it
in Phobos and presenting it as a professional way to code.
My take on this, is that uncaught exceptions are a program bug. Any
messages displayed to the user ought to come from a catch block that not
only prints the exception message (*without* things like line numbers
and stack traces, btw), but also provides context (e.g., "Error in
configuration file section 'abc': illegal field value" instead of just
"illegal field value" with no context of where it might have been
triggered).

Uncaught exceptions (which ideally should only be Errors, not
Exceptions) are a program bug that ought to be fixed. In the case that
somehow one managed to elude your catch blocks, the full debug infodump
(source file, line number, stack trace) is useful for users to hand back
to you in a bug report, so that you can track down the problem. The user
should not be expected to understand the infodump from an uncaught
exception, whereas a message printed from a catch block ought to be
user-understandable (like "can't open 'myphoto.jpg': file not found",
not "internal error on line 12345" which makes no sense to a user).


T
--
Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them.
bearophile via Digitalmars-d
2014-09-27 23:59:43 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Even if that is what you wanted, you won't get that from
FileException, as it will only show file/lines emanating from
calls inside std.file, not from higher level callers.
Can't we use the template arguments like __LINE__ to offer the
line of code in the IO function in user code?
Post by Walter Bright via Digitalmars-d
Besides, take a bit of care when formulating a string for
exceptions, and you won't have any trouble grepping for it.
This isn't rocket science.
The exception is generated inside library code, not in user code.
There's nothing to grep in the user code. The D script often
looks like this:

void main() {
import std.stdio;
auto file_name = "some_file;
// some code here
auto file_name1 = file_name ~ "1.txt";
auto f1 = File(file_name1);
// some code here
auto file_name2 = file_name ~ "2.txt";
auto f2 = File(file_name2, "w");
// some code here
}
Post by Walter Bright via Digitalmars-d
Presenting internal debugging data to users for
input/environmental errors is just bad programming practice.7
We shouldn't be enshrining it in Phobos and presenting it as a
professional way to code.
The file/line of code is not meant to replace serious practices
for dealing with I/O failures in serious D programs.

Bye,
bearophile
Walter Bright via Digitalmars-d
2014-09-28 00:54:09 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Even if that is what you wanted, you won't get that from FileException, as it
will only show file/lines emanating from calls inside std.file, not from
higher level callers.
Can't we use the template arguments like __LINE__ to offer the line of code in
the IO function in user code?
We could, but again, WHOA CAMEL!

Also, one should be careful in using __LINE__ in a template, as it will be
forced to generate a new instantiation with every use.
Post by Walter Bright via Digitalmars-d
Besides, take a bit of care when formulating a string for exceptions, and you
won't have any trouble grepping for it. This isn't rocket science.
The exception is generated inside library code, not in user code. There's
void main() {
import std.stdio;
auto file_name = "some_file;
// some code here
auto file_name1 = file_name ~ "1.txt";
auto f1 = File(file_name1);
// some code here
auto file_name2 = file_name ~ "2.txt";
auto f2 = File(file_name2, "w");
// some code here
}
Come on. What is hard about "Cannot open file `xxxx` in mode `xxxx`", then you
grep for "Cannot open file" ?
grep "Cannot open file" *.d
stdio.d: text("Cannot open file `", name, "' in mode `",
stdio.d:// new StdioException("Cannot open file `"~fName~"' for reading"));
stdio.d:// new StdioException("Cannot open file `"~fName~"' for reading"));

Besides, File doesn't throw a FileException and doesn't add caller's __FILE__
and __LINE__.

If you're seriously proposing adding __FILE__ and __LINE__ to every function
call, then D is the wrong language. This would have disastrous bloat and
performance problems. You mentioned Python and Ruby, both of which are well
known for miserable performance.

Building the equivalent of a symbolic debugger into RELEASED D code is just not
acceptable.
Post by Walter Bright via Digitalmars-d
Presenting internal debugging data to users for input/environmental errors is
just bad programming practice.7 We shouldn't be enshrining it in Phobos and
presenting it as a professional way to code.
The file/line of code is not meant to replace serious practices for dealing with
I/O failures in serious D programs.
That's exactly what you're doing with it.
Walter Bright via Digitalmars-d
2014-09-28 00:40:22 UTC
Permalink
Post by H. S. Teoh via Digitalmars-d
My take on this, is that uncaught exceptions are a program bug.
Not to me. Custom messages would be better, but the exception message should be
serviceable.
Post by H. S. Teoh via Digitalmars-d
Uncaught exceptions (which ideally should only be Errors, not
Exceptions) are a program bug that ought to be fixed. In the case that
somehow one managed to elude your catch blocks, the full debug infodump
(source file, line number, stack trace) is useful for users to hand back
to you in a bug report, so that you can track down the problem. The user
should not be expected to understand the infodump from an uncaught
exception, whereas a message printed from a catch block ought to be
user-understandable (like "can't open 'myphoto.jpg': file not found",
not "internal error on line 12345" which makes no sense to a user).
Whoa, Camel! You're again thinking of Exceptions as a debugging tool.
Timon Gehr via Digitalmars-d
2014-09-28 00:54:34 UTC
Permalink
Post by H. S. Teoh via Digitalmars-d
My take on this, is that uncaught exceptions are a program bug.
Not to me. ...
It is not worth fixing if a program terminates with a stack trace?
Walter Bright via Digitalmars-d
2014-09-28 01:16:50 UTC
Permalink
Post by Timon Gehr via Digitalmars-d
Post by H. S. Teoh via Digitalmars-d
My take on this, is that uncaught exceptions are a program bug.
Not to me. ...
It is not worth fixing if a program terminates with a stack trace?
I never was in favor of adding the stack trace output, either, for the same
reason. Exceptions are not programming bugs.

For Errors, a case can be made for them.
Sean Kelly via Digitalmars-d
2014-09-28 16:16:07 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Whoa, Camel! You're again thinking of Exceptions as a debugging tool.
They can be. What if an API you're using throws an exception you
didn't expect, and therefore don't handle? This might be
considered a logic error if the exception is recoverable and you
don't intend the program to abort from that operation.
Sean Kelly via Digitalmars-d
2014-09-28 16:23:40 UTC
Permalink
On Sunday, 28 September 2014 at 00:40:26 UTC, Walter Bright
Post by Walter Bright via Digitalmars-d
Whoa, Camel! You're again thinking of Exceptions as a
debugging tool.
They can be. What if an API you're using throws an exception
you didn't expect, and therefore don't handle? This might be
considered a logic error if the exception is recoverable and
you don't intend the program to abort from that operation.
Also, I think the idea that a program is created and shipped to
an end user is overly simplistic. In the server/cloud
programming world, when an error occurs, the client who submitted
the request will get a response appropriate for them and the
system will also generate log information intended for people
working on the system. So things like stack traces and assertion
failure information is useful even for production software. Same
with any critical system, as I'm sure you're aware. The systems
are designed to handle failures in specific ways, but they also
have to leave a breadcrumb trail so the underlying problem can be
diagnosed and fixed. Internal testing is never perfect, and
achieving a high coverage percentage is nearly impossible if the
system wasn't designed from the ground up to be testable in such
a way (mock frameworks and such).
Walter Bright via Digitalmars-d
2014-09-28 17:33:35 UTC
Permalink
Also, I think the idea that a program is created and shipped to an end user is
overly simplistic. In the server/cloud programming world, when an error occurs,
the client who submitted the request will get a response appropriate for them
and the system will also generate log information intended for people working on
the system. So things like stack traces and assertion failure information is
useful even for production software. Same with any critical system, as I'm sure
you're aware. The systems are designed to handle failures in specific ways, but
they also have to leave a breadcrumb trail so the underlying problem can be
diagnosed and fixed. Internal testing is never perfect, and achieving a high
coverage percentage is nearly impossible if the system wasn't designed from the
ground up to be testable in such a way (mock frameworks and such).
Then use assert(). That's just what it's for.
Sean Kelly via Digitalmars-d
2014-09-28 19:33:35 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Sean Kelly via Digitalmars-d
Also, I think the idea that a program is created and shipped
to an end user is overly simplistic. In the server/cloud
programming world, when an error occurs, the client who
submitted the request will get a response appropriate for them
and the system will also generate log information intended for
people working on the system. So things like stack traces and
assertion failure information is useful even for production
software. Same with any critical system, as I'm sure you're
aware. The systems are designed to handle failures in
specific ways, but they also have to leave a breadcrumb trail
so the underlying problem can be diagnosed and fixed.
Internal testing is never perfect, and achieving a high
coverage percentage is nearly impossible if the system wasn't
designed from the ground up to be testable in such a way (mock
frameworks and such).
Then use assert(). That's just what it's for.
What if I don't want to be forced to abort the program in the
event of such an error?
Walter Bright via Digitalmars-d
2014-09-28 20:13:49 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Then use assert(). That's just what it's for.
What if I don't want to be forced to abort the program in the event of such an
error?
Then we are back to the discussion about can a program continue after a logic
error is uncovered, or not.

In any program, the programmer must decide if an error is a bug or not, before
shipping it. Trying to avoid making this decision leads to confusion and using
the wrong techniques to deal with it.

A program bug is, by definition, unknown and unanticipated. The idea that one
can "recover" from it is fundamentally wrong. Of course, in D one can try and
recover from them anyway, but you're on your own trying that, just as you're on
your own when casting integers to pointers.

On the other hand, input/environmental errors must be anticipated and can often
be recovered from. But presenting debug traces to the users for these implies at
the very least a sloppily engineered product, in my not so humble opinion :-)
Joseph Rushton Wakeling via Digitalmars-d
2014-09-28 22:51:40 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Walter Bright via Digitalmars-d
Then use assert(). That's just what it's for.
What if I don't want to be forced to abort the program in the event of such an
error?
Then we are back to the discussion about can a program continue after a logic
error is uncovered, or not.
In any program, the programmer must decide if an error is a bug or not, before
shipping it. Trying to avoid making this decision leads to confusion and using
the wrong techniques to deal with it.
A program bug is, by definition, unknown and unanticipated. The idea that one
can "recover" from it is fundamentally wrong. Of course, in D one can try and
recover from them anyway, but you're on your own trying that, just as you're on
your own when casting integers to pointers.
Allowing for your "you can try ..." remarks, I still feel this doesn't really
cover the practical realities of how some applications need to behave.

Put it this way: suppose we're writing the software for a telephone exchange,
which is handling thousands of simultaneous calls. If an Error is thrown inside
the part of the code handling one single call, is it correct to bring down
everyone else's call too?

I appreciate that you might tell me "You need to find a different means of error
handling that can distinguish errors that are recoverable", but the bottom line
is, in such a scenario it's not possible to completely rule out an Error being
thrown (an obvious cause would be an assert that gets triggered because the
programmer forgot to put a corresponding enforce() statement at a higher level
in the code).

However, it's clearly very desirable in this use-case for the application to
keep going if at all possible and for any problem, even an Error, to be
contained in its local context if we can do so. (By "local context", in
practice this probably means a thread or fiber or some other similar programming
construct.)

Sean's touched on this in the current thread with his reference to Erlang, and I
remember that he and Dicebot brought the issue up in an earlier discussion on
the Error vs. Exception question, but I don't recall that discussion having any
firm conclusion, and I think it's important to address; we can't simply take "An
Error is unrecoverable" as a point of principle for every application.

(Related note: If I recall right, an Error or uncaught Exception thrown within a
thread or fiber will not actually bring the application down, only cause that
thread/fiber to hang, without printing any indication of anything going wrong.
So on a purely practical basis, it can be essential for the top-level code of a
thread or fiber to have a catch {} block for both Errors and Exceptions, just in
order to be able to report what has happened effectively.)
Walter Bright via Digitalmars-d
2014-09-29 00:09:57 UTC
Permalink
Post by Joseph Rushton Wakeling via Digitalmars-d
However, it's clearly very desirable in this use-case for the application to
keep going if at all possible and for any problem, even an Error, to be
contained in its local context if we can do so. (By "local context", in
practice this probably means a thread or fiber or some other similar programming
construct.)
If the program has entered an unknown state, its behavior from then on cannot be
predictable. There's nothing I or D can do about that. D cannot officially
endorse such a practice, though D being a systems programming language it will
let you do what you want.

I would not even consider such a practice for a program that is in charge of
anything that could result in injury, death, property damage, security breaches,
etc.
Sean Kelly via Digitalmars-d
2014-09-29 01:39:55 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Joseph Rushton Wakeling via Digitalmars-d
However, it's clearly very desirable in this use-case for the
application to
keep going if at all possible and for any problem, even an
Error, to be
contained in its local context if we can do so. (By "local
context", in
practice this probably means a thread or fiber or some other
similar programming
construct.)
If the program has entered an unknown state, its behavior from
then on cannot be predictable. There's nothing I or D can do
about that. D cannot officially endorse such a practice, though
D being a systems programming language it will let you do what
you want.
I would not even consider such a practice for a program that is
in charge of anything that could result in injury, death,
property damage, security breaches, etc.
Well... suppose you design a system with redundancy such that an
error in a specific process isn't enough to bring down the
system. Say it's a quorum method or whatever. In the instance
that a process goes crazy, I would argue that the system is in an
undefined state but a state that it's designed specifically to
handle, even if that state can't be explicitly defined at design
time. Now if enough things go wrong at once the whole system
will still fail, but it's about building systems with the
expectation that errors will occur. They may even be logic
errors--I think it's kind of irrelevant at that point.

Even a network of communicating processes, one getting in a bad
state can theoretically poison the entire system and you're often
not in a position to simply shut down the whole thing and wait
for a repairman. And simply rebooting the system if it's a bad
sensor that's causing the problem just means a pause before
another failure cascade. I think any modern program designed to
run continuously (increasingly the typical case) must be designed
with some degree of resiliency or self-healing in mind. And that
means planning for and limiting the scope of undefined behavior.
Walter Bright via Digitalmars-d
2014-09-29 02:57:02 UTC
Permalink
Well... suppose you design a system with redundancy such that an error in a
specific process isn't enough to bring down the system. Say it's a quorum
method or whatever. In the instance that a process goes crazy, I would argue
that the system is in an undefined state but a state that it's designed
specifically to handle, even if that state can't be explicitly defined at design
time. Now if enough things go wrong at once the whole system will still fail,
but it's about building systems with the expectation that errors will occur.
They may even be logic errors--I think it's kind of irrelevant at that point.
Even a network of communicating processes, one getting in a bad state can
theoretically poison the entire system and you're often not in a position to
simply shut down the whole thing and wait for a repairman. And simply rebooting
the system if it's a bad sensor that's causing the problem just means a pause
before another failure cascade. I think any modern program designed to run
continuously (increasingly the typical case) must be designed with some degree
of resiliency or self-healing in mind. And that means planning for and limiting
the scope of undefined behavior.
I've said that processes are different, because the scope of the effects is
limited by the hardware.

If a system with threads that share memory cannot be restarted, there are
serious problems with the design of it, because a crash and the necessary
restart are going to happen sooner or later, probably sooner.

I don't believe that the way to get 6 sigma reliability is by ignoring errors
and hoping. Airplane software is most certainly not done that way.

I recall Toyota got into trouble with their computer controlled cars because of
their idea of handling of inevitable bugs and errors. It was one process that
controlled everything. When something unexpected went wrong, it kept right on
operating, any unknown and unintended consequences be damned.

The way to get reliable systems is to design to accommodate errors, not pretend
they didn't happen, or hope that nothing else got affected, etc. In critical
software systems, that means shut down and restart the offending system, or
engage the backup.

There's no other way that works.
Sean Kelly via Digitalmars-d
2014-09-29 04:03:35 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I've said that processes are different, because the scope of
the effects is limited by the hardware.
If a system with threads that share memory cannot be restarted,
there are serious problems with the design of it, because a
crash and the necessary restart are going to happen sooner or
later, probably sooner.
Right. But if the condition that caused the restart persists,
the process can end up in a cascading restart scenario. Simply
restarting on error isn't necessarily enough.
Post by Walter Bright via Digitalmars-d
I don't believe that the way to get 6 sigma reliability is by
ignoring errors and hoping. Airplane software is most certainly
not done that way.
I believe I was arguing the opposite. More to the point, I think
it's necessary to expect undefined behavior to occur and to plan
for it. I think we're on the same page here and just
miscommunicating.
Post by Walter Bright via Digitalmars-d
I recall Toyota got into trouble with their computer controlled
cars because of their idea of handling of inevitable bugs and
errors. It was one process that controlled everything. When
something unexpected went wrong, it kept right on operating,
any unknown and unintended consequences be damned.
The way to get reliable systems is to design to accommodate
errors, not pretend they didn't happen, or hope that nothing
else got affected, etc. In critical software systems, that
means shut down and restart the offending system, or engage the
backup.
My point was that it's often more complicated than that. There
have been papers written on self-repairing systems, for example,
and ways to design systems that are inherently durable when it
comes to even internal errors. I think what I'm trying to say is
that simply aborting on error is too brittle in some cases,
because it only deals with one vector--memory corruption that is
unlikely to reoccur. But I've watched always-on systems fall
apart from some unexpected ongoing situation, and simply
restarting doesn't actually help.
Walter Bright via Digitalmars-d
2014-09-29 05:15:13 UTC
Permalink
Right. But if the condition that caused the restart persists, the process can
end up in a cascading restart scenario. Simply restarting on error isn't
necessarily enough.
When it isn't enough, use the "engage the backup" technique.
Post by Walter Bright via Digitalmars-d
I don't believe that the way to get 6 sigma reliability is by ignoring errors
and hoping. Airplane software is most certainly not done that way.
I believe I was arguing the opposite. More to the point, I think it's necessary
to expect undefined behavior to occur and to plan for it. I think we're on the
same page here and just miscommunicating.
Assuming that the program bug couldn't have affected other threads is relying on
hope. Bugs happen when the program went into an unknown and unanticipated state.
You cannot know, until after you debug it, what other damage the fault caused,
or what other damage caused the detected fault.
My point was that it's often more complicated than that. There have been papers
written on self-repairing systems, for example, and ways to design systems that
are inherently durable when it comes to even internal errors.
I confess much skepticism about such things when it comes to software. I do know
how reliable avionics software is done, and that stuff does work even in the
face of all kinds of bugs, damage, and errors. I'll be betting my life on that
tomorrow :-)

Would you bet your life on software that had random divide by 0 bugs in it that
were just ignored in the hope that they weren't serious? Keep in mind that
software is rather unique in that a single bit error in a billion bytes can
render the software utterly demented.

Remember the Apollo 11 lunar landing, when the descent computer software started
showing self-detected faults? Armstrong turned it off and landed manually. He
wasn't going to bet his ass that the faults could be ignored. You and I
wouldn't, either.
I think what I'm
trying to say is that simply aborting on error is too brittle in some cases,
because it only deals with one vector--memory corruption that is unlikely to
reoccur. But I've watched always-on systems fall apart from some unexpected
ongoing situation, and simply restarting doesn't actually help.
In such a situation, ignoring the error seems hardly likely to do any better.
Sean Kelly via Digitalmars-d
2014-09-29 19:09:32 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I confess much skepticism about such things when it comes to
software. I do know how reliable avionics software is done, and
that stuff does work even in the face of all kinds of bugs,
damage, and errors. I'll be betting my life on that tomorrow :-)
Would you bet your life on software that had random divide by 0
bugs in it that were just ignored in the hope that they weren't
serious? Keep in mind that software is rather unique in that a
single bit error in a billion bytes can render the software
utterly demented.
I'm not saying the errors should be ignored, but rather that
there are other approaches to handling errors besides (or in
addition to) terminating the process. For me, the single most
important thing is detecting errors as soon as possible so
corrective action can be taken before things go too far south (so
hooray for contracts!). From there, the proper response depends
on the error detected and the type of system I'm working on.
Like with persistent stateful systems, even if a restart occurs
can you assume that the persisted state is valid? With a mesh of
communicating systems, if one node goes insane, what impact might
it have on other nodes in the network? I think the definition of
what constitutes an interdependent system is application defined.

And yes, I know all about tiny bugs creating insane problems.
With event-based asynchronous programming, the most common
serious bugs I encounter memory corruption problems from dangling
pointers, and the only way to find and fix these is by analyzing
gigabytes worth of log files to try and unpack what happened
after the fact. Spending a day looking at the collateral damage
from what ultimately turns out to be a backwards conditional
expression in an error handler somewhere gives a pretty healthy
respect for the brittleness of memory unsafe code. This is one
area where having a GC is an enormous win.
Post by Walter Bright via Digitalmars-d
Remember the Apollo 11 lunar landing, when the descent computer
software started showing self-detected faults? Armstrong turned
it off and landed manually. He wasn't going to bet his ass that
the faults could be ignored. You and I wouldn't, either.
And this is great if there's a human available to take over. But
what if this were a space probe?
Post by Walter Bright via Digitalmars-d
I think what I'm trying to say is that simply aborting on
error is too brittle in some cases, because it only deals with
one vector--memory corruption that is unlikely to reoccur.
But I've watched always-on systems fall apart from some
unexpected ongoing situation, and simply restarting doesn't
actually help.
In such a situation, ignoring the error seems hardly likely to
do any better.
Again, not ignoring, but rather that a restart may not be the
appropriate response to the problem. Or it may be a part of the
appropriate response, but other things need to happen as well.
Paolo Invernizzi via Digitalmars-d
2014-09-29 07:52:32 UTC
Permalink
On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright
Right. But if the condition that caused the restart persists,
the process can end up in a cascading restart scenario. Simply
restarting on error isn't necessarily enough.
This can be mitigated: a cascade reboot would occur if the
problem affects the reboot sequence itself.

---
/Paolo
Sean Kelly via Digitalmars-d
2014-09-29 19:23:27 UTC
Permalink
On Monday, 29 September 2014 at 07:52:33 UTC, Paolo Invernizzi
Post by Paolo Invernizzi via Digitalmars-d
On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright
Right. But if the condition that caused the restart persists,
the process can end up in a cascading restart scenario.
Simply restarting on error isn't necessarily enough.
This can be mitigated: a cascade reboot would occur if the
problem affects the reboot sequence itself.
Or if an ongoing situation causes the problem to rapidly reoccur.
Look at most MMO game launches for example. Production load
hits and some process falls over in a weird way, which increases
load because everyone goes nuts trying to log back in, and when
the system comes back up it immediately falls over again. Rinse,
repeat.
Paolo Invernizzi via Digitalmars-d
2014-09-29 21:49:43 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
On Monday, 29 September 2014 at 07:52:33 UTC, Paolo Invernizzi
Post by Paolo Invernizzi via Digitalmars-d
On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright
Right. But if the condition that caused the restart
persists, the process can end up in a cascading restart
scenario. Simply restarting on error isn't necessarily
enough.
This can be mitigated: a cascade reboot would occur if the
problem affects the reboot sequence itself.
Or if an ongoing situation causes the problem to rapidly
reoccur.
Look at most MMO game launches for example. Production load
hits and some process falls over in a weird way, which increases
load because everyone goes nuts trying to log back in, and when
the system comes back up it immediately falls over again.
Rinse,
repeat.
Is it not better to throttle down the connection volumes before
it reach processes not being able to handle an overload in in a
correct way?
---
/Paolo
Sean Kelly via Digitalmars-d
2014-09-29 23:28:44 UTC
Permalink
On Monday, 29 September 2014 at 21:49:44 UTC, Paolo Invernizzi
Post by Paolo Invernizzi via Digitalmars-d
Is it not better to throttle down the connection volumes before
it reach processes not being able to handle an overload in in a
correct way?
Well, it many cases it isn't the pure load numbers that are the
problem but the interaction of the behavior of actual real users.
Like it isn't overly difficult to simulate a large volume of
traffic on a system, but it's extremely difficult to simulate
realistic behavior patters. I think in many cases the problem
ends up being unexpected interactions of different behaviors that
cause problems rather than simply high volume.
Bruno Medeiros via Digitalmars-d
2014-10-01 14:17:59 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I recall Toyota got into trouble with their computer controlled cars
because of their idea of handling of inevitable bugs and errors. It
was one process that controlled everything. When something unexpected
went wrong, it kept right on operating, any unknown and unintended
consequences be damned.
The way to get reliable systems is to design to accommodate errors,
not pretend they didn't happen, or hope that nothing else got
affected, etc. In critical software systems, that means shut down and
restart the offending system, or engage the backup.
My point was that it's often more complicated than that. There have
been papers written on self-repairing systems, for example, and ways to
design systems that are inherently durable when it comes to even
internal errors. I think what I'm trying to say is that simply aborting
on error is too brittle in some cases, because it only deals with one
vector--memory corruption that is unlikely to reoccur. But I've watched
always-on systems fall apart from some unexpected ongoing situation, and
simply restarting doesn't actually help.
Sean, I fully agree with the points you have been making so far.
But if Walter is fixated on thinking that all the practical uses of D
will be critical systems, or simple (ie, single-use, non-interactive)
command-line applications, it will be hard for him to comprehend the
whole point that "simply aborting on error is too brittle in some cases".

PS: Walter, what browser to you use?
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
bearophile via Digitalmars-d
2014-10-01 14:32:37 UTC
Permalink
Post by Bruno Medeiros via Digitalmars-d
But if Walter is fixated on thinking that all the practical
uses of D will be critical systems, or simple (ie, single-use,
non-interactive) command-line applications,
There's still some of way to go for D design to make it well fit
for high integrity systems (some people even use a restricted
subset of C for such purposes, but it's a bad language for it).

Bye,
bearophile
Joseph Rushton Wakeling via Digitalmars-d
2014-10-03 17:00:21 UTC
Permalink
Post by Walter Bright via Digitalmars-d
If the program has entered an unknown state, its behavior from then on cannot be
predictable. There's nothing I or D can do about that. D cannot officially
endorse such a practice, though D being a systems programming language it will
let you do what you want.
I would not even consider such a practice for a program that is in charge of
anything that could result in injury, death, property damage, security breaches,
etc.
I think I should clarify that I'm not asking you to say "I endorse catching
Errors". Your point about systems responsible for the safety of people or
property is very well made, and I'm fully in agreement with you about this.

What I'm asking you to consider is a use-case, one that I picked quite
carefully. Without assuming anything about how the system is architected, if we
have a telephone exchange, and an Error occurs in the handling of a single call,
it seems to me fairly unarguable that it's essential to avoid this bringing down
everyone else's call with it. That's not simply a matter of convenience -- it's
a matter of safety, because those calls might include emergency calls, urgent
business communications, or any number of other circumstances where dropping
someone's call might have severe negative consequences.

As I'm sure you realize, I also picked that particular use-case because it's one
where there is a well-known technological solution -- Erlang -- which has as a
key feature its ability to isolate different parts of the program, and to deal
with errors by bringing down the local process where the error occurred, rather
than the whole system. This is an approach which is seriously battle-tested in
production.

As I said, I'm not asking you to endorse catching Errors in threads, or other
gross simplifications of Erlang's approach. What I'm interested in are your
thoughts on how we might approach resolving the requirement for this kind of
stability and localization of error-handling with the tools that D provides.

I don't mind if you say to me "That's your problem" (which it certainly is:-),
but I'd like it to be clear that it _is_ a problem, and one that it's important
for D to address, given its strong standing in the development of
super-high-connectivity server applications.
Brad Roberts via Digitalmars-d
2014-10-03 17:38:21 UTC
Permalink
Post by Joseph Rushton Wakeling via Digitalmars-d
Post by Walter Bright via Digitalmars-d
If the program has entered an unknown state, its behavior from then on cannot be
predictable. There's nothing I or D can do about that. D cannot officially
endorse such a practice, though D being a systems programming language it will
let you do what you want.
I would not even consider such a practice for a program that is in charge of
anything that could result in injury, death, property damage, security breaches,
etc.
I think I should clarify that I'm not asking you to say "I endorse
catching Errors". Your point about systems responsible for the safety
of people or property is very well made, and I'm fully in agreement with
you about this.
What I'm asking you to consider is a use-case, one that I picked quite
carefully. Without assuming anything about how the system is
architected, if we have a telephone exchange, and an Error occurs in the
handling of a single call, it seems to me fairly unarguable that it's
essential to avoid this bringing down everyone else's call with it.
That's not simply a matter of convenience -- it's a matter of safety,
because those calls might include emergency calls, urgent business
communications, or any number of other circumstances where dropping
someone's call might have severe negative consequences.
As I'm sure you realize, I also picked that particular use-case because
it's one where there is a well-known technological solution -- Erlang --
which has as a key feature its ability to isolate different parts of the
program, and to deal with errors by bringing down the local process
where the error occurred, rather than the whole system. This is an
approach which is seriously battle-tested in production.
As I said, I'm not asking you to endorse catching Errors in threads, or
other gross simplifications of Erlang's approach. What I'm interested
in are your thoughts on how we might approach resolving the requirement
for this kind of stability and localization of error-handling with the
tools that D provides.
I don't mind if you say to me "That's your problem" (which it certainly
is:-), but I'd like it to be clear that it _is_ a problem, and one that
it's important for D to address, given its strong standing in the
development of super-high-connectivity server applications.
The part of Walter's point that is either deliberately overlooked or
somewhat misunderstood here is the notion of a fault domain. In a
typical unix or windows based environment, it's a process. A fault
within the process yields the aborting of the process but not all
processes. Erlang introduces within it's execution model a concept of a
process within the higher level notion of the os level process. Within
the erlang runtime it's individual processes run independently and can
each fail independently. The erlang runtime guarantees a higher level
of separation than a typical threaded java or c++ application. An error
within the erlang runtime itself would justifiably cause the entire
system to be halted. Just as within an airplane, to use Walter's
favorite analogy, the seat entertainment system is physically and
logically separated from flight control systems thus a fault within the
former has no impact on the latter.

So, where you have domains which must not impact each other, you reach
for tools that allow complete separation such that faults within one
CANNOT impact the other. You don't leave room for 'might not'.

Later,
Brad
Sean Kelly via Digitalmars-d
2014-10-03 18:20:09 UTC
Permalink
On Friday, 3 October 2014 at 17:38:40 UTC, Brad Roberts via
Post by Brad Roberts via Digitalmars-d
The part of Walter's point that is either deliberately
overlooked or somewhat misunderstood here is the notion of a
fault domain. In a typical unix or windows based environment,
it's a process. A fault within the process yields the aborting
of the process but not all processes. Erlang introduces within
it's execution model a concept of a process within the higher
level notion of the os level process. Within the erlang
runtime it's individual processes run independently and can
each fail independently. The erlang runtime guarantees a
higher level of separation than a typical threaded java or c++
application. An error within the erlang runtime itself would
justifiably cause the entire system to be halted. Just as
within an airplane, to use Walter's favorite analogy, the seat
entertainment system is physically and logically separated from
flight control systems thus a fault within the former has no
impact on the latter.
Yep. And I think it's a fair assertion that the default fault
domain in a D program is at the process level, since D is not
inherently memory safe. But I don't think the language should
necessarily make that assertion to the degree that no other
definition is possible.
ketmar via Digitalmars-d
2014-10-04 01:52:23 UTC
Permalink
On Sun, 28 Sep 2014 17:09:57 -0700
Post by Walter Bright via Digitalmars-d
If the program has entered an unknown state, its behavior from then
on cannot be predictable.
and D compiler itself contradicts this principle. why it tries to
"recover" from parsing/compiling errors? it should stop on the first
encountered error and not trying to "recover" itself from unknown state.
hate this. and it's inconsistent with your words.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141004/8f50b1f2/attachment.sig>
Brad Roberts via Digitalmars-d
2014-10-04 02:25:53 UTC
Permalink
Post by ketmar via Digitalmars-d
On Sun, 28 Sep 2014 17:09:57 -0700
Post by Walter Bright via Digitalmars-d
If the program has entered an unknown state, its behavior from then
on cannot be predictable.
and D compiler itself contradicts this principle. why it tries to
"recover" from parsing/compiling errors? it should stop on the first
encountered error and not trying to "recover" itself from unknown state.
hate this. and it's inconsistent with your words.
Where's the contradiction? The compilers state hasn't been corrupted
just because it encounters errors in the text file. In fact, it's
explicitly built to detect and handle them. There's not even a
contradiction in making assumptions about what that input could have
been and attempting to continue based on those assumptions. At no time
in there is the compilers internal state corrupted.

And in direct affirmation of the principle, the compiler has numerous
asserts scattered around that _do_ abort compilation should an
unexpected and invalid state be detected.
ketmar via Digitalmars-d
2014-10-04 03:15:54 UTC
Permalink
On Fri, 03 Oct 2014 19:25:53 -0700
Post by Brad Roberts via Digitalmars-d
Where's the contradiction? The compilers state hasn't been corrupted
just because it encounters errors in the text file.
but compiler is in unknown state. it can't do telepathy, and it's tries
are annoying. there is no reason to guess what code programmer meant to
write, it's just a source of mystic/nonsensical error messages
("garbage", in other words).

the original source of "trying to recover and continue analysis" was
slow compilation times, it was really painful to restart compiler after
each error. but D compilation times are good enough to stop this
"guess-and-miss" nonsence. and we have good IDEs that can analyse code
on background and highligh errors, so there are virtually no reasons
for telepathy left.

yet many compilers (including D) still tries to do telepathy (and
fails). 'cmon, it's best to improve compiling times, not attempting to
guess. PL/1 fails at this, and all other compilers since then fails too.

it's strange to me that Walter telling us that program should stop once
it enters unknown state, but forcing D compiler to make uneducated
guesses when D compiler enters unknown state. something is very wrong
with one of this things.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141004/943f6684/attachment.sig>
eles via Digitalmars-d
2014-10-04 04:10:49 UTC
Permalink
On Saturday, 4 October 2014 at 03:16:08 UTC, ketmar via
Post by ketmar via Digitalmars-d
On Fri, 03 Oct 2014 19:25:53 -0700
Brad Roberts via Digitalmars-d <digitalmars-d at puremagic.com>
Post by Brad Roberts via Digitalmars-d
Where's the contradiction? The compilers state hasn't been
corrupted just because it encounters errors in the text file.
but compiler is in unknown state.
It's not. It just detected that another system would enter in an
unknown state if built. The compiler is like the engineer that
examines the design of a project and discovers an error on it, so
it refuses to build the product. Is the engineer in an unknown
state?

No, it is, just like the compiler is, outside its/his/her normal
execution flow, wich is "take the design and build the product".
It is a known state of the compiler/engineer, namely the error
processing path.

While on the error processing path you allow yourself to be
slower and take time to guess. This is important. Just as for the
engineer, this will educate the designers and will not come again
so often with the same design mistake. Then, working/building
fast is measured on the normal execution path, not on the error
recovery path since the latters is assumed to occur quite rarely
(in any case, its seen like an exceptional issue).

Of course, the engineer has to handle the work conflict with the
designer in a polite form. For the compiler this means printing a
nice error message to the user.

Guessing might not be good, but it is nice effort to do. Do you
really miss the super-cryptic C (let's not even talk about C++)
error messages that you sometimes receive?
ketmar via Digitalmars-d
2014-10-04 04:26:33 UTC
Permalink
On Sat, 04 Oct 2014 04:10:49 +0000
Post by eles via Digitalmars-d
Post by ketmar via Digitalmars-d
Post by Brad Roberts via Digitalmars-d
Where's the contradiction? The compilers state hasn't been
corrupted just because it encounters errors in the text file.
but compiler is in unknown state.
It's not. It just detected that another system would enter in an
unknown state if built. The compiler is like the engineer that
examines the design of a project and discovers an error on it, so
it refuses to build the product. Is the engineer in an unknown
state?
sorry. i meant that compiler WILL be in unknown state if it will
continute to processing invalid source. that's why it should stop right
after the first error.
Post by eles via Digitalmars-d
Guessing might not be good, but it is nice effort to do. Do you
really miss the super-cryptic C (let's not even talk about C++)
error messages that you sometimes receive?
yes. DMD attempts to 'guess' what identifier i mistyped drives me
crazy. just shut up and stop after "unknown identifier", you robot,
don't try to show me your artificial idiocity!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141004/3c5e55f0/attachment.sig>
eles via Digitalmars-d
2014-10-04 04:36:48 UTC
Permalink
On Saturday, 4 October 2014 at 04:26:45 UTC, ketmar via
Post by ketmar via Digitalmars-d
On Sat, 04 Oct 2014 04:10:49 +0000
Post by eles via Digitalmars-d
Post by ketmar via Digitalmars-d
Post by Brad Roberts via Digitalmars-d
Where's the contradiction? The compilers state hasn't been
corrupted just because it encounters errors in the text
file.
but compiler is in unknown state.
It's not. It just detected that another system would enter in
an unknown state if built. The compiler is like the engineer
that examines the design of a project and discovers an error
on it, so it refuses to build the product. Is the engineer in
an unknown state?
sorry. i meant that compiler WILL be in unknown state if it will
continute to processing invalid source. that's why it should
stop right
after the first error.
No. It might produce an invalid product. Just like a real
engineer could produce a flawed product on the basis of wrong
designs.

Yes, you might reach a state where you are no longer be able to
continue because physically impossible. "Build a round square".
Both the engineer and the compiler will bark when they see this.
Post by ketmar via Digitalmars-d
Post by eles via Digitalmars-d
Guessing might not be good, but it is nice effort to do. Do
you really miss the super-cryptic C (let's not even talk about
C++) error messages that you sometimes receive?
yes. DMD attempts to 'guess' what identifier i mistyped drives
me
crazy. just shut up and stop after "unknown identifier", you
robot,
don't try to show me your artificial idiocity!
Could we add a flag to the compiler for that?

-cassandramode=[yes/no/whatever]

Joke :)

Anyway, the Cassandra name for the compiler is just perfect: it
might be right, but you won't believe!
ketmar via Digitalmars-d
2014-10-04 05:01:51 UTC
Permalink
On Sat, 04 Oct 2014 04:36:48 +0000
Post by eles via Digitalmars-d
Post by ketmar via Digitalmars-d
sorry. i meant that compiler WILL be in unknown state if it will
continute to processing invalid source. that's why it should
stop right
after the first error.
No. It might produce an invalid product. Just like a real
engineer could produce a flawed product on the basis of wrong
designs.
i can't see any sane reason to process garbage data, 'cause result is
known to be garbage too.
Post by eles via Digitalmars-d
Could we add a flag to the compiler for that?
-cassandramode=[yes/no/whatever]
Joke :)
but this will be fine. let beginners get suggestions about importing
'std.stdio' on unknown 'writeln', but don't spit that on me. i tend to
read compiler messages, and additional noise is annoying.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141004/79c503a5/attachment.sig>
eles via Digitalmars-d
2014-10-04 05:19:32 UTC
Permalink
On Saturday, 4 October 2014 at 05:02:04 UTC, ketmar via
Post by ketmar via Digitalmars-d
On Sat, 04 Oct 2014 04:36:48 +0000
Post by eles via Digitalmars-d
Post by ketmar via Digitalmars-d
sorry. i meant that compiler WILL be in unknown state if it
will
continute to processing invalid source. that's why it should
stop right
after the first error.
No. It might produce an invalid product. Just like a real
engineer could produce a flawed product on the basis of wrong
designs.
i can't see any sane reason to process garbage data, 'cause
result is
known to be garbage too.
The same reason why a schoolteacher will process garbage exam
sheets.
ketmar via Digitalmars-d
2014-10-04 05:31:00 UTC
Permalink
On Sat, 04 Oct 2014 05:19:32 +0000
Post by eles via Digitalmars-d
Post by ketmar via Digitalmars-d
i can't see any sane reason to process garbage data, 'cause
result is
known to be garbage too.
The same reason why a schoolteacher will process garbage exam
sheets.
it can be true if DMD will has advanced AI someday -- to clearly
understand what programmer wants to write. but then there will be no
reason to emit error messages. and to writing code manually too.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141004/c0d13a48/attachment.sig>
Walter Bright via Digitalmars-d
2014-10-04 08:06:14 UTC
Permalink
there is no reason to guess what code programmer meant to write,
It doesn't do that anymore (at least mostly not). The compiler only continues to
issue error messages for code that is not dependent on the code that caused an
error message.
Joseph Rushton Wakeling via Digitalmars-d
2014-09-28 23:18:02 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Also, I think the idea that a program is created and shipped to an end user is
overly simplistic. In the server/cloud programming world, when an error occurs,
the client who submitted the request will get a response appropriate for them
and the system will also generate log information intended for people working on
the system. So things like stack traces and assertion failure information is
useful even for production software. Same with any critical system, as I'm sure
you're aware. The systems are designed to handle failures in specific ways, but
they also have to leave a breadcrumb trail so the underlying problem can be
diagnosed and fixed. Internal testing is never perfect, and achieving a high
coverage percentage is nearly impossible if the system wasn't designed from the
ground up to be testable in such a way (mock frameworks and such).
Then use assert(). That's just what it's for.
I don't follow this point. How can this approach work with programs that are
built with the -release switch?

Moreover, Sean's points here are absolutely on the money -- there are cases
where the "users" of a program may indeed want to see traces even for
anticipated errors. And even if you design a nice structure of throwing and
catching exceptions so that the simple error message _always_ gives good enough
context to understand what went wrong, you still have the other issue that Sean
raised -- of an exception accidentally escaping its intended scope, because you
forgot to handle it -- when a trace may be extremely useful.

Put it another way -- I think you make a good case that stack traces for
exceptions should be turned off by default (possibly just in -release mode?),
but if that happens I think there's also a good case for a build flag that
ensures stack traces _are_ shown for Exceptions as well as Errors.
ketmar via Digitalmars-d
2014-09-29 01:44:12 UTC
Permalink
On Mon, 29 Sep 2014 01:18:02 +0200
Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d at puremagic.com>
Post by Joseph Rushton Wakeling via Digitalmars-d
Post by Walter Bright via Digitalmars-d
Then use assert(). That's just what it's for.
I don't follow this point. How can this approach work with programs
that are built with the -release switch?
don't use "-release" switch. the whole concept of "release version" is
broken by design. ship what you debugged, not what you think you
debugged.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/73a9dac7/attachment.sig>
Walter Bright via Digitalmars-d
2014-09-29 00:47:56 UTC
Permalink
Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with programs that are
built with the -release switch?
All -release does is not generate code for assert()s. To leave the asserts in,
do not use -release. If you still want the asserts to be in even with -release,

if (condition) assert(0);
Post by Joseph Rushton Wakeling via Digitalmars-d
Moreover, Sean's points here are absolutely on the money -- there are cases
where the "users" of a program may indeed want to see traces even for
anticipated errors.
And even if you design a nice structure of throwing and
catching exceptions so that the simple error message _always_ gives good enough
context to understand what went wrong, you still have the other issue that Sean
raised -- of an exception accidentally escaping its intended scope, because you
forgot to handle it -- when a trace may be extremely useful.
Put it another way -- I think you make a good case that stack traces for
exceptions should be turned off by default (possibly just in -release mode?),
but if that happens I think there's also a good case for a build flag that
ensures stack traces _are_ shown for Exceptions as well as Errors.
The -g switch should take care of that. It's what I use when I need a stack
trace, as there are many ways a program can fail (not just Errors).
Timon Gehr via Digitalmars-d
2014-09-29 04:06:37 UTC
Permalink
Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with programs that are
built with the -release switch?
All -release does is not generate code for assert()s. ...
(Euphemism for undefined behaviour.)
Timon Gehr via Digitalmars-d
2014-09-29 04:07:43 UTC
Permalink
Post by Timon Gehr via Digitalmars-d
Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with programs that are
built with the -release switch?
All -release does is not generate code for assert()s. ...
(Euphemism for undefined behaviour.)
Also, -release additionally removes contracts, in particular invariant
calls, and enables version(assert).
Johannes Pfau via Digitalmars-d
2014-09-29 08:18:27 UTC
Permalink
Am Sun, 28 Sep 2014 17:47:56 -0700
Post by Walter Bright via Digitalmars-d
Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with
programs that are built with the -release switch?
All -release does is not generate code for assert()s. To leave the
asserts in, do not use -release. If you still want the asserts to be
in even with -release,
if (condition) assert(0);
Right now, but some time ago there was a huge debate whether it should
be valid for the compiler to optimize based on asserts.

I wonder if these 'use asserts for stack traces' and 'an assert is
always supposed to pass, so it's valid to assume the condition holds
(in release)' notions can go together. I guess it might at least lead
to programs that are unusable when compiled with -release.
Daniel N via Digitalmars-d
2014-09-29 08:38:48 UTC
Permalink
Post by Johannes Pfau via Digitalmars-d
Right now, but some time ago there was a huge debate whether it should
be valid for the compiler to optimize based on asserts.
I wonder if these 'use asserts for stack traces' and 'an assert is
always supposed to pass, so it's valid to assume the condition
holds
(in release)' notions can go together. I guess it might at
least lead
to programs that are unusable when compiled with -release.
For this reason I think it makes more sense to use abort() if you
plan to use "-release".
ketmar via Digitalmars-d
2014-09-29 09:01:29 UTC
Permalink
On Mon, 29 Sep 2014 10:18:27 +0200
Post by Johannes Pfau via Digitalmars-d
I wonder if these 'use asserts for stack traces' and 'an assert is
always supposed to pass, so it's valid to assume the condition holds
(in release)' notions can go together. I guess it might at least lead
to programs that are unusable when compiled with -release.
compiler never optimizes away `assert(0)`, AFAIR. `assert(0)` is a
special thing, it means `abort()`.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/108a2330/attachment.sig>
Joseph Rushton Wakeling via Digitalmars-d
2014-09-29 15:39:06 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Joseph Rushton Wakeling via Digitalmars-d
I don't follow this point. How can this approach work with
programs that are
built with the -release switch?
All -release does is not generate code for assert()s. To leave
the asserts in, do not use -release. If you still want the
asserts to be in even with -release,
if (condition) assert(0);
The reason I queried your approach here is because I feel you're
conflating two things:

* the _definition_ of an Exception vs. an Error, on which we
100%
agree: the former as an anticipated possibility which a
program
is committed to try and handle, the latter a failure which
is
fundamentally wrong and should not happen under any
conditions.

* the way in which a program should report these different
kinds
of error.

You seem to be advocating that, by definition, Exceptions and
Errors should be reported differently (one without, and one with,
a trace). I don't at all object to that as a sensible default,
but I think that the ultimate decision on how Exceptions and
Errors report themselves should be in the hands of the program
developer, depending on the use-case of the application.
via Digitalmars-d
2014-09-29 16:32:14 UTC
Permalink
On Monday, 29 September 2014 at 15:39:08 UTC, Joseph Rushton
with, a trace). I don't at all object to that as a sensible
default, but I think that the ultimate decision on how
Exceptions and Errors report themselves should be in the hands
of the program developer, depending on the use-case of the
application.
That's a very sensible argument. From a pragmatic point of view I
am usually not interested in the whole stack trace.

I am primarily interested in where it was thrown, where it was
turned into an error and where it was logged + "function call
input" at that point.

Maybe a trade off can be found that is less costly than building
a full stack trace.
Walter Bright via Digitalmars-d
2014-09-28 17:32:14 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
Post by Walter Bright via Digitalmars-d
Whoa, Camel! You're again thinking of Exceptions as a debugging tool.
They can be.
Of course they can be. But it's inappropriate to use them that way, and we
should not be eschewing such in the library.
Post by Sean Kelly via Digitalmars-d
What if an API you're using throws an exception you didn't expect,
and therefore don't handle?
Then the app user sees the error message. This is one of the cool things about D
- I can write small apps with NO error handling logic in it, and I still get
appropriate and friendly messages when things go wrong like missing files.

That is, until recently, when I get a bunch of debug stack traces and internal
file/line messages, which are of no use at all to an app user and look awful.
Post by Sean Kelly via Digitalmars-d
This might be considered a logic error if the
exception is recoverable and you don't intend the program to abort from that
operation.
Adding file/line to all exceptions implies that they are all bugs, and
encourages them to be thought of as bugs and debugging tools, when they are NOT.
Exceptions are for:

1. enabling recovery from input/environmental errors
2. reporting input/environmental errors to the app user
3. making input/environmental errors not ignorable by default

They are not for detecting logic errors. Assert is designed for that.
H. S. Teoh via Digitalmars-d
2014-09-28 20:56:27 UTC
Permalink
[...]
Post by Walter Bright via Digitalmars-d
Post by Sean Kelly via Digitalmars-d
What if an API you're using throws an exception you didn't expect,
and therefore don't handle?
Then the app user sees the error message. This is one of the cool
things about D - I can write small apps with NO error handling logic
in it, and I still get appropriate and friendly messages when things
go wrong like missing files.
That is, until recently, when I get a bunch of debug stack traces and
internal file/line messages, which are of no use at all to an app user
and look awful.
It looks even more awful when the person who wrote the library code is
Russian, and the user speaks English, and when an uncaught exception
terminates the program, you get a completely incomprehensible message in
a language you don't know. Not much different from a line number and
filename that has no meaning for a user.

That's why I said, an uncaught exception is a BUG. The only place where
user-readable messages can be output is in a catch block where you
actually have the chance to localize the error string. But if no catch
block catches it, then by definition it's a bug, and you might as while
print some useful info with it that your users can send back to you,
rather than unhelpful bug reports of the form "the program crashed with
error message 'internal error'". Good luck finding where in the code
that is. (And no, grepping does not work -- the string 'internal error'
could have come from a system call or C library error code translated by
a generic code-to-message function, which could've been called from
*anywhere*.)
Post by Walter Bright via Digitalmars-d
Post by Sean Kelly via Digitalmars-d
This might be considered a logic error if the exception is
recoverable and you don't intend the program to abort from that
operation.
Adding file/line to all exceptions implies that they are all bugs, and
encourages them to be thought of as bugs and debugging tools, when
1. enabling recovery from input/environmental errors
2. reporting input/environmental errors to the app user
3. making input/environmental errors not ignorable by default
They are not for detecting logic errors. Assert is designed for that.
I do not condone adding file/line to exception *messages*. Catch blocks
can print / translate those messages, which can be made user-friendly,
but if the program failed to catch an exception, you're already screwed
anyway so why not provide more info rather than less?

Unless, of course, you're suggesting that we put this around every
main() function:

void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}


T
--
I think Debian's doing something wrong, `apt-get install pesticide', doesn't seem to remove the bugs on my system! -- Mike Dresser
Sean Kelly via Digitalmars-d
2014-09-28 21:16:57 UTC
Permalink
On Sunday, 28 September 2014 at 20:58:20 UTC, H. S. Teoh via
Post by H. S. Teoh via Digitalmars-d
That's why I said, an uncaught exception is a BUG. The only
place where
user-readable messages can be output is in a catch block where
you
actually have the chance to localize the error string. But if
no catch
block catches it, then by definition it's a bug, and you might
as while
print some useful info with it that your users can send back to
you,
rather than unhelpful bug reports of the form "the program
crashed with
error message 'internal error'".
Pretty much every system should generate a localized error
message for the user and a detailed log of the problem for the
programmer. Ideally, the user message will indicate how to
provide the detailed information to the developer so the problem
can be fixed.

The one case where uncaught exceptions aren't really a bug is
with programs that aren't being used outside the group that
developed them. In these cases, the default behavior is pretty
much exactly what's desired--a message, file/line info, and a
stack trace. Which is why it's there. The vast bulk of today's
shipping code doesn't run from the command line anyway, so the
default exception handler should be practically irrelevant.
Cliff via Digitalmars-d
2014-09-28 21:22:55 UTC
Permalink
On Sunday, 28 September 2014 at 20:58:20 UTC, H. S. Teoh via
Post by H. S. Teoh via Digitalmars-d
I do not condone adding file/line to exception *messages*.
Catch blocks
can print / translate those messages, which can be made
user-friendly,
but if the program failed to catch an exception, you're already
screwed
anyway so why not provide more info rather than less?
Unless, of course, you're suggesting that we put this around
every
void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}
In our production C# code, we had a few practices which might be
applicable here:

1. main() definitely had a top-level try/catch handler to produce
useful output messages. Because throwing an uncaught exception
out to the user *is* a bug, we naturally want to not just toss
out a stack trace but information on what to do with it should a
user encounter it. Even better if there is additional runtime
information which can be provided for a bug report.

2. We also registered a top-level unhandled exception handler on
the AppDomain (equivalent to a process in .NET, except that
multiple AppDomains may exist within a single OS process), which
allows the catching to exceptions which would otherwise escape
background threads. Depending on the nature of the application,
these could be logged to some repository to which the user could
be directed. It's hard to strictly automate this because exactly
what you can do with an exception which escapes a thread will be
application dependent. In our case, these exceptions were
considered bugs, were considered to be unrecoverable and resulted
in a program abort with a user message indicating where to find
the relevant log outputs and how to contact us.

3. For some cases, throwing an exception would also trigger an
application dump suitable for post-mortem debugging from the
point the exception was about to be thrown. This functionality
is, of course, OS-specific, but helped us on more than a few
occasions by eliminating the need to try to pre-determine which
information was important and which was not so the exception
could be usefully populated.

I'm not a fan of eliminating the stack from exceptions. While
exceptions should not be used to catch logic errors, an uncaught
exception is itself a logic error (that is, one has omitted some
required conditions in their code) and thus the context of the
error needs to be made available somehow.
Walter Bright via Digitalmars-d
2014-09-28 22:59:45 UTC
Permalink
Post by H. S. Teoh via Digitalmars-d
It looks even more awful when the person who wrote the library code is
Russian, and the user speaks English, and when an uncaught exception
terminates the program, you get a completely incomprehensible message in
a language you don't know. Not much different from a line number and
filename that has no meaning for a user.
I cannot buy into the logic that since Russian error messages are
incomprehensible to me, that therefore incomprehensible messages are ok.
Post by H. S. Teoh via Digitalmars-d
That's why I said, an uncaught exception is a BUG.
It's a valid opinion, but is not the way D is designed to work.
Post by H. S. Teoh via Digitalmars-d
The only place where
user-readable messages can be output is in a catch block where you
actually have the chance to localize the error string. But if no catch
block catches it, then by definition it's a bug, and you might as while
print some useful info with it that your users can send back to you,
rather than unhelpful bug reports of the form "the program crashed with
error message 'internal error'".
If anyone is writing code that throws an Exception with "internal error", then
they are MISUSING exceptions to throw on logic bugs. I've been arguing this all
along.
Post by H. S. Teoh via Digitalmars-d
if the program failed to catch an exception, you're already screwed
anyway
This is simply not true. One can write utilities with no caught exceptions at
all, and yet have the program emit user friendly messages about "disk full" and
stuff like that.
Post by H. S. Teoh via Digitalmars-d
so why not provide more info rather than less?
Because having an internal stack dump presented to the app user for when he,
say, puts in invalid command line arguments, is quite inappropriate.
Post by H. S. Teoh via Digitalmars-d
Unless, of course, you're suggesting that we put this around every
void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}
I'm not suggesting that Exceptions are to be thrown on programmer screwups - I
suggest the OPPOSITE.
Timon Gehr via Digitalmars-d
2014-09-29 04:16:18 UTC
Permalink
Post by Walter Bright via Digitalmars-d
...
Post by H. S. Teoh via Digitalmars-d
Unless, of course, you're suggesting that we put this around every
void main() {
try {
...
} catch(Exception e) {
assert(0, "Unhandled exception: I screwed up");
}
}
I'm not suggesting that Exceptions are to be thrown on programmer
screwups - I suggest the OPPOSITE.
He does not suggest that Exceptions are to be thrown on programmer
screw-ups, but rather that the thrown exception itself is the screw-up,
with a possibly complex cause.

It is not:

if(screwedUp()) throw Exception("");


It is rather:

void foo(int x){
if(!test(x)) throw Exception(""); // this may be an expected code
path for some callers
}

void bar(){
// ...
int y=screwUp();
foo(y); // yet it is unexpected here
}
via Digitalmars-d
2014-09-29 04:31:42 UTC
Permalink
Post by Walter Bright via Digitalmars-d
If anyone is writing code that throws an Exception with
"internal error", then they are MISUSING exceptions to throw on
logic bugs. I've been arguing this all along.
Nothing wrong with it. Quite common and useful for a non-critical
web service to log the exception, then re-throw something like
"internal error", catch the internal error at the root and
returning the appropriate 5xx HTTP response, then keep going.

You are arguing as if it is impossible to know whether the logic
error is local to the handler, or not, with a reasonable
probability. "Division by zero" is usually not a big deal, but it
is a logic error. No need to shut down the service.
Post by Walter Bright via Digitalmars-d
I'm not suggesting that Exceptions are to be thrown on
programmer screwups - I suggest the OPPOSITE.
It is impossible to verify what the source is. It might be a bug
in a boolean expression leading to a throw when the system is ok.

assert()s should also not be left in production code. They are
not for catching runtime errors, but for testing at the expense
of performance.

Uncaught exceptions should be re-thrown higher up in the call
chain to a different error level based on the possible impact on
the system. Getting an unexpected mismatch exception in a
form-validator is not a big deal. Getting out-of-bounds error in
main storage is a big deal. Whether it is a big deal can only be
decided at the higher level.

It is no doubt useful to be able to obtain a stack trace so that
you can log it when an exception turns out to fall into the "big
deal" category and therefore should be re-thrown as a critical
failture. The deciding factor should be performance.
Walter Bright via Digitalmars-d
2014-09-29 04:57:44 UTC
Permalink
On 9/28/2014 9:31 PM, "Ola Fosheim GrÞstad"
Nothing wrong with it. Quite common and useful for a non-critical web service to
log the exception, then re-throw something like "internal error", catch the
internal error at the root and returning the appropriate 5xx HTTP response, then
keep going.
Lots of bad practices are commonplace.
You are arguing as if it is impossible to know whether the logic error is local
to the handler, or not, with a reasonable probability.
You're claiming to know that a program in an unknown and unanticipated state is
really in a known state. It isn't.
assert()s should also not be left in production code. They are not for catching
runtime errors, but for testing at the expense of performance.
Are you really suggesting that asserts should be replaced by thrown exceptions?
I suspect we have little common ground here.
Uncaught exceptions should be re-thrown higher up in the call chain to a
different error level based on the possible impact on the system. Getting an
unexpected mismatch exception in a form-validator is not a big deal. Getting
out-of-bounds error in main storage is a big deal. Whether it is a big deal can
only be decided at the higher level.
A vast assumption here that you know in advance what bugs you're going to have
and what causes them.
It is no doubt useful to be able to obtain a stack trace so that you can log it
when an exception turns out to fall into the "big deal" category and therefore
should be re-thrown as a critical failture. The deciding factor should be
performance.
You're using exceptions as a program bug reporting mechanism. Whoa camel, indeed!
via Digitalmars-d
2014-09-29 06:08:53 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Lots of bad practices are commonplace.
This is not an argument, it is a postulate.
Post by Walter Bright via Digitalmars-d
Post by via Digitalmars-d
You are arguing as if it is impossible to know whether the
logic error is local
to the handler, or not, with a reasonable probability.
You're claiming to know that a program in an unknown and
unanticipated state is really in a known state. It isn't.
It does not have to be known, it is sufficient that it is
isolated or that it is improbable to be global or that it is of
low impact to long term integrity.
Post by Walter Bright via Digitalmars-d
Are you really suggesting that asserts should be replaced by
thrown exceptions? I suspect we have little common ground here.
No, regular asserts should not be caught except for mailing the
error log to the developer. They are for testing only.

Pre/postconditions between subsystems are on a different level
though. They should not be conflated with regular asserts.
Post by Walter Bright via Digitalmars-d
A vast assumption here that you know in advance what bugs
you're going to have and what causes them.
I know in advance that a "divison-by-zero" error is of limited
scope with high probability or that an error in a strictly pure
validator is of low impact with high probability. I also know
that any sign of a flaw in a transaction engine is a critical
error that warrants a shutdown.

We know in advance that all programs above low complexity will
contain bugs, most of them innocent and they are not a good
excuse of shutting down the entire service for many services.

If you have memory safety, reasonable isolation and well tested
global data-structures it is most desirable to keep the system
running if it is incapable of corrupting a critical database.
Post by Walter Bright via Digitalmars-d
You're using exceptions as a program bug reporting mechanism.
Uncaught exceptions are bugs and should be logged as such. If a
form validator throws an unexpected exception then it is a bug.
It makes the validation questionable, but does not affect the
rest of the system. It is a non-critical bug that needs attention.
Post by Walter Bright via Digitalmars-d
Whoa camel, indeed!
By your line of reasoning no software should ever be shipped,
without a formal proof, because they most certainly will be buggy
and contain unspecified undetected state.

Keep in mind that a working program, in the real world, is a
program that provides reasonable output for reasonable input.
Total correctness is a pipe dream, it is not within reach for
most real programs. Not even with formal proofs.
Johannes Pfau via Digitalmars-d
2014-09-29 08:27:16 UTC
Permalink
Am Sun, 28 Sep 2014 15:59:45 -0700
Post by Walter Bright via Digitalmars-d
Post by H. S. Teoh via Digitalmars-d
if the program failed to catch an exception, you're already screwed
anyway
This is simply not true. One can write utilities with no caught
exceptions at all, and yet have the program emit user friendly
messages about "disk full" and stuff like that.
You're always thinking of simple console apps but this is the only place
where the default 'print exception to console' strategy works.

In a daemon which logs to syslog or in a GUI application or a game an
uncaught 'disk full exception' would go completely unnoticed and that's
definitely a bug.
Walter Bright via Digitalmars-d
2014-09-29 09:06:52 UTC
Permalink
Post by Johannes Pfau via Digitalmars-d
In a daemon which logs to syslog or in a GUI application or a game an
uncaught 'disk full exception' would go completely unnoticed and that's
definitely a bug.
Failure to respond properly to an input/environmental error is a bug. But the
input/environmental error is not a bug. If it was, then the program should
assert on the error, not throw.
Bruno Medeiros via Digitalmars-d
2014-10-01 13:21:56 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Johannes Pfau via Digitalmars-d
In a daemon which logs to syslog or in a GUI application or a game an
uncaught 'disk full exception' would go completely unnoticed and that's
definitely a bug.
Failure to respond properly to an input/environmental error is a bug.
But the input/environmental error is not a bug. If it was, then the
program should assert on the error, not throw.
I agree. And isn't that exactly what Teoh said then:
"That's why I said, an uncaught exception is a BUG. "

I think people should be more careful with the term "uncaught exception"
because it's not very precise.
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
Steven Schveighoffer via Digitalmars-d
2014-09-28 01:24:00 UTC
Permalink
Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
Library code often cannot make that choice. The issue with exceptions
vs. errors is that often you don't know where the input comes from.

e.g.:

auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception

How does File know what it's target file name came from?

-Steve
Walter Bright via Digitalmars-d
2014-09-28 01:52:11 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
Library code often cannot make that choice.
The issue with exceptions vs. errors
is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?
If the app is concerned about invalid filenames as bugs, you should scrub the
filenames first. I.e. the interface is defined improperly if the code confuses a
programming bug with input errors.
Steven Schveighoffer via Digitalmars-d
2014-09-29 11:47:52 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
Library code often cannot make that choice.
The issue with exceptions vs. errors
is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?
If the app is concerned about invalid filenames as bugs, you should
scrub the filenames first. I.e. the interface is defined improperly if
the code confuses a programming bug with input errors.
OK, so if you want to avoid improperly having errors/exceptions, don't
put bugs into your code.

A simple plan!

-Steve
Walter Bright via Digitalmars-d
2014-09-29 11:53:05 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
Post by Walter Bright via Digitalmars-d
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
Library code often cannot make that choice.
The issue with exceptions vs. errors
is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?
If the app is concerned about invalid filenames as bugs, you should
scrub the filenames first. I.e. the interface is defined improperly if
the code confuses a programming bug with input errors.
OK, so if you want to avoid improperly having errors/exceptions, don't put bugs
into your code.
A simple plan!
Validating user input is not the same thing as removing all the logic bugs from
the program.
Steven Schveighoffer via Digitalmars-d
2014-09-29 12:03:56 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Walter Bright via Digitalmars-d
If the app is concerned about invalid filenames as bugs, you should
scrub the filenames first. I.e. the interface is defined improperly if
the code confuses a programming bug with input errors.
OK, so if you want to avoid improperly having errors/exceptions, don't put bugs
into your code.
A simple plan!
Validating user input is not the same thing as removing all the logic
bugs from the program.
What if it's not user input?

-Steve
Jacob Carlborg via Digitalmars-d
2014-09-28 09:01:13 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
Library code often cannot make that choice. The issue with exceptions
vs. errors is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?
Both of theses should throw an exception. Most stuff related to file
operations should throw an exception, not an error.
--
/Jacob Carlborg
Steven Schveighoffer via Digitalmars-d
2014-09-29 11:51:08 UTC
Permalink
Post by Jacob Carlborg via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
Library code often cannot make that choice. The issue with exceptions
vs. errors is that often you don't know where the input comes from.
auto f = File(someInternalStringThatIsCorrupted) -> error
auto f = File(argv[1]) -> exception
How does File know what it's target file name came from?
Both of theses should throw an exception. Most stuff related to file
operations should throw an exception, not an error.
That makes no sense. The opening of the file is subject to issues with
the filesystem, which means they may be environmental or user errors,
not programming errors. But that doesn't mean the opening of the file
failed because the file doesn't exist, it could be an error in how you
construct the file name.

What about:

File f;
f.open(null);

Is that an environmental error or User error?

-Steve
Walter Bright via Digitalmars-d
2014-09-29 11:54:39 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
File f;
f.open(null);
Is that an environmental error or User error?
Passing invalid arguments to a function is a programming bug.
Steven Schveighoffer via Digitalmars-d
2014-09-29 12:04:32 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
File f;
f.open(null);
Is that an environmental error or User error?
Passing invalid arguments to a function is a programming bug.
That throws an exception. My point exactly.

-Steve
Jacob Carlborg via Digitalmars-d
2014-09-29 14:48:12 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
That makes no sense. The opening of the file is subject to issues with
the filesystem, which means they may be environmental or user errors,
not programming errors. But that doesn't mean the opening of the file
failed because the file doesn't exist, it could be an error in how you
construct the file name.
File f;
f.open(null);
Is that an environmental error or User error?
That depends on what "open" expect from its argument. In this particular
case, in D, "null" is the same as the empty string. I don't see why that
technically shouldn't be allowed.

Of course, you can specify that "open" expects a string argument with
the length that is greater then 0, in that case it's a bug by the
programmer that uses the "open" function.
--
/Jacob Carlborg
Steven Schveighoffer via Digitalmars-d
2014-09-29 15:13:59 UTC
Permalink
Post by Jacob Carlborg via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
That makes no sense. The opening of the file is subject to issues with
the filesystem, which means they may be environmental or user errors,
not programming errors. But that doesn't mean the opening of the file
failed because the file doesn't exist, it could be an error in how you
construct the file name.
File f;
f.open(null);
Is that an environmental error or User error?
That depends on what "open" expect from its argument. In this particular
case, in D, "null" is the same as the empty string. I don't see why that
technically shouldn't be allowed.
My entire point is, it doesn't matter what is expected or what is
treated as "correct." what matters is where the input CAME from. To a
library function, it has no idea. There is no extra type info saying
"this parameter comes from user input."
Post by Jacob Carlborg via Digitalmars-d
Of course, you can specify that "open" expects a string argument with
the length that is greater then 0, in that case it's a bug by the
programmer that uses the "open" function.
Is it? I can think of cases where it's programmer error, and cases where
it's user error.

There are also better example functions, but I'm focusing on File
because that's what this thread is about.

The largest issue I see with this whole scheme is that exceptions can be
turned into errors, but not the reverse. Once an error is thrown, it's
pretty much game over. So defensive coding would suggest when you don't
know the answer, throw an exception, and something higher up would say
"Oh, that is really a program error, rethrowing"

But expecting developers to do this at EVERY CALL is really impossible.

My expectation is that an exception is really an error unless caught. It
would be nice to postpone generating the stack trace unless the
exception is caught outside the main function. I don't know enough about
how exceptions work to know if this is possible or not.

-Steve
Jeremy Powers via Digitalmars-d
2014-09-29 18:43:41 UTC
Permalink
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d <
My entire point is, it doesn't matter what is expected or what is treated
as "correct." what matters is where the input CAME from. To a library
function, it has no idea. There is no extra type info saying "this
parameter comes from user input."
From the method's view, parameters passed in are user input. Full stop.
One thing that seems to be talked around a bit here is the
separation/encapsulation of things. It is perfectly fine, and expected,
for a library method to throw exceptions on bad input to the method - even
if this input turns out to be a programming bug elsewhere. From the
standpoint of the method, it does not know (and does not care) where the
thing ultimately came from - all it knows is that it is input here, and it
is wrong.

If you call a method with bad input, and fail to catch the resulting
exception, then _that_ is a bug, not the method throwing. It may be
perfectly recoverable to ignore/retry/whatever, or it may be a symptom of
something that should abort the program. But the method throwing does not
know or care.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/a79b902b/attachment.html>
Jeremy Powers via Digitalmars-d
2014-09-29 18:59:49 UTC
Permalink
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d <
Post by Steven Schveighoffer via Digitalmars-d
The largest issue I see with this whole scheme is that exceptions can be
turned into errors, but not the reverse. Once an error is thrown, it's
pretty much game over. So defensive coding would suggest when you don't
know the answer, throw an exception, and something higher up would say "Oh,
that is really a program error, rethrowing"
But expecting developers to do this at EVERY CALL is really impossible.
And this is an argument for checked exceptions - being able to explicitly
state 'these are known fatal cases for this component, you should deal with
them appropriately' when defining a method. Cuts down the catch/check to
just the common cases, and makes such cases explicit to the caller.
Anything not a checked exception falls into the 'error, abort!' path.
(Memory corruption etc. being abort scenarios)

If I really needed to write a robust program in D right now, I would
(attempt) to wrap every call in a try/catch, and check if the thrown
exception was of a handleable type. But knowing which types for which
methods would lead me to basically hacking up some documentation-enforced
checked exceptions, and being entirely unmaintainable.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/a67801ce/attachment.html>
Steven Schveighoffer via Digitalmars-d
2014-09-29 18:58:40 UTC
Permalink
Post by Jeremy Powers via Digitalmars-d
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d
My entire point is, it doesn't matter what is expected or what is
treated as "correct." what matters is where the input CAME from. To
a library function, it has no idea. There is no extra type info
saying "this parameter comes from user input."
From the method's view, parameters passed in are user input. Full stop.
This is missing the point of an exception. An uncaught exception is an
error which crashes the program. If you catch the exception, you can
handle it, but if you don't expect it, then it's a bug. Any uncaught
exceptions are BY DEFINITION programming errors.
Post by Jeremy Powers via Digitalmars-d
One thing that seems to be talked around a bit here is the
separation/encapsulation of things. It is perfectly fine, and expected,
for a library method to throw exceptions on bad input to the method -
even if this input turns out to be a programming bug elsewhere. From
the standpoint of the method, it does not know (and does not care) where
the thing ultimately came from - all it knows is that it is input here,
and it is wrong.
What is being discussed here is removing the stack trace and printout
when an exception is thrown.

Imagine this error message:

myprompt> ./mycoolprogram
(1 hour later)
FileException: Error opening file xgghfsnbuer
myprompt>

Now what? xgghfsnbuer may not even be in the code anywhere. There may be
no hints at all as to what caused it to happen. You don't even know
which line of code YOU wrote that was causing the issue! You have to
examine every File open, and every call that may have opened a file, and
see if possibly that file name was passed into it.

Whereas if you get a trace, you can at least see where the exception
occurred, and start from there.

Now, RELYING on this printout to be your interface to the user, that is
incorrect design, I will agree. But one cannot possibly be expected to
handle every possible exception at every possible call so one can throw
an error in the cases where it actually is an error. D doesn't even
require listing the exceptions that may be thrown on the API (and no,
I'm not suggesting it should).
Post by Jeremy Powers via Digitalmars-d
If you call a method with bad input, and fail to catch the resulting
exception, then _that_ is a bug, not the method throwing. It may be
perfectly recoverable to ignore/retry/whatever, or it may be a symptom
of something that should abort the program. But the method throwing
does not know or care.
Sure, but it doesn't happen. Just like people do not check return values
from syscalls.

The benefit of the exception printing is at least you get a trace of
where things went wrong when you didn't expect them to. Ignoring a
call's return value doesn't give any notion something is wrong until
much later.

-Steve
Jeremy Powers via Digitalmars-d
2014-09-29 19:32:10 UTC
Permalink
On Mon, Sep 29, 2014 at 11:58 AM, Steven Schveighoffer via Digitalmars-d <
Post by Steven Schveighoffer via Digitalmars-d
Post by Jeremy Powers via Digitalmars-d
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d
My entire point is, it doesn't matter what is expected or what is
treated as "correct." what matters is where the input CAME from. To
a library function, it has no idea. There is no extra type info
saying "this parameter comes from user input."
From the method's view, parameters passed in are user input. Full stop.
This is missing the point of an exception. An uncaught exception is an
error which crashes the program. If you catch the exception, you can handle
it, but if you don't expect it, then it's a bug. Any uncaught exceptions
are BY DEFINITION programming errors.
I agree (except the bit about missing the point). The point I wanted to
make was that encapsulation means what is a fatal error to one part of a
program may be easily handled by the containing part. Just because an
exception is thrown somewhere does not mean the program is broken - it is
the failure to handle the exception (explicit or inadvertent) that
indicates an error.


What is being discussed here is removing the stack trace and printout when
Post by Steven Schveighoffer via Digitalmars-d
an exception is thrown.
....
Sure, but it doesn't happen. Just like people do not check return values
Post by Steven Schveighoffer via Digitalmars-d
from syscalls.
The benefit of the exception printing is at least you get a trace of where
things went wrong when you didn't expect them to. Ignoring a call's return
value doesn't give any notion something is wrong until much later.
-Steve
I absolutely do not want a removal of stack trace information. If an
uncaught exception bubbles up and terminates the program, this is a bug and
I sure as hell want to know as much about it as possible. If having such
information presented to the end user is unacceptable, then wrap and spew
something better.

Ignoring an error return value is like ignoring an exception - bad news,
and indicative of a broken program.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/c385d0b3/attachment.html>
Bruno Medeiros via Digitalmars-d
2014-10-01 13:47:48 UTC
Permalink
Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
Steven Schveighoffer via Digitalmars-d
2014-10-01 13:55:40 UTC
Permalink
Post by Bruno Medeiros via Digitalmars-d
Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.
No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.

-Steve
Bruno Medeiros via Digitalmars-d
2014-10-01 14:36:51 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
Post by Bruno Medeiros via Digitalmars-d
Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.
No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.
-Steve
Well, at the very least it's bad UI design for sure (textual UI is still
UI).
But it's only a *bug* if it's not the behavior the programmer intended.
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
Steven Schveighoffer via Digitalmars-d
2014-10-01 14:46:50 UTC
Permalink
Post by Bruno Medeiros via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
Post by Bruno Medeiros via Digitalmars-d
Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily.
For some applications (for example simple console apps), you can
consider the D runtime's default exception handler to be an appropriate
way to respond to the exception.
No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.
-Steve
Well, at the very least it's bad UI design for sure (textual UI is still
UI).
But it's only a *bug* if it's not the behavior the programmer intended.
Sure, one could also halt a program by reading a null pointer on
purpose. This is a grey area that I think reasonable people can
correctly call a bug if they so wish, despite the intentions of the
developer.

-Steve
Paolo Invernizzi via Digitalmars-d
2014-10-01 19:24:36 UTC
Permalink
On Wednesday, 1 October 2014 at 14:46:50 UTC, Steven
Post by Steven Schveighoffer via Digitalmars-d
This is a grey area that I think reasonable people
can correctly call a bug if they so wish, despite the
intentions of the developer.
Correctly?

In a discussion, It's amazing how difficult it is to agree also
on simple words meaning: an _intentional programmer behaviour_ a
bug?

Whah ;-P

---
/Paolo
Steven Schveighoffer via Digitalmars-d
2014-10-01 19:56:25 UTC
Permalink
Post by Paolo Invernizzi via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
This is a grey area that I think reasonable people
can correctly call a bug if they so wish, despite the intentions of
the developer.
Correctly?
In a discussion, It's amazing how difficult it is to agree also on
simple words meaning: an _intentional programmer behaviour_ a bug?
More appropriately, it's not a bug but an incorrect design. I still
would call it a bug, and I'm 99% sure the users would report it as a bug :)

-Steve
Andrej Mitrovic via Digitalmars-d
2014-10-01 15:00:42 UTC
Permalink
On 10/1/14, Steven Schveighoffer via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.
So when they file a bug report are you going to also ask them to run
the debugger so they capture the stack trace and file that to you?
Come on.
Steven Schveighoffer via Digitalmars-d
2014-10-01 15:21:22 UTC
Permalink
Post by Andrej Mitrovic via Digitalmars-d
On 10/1/14, Steven Schveighoffer via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
No, this is lazy/incorrect coding. You don't want your user to see an
indecipherable stack trace on purpose.
So when they file a bug report are you going to also ask them to run
the debugger so they capture the stack trace and file that to you?
Come on.
No what I mean is:

./niftyapp badfilename.txt

Result should be:

Error: Could not open badfilename.txt, please check and make sure the
file exists and is readable.

Not:

std.exception.ErrnoException at std/stdio.d(345): Cannot open file
`badfilename.txt' in mode `rb' (No such file or directory)
----------------
5 testexception 0x0000000104fad02d ref
std.stdio.File std.stdio.File.__ctor(immutable(char)[], const(char[])) + 97
6 testexception 0x0000000104f8d735 _Dmain + 69
7 testexception 0x0000000104f9f771 void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).runAll().void __lambda1() + 33
8 testexception 0x0000000104f9f6bd void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) + 45
9 testexception 0x0000000104f9f71d void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).runAll() + 45
10 testexception 0x0000000104f9f6bd void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) + 45
11 testexception 0x0000000104f9f639 _d_run_main + 449
12 testexception 0x0000000104f8d75c main + 20
13 libdyld.dylib 0x00007fff8fb2a5fd start + 1
14 ??? 0x0000000000000001 0x0 + 1

If it's an error due to *user input*, you should not rely on the
exception handling of the runtime, you should have a more user-friendly
message.

Obviously, if you fail to handle it, the full trace happens, and then
you must fix that in your code.

It's for your benefit too :) This way you get less nuisance
troubleshooting calls since the error message is clearer.

-Steve
Sean Kelly via Digitalmars-d
2014-09-29 19:28:31 UTC
Permalink
On Monday, 29 September 2014 at 18:59:59 UTC, Jeremy Powers via
Post by Jeremy Powers via Digitalmars-d
And this is an argument for checked exceptions - being able to
explicitly state 'these are known fatal cases for this
component, you should deal with them appropriately' when
defining a method. Cuts down the catch/check to just the
common cases, and makes such cases explicit to the caller.
Anything not a checked exception falls into the 'error, abort!'
path. (Memory corruption etc. being abort scenarios)
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.
Jeremy Powers via Digitalmars-d
2014-09-29 19:44:14 UTC
Permalink
On Mon, Sep 29, 2014 at 12:28 PM, Sean Kelly via Digitalmars-d <
Post by Sean Kelly via Digitalmars-d
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.
I've heard this before, but have not seen a reasonable argument as to why
they are a failure. Last time this was discussed a link to a blog was
provided, with lots of discussion there - which as far as I could tell
boiled down to 'catching exceptions is ugly, and people just do the wrong
thing anyway which is ugly when you have checked exceptions.'

I am unlucky enough to write Java all day, and from my standpoint checked
exceptions are a huge win. There are certain edges which can catch you,
but they are immensely useful in developing robust programs. Basically
checked exceptions -> recoverable problems, unchecked ->
unrecoverable/programming errors (like asserts or memory errors).

Note I am not advocating adding checked exceptions to D (though I would
like it). Point is to acknowledge that there are different kinds of
exceptions, and an exception for one part of the code may not be a problem
for the bit that invokes it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140929/8c59e583/attachment-0001.html>
Steven Schveighoffer via Digitalmars-d
2014-09-30 12:43:37 UTC
Permalink
Post by Jeremy Powers via Digitalmars-d
On Mon, Sep 29, 2014 at 12:28 PM, Sean Kelly via Digitalmars-d
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.
I've heard this before, but have not seen a reasonable argument as to
why they are a failure. Last time this was discussed a link to a blog
was provided, with lots of discussion there - which as far as I could
tell boiled down to 'catching exceptions is ugly, and people just do the
wrong thing anyway which is ugly when you have checked exceptions.'
I am unlucky enough to write Java all day, and from my standpoint
checked exceptions are a huge win. There are certain edges which can
catch you, but they are immensely useful in developing robust programs.
Basically checked exceptions -> recoverable problems, unchecked ->
unrecoverable/programming errors (like asserts or memory errors).
Well, the failure comes from the effort to effect a certain behavior.

Sun was looking to make programmers more diligent about handling errors.
However, humans are lazy worthless creatures. What ends up happening is,
the compiler complains they aren't handling an exception. They can't see
any reason why the exception would occur, so they simply catch and
ignore it to shut the compiler up.

In 90% of cases, they are right -- the exception will not occur. But
because they have been "trained" to simply discard exceptions, it ends
up defeating the purpose for the 10% of the time that they are wrong.

If you have been able to resist that temptation and handle every
exception, then I think you are in the minority. But I have no evidence
to back this up, it's just a belief.
Post by Jeremy Powers via Digitalmars-d
Note I am not advocating adding checked exceptions to D (though I would
like it). Point is to acknowledge that there are different kinds of
exceptions, and an exception for one part of the code may not be a
problem for the bit that invokes it.
I think this is appropriate for a lint tool for those out there like
yourself who want that information. But requiring checked exceptions is
I think a futile attempt to outlaw natural human behavior.

-Steve
Jeremy Powers via Digitalmars-d
2014-09-30 19:18:00 UTC
Permalink
On Tue, Sep 30, 2014 at 5:43 AM, Steven Schveighoffer via Digitalmars-d <
Post by Steven Schveighoffer via Digitalmars-d
...
Well, the failure comes from the effort to effect a certain behavior.
Sun was looking to make programmers more diligent about handling errors.
However, humans are lazy worthless creatures. What ends up happening is,
the compiler complains they aren't handling an exception. They can't see
any reason why the exception would occur, so they simply catch and ignore
it to shut the compiler up.
In 90% of cases, they are right -- the exception will not occur. But
because they have been "trained" to simply discard exceptions, it ends up
defeating the purpose for the 10% of the time that they are wrong.
That's the argument, but it doesn't seem valid to me. Without checked
exceptions, you will always be ignoring exceptions. With checked
exceptions, you have to explicitly ignore (some) exceptions, and when you
do it is immediately obvious in the code. You go from everyone ignoring
exceptions all the time, to some people ignoring them - and being able to
easily notice and call out such.

Anyone 'trained' to ignore checked exceptions are simply shooting
themselves in the foot - same as if there were no checked exceptions, but
with more verbosity. This is not a failure of checked exceptions, but a
failure of people to use a language feature properly. (Which, yeah, meta is
a failure of the feature... not going to go there)
Post by Steven Schveighoffer via Digitalmars-d
If you have been able to resist that temptation and handle every
exception, then I think you are in the minority. But I have no evidence to
back this up, it's just a belief.
In my world of professional java, ignoring exceptions is an immediate,
obvious indicator of bad code. You will be called on it, and chastised
appropriately. So from my standpoint, Sun was successful in making
programmers more diligent about handling errors.
Post by Steven Schveighoffer via Digitalmars-d
Note I am not advocating adding checked exceptions to D (though I would
Post by Jeremy Powers via Digitalmars-d
like it). Point is to acknowledge that there are different kinds of
exceptions, and an exception for one part of the code may not be a
problem for the bit that invokes it.
I think this is appropriate for a lint tool for those out there like
yourself who want that information. But requiring checked exceptions is I
think a futile attempt to outlaw natural human behavior.
Perhaps I shouldn't have mentioned checked exceptions at all, seem to be
distracting from what I wanted to say. The important bit I wanted to bring
to the discussion is that not all exceptions are the same, and different
sections of code have their own ideas of what is a breaking problem. A
module/library/component/whatever treats any input into itself as its
input, and thus appropriately throws exceptions on bad input. But code
using that whatever may be perfectly fine handling exceptions coming from
there.

Exceptions need to be appropriate to the given abstraction, and dealt with
by the user of that abstraction.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140930/5271274d/attachment.html>
Bruno Medeiros via Digitalmars-d
2014-10-01 14:24:47 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
Checked exceptions are good in theory but they failed utterly in
Java. I'm not interested in seeing them in D.
That is the conventional theory, the established wisdom.
But the more I become experienced with Java, over the years, I've become
convinced otherwise.

What has failed is not the concept of checked exceptions per se, but
mostly, the failure of Java programmers to use checked exceptions
effectively, and properly design their code around this paradigm.

Like Jeremy mentioned, if one puts catch blocks right around the
function that throws an exception, and just swallow/forget it there
without doing anything else, then it's totally the programmers fault for
being lazy.

If one is annoyed that often, adding a throws clause in a function will
require adding the same throws clause function to several other
functions, well, that is editing work you have to accept for the sake of
more correctness. But also one should understand there are ways to
mitigate this editing work:

First point is that in a lot of code, is better to have a function throw
just one generic (but checked) exception, that can wrap any other
specific errors/exceptions. If you are doing an operation that can throw
File-Not-Found, Invalid-Path, No-Permissions, IO-Exception, etc., then
often all of these will be handled in the same user-reporting code, so
they could be wrapped under a single exception that would be used in the
throws clause. And so the whole function call chain doesn't need to be
modified every time a new exception is added or removed.

If you're thinking that means adding a "throws Exception" to such
functions in Java, then no. Because this will catch RuntimeExceptions
too (the unchecked exceptions of Java), and these you often want to
handle elsewhere than where you handle the checked exceptions. In this
regard, Java does have a design fault, IMO, which is that there is no
common superclass for checked Exceptions. (there is only for unchecked
exceptions)

The second point, is that even adding (or modifying) the throws clause
of function signatures cause be made much easier with an IDE, and in
particular Eclipse JDT helps a lot. If you have an error in the editor
about a checked exception that is not caught or thrown, you can just
press Ctrl-1 to automatically add either a throws clause, or a
surrounding try-catch block.
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jeremy Powers via Digitalmars-d
2014-10-01 22:42:17 UTC
Permalink
On Wed, Oct 1, 2014 at 7:24 AM, Bruno Medeiros via Digitalmars-d <
Post by Bruno Medeiros via Digitalmars-d
What has failed is not the concept of checked exceptions per se, but
mostly, the failure of Java programmers to use checked exceptions
effectively, and properly design their code around this paradigm.
This.

I have seen many java programs (and their programmers) fail utterly at
using exceptions. Like OO, the design of the system has to leverage it
properly, and there are places where people can easily get tripped up - but
when used well, can be immensely useful.

Error handling is part of an API, and exceptions are error handling, so
should be considered when designing API. Checked exceptions are a
language-supported way to do this.

For those that consider checked exceptions a failure: what other feature(s)
would work instead?


NB:
If you see "throws Exception" in java code chances are the code is broken,
same as if you see "catch (Exception" - this tells you nothing about the
exception that happened, and hence you can do nothing with it. So you
either swallow (and silently break in many cases) or rethrow (and break for
things you needn't have). As mentioned, the standard way to avoid this is
to have a parent exception type appropriate to the abstraction in the API,
and throw subtypes in the implementation. Among other things, this means
you can change the implementation to throw different exceptions without
breaking any users (who will already be catching the parent exception).
Adding/modifying a throws clause is an API-breaking change, so should be
avoided however easy it is in the IDE.
(Yes, I'm biased to writing libraries consumed by others)

NNB: Retrofitting a program to use proper exception handling is much harder
than it is to design it the right way from scratch. I'm going to compare
to OO again: don't consider OO broken because people use inheritance when
they want ownership, and it is painful to fix later.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20141001/93b0b551/attachment.html>
Sean Kelly via Digitalmars-d
2014-10-01 23:00:41 UTC
Permalink
On Wednesday, 1 October 2014 at 22:42:27 UTC, Jeremy Powers via
Post by Jeremy Powers via Digitalmars-d
If you see "throws Exception" in java code chances are the code
is broken,
same as if you see "catch (Exception" - this tells you nothing
about the
exception that happened, and hence you can do nothing with it.
So you
either swallow (and silently break in many cases) or rethrow
(and break for things you needn't have). As mentioned, the
standard way to avoid this is to have a parent exception type
appropriate to the abstraction in the API, and throw subtypes
in the implementation. Among other things, this means you can
change the implementation to throw different exceptions without
breaking any users (who will already be catching the parent
exception).
...while in Phobos, most of the subtyped exceptions were
eliminated a while back in favor of just always throwing
Exception.
David Nadlinger via Digitalmars-d
2014-10-01 23:19:15 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
...while in Phobos, most of the subtyped exceptions were
eliminated a while back in favor of just always throwing
Exception.
What are you referring to specifically? Compared to Tango, yes,
Phobos might have a lot fewer concrete exception types. But I
don't recall actually eliminating existing ones.

David
Jacob Carlborg via Digitalmars-d
2014-10-02 06:39:16 UTC
Permalink
What are you referring to specifically? Compared to Tango, yes, Phobos
might have a lot fewer concrete exception types. But I don't recall
actually eliminating existing ones.
It happens implicitly when using "enforce". By default it will throw an
instance of Exception. In most cases I don't think the developer bothers
with specifying a specific exception type.

I really, really hate this. It makes it basically impossible to do any
form of error handling correctly. I think Exception should be an
interface or abstract class.
--
/Jacob Carlborg
David Nadlinger via Digitalmars-d
2014-10-03 12:36:49 UTC
Permalink
Post by Jacob Carlborg via Digitalmars-d
Post by David Nadlinger via Digitalmars-d
What are you referring to specifically? Compared to Tango,
yes, Phobos
might have a lot fewer concrete exception types. But I don't
recall actually eliminating existing ones.
It happens implicitly when using "enforce".
I'm well aware of that. But as
Post by Jacob Carlborg via Digitalmars-d
...while in Phobos, most of the subtyped exceptions were
eliminated a while back in favor of just always throwing
Exception.
you are saying that specific exceptions were replaced by enforce?
I can't recall something like this happening.

David
Jacob Carlborg via Digitalmars-d
2014-10-03 19:35:01 UTC
Permalink
you are saying that specific exceptions were replaced by enforce? I
can't recall something like this happening.
I have no idea about this but I know there are a lot of "enforce" in
Phobos and it sees to be encouraged to use it. Would be really sad if
specific exceptions were deliberately replaced with less specific
exceptions.
--
/Jacob Carlborg
Andrei Alexandrescu via Digitalmars-d
2014-10-01 23:19:32 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
On Wednesday, 1 October 2014 at 22:42:27 UTC, Jeremy Powers via
Post by Jeremy Powers via Digitalmars-d
If you see "throws Exception" in java code chances are the code is
broken,
same as if you see "catch (Exception" - this tells you nothing about the
exception that happened, and hence you can do nothing with it. So you
either swallow (and silently break in many cases) or rethrow (and
break for things you needn't have). As mentioned, the standard way to
avoid this is to have a parent exception type appropriate to the
abstraction in the API, and throw subtypes in the implementation.
Among other things, this means you can change the implementation to
throw different exceptions without breaking any users (who will
already be catching the parent exception).
....while in Phobos, most of the subtyped exceptions were
eliminated a while back in favor of just always throwing
Exception.
My recollection is that was only talked about. Anyhow, one thing is
clear - as of now there are no clear idioms and successful techniques
for handling errors with exceptions (including the use of subtyping). --
Andrei
Jacob Carlborg via Digitalmars-d
2014-10-02 06:36:27 UTC
Permalink
Post by Andrei Alexandrescu via Digitalmars-d
My recollection is that was only talked about. Anyhow, one thing is
clear - as of now there are no clear idioms and successful techniques
for handling errors with exceptions (including the use of subtyping). --
Andrei
I think most error handling is done with "enforce", this will, by
default, throw an instance of Exception. I really, really hate this. I
think Exception should be an abstract class or an interface.
--
/Jacob Carlborg
Jacob Carlborg via Digitalmars-d
2014-10-01 19:27:35 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
Is it? I can think of cases where it's programmer error, and cases where
it's user error.
When would it be a user error?
--
/Jacob Carlborg
Steven Schveighoffer via Digitalmars-d
2014-10-01 19:57:59 UTC
Permalink
Post by Jacob Carlborg via Digitalmars-d
Post by Steven Schveighoffer via Digitalmars-d
Is it? I can think of cases where it's programmer error, and cases where
it's user error.
When would it be a user error?
./progThatExpectsFilename ""

-Steve
Jacob Carlborg via Digitalmars-d
2014-10-02 06:45:21 UTC
Permalink
Post by Steven Schveighoffer via Digitalmars-d
./progThatExpectsFilename ""
-Steve
It's the developer's responsibility to make sure a value like that never
reaches the "File" constructor. That is, the developer of the
"progThatExpectsFilename" application that uses "File". Not the
developer of "File".

Although, I don't see why you shouldn't be able to pass an empty string
to "File". You'll just get an exception, "cannot open file ''". This is
exactly what happens in Bash:

$ echo "asd" > ""
-bash: : No such file or directory
--
/Jacob Carlborg
ketmar via Digitalmars-d
2014-09-28 01:30:58 UTC
Permalink
On Sat, 27 Sep 2014 16:15:40 -0700
After all, what would you think of a compiler that spewed out
dmd test.d
test.d(15) Error: missing } thrown from dmd/src/parse.c(283)
?
"wow, that's cool! one more pleasing feature!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20140928/e12d80ff/attachment.sig>
luka8088 via Digitalmars-d
2014-09-28 09:20:25 UTC
Permalink
Post by Walter Bright via Digitalmars-d
https://www.youtube.com/watch?v=hBhlQgvHmQ0
In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
assert()s and contracts are for debugging programs.
dmd test.d
test.d(15) Error: missing } thrown from dmd/src/parse.c(283)
?
https://issues.dlang.org/show_bug.cgi?id=13543
As for the programmer wanting to know where the message "missing }" came from,
li
grep -r dmd/src/*.c "missing }"
works nicely. I do that sort of thing all the time. It really isn't a problem.
We has this issue at work (we are working with php). We outputted a
stack trace for both exceptions and asserts but the lines that should be
addressed are not always so obvious.

I found a solution and it works great for us. All library code is marked
appropriately so when stack is outputted it is shadows out (with gray
color) all the lines in library code and point out first non-library
line from the top of the stack. In 95% of the time it is the line that
the programmer should look into. Other 5% is the time when it shows the
line where programmer is forwarding a call to the library but turns out
to be ok as it turns out to be much more comprehensible than the entire
stack. One note worth mentioning is that juniors have much easier time
understanding which lines concern them, and from that I can only
conclude that such approach is more intuitive.

Marking is done on namespace level so it can be easily disabled for
entire namespace.

I think outputting a stack trace for asserts is a must because of that
5%. And for exceptions I agree completely with your arguments and I
think that there is no need for stack.

From my experience this has been a good approach and I think is worth
considering.
bearophile via Digitalmars-d
2014-09-28 09:28:13 UTC
Permalink
All library code is marked appropriately so when stack is
outputted it is shadows out (with gray color) all the lines in
library code and point out first non-library line from the top
of the stack. In 95% of the time it is the line that the
programmer should look into. Other 5% is the time when it shows
the line where programmer is forwarding a call to the library
but turns out to be ok as it turns out to be much more
comprehensible than the entire stack. One note worth mentioning
is that juniors have much easier time understanding which lines
concern them, and from that I can only conclude that such
approach is more intuitive.
This looks like a little enhancement request, to colourize
differently the parts of the stack trace of the library
code/runtime from the user code.
And for exceptions I agree completely with your arguments and I
think that there is no need for stack.
I think Walter is not suggesting to remove the stack trace for
exceptions.

Bye,
bearophile
Walter Bright via Digitalmars-d
2014-09-28 17:36:12 UTC
Permalink
And for exceptions I agree completely with your arguments and I think that
there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions.
I suggest removal of stack trace for exceptions, but leaving them in for asserts.

Asserts are a deliberately designed debugging tool. Exceptions are not.
Andrei Alexandrescu via Digitalmars-d
2014-09-28 17:49:10 UTC
Permalink
Post by Walter Bright via Digitalmars-d
And for exceptions I agree completely with your arguments and I think that
there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions.
I suggest removal of stack trace for exceptions, but leaving them in for asserts.
Asserts are a deliberately designed debugging tool. Exceptions are not.
I'm fine with that philosophy, with the note that it's customary
nowadays to inflict things like the stack trace on the user. -- Andrei
bearophile via Digitalmars-d
2014-09-28 18:25:25 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving
them in for asserts.
I suggest to keep stack trace for both cases, and improve it with
colors :-) Another possibility is to keep the stack trace for
exceptions in nonrelease mode only.
Post by Walter Bright via Digitalmars-d
Asserts are a deliberately designed debugging tool. Exceptions
are not.
Exceptions are often used to help debugging... We have even
allowed exceptions inside D contracts (but I don't know why).

Bye,
bearophile
Walter Bright via Digitalmars-d
2014-09-28 19:32:03 UTC
Permalink
Post by bearophile via Digitalmars-d
Exceptions are often used to help debugging...
https://www.youtube.com/watch?v=hBhlQgvHmQ0
luka8088 via Digitalmars-d
2014-09-28 20:02:35 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Post by bearophile via Digitalmars-d
Exceptions are often used to help debugging...
https://www.youtube.com/watch?v=hBhlQgvHmQ0
Example exception messages:

Unable to connect to database
Invalid argument count
Invalid network package format

All this messages do not require a stack trace as they do not require
code fixes, they indicate an issue outside the program itself. If stack
trace is required then assert should have been used instead.

Or to better put it: can anyone give an example of exception that would
require stack trace?
Jacob Carlborg via Digitalmars-d
2014-09-28 19:33:33 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving them in for asserts.
If you don't like the stack track, just wrap the "main" function in a
try-catch block, catch all exceptions and print the error message.
--
/Jacob Carlborg
Walter Bright via Digitalmars-d
2014-09-28 20:14:43 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving them in for asserts.
If you don't like the stack track, just wrap the "main" function in a try-catch
block, catch all exceptions and print the error message.
That's what the runtime that calls main() is supposed to do.
Marco Leise via Digitalmars-d
2014-10-01 12:14:15 UTC
Permalink
Am Sun, 28 Sep 2014 13:14:43 -0700
Post by Walter Bright via Digitalmars-d
Post by Walter Bright via Digitalmars-d
I suggest removal of stack trace for exceptions, but leaving them in for asserts.
If you don't like the stack track, just wrap the "main" function in a try-catch
block, catch all exceptions and print the error message.
That's what the runtime that calls main() is supposed to do.
Guys, a druntime flag could settle matters in 10 minutes.
But this topic is clearly about the right school of thought.

I use contracts to check for logical errors, like when an
argument must not be null or a value less than the length of
some data structure.
I use exceptions to check for invalid input and the return
values of external libraries. External libraries can be
anything from my own code in the same project to OpenGL from
vendor XY. They could error out on valid input (if we leave
out of memory aside for now), because of bugs or incorrect
assumptions of the implementation.
If that happens and all I get is:
"Library XY Exception: code 0x13533939 (Invalid argument)."
I'm at a loss, where the library might have had a hickup.
Did some function internally handle a uint as an int and
wrapped around?
Maybe with std.logger we will see single line messages on the
terminal and multi-line exception traces in the logs (which by
default print to stderr as well). And then this discussion can
be resolved.
--
Marco
Sean Kelly via Digitalmars-d
2014-09-28 19:34:27 UTC
Permalink
Post by Walter Bright via Digitalmars-d
And for exceptions I agree completely with your arguments and I think that
there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions.
I suggest removal of stack trace for exceptions, but leaving
them in for asserts.
Asserts are a deliberately designed debugging tool. Exceptions
are not.
Fair. So we generate traces for Errors but not Exceptions.
Ary Borenszweig via Digitalmars-d
2014-09-28 15:10:24 UTC
Permalink
Post by Walter Bright via Digitalmars-d
https://www.youtube.com/watch?v=hBhlQgvHmQ0
In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
assert()s and contracts are for debugging programs.
For me, assert is useless.

We are developing a language using LLVM as our backend. If you give LLVM
something it doesn't like, you get something this:

~~~
Assertion failed: (S1->getType() == S2->getType() && "Cannot create
binary operator with two operands of differing type!"), function Create,
file Instructions.cpp, line 1850.

Abort trap: 6
~~~

That is what the user gets when there is a bug in the compiler, at least
when we are generating invalid LLVM code. And that's one of the good
paths, if you compiled LLVM with assertions, because otherwise I guess
it's undefined behaviour.

What I'd like to do, as a compiler, is to catch those errors and tell
the user: "You've found a bug in the app, could you please report it in
this URL? Thank you.". We can't: the assert is there and we can't change it.

Now, this is when you interface with C++/C code. But inside our language
code we always use exceptions so that programmers can choose what to do
in case of an error. With assert you loose that possibility.

Raising an exception is costly, but that should happen in exceptional
cases. Installing an exception handler is cost-free, so I don't see why
there is a need for a less powerful construct like assert.
Xiao Xie via Digitalmars-d
2014-09-28 16:25:41 UTC
Permalink
On Sunday, 28 September 2014 at 15:10:26 UTC, Ary Borenszweig
Post by Ary Borenszweig via Digitalmars-d
What I'd like to do, as a compiler, is to catch those errors
and tell the user: "You've found a bug in the app, could you
please report it in this URL? Thank you.". We can't: the assert
is there and we can't change it.
Why is SIGABRT handler not working for your usecase? print and
exit?
Walter Bright via Digitalmars-d
2014-09-28 17:40:47 UTC
Permalink
Post by Ary Borenszweig via Digitalmars-d
For me, assert is useless.
We are developing a language using LLVM as our backend. If you give LLVM
~~~
Assertion failed: (S1->getType() == S2->getType() && "Cannot create binary
operator with two operands of differing type!"), function Create, file
Instructions.cpp, line 1850.
Abort trap: 6
~~~
That is what the user gets when there is a bug in the compiler, at least when we
are generating invalid LLVM code. And that's one of the good paths, if you
compiled LLVM with assertions, because otherwise I guess it's undefined behaviour.
"You've found a bug in the app, could you please report it in this URL? Thank
you.". We can't: the assert is there and we can't change it.
You can hook D's assert and do what you want with it.
Post by Ary Borenszweig via Digitalmars-d
Now, this is when you interface with C++/C code. But inside our language code we
always use exceptions so that programmers can choose what to do in case of an
error. With assert you loose that possibility.
If you want to use Exceptions for debugging in your code, I won't try and stop
you. But using them for debugging in official Phobos I strongly object to.
Post by Ary Borenszweig via Digitalmars-d
Installing an exception handler is cost-free,
Take a look at the assembler dump from std.file.copy() that I posted in the
other thread.
Post by Ary Borenszweig via Digitalmars-d
so I don't see why there is a need
for a less powerful construct like assert.
Exceptions are meant for RECOVERABLE errors. If you're using them instead of
assert for logic bugs, you're looking at undefined behavior. Logic bugs are not
recoverable.
Sean Kelly via Digitalmars-d
2014-09-28 19:38:09 UTC
Permalink
Post by Walter Bright via Digitalmars-d
You can hook D's assert and do what you want with it.
With the caveat that you must finish by either exiting the app or
throwing an exception, since the compiler doesn't generate a
stack frame that can be returned from.
Post by Walter Bright via Digitalmars-d
Exceptions are meant for RECOVERABLE errors. If you're using
them instead of assert for logic bugs, you're looking at
undefined behavior. Logic bugs are not recoverable.
In a multithreaded program, does this mean that the thread must
be terminated or the entire process? In a multi-user system,
does this mean the transaction or the entire process? The scope
of a logic bug can be known to be quite limited. Remember my
earlier point about Erlang, where a "process" there is actually
just a logical thread in the VM.
Walter Bright via Digitalmars-d
2014-09-28 20:31:01 UTC
Permalink
Post by Walter Bright via Digitalmars-d
Exceptions are meant for RECOVERABLE errors. If you're using them instead of
assert for logic bugs, you're looking at undefined behavior. Logic bugs are
not recoverable.
In a multithreaded program, does this mean that the thread must be terminated or
the entire process? In a multi-user system, does this mean the transaction or
the entire process? The scope of a logic bug can be known to be quite limited.
Remember my earlier point about Erlang, where a "process" there is actually just
a logical thread in the VM.
This has been asked many times before.

If the threads share memory, the only robust choice is to terminate all the
threads and the application.

If the thread is in another process, where the memory is not shared, then
terminating and possibly restarting that process is quite acceptable.
The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of inserting
recovery code for unknown problems. I can't really accept that one has "unknown
bugs of known scope".
Sean Kelly via Digitalmars-d
2014-09-28 20:50:29 UTC
Permalink
Post by Walter Bright via Digitalmars-d
If the threads share memory, the only robust choice is to
terminate all the threads and the application.
If the thread is in another process, where the memory is not
shared, then terminating and possibly restarting that process
is quite acceptable.
The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already
instead of inserting recovery code for unknown problems. I
can't really accept that one has "unknown bugs of known scope".
Well, say you're using SafeD or some other system where you know
that memory corruption is not possible (pure functional
programming, for example). In this case, if you know what data a
particular execution flow touches, you know the scope of the
potential damage. And if the data touched is all either shared
but read-only or generated during the processing of the request,
you can be reasonably certain that nothing outside the scope of
the transaction has been adversely affected at all.
Dmitry Olshansky via Digitalmars-d
2014-09-28 21:16:20 UTC
Permalink
Post by Walter Bright via Digitalmars-d
If the threads share memory, the only robust choice is to terminate
all the threads and the application.
If the thread is in another process, where the memory is not shared,
then terminating and possibly restarting that process is quite
acceptable.
The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of
inserting recovery code for unknown problems. I can't really accept
that one has "unknown bugs of known scope".
Well, say you're using SafeD or some other system where you know that
memory corruption is not possible (pure functional programming, for
example).
In this case, if you know what data a particular execution
flow touches, you know the scope of the potential damage. And if the
data touched is all either shared but read-only or generated during the
processing of the request, you can be reasonably certain that nothing
outside the scope of the transaction has been adversely affected at all.
not possible / highly unlikely (i.e. bug in VM or said system)

But otherwise agreed, dropping the whole process is not always a good
idea or it easily becomes a DoS attack vector in a public service.
--
Dmitry Olshansky
Sean Kelly via Digitalmars-d
2014-09-28 21:21:25 UTC
Permalink
On Sunday, 28 September 2014 at 21:16:51 UTC, Dmitry Olshansky
Post by Dmitry Olshansky via Digitalmars-d
But otherwise agreed, dropping the whole process is not always
a good idea or it easily becomes a DoS attack vector in a
public service.
What I really want to work towards is the Erlang model where an
app is a web of communicating processes (though Erlang processes
are effectively equivalent to D objects). Then, killing a
process on an error is absolutely correct. It doesn't affect the
resilience of the system. But if these processes are actually
threads or fibers with memory protection, things get a lot more
complicated. I really need to spend some time investigating how
modern Linux systems handle tons of processes running on them and
try to find a happy medium.
Dmitry Olshansky via Digitalmars-d
2014-09-28 21:28:32 UTC
Permalink
Post by Dmitry Olshansky via Digitalmars-d
But otherwise agreed, dropping the whole process is not always a good
idea or it easily becomes a DoS attack vector in a public service.
What I really want to work towards is the Erlang model where an app is a
web of communicating processes (though Erlang processes are effectively
equivalent to D objects). Then, killing a process on an error is
absolutely correct. It doesn't affect the resilience of the system.
But if these processes are actually threads or fibers with memory
protection, things get a lot more complicated.
One thing I really appreciated about JVM is exactly the memory safety
with ability to handle this pretty much in the same way Erlang does.
I really need to spend
some time investigating how modern Linux systems handle tons of
processes running on them and try to find a happy medium.
Keep us posted.
--
Dmitry Olshansky
Walter Bright via Digitalmars-d
2014-09-28 22:00:22 UTC
Permalink
Post by Walter Bright via Digitalmars-d
The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of
inserting recovery code for unknown problems. I can't really accept that one
has "unknown bugs of known scope".
Well, say you're using SafeD or some other system where you know that memory
corruption is not possible (pure functional programming, for example). In this
case, if you know what data a particular execution flow touches, you know the
scope of the potential damage. And if the data touched is all either shared but
read-only or generated during the processing of the request, you can be
reasonably certain that nothing outside the scope of the transaction has been
adversely affected at all.
You may know the error is not a memory corrupting one, but that doesn't mean
there aren't non-corrupting changes to the shared memory that would result in
additional unexpected failures. Also, the logic bug may be the result of an
@system part of the code going wrong. You do not know, because YOU DO NOT KNOW
the cause the error. And if you knew the cause, you wouldn't need a stack trace
to debug it anyway.

I.e. despite being 'safe' it does not imply the program is in a predictable or
anticipated state.

I can't get behind the notion of "reasonably certain". I certainly would not use
such techniques in any code that needs to be robust, and we should not be using
such cowboy techniques in Phobos nor officially advocate their use.
Sean Kelly via Digitalmars-d
2014-09-29 01:17:03 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I can't get behind the notion of "reasonably certain". I
certainly would not use such techniques in any code that needs
to be robust, and we should not be using such cowboy techniques
in Phobos nor officially advocate their use.
I think it's a fair stance not to advocate this approach. But as
it is I spend a good portion of my time diagnosing bugs in
production systems based entirely on archived log data, and
analyzing the potential impact on the system to determine the
importance of a hot fix. The industry seems to be moving towards
lowering the barrier between engineering and production code
(look at what Netflix has done for example), and some of this
comes from an isolation model akin to the Erlang approach, but
the typical case is still that hot fixing code is incredibly
expensive and so you don't want to do it if it isn't necessary.
For me, the correct approach may simply be to eschew assert() in
favor of enforce() in some cases. But the direction I want to be
headed is the one you're encouraging. I simply don't know if
it's practical from a performance perspective. This is still
developing territory.
Walter Bright via Digitalmars-d
2014-09-29 03:04:10 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I can't get behind the notion of "reasonably certain". I certainly would not
use such techniques in any code that needs to be robust, and we should not be
using such cowboy techniques in Phobos nor officially advocate their use.
I think it's a fair stance not to advocate this approach. But as it is I spend
a good portion of my time diagnosing bugs in production systems based entirely
on archived log data, and analyzing the potential impact on the system to
determine the importance of a hot fix. The industry seems to be moving towards
lowering the barrier between engineering and production code (look at what
Netflix has done for example), and some of this comes from an isolation model
akin to the Erlang approach, but the typical case is still that hot fixing code
is incredibly expensive and so you don't want to do it if it isn't necessary.
For me, the correct approach may simply be to eschew assert() in favor of
enforce() in some cases. But the direction I want to be headed is the one
you're encouraging. I simply don't know if it's practical from a performance
perspective. This is still developing territory.
You've clearly got a tough job to do, and I understand you're doing the best you
can with it. I know I'm hardcore and uncompromising on this issue, but that's
where I came from (the aviation industry).

I know what works (airplanes are incredibly safe) and what doesn't work
(Toyota's approach was in the news not too long ago). Deepwater Horizon and
Fukushima are also prime examples of not dealing properly with modest failures
that cascaded into disaster.
Kagamin via Digitalmars-d
2014-10-03 11:27:18 UTC
Permalink
Post by Walter Bright via Digitalmars-d
You've clearly got a tough job to do, and I understand you're
doing the best you can with it. I know I'm hardcore and
uncompromising on this issue, but that's where I came from (the
aviation industry).
I know what works (airplanes are incredibly safe) and what
doesn't work (Toyota's approach was in the news not too long
ago). Deepwater Horizon and Fukushima are also prime examples
of not dealing properly with modest failures that cascaded into
disaster.
Do you interpret airplane safety right? As I understand,
airplanes are safe exactly because they recover from assert
failures and continue operation. Your suggestion is when seat 2A
creaks, shut down the whole airplane. In reality airplanes
continue to operate until there's zero physical resource to
operate. Fukushima caused disaster because it didn't try to
handle failure. But this is your idea that one can do nothing
meaningful on failure, and Fukushima did just that: nothing.

Termination of the process is the safe default, especially in the
case of client software, but servers should probably terminate
failed request, gracefully clean up and continue operation, like
airplanes.
Jacob Carlborg via Digitalmars-d
2014-10-03 12:16:30 UTC
Permalink
Do you interpret airplane safety right? As I understand, airplanes are
safe exactly because they recover from assert failures and continue
operation. Your suggestion is when seat 2A creaks, shut down the whole
airplane. In reality airplanes continue to operate until there's zero
physical resource to operate.
I have no idea of airplane works but I think Walter usual says they have
at least three backup systems. If one system fails, shut it down and
switch to the backup.
--
/Jacob Carlborg
Sean Kelly via Digitalmars-d
2014-10-03 15:43:57 UTC
Permalink
Post by Jacob Carlborg via Digitalmars-d
Post by Kagamin via Digitalmars-d
Do you interpret airplane safety right? As I understand,
airplanes are
safe exactly because they recover from assert failures and
continue
operation. Your suggestion is when seat 2A creaks, shut down
the whole airplane. In reality airplanes continue to operate
until there's zero physical resource to operate.
I have no idea of airplane works but I think Walter usual says
they have at least three backup systems. If one system fails,
shut it down and switch to the backup.
My point, and I think Kagamin's as well, is that the entire plane
is a system and the redundant internals are subsystems. They may
not share memory, but they are wired to the same sensors, servos,
displays, etc. Thus the point about shutting down the entire
plane as a result of a small failure is fair.
via Digitalmars-d
2014-10-03 16:10:58 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
My point, and I think Kagamin's as well, is that the entire
plane is a system and the redundant internals are subsystems.
They may not share memory, but they are wired to the same
sensors, servos, displays, etc. Thus the point about shutting
down the entire plane as a result of a small failure is fair.
An airplane is a bad analogy for a regular server. You have
redundant backups everywhere and you are not allowed to take off
at the smallest sign of deviation from normal operation. You will
never see D in a fighter jet (and you can probably not fly it
without the controller in operation either, your only choice is
to send the plane into the ocean and escape in a parachute).

I think Walter forgets that you ensure integrity of a complex
system of servers by utilizing a rock solid proven transaction
database/task-scheduler for handling all critical information. If
that fails, you probably should shut down everything, roll back
to the last backup and reboot.

But you don't shut down a restaurant because the waiter forgets
to write down an order every once in a while, you shut it down if
the kitchen is unsuitable for preparing food. After sanitizing
the kitchen you open the restaurant again. You also don't fire
the sloppy waiter until you have a better waiter at hand

Piotrek via Digitalmars-d
2014-10-03 17:33:31 UTC
Permalink
On Friday, 3 October 2014 at 16:11:00 UTC, Ola Fosheim GrÞstad
Post by via Digitalmars-d
Post by Sean Kelly via Digitalmars-d
My point, and I think Kagamin's as well, is that the entire
plane is a system and the redundant internals are subsystems.
They may not share memory, but they are wired to the same
sensors, servos, displays, etc. Thus the point about shutting
down the entire plane as a result of a small failure is fair.
An airplane is a bad analogy for a regular server. You have
redundant backups everywhere and you are not allowed to take
off at the smallest sign of deviation from normal operation.
That depends on design (logic). Ever heard of this?

http://www.reddit.com/r/programming/comments/1ax0oa/how_kdes_1500_git_repositories_almost_were_lost/
Post by via Digitalmars-d
I think Walter forgets that you ensure integrity of a complex
system of servers by utilizing a rock solid proven transaction
database/task-scheduler for handling all critical information.
If that fails, you probably should shut down everything, roll
back to the last backup and reboot.
I agree with Walter wholeheartedly. If I get him correctly he
speaks about distinction between the program logic and input
errors. Not about recovery strategies/decisions.
Post by via Digitalmars-d
But you don't shut down a restaurant because the waiter forgets
to write down an order every once in a while, you shut it down
if the kitchen is unsuitable for preparing food. After
sanitizing the kitchen you open the restaurant again. You also
don't fire the sloppy waiter until you have a better waiter at
hand

Let me play the game of finding analogies ;)
IMO, an exception is more suitable for the analogy with waiter
and dirty kitchen.
A logic error would be a case when you think you are running a
garage but suddenly you noticed your stuff is selling meals and
is wearing chef's uniforms.

Piotrek
via Digitalmars-d
2014-10-03 19:05:49 UTC
Permalink
Post by Piotrek via Digitalmars-d
That depends on design (logic). Ever heard of this?
http://www.reddit.com/r/programming/comments/1ax0oa/how_kdes_1500_git_repositories_almost_were_lost/
How is not having redundant storage, logging or backup related to
this?

This is a risk assessment scenario: «What are we willing to loose
compared to what it costs to mitigate the risks by investing in
additional resources?»
Post by Piotrek via Digitalmars-d
A logic error would be a case when you think you are running a
garage but suddenly you noticed your stuff is selling meals and
is wearing chef's uniforms.
But it is a business decision whether it is better to take
amazon.com off the network for a week or just let their search
engine occasionally serve food instead of books as search
results. Not an engineering decision.

It is a business decision whether it is better for a game to
corrupt 1% of user accounts and let customer support manually
build them back up than to take the game off the network until
the problem is fixed. You would probably have heavier load on
customer support and loose more subscriptions by taking the game
off the network than giving those 1% one year of free game play
as a compensation.

If you have a logic error in a functional routine, it is local.
It might not matter, it might be expected.

Logic errors do not imply memory corruption.
Memory corruption does not imply that exceptions are thrown.

Even if memory corruption would lead to exceptions being thrown
in 30% of the cases, you'd still have 70% of cases where memory
corruption goes undetected. So if that is a concern, you need to
focus elsewhere.

You have to think about this in probabilistic terms and relate it
to business decisions.

Defining thresholds for acceptable reliability is not an
engineering decision. An engineering decision is to use isolates,
Erlang, Haskell etc to achieve the thresholds set as acceptable
reliability/quality viewed from a business point of view.
Piotrek via Digitalmars-d
2014-10-03 18:00:57 UTC
Permalink
Post by Sean Kelly via Digitalmars-d
My point, and I think Kagamin's as well, is that the entire
plane is a system and the redundant internals are subsystems.
They may not share memory, but they are wired to the same
sensors, servos, displays, etc. Thus the point about shutting
down the entire plane as a result of a small failure is fair.
This "real life" example:

http://en.wikipedia.org/wiki/Air_France_Flight_447

I just pick some interesting statements (there are other factors
described as well):

"temporary inconsistency between the measured speeds, likely as a
result of the obstruction of the pitot tubes by ice crystals,
causing autopilot disconnection and reconfiguration to alternate
law;"


And as I can see it, all subsystems related to the "small
failure" was shut down. But what is also important information
was not clearly provided to the pilots:

"Despite the fact that they were aware that altitude was
declining rapidly, the pilots were unable to determine which
instruments to trust: it may have appeared to them that all
values were incoherent"

"the cockpit lacked a clear display of the inconsistencies in
airspeed readings identified by the flight computers;"

Piotrek
Sean Kelly via Digitalmars-d
2014-10-03 18:17:48 UTC
Permalink
Post by Piotrek via Digitalmars-d
And as I can see it, all subsystems related to the "small
failure" was shut down. But what is also important information
"Despite the fact that they were aware that altitude was
declining rapidly, the pilots were unable to determine which
instruments to trust: it may have appeared to them that all
values were incoherent"
"the cockpit lacked a clear display of the inconsistencies in
airspeed readings identified by the flight computers;"
There's a similar issue with nuclear reactors, which is that
there are so many blinky lights and such that it can be
impossible to spot or prioritize problems in a failure scenario.
I know there have been articles written on revisions of user
interface design in reactors specifically to deal with this
issue, and I suspect the ideas are applicable to error handling
in general.
Paolo Invernizzi via Digitalmars-d
2014-10-03 20:31:40 UTC
Permalink
Post by Piotrek via Digitalmars-d
http://en.wikipedia.org/wiki/Air_France_Flight_447
I just pick some interesting statements (there are other
"temporary inconsistency between the measured speeds, likely as
a result of the obstruction of the pitot tubes by ice crystals,
causing autopilot disconnection and reconfiguration to
alternate law;"
And as I can see it, all subsystems related to the "small
failure" was shut down. But what is also important information
"Despite the fact that they were aware that altitude was
declining rapidly, the pilots were unable to determine which
instruments to trust: it may have appeared to them that all
values were incoherent"
"the cockpit lacked a clear display of the inconsistencies in
airspeed readings identified by the flight computers;"
Piotrek
As one that has read the original report integrally, I think that
you have taken a bad example: despite the autopilot was
disengaged, the stall alarm ringed a pletora of times.

There's no real alternative to the disengagement of the autopilot
is that fundamental parameter is compromised.

It took the captain only a few moment to understand the problem
(read the voice-recording transcription), but it was too late...

---
/Paolo
Piotrek via Digitalmars-d
2014-10-03 22:27:43 UTC
Permalink
Post by Paolo Invernizzi via Digitalmars-d
As one that has read the original report integrally, I think
that you have taken a bad example: despite the autopilot was
disengaged, the stall alarm ringed a pletora of times.
My point was that the broken speed indicators shut down the
autopilot systems.

Piotrek
Piotrek via Digitalmars-d
2014-10-03 22:27:43 UTC
Permalink
Post by Paolo Invernizzi via Digitalmars-d
As one that has read the original report integrally, I think
that you have taken a bad example: despite the autopilot was
disengaged, the stall alarm ringed a pletora of times.
My point was that the broken speed indicators shut down the
autopilot systems.

Piotrek
eles via Digitalmars-d
2014-10-04 05:26:50 UTC
Permalink
Post by Paolo Invernizzi via Digitalmars-d
As one that has read the original report integrally, I think
that you have taken a bad example: despite the autopilot was
disengaged, the stall alarm ringed a pletora of times.
There's no real alternative to the disengagement of the
autopilot is that fundamental parameter is compromised.
It took the captain only a few moment to understand the problem
(read the voice-recording transcription), but it was too late...
For the curious, the flight analysis here:

http://www.popularmechanics.com/technology/aviation/crashes/what-really-happened-aboard-air-france-447-6611877

Captain's first error was to leave the cockpit when approaching
storm. Second, was to give command to the lower-experienced
co-pilots. Not only for quality of the flight, but also for the
quality of the team. His third error was to have neglected the
fact that the radar was not correctly set up. And his most
important (and final) error was to not take commands back while
coming back in to the cockpit.

And it was airliner's fault to embark on transatlantic flights
just one experienced man and two very low experienced copilots.

This is a beginner mistake:

"the insanity of pulling back on the controls while stalled"

and this passage resumes it quite well:

"the captain of the flight makes no attempt to physically take
control of the airplane. Had Dubois done so, he almost certainly
would have understood, as a pilot with many hours flying light
airplanes, the insanity of pulling back on the controls while
stalled"

If you read the analysis you get scared.
Walter Bright via Digitalmars-d
2014-10-04 08:08:50 UTC
Permalink
Do you interpret airplane safety right? As I understand, airplanes are safe
exactly because they recover from assert failures and continue operation.
Nope. That's exactly 180 degrees from how it works.

Any airplane system that detects a fault shuts itself down and the backup is
engaged. No way in hell is software allowed to continue that asserted.
Bruno Medeiros via Digitalmars-d
2014-10-01 13:17:07 UTC
Permalink
Post by Walter Bright via Digitalmars-d
I can't get behind the notion of "reasonably certain". I certainly would
not use such techniques in any code that needs to be robust,
I know I'm hardcore and uncompromising on this issue, but that's where I
came from (the aviation industry).
Walter, you do understand that not all software has to be robust - in
the critical systems sense - to be quality software? And that in fact,
the majority of software is not critical systems software?...

I was under the impression that D was meant to be a general purpose
language, not a language just for critical systems. Yet, on language
design issues, you keep making a series or arguments and points that
apply *only* to critical systems software.
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
Bruno Medeiros via Digitalmars-d
2014-10-01 13:44:27 UTC
Permalink
Post by Walter Bright via Digitalmars-d
https://www.youtube.com/watch?v=hBhlQgvHmQ0
In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.
When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.
This is incorrect.

Yes, the primary purpose of Exceptions is not for debugging, but to
report exceptional state to the user (or some other component of the
system).

But they also have a purpose for debugging, particularly the stack
traces of exceptions. Take what you said:

"Failure to respond properly to an input/environmental error is a bug.
But the input/environmental error is not a bug. If it was, then the
program should assert on the error, not throw. "

So, some component/function Foo detects an environmental error, and
throws and Exception, accordingly. Foo is not responsible for handling
these errors, but some other component is.

Component/function Bar is the one that should handle such an error (for
example, it should display a dialog to the user, and continue the
application). But due to a bug, it doesn't do so, and the Exception goes
all the way through main().

The programmer notices this happening, and clearly recognizes it's a bug
(but doesn't know where the bug is, doesn't know that it's Bar that
should be handling it). Now, what is best, to just have the Exception
message (something like "File not found") displayed to the programmer -
or even an end-user that could report a bug -, or to have the stack
trace of the Exception so that the programmer can more easily look at
which function should be handling it?
--
Bruno Medeiros
https://twitter.com/brunodomedeiros
Loading...