|  | core features and functions required in modern programming l | | |
| | | Guest |  |
|  Posted: Tue Aug 05, 2008 10:35 am Post subject: Re: core features and functions required in modern programmi |  |
| |  | |
On 5 Aug., 05:29, "robertwess...@yahoo.com" <robertwess...@yahoo.com>
wrote:
| Quote: | On Aug 4, 8:41 pm, Joe Wright <joewwri...@comcast.net> wrote:
In general, I feel that floating point gets a bad rap for being
approximate rather than exact. FP can be as exact as you want it to be.
You should try doing an acceptable job dealing with currency in
(binary) float. Trust me - it's hard.
|
I have seen quite often the use of float to represent monetary
amounts. The people draw the wrong conclusion that numbers
with decimal point must be represented as float. Later when
inaccuracies show up they start to use all sorts of rounding in
between and still do not succeed to get the correct results.
Such inaccuracies happen even with simple operations like
summing up a large list of values.
I have removed such inaccuracies by deciding for a unit of 1c or
0.01c and using an int or long to represent the amount.
I don't say that this solution works always but I was able to remove
all inaccuracies once and for all.
Greetings Thomas Mertes
Seed7 Homepage: LINK
Seed7 - The extensible programming language: User defined statements
and operators, abstract data types, templates without special
syntax, OO with interfaces and multiple dispatch, statically typed,
interpreted or compiled, portable, runs under linux/unix/windows. |
| |
| | | Richard Heathfield | |
| | Posted: Tue Aug 05, 2008 12:15 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
thomas.mertes@gmx.at said:
<snip>
| Quote: | AFAIK there are exact rules (by law) how this calculation should be
done and at which places rounding should take place to some given
precision
|
Right.
| Quote: | (E.g.: The account is never allowed to carry any sub cent
amounts).
|
Less right.  I mean yes, there could be situations where that's true,
but banking and insurance aren't two of them.
| Quote: | I cannot see any advantage that floats could give me here,
but I have seen the disadvantages of floats and monetary amounts
in the past.
|
In the UK all intermediate financial results (in banking and insurance, at
least) have to be accurate to within plus or minus a millionth of a penny.
If you take this as your base unit, then an unsigned 32-bit integer can't
even store a 43-pound (or dollar) balance.
| Quote: | If you can do this calculation accurately in 32-bit integers (and that
also means no bignums, by the way), you deserve a pat on the back.
So use 64-bit integers.
|
In practice, in the real financial world, doubles are routinely used. (So,
at the other extreme, is binary coded decimal!)
| Quote: | BTW.: Isn't something as simple as 0.1 not representable exactly as
IEEE 754 floating point?
|
It's certainly not representable in pure binary. The closest you can get in
16 bits, for example, is 6553/65536, which is 0.0999908447265625 - not too
shabby, really. On my system, a double appears to respond to a request to
store 0.1 by storing this instead:
0.1000000000000000055511151231257827021181583404541015625
which is certainly adequate for financial work.
| Quote: | With the following Seed7 program I found out that 8071 summations
of 0.1 with single precision floating points (32 bits wide) cause
an error of 0.1 (when rounded to 0.1):
|
Yes. Single-precision floats don't cut it.
| Quote: | Accountants sometimes spend days with searching for some tiny amount
in some big calculation (such as a 1? difference in calculations
with a total amount of 1234567?).
I have removed such inaccuracies by deciding for a unit of 1c or
0.01c and using an int or long to represent the amount.
I don't say that this solution works always but I was able to remove
all inaccuracies once and for all.
It doesn't work for rollup calcs in the general case.
Why?
|
Let's imagine a bank account that pays 5% pa, calculated daily (so that's
0.01336806171134% per day on the close-of-day balance, applied 365 days a
year or 366 days in leap years) on accounts with 500 or more coming in
each month, but the interest rate is significantly higher for balances of
300,000 or more. A customer opened an account on 5/1/2001 with the minimum
balance of 10, but later that same day day he paid in his wages of 500,
and every 30th day thereafter he has paid in 500. (On the 900th day that
the account was open, the customer received an inheritance of 125537.20
which he paid into the account. This is the only break in the savings
pattern, though.) Assuming he keeps up this pattern, on what day will the
new rate cut in for this customer?
If you do this calculation using 32-bit integers, I would be very surprised
indeed if you manage to get the correct date.
--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999 |
| |
| | | Bartc | |
| | Posted: Tue Aug 05, 2008 12:24 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
"Richard Heathfield" <rjh@see.sig.invalid> wrote in message
news:fOOdndxzcNDHwgXVnZ2dnUVZ8sXinZ2d@bt.com...
| Quote: | Let's imagine a bank account that pays 5% pa, calculated daily (so that's
0.01336806171134% per day on the close-of-day balance, applied 365 days a
year or 366 days in leap years) on accounts with 500 or more coming in
each month, but the interest rate is significantly higher for balances of
300,000 or more. A customer opened an account on 5/1/2001 with the minimum
balance of 10, but later that same day day he paid in his wages of 500,
and every 30th day thereafter he has paid in 500. (On the 900th day that
the account was open, the customer received an inheritance of 125537.20
which he paid into the account. This is the only break in the savings
pattern, though.) Assuming he keeps up this pattern, on what day will the
new rate cut in for this customer?
|
On the day the balance reaches or passes 300,000. If the actual day that
happens is wrong by a day or two (and I make it a couple of decades in the
future), who's ever going to know?
--
Bartc |
| |
| | | Richard Heathfield | |
| | Posted: Tue Aug 05, 2008 12:34 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
Bartc said:
| Quote: | "Richard Heathfield" <rjh@see.sig.invalid> wrote in message
news:fOOdndxzcNDHwgXVnZ2dnUVZ8sXinZ2d@bt.com...
Let's imagine a bank account that pays 5% pa, calculated daily (so
that's 0.01336806171134% per day on the close-of-day balance, applied
365 days a year or 366 days in leap years) on accounts with 500 or more
coming in each month, but the interest rate is significantly higher for
balances of 300,000 or more. A customer opened an account on 5/1/2001
with the minimum balance of 10, but later that same day day he paid in
his wages of 500, and every 30th day thereafter he has paid in 500. (On
the 900th day that the account was open, the customer received an
inheritance of 125537.20 which he paid into the account. This is the
only break in the savings pattern, though.) Assuming he keeps up this
pattern, on what day will the new rate cut in for this customer?
On the day the balance reaches or passes 300,000. If the actual day that
happens is wrong by a day or two (and I make it a couple of decades in
the future),
|
You are out by a number of *years*.
| Quote: | who's ever going to know?
|
The internal auditors, who can slap your wrist if you get it wrong. The
external auditors, who might well get you fired if you get it wrong. The
customer (if he's sufficiently clued-up, and he might well be - bankers
and actuaries have bank accounts too!), who can take you to court if you
get it wrong. Or the Financial Services Authority, who can actually close
you down if you get it wrong.
--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999 |
| |
| | | Guest | |
| | Posted: Tue Aug 05, 2008 1:07 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
On 5 Aug., 12:51, Richard Heathfield <r...@see.sig.invalid> wrote:
| Quote: | thomas.mer...@gmx.at said:
On 5 Aug., 05:29, "robertwess...@yahoo.com" <robertwess...@yahoo.com
wrote:
snip
You should try doing an acceptable job dealing with currency in
(binary) float. Trust me - it's hard.
I have seen quite often the use of float to represent monetary
amounts. The people draw the wrong conclusion that numbers
with decimal point must be represented as float.
Sometimes it's unavoidable.
I had a case where it was necessary to create a float just to enter |
a value in a database table.
| Quote: | Later when
inaccuracies show up they start to use all sorts of rounding in
between and still do not succeed to get the correct results.
Such inaccuracies happen even with simple operations like
summing up a large list of values.
They certainly happen when /multiplying/ a list of values. Consider, for
example, a loan of 270000 (local currency units) over 25 years at 6.25%
calculated daily and added annually, repayments to be made monthly, and
all repayments to be the same amount except for the last, which is
permitted to be slightly but not drastically lower to make everything add
up.
AFAIK there are exact rules (by law) how this calculation should be |
done and at which places rounding should take place to some given
precision (E.g.: The account is never allowed to carry any sub cent
amounts). I cannot see any advantage that floats could give me here,
but I have seen the disadvantages of floats and monetary amounts
in the past.
| Quote: | If you can do this calculation accurately in 32-bit integers (and that also
means no bignums, by the way), you deserve a pat on the back.
So use 64-bit integers. |
With integers you don't have the problem that you carry some minimal
values around that show up at a totally unrelated place. You can
print all your calculations and the people can verify all the steps
with manual calculation and nobody will find a place where one cent
(or 0.01c or whatever) suddenly shows up without any explanation.
The formal rules for such calculations were defined with manual
calculations (on paper) in mind and not with IEEE 754 floating
points as concept.
BTW.: Isn't something as simple as 0.1 not representable exactly as
IEEE 754 floating point?
With the following Seed7 program I found out that 8071 summations
of 0.1 with single precision floating points (32 bits wide) cause
an error of 0.1 (when rounded to 0.1):
$ include "seed7_05.s7i";
include "float.s7i";
const proc: main is func
local
var float: accumulator is 0.0;
var integer: count is 0;
var integer: number is 0;
var string: stri is "";
begin
for number range 10 to 8080 do
accumulator := 0.0;
for count range 1 to number do
accumulator +:= 0.1;
end for;
stri := str(number);
stri := stri[.. pred(length(stri))] & "." &
stri[length(stri) len 1];
if stri <> (accumulator digits 1) then
writeln(number <& " " <& accumulator <& " " <& stri);
end if;
end for;
end func;
I verified this with the following C program:
# include <stdio.h>
int main (void)
{
float accumulator = 0.0;
int max_count = 8071;
int count;
for (count = 1; count <= max_count; count++) {
accumulator += 0.1;
} /* for */
printf("%d * 0.1 = %0.14f rounded = %0.1f should be %d.%d\n",
max_count, accumulator, accumulator,
max_count / 10, max_count % 10);
return 0;
}
Accountants sometimes spend days with searching for some tiny amount
in some big calculation (such as a 1€ difference in calculations
with a total amount of 1234567€).
| Quote: | I have removed such inaccuracies by deciding for a unit of 1c or
0.01c and using an int or long to represent the amount.
I don't say that this solution works always but I was able to remove
all inaccuracies once and for all.
It doesn't work for rollup calcs in the general case.
|
Why?
Greetings Thomas Mertes
Seed7 Homepage: LINK
Seed7 - The extensible programming language: User defined statements
and operators, abstract data types, templates without special
syntax, OO with interfaces and multiple dispatch, statically typed,
interpreted or compiled, portable, runs under linux/unix/windows. |
| |
| | | Bartc | |
| | Posted: Tue Aug 05, 2008 1:30 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
"Richard Heathfield" <rjh@see.sig.invalid> wrote in message
news:Cdadnd0Me99A_gXVnZ2dnUVZ8h2dnZ2d@bt.com...
| Quote: | Bartc said:
"Richard Heathfield" <rjh@see.sig.invalid> wrote in message
news:fOOdndxzcNDHwgXVnZ2dnUVZ8sXinZ2d@bt.com...
Let's imagine a bank account that pays 5% pa, calculated daily (so
that's 0.01336806171134% per day on the close-of-day balance, applied
365 days a year or 366 days in leap years) on accounts with 500 or more
coming in each month, but the interest rate is significantly higher for
balances of 300,000 or more. A customer opened an account on 5/1/2001
with the minimum balance of 10, but later that same day day he paid in
his wages of 500, and every 30th day thereafter he has paid in 500. (On
the 900th day that the account was open, the customer received an
inheritance of 125537.20 which he paid into the account. This is the
only break in the savings pattern, though.) Assuming he keeps up this
pattern, on what day will the new rate cut in for this customer?
On the day the balance reaches or passes 300,000. If the actual day that
happens is wrong by a day or two (and I make it a couple of decades in
the future),
You are out by a number of *years*.
|
That was just a guess, and I forgot the interest is much higher from day
900.
My code calculates it as 4384th day of opening (5 Jan 2013), without
applying rounding. Rounding each day to 0.01 units delayed it by 1 day.
Calculations using 64-bit 'doubles'.
What was the real answer?
--
Bartc |
| |
| | | Richard Heathfield | |
| | Posted: Tue Aug 05, 2008 2:01 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
thomas.mertes@gmx.at said:
| Quote: | On 5 Aug., 16:15, Richard Heathfield <r...@see.sig.invalid> wrote:
thomas.mer...@gmx.at said:
snip
AFAIK there are exact rules (by law) how this calculation should be
done and at which places rounding should take place to some given
precision
Right.
(E.g.: The account is never allowed to carry any sub cent
amounts).
Less right. I mean yes, there could be situations where that's
true, but banking and insurance aren't two of them.
I cannot see any advantage that floats could give me here,
but I have seen the disadvantages of floats and monetary amounts
in the past.
In the UK all intermediate financial results (in banking and insurance,
at least) have to be accurate to within plus or minus a millionth of a
penny. If you take this as your base unit, then an unsigned 32-bit
integer can't even store a 43-pound (or dollar) balance.
An IEEE 754 double-precision 64 bit has a mantissa of 52 bits. It
would not be precise to a millionth of a penny at approx 45035996.
|
Right. I'm not saying 64-bit doubles are always good enough. They aren't.
What I'm saying is that there are situations where 32-bit integers won't
cut it.
<snip>
| Quote: | If you get a bill you don't expect the amounts to be printed with an
accuracy of a millionth of a cent/penny.
|
Right - if you're just dealing with ordinary transactions where interest is
not an issue, and where the values aren't too colossal, integers work just
fine.
<snip>
| Quote: | BTW.: Isn't something as simple as 0.1 not representable exactly as
IEEE 754 floating point?
It's certainly not representable in pure binary. The closest you can get
in 16 bits, for example, is 6553/65536, which is 0.0999908447265625 -
not too shabby, really. On my system, a double appears to respond to a
request to store 0.1 by storing this instead:
0.1000000000000000055511151231257827021181583404541015625
For single precision 0.1 is stored as approx
0.10000000149 ...
|
Yes, but you don't use single precision if you care about the accuracy of a
long rollup calculation!
| Quote: | From that most people would guess that summing up this value a
million times would not cause a difference of 0.1 . But as my test
showed 8071 summations are enough to create such a difference.
|
This demonstrates why most people should leave financial programming well
alone. :-)
| Quote: | Therefore I am not so sure that you are at the safe side with
doubles.
|
It is wise to doubt when you don't know. But I *am* sure - because I've
been there, done that, had the fights with the actuarial people, been
through the audit process, the lot.
<snip>
--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999 |
| |
| | | Richard Heathfield | |
| | Posted: Tue Aug 05, 2008 2:06 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
Bartc said:
| Quote: |
"Richard Heathfield" <rjh@see.sig.invalid> wrote in message
news:Cdadnd0Me99A_gXVnZ2dnUVZ8h2dnZ2d@bt.com...
Bartc said:
"Richard Heathfield" <rjh@see.sig.invalid> wrote in message
news:fOOdndxzcNDHwgXVnZ2dnUVZ8sXinZ2d@bt.com...
Let's imagine a bank account that pays 5% pa, calculated daily (so
that's 0.01336806171134% per day on the close-of-day balance, applied
365 days a year or 366 days in leap years) on accounts with 500 or
more coming in each month, but the interest rate is significantly
higher for balances of 300,000 or more. A customer opened an account
on 5/1/2001 with the minimum balance of 10, but later that same day
day he paid in his wages of 500, and every 30th day thereafter he has
paid in 500. (On the 900th day that the account was open, the customer
received an inheritance of 125537.20 which he paid into the account.
This is the only break in the savings pattern, though.) Assuming he
keeps up this pattern, on what day will the new rate cut in for this
customer?
On the day the balance reaches or passes 300,000. If the actual day
that happens is wrong by a day or two (and I make it a couple of
decades in the future),
You are out by a number of *years*.
That was just a guess,
|
Ah! :-)
| Quote: | and I forgot the interest is much higher from day 900.
|
Aye.
| Quote: |
My code calculates it as 4384th day of opening (5 Jan 2013), without
applying rounding. Rounding each day to 0.01 units delayed it by 1 day.
Calculations using 64-bit 'doubles'.
|
With 64-bit doubles, I'd expect you to get the right answer, and indeed you
did.
| Quote: | What was the real answer?
|
As you calculated. 4384 days is precisely 12 years (including 3 leap
years).
I would expect integer-based calcs to slip the date by at least one day and
possibly more.
--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999 |
| |
| | | Guest | |
| | Posted: Tue Aug 05, 2008 3:30 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
On 5 Aug., 16:15, Richard Heathfield <r...@see.sig.invalid> wrote:
| Quote: | thomas.mer...@gmx.at said:
snip
AFAIK there are exact rules (by law) how this calculation should be
done and at which places rounding should take place to some given
precision
Right.
(E.g.: The account is never allowed to carry any sub cent
amounts).
Less right. I mean yes, there could be situations where that's true,
but banking and insurance aren't two of them.
I cannot see any advantage that floats could give me here,
but I have seen the disadvantages of floats and monetary amounts
in the past.
In the UK all intermediate financial results (in banking and insurance, at
least) have to be accurate to within plus or minus a millionth of a penny.
If you take this as your base unit, then an unsigned 32-bit integer can't
even store a 43-pound (or dollar) balance.
|
An IEEE 754 double-precision 64 bit has a mantissa of 52 bits. It
would not be precise to a millionth of a penny at approx 45035996.
For an account this is much, but for an insurance company it may be
necessary to deal with such an amount.
If you get a bill you don't expect the amounts to be printed with an
accuracy of a millionth of a cent/penny. I guess that the smallest
amount allowed on bills (by law) in the final sums will be a cent
(or penny). For other lines there will be let's say 0.01c precision.
The bill can contain many lines with subtotals and totals as netto
and brutto (incl. tax). You can always take your calculator (or a
sheet of paper) and recalculate the total sum (according to some
rules (e.g. sum up the netto ammounts and calculate the tax from the
sum)). If you cannot verify the sum up to the cent the company has
troubles. In my country there are really people who check bills this
way.
I was involved in creating such bills for a mobile phone company.
The bills for big customers contain lots and lots of lines. As long
as doubles were used, some tiny (cent) differences showed up from
time to time. In the moment I switched to long integers this
differences disapeared and they never came back.
| Quote: | If you can do this calculation accurately in 32-bit integers (and that
also means no bignums, by the way), you deserve a pat on the back.
So use 64-bit integers.
In practice, in the real financial world, doubles are routinely used. (So,
at the other extreme, is binary coded decimal!)
BTW.: Isn't something as simple as 0.1 not representable exactly as
IEEE 754 floating point?
It's certainly not representable in pure binary. The closest you can get in
16 bits, for example, is 6553/65536, which is 0.0999908447265625 - not too
shabby, really. On my system, a double appears to respond to a request to
store 0.1 by storing this instead:
0.1000000000000000055511151231257827021181583404541015625
|
For single precision 0.1 is stored as approx
0.10000000149 ...
From that most people would guess that summing up this value a
million times would not cause a difference of 0.1 . But as my test
showed 8071 summations are enough to create such a difference.
Therefore I am not so sure that you are at the safe side with
doubles.
| Quote: | which is certainly adequate for financial work.
With the following Seed7 program I found out that 8071 summations
of 0.1 with single precision floating points (32 bits wide) cause
an error of 0.1 (when rounded to 0.1):
Yes. Single-precision floats don't cut it.
|
I am too lazy to test it, but I guess that double precision floats
do it neither.
Greetings Thomas Mertes
Seed7 Homepage: LINK
Seed7 - The extensible programming language: User defined statements
and operators, abstract data types, templates without special
syntax, OO with interfaces and multiple dispatch, statically typed,
interpreted or compiled, portable, runs under linux/unix/windows. |
| |
| | | James Harris | |
| | Posted: Tue Aug 05, 2008 4:19 pm Post subject: Re: core features and functions required in modern programmi | |
| | | |
On 5 Aug, 15:24, "Bartc" <b...@freeuk.com> wrote:
| Quote: | "Richard Heathfield" <r...@see.sig.invalid> wrote in message
news:fOOdndxzcNDHwgXVnZ2dnUVZ8sXinZ2d@bt.com...
Let's imagine a bank account that pays 5% pa, calculated daily (so that's
0.01336806171134% per day on the close-of-day balance, applied 365 days a
year or 366 days in leap years) on accounts with 500 or more coming in
each month, but the interest rate is significantly higher for balances of
300,000 or more. A customer opened an account on 5/1/2001 with the minimum
balance of 10, but later that same day day he paid in his wages of 500,
and every 30th day thereafter he has paid in 500. (On the 900th day that
the account was open, the customer received an inheritance of 125537.20
which he paid into the account. This is the only break in the savings
pattern, though.) Assuming he keeps up this pattern, on what day will the
new rate cut in for this customer?
On the day the balance reaches or passes 300,000. If the actual day that
happens is wrong by a day or two (and I make it a couple of decades in the
future), who's ever going to know?
|
Whoa! Alarm bells should be ringing loudly here. "Who's ever going to
know?"??? I guess most people would not want their banks to get the
figures almost right. It's part of their duty of trust to calculate
correctly or at least to within defined parameters.
FWIW in my experience of writing software in the UK banking industry
we used decimal arithmetic for financial amounts. It was so long ago
that I cannot recall whether we calculated to pence or to some
specific decimal fraction of a penny, but I'm sure details of the
calculations were prescribed.
Imagine the bad publicity if a bank was found to have miscalculated
even slightly in their favour for a number of years. It's not so much
the financial losses to customers if they are negligible but the
appearance of incompetence of the bank. If banks can't calculate
finances correctly, ....
BTW, have you seen HawkEye used in tennis tournaments to rule on
whether a ball was in or out? I can't help feeling I'd rather see
three such machines from different manufacturers each give a verdict.
Then a majority decision could be taken. And the closeness of the
other two should give a measure of confidence or lack thereof in the
trajectories shown. At the moment full trust is placed in one computer
program. Not ideal, though it is the same for both players. OK I know
this is a little OT. |
| |
|
|
