|  | Is try-except slow? | | |
| | | ssecorp |  |
|  Posted: Tue Sep 02, 2008 11:44 pm Post subject: Is try-except slow? |  |
or why does this take so god damn long time?
and if I run into an IndexError it break out of the inner loop right?
so having range up to 10000000 or 1000 wouldn't matter if biggest
working x is 800?
def getPixels(fileName):
im = PIL.Image.open(fileName)
colors = []
for y in range(1, 1000):
row = []
for x in range(1, 1000):
try:
color = im.getpixel((x,y))
row.append(color)
except IndexError:
break
colors.append(row)
return numpy.array(colors) |
| |
| | | Robert Kern | |
| | Posted: Tue Sep 02, 2008 11:44 pm Post subject: Re: Is try-except slow? | |
| |  | |
ssecorp wrote:
| Quote: | or why does this take so god damn long time?
|
Several reasons. One of which is that try: except: is slow.
| Quote: | and if I run into an IndexError it break out of the inner loop right?
so having range up to 10000000 or 1000 wouldn't matter if biggest
working x is 800?
|
try: except: needs to set up a few things before the try: suite is executed. You
pay this cost regardless of whether the exception fires or not.
| Quote: | def getPixels(fileName):
im = PIL.Image.open(fileName)
colors = []
for y in range(1, 1000):
row = []
for x in range(1, 1000):
try:
color = im.getpixel((x,y))
row.append(color)
except IndexError:
break
|
Images have a size tuple attached to them. I recommend using that to get the
upper bounds of the iterations.
| Quote: | colors.append(row)
return numpy.array(colors)
|
Or, with reasonably modern versions of PIL and numpy:
In [1]: import Image
In [2]: im = Image.open('lena.png')
In [3]: import numpy
In [4]: numpy.asarray(im)
Out[4]:
array([[[228, 134, 132],
[228, 134, 132],
[228, 135, 130],
...,
[244, 151, 136],
[227, 132, 114],
[197, 102, 82]],
[[228, 135, 130],
[228, 135, 130],
[228, 135, 130],
...,
[237, 145, 124],
[219, 127, 102],
[191, 100, 73]],
[[227, 134, 129],
[227, 134, 129],
[227, 134, 127],
...,
[236, 147, 117],
[216, 130, 97],
[192, 106, 71]],
...,
[[ 87, 22, 56],
[ 89, 24, 58],
[ 90, 25, 59],
...,
[178, 67, 76],
[180, 65, 72],
[179, 62, 70]],
[[ 87, 22, 56],
[ 88, 23, 57],
[ 90, 25, 59],
...,
[183, 68, 75],
[188, 67, 72],
[190, 67, 72]],
[[ 86, 21, 55],
[ 88, 23, 57],
[ 90, 25, 59],
...,
[186, 70, 73],
[193, 70, 73],
[195, 71, 73]]], dtype=uint8)
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco |
| |
| | | John Machin | |
| | Posted: Wed Sep 03, 2008 12:36 am Post subject: Re: Is try-except slow? | |
| | | |
On Sep 3, 9:44 am, ssecorp <circularf...@gmail.com> wrote:
| Quote: | or why does this take so god damn long time?
|
Because your code does so many god damn unnecessary things. Apart from
the fact (as pointed out already by Robert) that you are needlessly
finding the sizes (Y, X) that are already available:
(1) You are executing a try block Y*X (approx) times instead of the Y+X
+2 times it would take if you were to do preliminary probes to find Y
and X
(2) You are doing range(1, 1000) Y times instead of once [see question
below]
(3) You are doing the method lookup im.getpixel Y*X times instead of
once
(4) you are doing the method lookup row.append Y*X times instead of Y
times
| Quote: | and if I run into an IndexError it break out of the inner loop right?
so having range up to 10000000 or 1000 wouldn't matter if biggest
working x is 800?
def getPixels(fileName):
im = PIL.Image.open(fileName)
colors = []
for y in range(1, 1000):
|
Are you sure that both occurrences of range(1, 1000) shouldn't be
range(1000)?
| Quote: | row = []
for x in range(1, 1000):
try:
color = im.getpixel((x,y))
row.append(color)
except IndexError:
break
colors.append(row)
return numpy.array(colors)
|
and it appears that you haven't bothered to read the manual section on
Image.getpixel:
"""
Note that this method is rather slow; if you need to process larger
parts of an image from Python, you can either use pixel access objects
(see load), or the getdata method.
""" |
| |
| | | process | |
| | Posted: Wed Sep 03, 2008 1:00 am Post subject: Re: Is try-except slow? | |
| | | |
On Sep 3, 2:36 am, John Machin <sjmac...@lexicon.net> wrote:
| Quote: | On Sep 3, 9:44 am, ssecorp <circularf...@gmail.com> wrote:
or why does this take so god damn long time?
Because your code does so many god damn unnecessary things. Apart from
the fact (as pointed out already by Robert) that you are needlessly
finding the sizes (Y, X) that are already available:
(1) You are executing a try block Y*X (approx) times instead of the Y+X
+2 times it would take if you were to do preliminary probes to find Y
and X
(2) You are doing range(1, 1000) Y times instead of once [see question
below]
(3) You are doing the method lookup im.getpixel Y*X times instead of
once
(4) you are doing the method lookup row.append Y*X times instead of Y
times
and if I run into an IndexError it break out of the inner loop right?
so having range up to 10000000 or 1000 wouldn't matter if biggest
working x is 800?
def getPixels(fileName):
im = PIL.Image.open(fileName)
colors = []
for y in range(1, 1000):
Are you sure that both occurrences of range(1, 1000) shouldn't be
range(1000)?
row = []
for x in range(1, 1000):
try:
color = im.getpixel((x,y))
row.append(color)
except IndexError:
break
colors.append(row)
return numpy.array(colors)
and it appears that you haven't bothered to read the manual section on
Image.getpixel:
"""
Note that this method is rather slow; if you need to process larger
parts of an image from Python, you can either use pixel access objects
(see load), or the getdata method.
"""
|
how could I do getpixel once when x and y s changing?
anyway I rewrote and saw I did a lot of stupid stuff. this is fast:
def getPixels5(fileName):
im = PIL.Image.open(fileName)
colors = []
r, c = im.size
for y in range(0, c):
row = []
for x in range(0, r):
color = im.getpixel((x,y))
row.append(color)
colors.append(row)
return numpy.array(colors)
but I don't need it anuyway apparently since there already was such
methods  |
| |
| | | process | |
| | Posted: Wed Sep 03, 2008 1:14 am Post subject: Re: Is try-except slow? | |
is this faster btw? I guess big doesn't help, it's only retrieved once
anyway? But is rows retrieved in every loop? the python interpreter
aint too smart?
def getPixels(fileName):
im = PIL.Image.open(fileName)
colors = []
r, c = im.size
big = range(0, c)
rows = range(0, r)
for y in big:
row = []
for x in rows:
color = im.getpixel((x,y))
row.append(color)
colors.append(row)
return numpy.array(colors) |
| |
| | | John Machin | |
| | Posted: Wed Sep 03, 2008 1:45 am Post subject: Re: Is try-except slow? | |
On Sep 3, 11:00 am, process <circularf...@gmail.com> wrote:
| Quote: | how could I do getpixel once when x and y s changing?
|
I was not referring to *calling* im.getpixel, I was referring to
looking up getpixel as an attribute of im. How? See below.
| Quote: |
anyway I rewrote and saw I did a lot of stupid stuff. this is fast:
def getPixels5(fileName):
im = PIL.Image.open(fileName)
colors = []
r, c = im.size
im_getpixel = im.getpixel
for y in range(0, c):
row = []
for x in range(0, r):
color = im.getpixel((x,y))
color = im_getpixel((x, y))
row.append(color)
colors.append(row)
return numpy.array(colors)
but I don't need it anuyway apparently since there already was such
methods |
|
| |
| | | John Machin | |
| | Posted: Wed Sep 03, 2008 1:54 am Post subject: Re: Is try-except slow? | |
On Sep 3, 11:14 am, process <circularf...@gmail.com> wrote:
| Quote: | is this faster btw?
|
Time it and see.
| Quote: | I guess big doesn't help, it's only retrieved once
anyway?
|
Correct.
| Quote: | But is rows retrieved in every loop?
|
Of course. Read your own code! The point is that retrieving the list
referenced by "rows" is much faster than creating it needlessly each
time.
| Quote: | the python interpreter
aint too smart?
|
It's smarter than some. The trade-off for having a very dynamic
language is that some compile-time optimisations are more difficult.
If you are interested in work (being)? done in this area, look at
psyco and PyPy. |
| |
| | | Steven D'Aprano | |
| | Posted: Wed Sep 03, 2008 3:43 am Post subject: Re: Is try-except slow? | |
| | | |
On Tue, 02 Sep 2008 18:56:48 -0500, Robert Kern wrote:
| Quote: | ssecorp wrote:
or why does this take so god damn long time?
Several reasons. One of which is that try: except: is slow.
|
I beg to differ. Setting up a try...except block is very fast. Here's an
example in Python 2.5:
| Quote: | from timeit import Timer
Timer('len("abc")').repeat()
[0.27346706390380859, 0.1530919075012207, 0.14886784553527832]
Timer('''try:
.... len("abc") |
.... except:
.... pass
.... ''').repeat()
[0.27847194671630859, 0.19191384315490723, 0.19077491760253906]
The difference (approx 0.04 microseconds) applicable to setting up the
try...except block is trivial, of the same magnitude as a pass statement:
| Quote: | Timer('pass').repeat()
[0.059719085693359375, 0.060056924819946289, 0.059512138366699219] |
However, *catching* the exception may be relatively slow:
| Quote: | Timer('''try:
.... len(abc) # raise a NameError |
.... except:
.... pass
.... ''').repeat()
[3.2067418098449707, 2.7088210582733154, 1.9558219909667969]
--
Steven |
| |
| | | Fredrik Lundh | |
| | Posted: Wed Sep 03, 2008 4:36 pm Post subject: Re: Is try-except slow? | |
process wrote:
| Quote: | is this faster btw? I guess big doesn't help, it's only retrieved once
anyway? But is rows retrieved in every loop? the python interpreter
aint too smart?
def getPixels(fileName):
im = PIL.Image.open(fileName)
colors = []
r, c = im.size
big = range(0, c)
rows = range(0, r)
for y in big:
row = []
for x in rows:
color = im.getpixel((x,y))
row.append(color)
colors.append(row)
return numpy.array(colors)
|
you'd probably get more done if you read the replies you get a bit more
carefully. Robert Kern suggesting using numpy.asarray earlier:
def getPixels(fileName):
im = PIL.Image.open(fileName)
return numpy.asarray(im)
if you want to work with pixels on the Python level, use im.getdata() or
the pixel access object returned by im.load().
</F> |
| |
| | | Robert Kern | |
| | Posted: Wed Sep 03, 2008 5:47 pm Post subject: Re: Is try-except slow? | |
| | | |
Steven D'Aprano wrote:
| Quote: | On Tue, 02 Sep 2008 18:56:48 -0500, Robert Kern wrote:
ssecorp wrote:
or why does this take so god damn long time?
Several reasons. One of which is that try: except: is slow.
I beg to differ. Setting up a try...except block is very fast. Here's an
example in Python 2.5:
from timeit import Timer
Timer('len("abc")').repeat()
[0.27346706390380859, 0.1530919075012207, 0.14886784553527832]
Timer('''try:
... len("abc")
... except:
... pass
... ''').repeat()
[0.27847194671630859, 0.19191384315490723, 0.19077491760253906]
The difference (approx 0.04 microseconds) applicable to setting up the
try...except block is trivial, of the same magnitude as a pass statement:
Timer('pass').repeat()
[0.059719085693359375, 0.060056924819946289, 0.059512138366699219]
However, *catching* the exception may be relatively slow:
Timer('''try:
... len(abc) # raise a NameError
... except:
... pass
... ''').repeat()
[3.2067418098449707, 2.7088210582733154, 1.9558219909667969]
|
You're right. My mistake. I was misremembering Guido's old essay about try:
except: versus if: else:.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco |
| |
|
|
