Graph based Image Segmentation

My GSoC project this year is Graph based segmentation algorithms using region adjacency graphs. Community binding period is coming to an end. I have experimented a bit with Region Adjacency Graphs (RAGs) and Minimum Spanning Trees (MSTs) with this ugly piece of Python code.  I will try to describe in brief what I plan to do during this GSoC period.

Certain image segmentation algorithms have a tendency to over segment an image. They divide a region as perceived by humans into two or more regions. This is because they tend to favor small regions of similar color. But in the real world one object might have different shades of the same color or different colors all together. Here is an example using SLIC. In broad terms SLIC is k-means done on (X,Y, Z ) color space

We consider each of these regions as a vertex in a graph. Each region is connected to all the regions that touch it. Similar regions are joined with an edge of less weight. Dissimilar regions are joined with edges oh high weight. One measure of dissimilarity might be difference in the mean color. See the below example.

If we remove the edges with higher weights in an appropriate manner, the regions remaining connected would belong to the same object. Thus in this case the face, the hat, the hair might be finally one connected subgraph of regions. Over the next two weeks I will try to take an over segmented image and build its RAG. As a proof of concept of the underlying data structures and algorithms I will apply a threshold and remove the edges with weights higher than it. Later on I will move onto to more complicated selection procedures including N-cut and if my MST experiments yield good results an MST based procedure.

The with statement

Python’s with statement has bewildered me for a while. I never really felt the need to use it. So in this post I’m going to try to explore a bit more about it. A common example of with is to use it for files

```
with open("file.txt","w") as f:
f.write("hello\n")
f.write("bye")

```

Let’s create our own class and see if we can use it with the with statement.

```
class MyClass:
def __init__(self):
print "Initializing an Awesome Class"

def doSomething(self):
print "Doing Something"

with MyClass() as myObject:
myObject.doSomething()

```

And…..we have an error

```
Initializing an Awesome Class
Traceback (most recent call last):
File "test.py", line 10, in <module>
with MyClass() as myObject:
AttributeError: MyClass instance has no attribute '__exit__'

```

Upon closer inspection you’ll notice that Python complains about our Class not having an __exit__ method, so let’s give it one.

```

class MyClass:
def __init__(self):
print "Initializing an Awesome Class"

def doSomething(self):
print "Doing Something"

def __exit__(self):
print "Just Exiting as one should"

with MyClass() as myObject:
myObject.doSomething()

```

Arrgghhh….another error

This time Python complains about an __enter__ method. All right Mr. Python Interpreter, as you say.

```

class MyClass:
def __init__(self):
print "Initializing an Awesome Class"

def doSomething(self):
print "Doing Something"

def __exit__(self):
print "Just Exiting as one should"

def __enter__(self):
print "Knock Knock"

with MyClass() as myObject:
myObject.doSomething()

```

What now ?

```
Initializing an Awesome Class
Knock Knock
Traceback (most recent call last):
File "test.py", line 16, in <module>
myObject.doSomething()
TypeError: __exit__() takes exactly 1 argument (4 given)

```

So __enter__ is working fine,but Python is supplying 4 arguments to __exit__
Let’s change the prototype and see what those 4 arguments are.

```class MyClass:
def __init__(self):
print "Initializing an Awesome Class"

def doSomething(self):
print "Doing Something"

def __exit__(self,arg1,arg2,arg3):
print "Just Exiting as one should"
print 'arg1 = ',arg1,'of type ',type(arg1)
print 'arg2 = ',arg2,'of type ',type(arg2)
print 'arg3 = ',arg3,'of type ',type(arg3)

def __enter__(self):
print "Knock Knock"

with MyClass() as myObject:
myObject.doSomething()
```

And………another error

We can infer from this traceback that myObject is None. The __exit__ method tells us more about the error that occurred . This highlights the philosophy behind the __exit__ method. It is supposed to handle any errors that might occur in the with clause. The arguments to it are the type of error, the error object and the traceback object respectively. It might also be used to perform clean up if no errors occur. For eg: Closing a file.

The with clause relies on __enter__ and __exit__ . The object returned by one of these must go into myObject. Since __exit__ would be performed at the end, this leaves __enter__ to return the required object.

I am just returning the object itself, but it can return anything that can be intended to be used in the with clause.

```class MyClass:
def __init__(self):
print "Initializing an Awesome Class"

def doSomething(self):
print "Doing Something"

def __exit__(self,arg1,arg2,arg3):
print "Just Exiting as one should"
print 'arg1 = ',arg1,'of type ',type(arg1)
print 'arg2 = ',arg2,'of type ',type(arg2)
print 'arg3 = ',arg3,'of type ',type(arg3)

def __enter__(self):
print "Knock Knock"
return self

with MyClass() as myObject:
myObject.doSomething()
```

Finalllyyy

```Initializing an Awesome Class
Knock Knock
Doing Something
Just Exiting as one should
arg1 =  None of type  <type 'NoneType'>
arg2 =  None of type  <type 'NoneType'>
arg3 =  None of type  <type 'NoneType'>
```

As you can see, since no errors occur, the __exit__ method is supplied all None arguments.

The point behind explaining it this way is to highlight Python’s amazing debugging ability. We have seen the usage and predicted the philosophy behind the with clause without referring the docs. But I’ll highly recommend reading them to get the complete picture.

What I’m using

For anyone ( specially non-programmers ) who want to take a shot at Image processing SimpleCV is the easiest way. It’s easy to use, written for Python and reasonably fast for most applications. I happen to be a GSoC student for the same this year and I’m working to improve it’s display system ( more about it here )

You can visit the Homepage to read up and pre-configured installers.

Or to be extra awesome you can clone the latest source code from here.

Let’s get going

```>>> from SimpleCV import *
```

Start by importing all names in the SimpleCV namespace.

```>>> img = Image('lenna') # strings like 'lenna' , 'lyle'
#and 'logo' load default stored images
>>> img.show()  # calling show() displays the image
>>> img = Image('lyle')
>>> img.show()
>>> img = Image('file.png') # specifying a filename
#will load an image from the file if it's present
>>> img.show()
```

The R,G,B colorspace

Images are stored in memory by storing the amount of Red, Green and Blue Colors. Let’s see this in action

```>>> img = Image((500,500))
>>> img.show()
```

The above code will create a small image 500×500 pixels wide. It’s all black initially :(.

```>>> print img[25,37]
(0.0, 0.0, 0.0)
>>> print img[25,37,0]
```

The red,green and clue color values are stored in this case as an integer between 0 and 255. 0 means that color isn’t present and 255 means the color is present as much as can be.The images can be accessed as matrices . img[x,y] signifies a pixel in the row number x and column number y. Each pixel is associated with 3 values, for R,G and B in that order. As python correctly tells us, all 3 colors are 0. The same is true for all x and y values as of now.

```>>> for i in range(100,200):
>>>     for j in range(30,70):
>>>         img[i,j] = 255,0,0
>>> img.show()
```

If you run the above code you’ll notice a small, bright red rectangle on the image. The code sets the red value of all pixels in the range (100:200,30:70)

Please note, this in not the correct way to draw shapes, I’m just using for loops for illustrations

```>>> for i in range(400,450):
>>>     for j in range(130,170):
>>>         img[i,j] = 128,0,0
>>> img.show()
```

This code draws a second rectangle. But this time around the rectangle is a little dull. That’s because the red value is 128.

```>>> for i in range(0,50):
>>>     for j in range(200,300):
>>>         img[i,j] = 0,255,0
>>> img.show()
```

Wow ! This gives us green
We can have many combinations of colors ( 16581375 to be precise ). The following snippet shows a few examples

```from SimpleCV import *

def paintSquare(img,x,y,color):
"""
draws a square at x,y in img, with the given color of 50px X 50px
"""
for i in range(x,x+50):
for j in range(y,y+50):
img[i,j] = color

img = Image((500,500))

# red
color = (255,0,0)
paintSquare(img,50,34,color)

# green
color = (0,255,0)
paintSquare(img,156,200,color)

#blue
color = (0,0,255)
paintSquare(img,300,340,color)

#yellow
color = (255,255,0)
paintSquare(img,450,300,color)

#pink
color = (255,0,255)
paintSquare(img,30,100,color)

#white
color = (255,255,255)
paintSquare(img,50,34,color)

#gray (ish)
color = (100,100,100)
paintSquare(img,450,200,color)

#aqua
color = (0,255,255)
paintSquare(img,100,100,color)

#skintone
color = (239,208,207)
paintSquare(img,440,440,color)

img.show()
input()
```

The code snippet contains a function paintSquare which paints a 50 x 50 px square at the given position.

Note that all 255 shades of gray(:D) have equal R,G,B values, with (0,0,0) being black and (255,255,255) being white. (120,120,120) will be greyish.

After This

This was a post to illustrate the very basic fundamentals of images. You can go to http://tutorial.simplecv.org/en/latest/ to learn more things. If you would like to see anything else explained by me , comment below.

NITK Automata 2012 Winning Run

Me and 2 of my friends , Dhruv and Sanket participated at NIT Surathkal in a contest called Automata. The task was to get the robot to traverse the maze. A Laptop was connected to an overhead camera . The laptop solved the maze, plotted a virtual path and directed the robot successfully to the center.

How  the computer saw the maze

The three yellow dots are the center of the maze and the two colored blobs on the robot. The green dots represent the path that the robot is supposed to follow.

Things we used

Python, OpenCV for Python and wxWidgets.The computer directed the robot over Bluetooth, communicating through pySerial.

Ultra Quick GUIs with wxFormBuilder/Python

I’ve been a fan of wxFormBuilder for a a long time. It allows programmers like me to develop highly polished GUIs for Real World Applications or quick, temporary GUIs just to get the job done. To follow the below steps won’t take more than 15 minutes. The Goal is to build a usable calculator using wxPython and wxFoemBuilder

Things you’ll need

Any versions will do as long as they are inter compatible. I used Python 2.7.3 (32-bit), wxPython 2.8.12.1 (msw-unicode) and wxFormBuilder 3.2.3-beta (unicode)

Install all the above in thee order mentioned and we should be good to go.

Enter wxFormBuilder

Start wxFormBuilder and you’ll be greeted with New Project with a blank grey area in the center. Go to the Object Properties Window on the right and change name,file and set code generation to Python

From the Forms tab Choose Frame

Choose The Recently Added Frame from the object Tree and change name to MainFrame

Add a wxBoxSizer from Layout tab , make sure that orient is wxVERTICAL from the Object Properties

From the Common tab add TexCtrl and 2 Buttons.

Select each of these 3 elements and enable Expand and Stretch

Select the TextCtrl and change the name to text

Choose any button and rename it to solveButton and change the label to Solve

In the Events Tab change the OnButtonClick value to solveFunc. This will be the function called when the button is clicked.

Similarly rename the other Button to clearButton and change OnButtonCick to clearFunc

Your window should be looking similar to this by now.

Save The Project and hit F8 to generate code. You should end up with a file called gui.py in the directory you saved in. This file holds the code for generating the graphics.

Almost There

The final bit is writing the python code. Create a new python file in the same directory, and copy this code into it

```#importing wx files
import wx

#import the newly created GUI file
import gui

#importing * : to enable writing sin(13) instead of math.sin(13)
from math import *

#inherit from the MainFrame created in wxFowmBuilder and create CalcFrame
class CalcFrame(gui.MainFrame):
#constructor
def __init__(self,parent):
#initialize parent class
gui.MainFrame.__init__(self,parent)

#what to when 'Solve' is clicked
#wx calls this function with and 'event' object
def solveFunc(self,event):
try:
#evaluate the string in 'text' and put the answer back
ans = eval(self.text.GetValue())
self.text.SetValue (str(ans))
except Exception:
print 'error'
#put a blank string in text when 'Clear' is clicked
def clearFunc(self,event):
self.text.SetValue(str(''))

#mandatory in wx, create an app, False stands for not deteriction stdin/stdout
#refer manual for details
app = wx.App(False)

#create an object of CalcFrame
frame = CalcFrame(None)
#show the frame
frame.Show(True)
#start the applications
app.MainLoop()

```

The File contains less than 20 Lines of functional code. All hail Python !!

If you are using IDLE on windows you can run this by clicking on Run Module and on Linux you can use python file.py where file.py is the file with the above contents. ( Thanks No more commandline )

Usage

Insert any valid mathematical expression in the Text Box and get the result

Shallow Copy in Python

When you say something-else = something in Python creates a shallow copy of that something. This might be new for someone migrating from a C/C++ background and it might take a newbie a long time to realize ( sometimes after a lot of hairloss ).

When you write y=x, y will now point to the location of the object x was pointing to.This can be shown with the help of the id() built-in function

```>>> a = [0,1]
>>> b = a
>>> b[0] = 1
>>> print b
[1, 1]
>>> print a
[1, 1]
>>> print id(a),id(b)
44504048 44504048
```

This might always be that apparent, observe the following code

The reason is, Integers unlike Lists are immutable.When we try to change the value of y , a new integer object is created and assigned to y, since changing its value isn’t possible.

```>>> x = 100
>>> y = x
>>> print id(x),id(y)
31959540 31959540
>>> y = 200
>>> print y
200
>>> print x
100
>>> print id(x),id(y)
31959540 31960324
```

Leave comments for any further clarification or to point out errors .