Python + PostgreSQL + Non English chars = Error?

[selimhanov]

Hi,
It must be a simple issue, but couldn't find any answer googling.
Here is the problem:
I have a simple test application with a TextCtrl, Button and StaticText. When I press the Button, it must query PostgreSQL with ...

SELECT name FROM contacts WHERE name LIKE '%"+TextCtrl.Value+"%'

and return the result.
Everything works fine when I run the application in Eclipse. But when I run the application in Terminal (by the way, I'm using Ubuntu(karmic) ) using command...

python UnicodeTest.py

everything works fine only if I enter English symbols but not Cyrillic (e.g. Я or Д). Entering non-English symbols ends with...

Traceback (most recent call last):
  File "UnicodeTest.py", line 28, in onClick
    db.execute("SELECT name FROM contacts WHERE name LIKE '%"+input.Value+"%'")
  File "/usr/lib/python2.6/dist-packages/pgdb.py", line 259, in execute
    self.executemany(operation, (params,))
  File "/usr/lib/python2.6/dist-packages/pgdb.py", line 291, in executemany
    raise OperationalError("internal error in '%s': %s" % (sql, err))
pg.OperationalError

Ok, here is the complete test code:

'''
Created on Jan 8, 2010

@author: selim
'''
import pgdb
import wx

class testApp(wx.App):
    def OnInit(self):
        ''' Creating interface '''
        frame = wx.Frame(None,wx.ID_ANY,"Testing non-unicode chars")
        panel = wx.Panel(frame)
        input = wx.TextCtrl(panel, wx.ID_ANY)
        out = wx.StaticText(panel, wx.ID_ANY)
        but = wx.Button(panel,wx.ID_ANY,"Test")

        sizer = wx.FlexGridSizer(rows=2,cols=2,vgap=5,hgap=5)
        sizer.Add(input)
        sizer.Add(but)
        sizer.Add(out)
        panel.SetSizer(sizer)
        sizer.Fit(frame)
        
        ''' And here the most interesting part is '''
        def onClick(self,e=None):
            conn = pgdb.connect(host="localhost:5433",database="database",user="user", password="password")
            db = conn.cursor()
            db.execute("SELECT name FROM contacts WHERE name LIKE '%"+input.Value+"%'")
            dbName = db.fetchone()
            if(dbName):
                out.Label = dbName[0]

        but.Bind(wx.EVT_BUTTON,onClick)
        frame.Show()
        return True
    
if __name__ == '__main__':
    app = testApp()
    app.MainLoop()

Any help will be much appreciated

[Pupo]

Hi.
Try adding this at top of your .py file:

# -*- coding: utf-8 -*-

Cheers.

Hi,
It must be a simple issue, but couldn't find any answer googling.
Here is the problem:
I have a simple test application with a TextCtrl, Button and StaticText. When I press the Button, it must query PostgreSQL with ...SELECT name FROM contacts WHERE name LIKE '%"+TextCtrl.Value+"%' and return the result.
Everything works fine when I run the application in Eclipse. But when I run the application in Terminal (by the way, I'm using Ubuntu(karmic) ) using command... python UnicodeTest.py everything works fine only if I enter English symbols but not Cyrillic (e.g. Я or Д). Entering non-English symbols ends with...

Traceback (most recent call last):
  File "UnicodeTest.py", line 28, in onClick
    db.execute("SELECT name FROM contacts WHERE name LIKE '%"+input.Value+"%'")
  File "/usr/lib/python2.6/dist-packages/pgdb.py", line 259, in execute
    self.executemany(operation, (params,))
  File "/usr/lib/python2.6/dist-packages/pgdb.py", line 291, in executemany
    raise OperationalError("internal error in '%s': %s" % (sql, err))
pg.OperationalError

Ok, here is the complete test code:

'''
Created on Jan 8, 2010

@author: selim
'''
import pgdb
import wx

class testApp(wx.App):
    def OnInit(self):
        ''' Creating interface '''
        frame = wx.Frame(None,wx.ID_ANY,"Testing non-unicode chars")
        panel = wx.Panel(frame)
        input = wx.TextCtrl(panel, wx.ID_ANY)
        out = wx.StaticText(panel, wx.ID_ANY)
        but = wx.Button(panel,wx.ID_ANY,"Test")

        sizer = wx.FlexGridSizer(rows=2,cols=2,vgap=5,hgap=5)
        sizer.Add(input)
        sizer.Add(but)
        sizer.Add(out)
        panel.SetSizer(sizer)
        sizer.Fit(frame)

        ''' And here the most interesting part is '''
        def onClick(self,e=None):
            conn = pgdb.connect(host="localhost:5433",database="database",user="user", password="password")
            db = conn.cursor()
            db.execute("SELECT name FROM contacts WHERE name LIKE '%"+input.Value+"%'")
            dbName = db.fetchone()
            if(dbName):
                out.Label = dbName[0]

        but.Bind(wx.EVT_BUTTON,onClick)
        frame.Show()
        return True

if __name__ == '__main__':
    app = testApp()
    app.MainLoop()

Any help will be much appreciated

[selimhanov]

Thank you, Pupo for quick reply. I added what you suggested and it now starts with

# -*- coding: utf-8 -*-
'''
Created on Jan 8, 2010

@author: selim
'''
import pgdb
import wx

... after testing again I got the same error in the Terminal

selim@selim-laptop:~/EclipseWorkspace/test/src$ python UnicodeTest.py 
Traceback (most recent call last):
  File "UnicodeTest.py", line 29, in onClick
    db.execute("SELECT name FROM contacts WHERE name LIKE '%"+input.Value+"%'")
  File "/usr/lib/python2.6/dist-packages/pgdb.py", line 259, in execute
    self.executemany(operation, (params,))
  File "/usr/lib/python2.6/dist-packages/pgdb.py", line 291, in executemany
    raise OperationalError("internal error in '%s': %s" % (sql, err))
pg.OperationalError

It's weird, istn't it?! Any more ideas?

[Pupo]

If that's not enought, you'll must modify the line:
query = input.GetValue().encode('utf8')
db.execute("SELECT name FROM contacts WHERE name LIKE '%"+query+"%'")
Hope that helps you.

[selimhanov]

Great! Worked perfectly. :)

After your previous post I started to read about PostgreSQL tricks like "SET CLIENT_ENCODING TO 'value'; " and etc., but what you suggested worked like a charm.

Thanks a lot, Pupo!

[Pupo]

Glad to serv you, i always have same problems because i'm Spanish parlant, so i use unicode chars too.
See you