Member Avatar for jamesjohnson25

Value of

set(sum(french,())) = set(['mardi', 'pass\xc3\xa9', 'voyez', 'envoy\xc3\xa9', 'membres', 's\xc3\xa9lection', 'peut', 'remplissaient', '\xc3\xa9t\xc3\xa9', 'prononcent', 'travaux', 'd\xc3\xa9terminent', 'trop', 'lib\xc3\xa9raux', 'd\xc3\xa9clar\xc3\xa9', 'dont', 'le', 'mais', 'la', '(', ',', 'internationales', 'Les',.....])

tm = {('se', 'est', '-', 'il', 'pass\xc3\xa9'): [phrase(english='has happened', logprob=0.0)], ('pos\xc3\xa9e',): [phrase(english='asked', logprob=-0.261521458626)], ('le', 'cours', 'de', 'les', 'deux', 'prochaines'): [phrase(english='the next two', logprob=0.0)], ('sujet', 'de'): [phrase(english='about', logprob=-0.390253186226)], ('pla\xc3\xaet',): [phrase(english='pleasure', logprob=-0.0914471149445)],.....}

Snippet

for word in set(sum(french,())):
  if (word,) not in tm:
    tm[(word,)] = [models.phrase(word, 0.0)]

Q . when comes to if condition , what exaclty is it trying to do ?

Q . Is it comparing the whole tuple in tm ?

  • 2 Contributors
  • 1 Reply
  • 261 Views
  • 1 Hour Discussion Span
  • Latest Post Latest Post by Gribouillis

Dani AI

Generated

Short answer: the if is checking for a dictionary key that is a one-element tuple containing the current word. If that exact tuple key is missing, the code creates a default value for that key. This matters because a bare string and a one-element tuple are different objects in Python — they do not compare equal and have different hashes, so they are different dictionary keys.

A few practical notes and tips (building on comment about encodings and 's snippet):

  • A one-element tuple is written with a trailing comma; it is immutable and hashable, so it can be used as a dict key (lists cannot). To see the difference, inspect types and equality in the REPL (print type()/repr() or use isinstance()) rather than guessing which form is stored as a key.

  • Instead of the explicit membership test + assignment, consider helpers that simplify “create if missing” logic:

    # setdefault example
key = (w,)
tm.setdefault(key, []).append(make_entry(w))

    Or use a defaultdict(list) and then append; both avoid repeating lookup/assignment.

  • About flattening the input: using repeated tuple concatenation (sum over tuples) can be slow on large inputs. Prefer a memory-efficient flatten with itertools.chain.from_iterable or a set comprehension:

    from itertools import chain
unique_words = set(chain.from_iterable(french))

  • Encoding: follow — decode byte strings to Unicode as early as possible (open files with an encoding argument in Python 3, or decode bytes in Python 2). When debugging key problems, print repr(key) and type(key) or list(tm.keys()) to confirm you really have tuple keys and not bare strings.

These checks will make it clear why the code uses a tuple key and will point you to simpler, faster alternatives for populating the dictionary.

Member Avatar for Gribouillis

The if condition looks if the dictionary tm contains a key which is a tuple of length 1 containing the word as sole item. If it does not, a value is associated to this key in this dictionary.

It seems strange to me that you don't work with unicode strings instead of utf8-encoded strings. For exemple in python 2:

>>> s = 'pass\xc3\xa9'
>>> t = s.decode('utf8')
>>> print(t)
passé
>>> print(repr(t))
u'pass\xe9'
Edited by Gribouillis
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.