Getting Distinct value in Python

An answer to this question on Stack Overflow.

Question

I have a list

[<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>,
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>,
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>]

I want the distinct value of this Expected Result is

[
 <Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
 <Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>
]

This data is being pulled from Database. Please don't advice me to use the DISTINCT keyword on the database. Its been sorted by another column. Using distinct will remove the sorted result.

Can it be done in python. Or do i have to write a for loop to do the same thing?

MyCODE

entries=db.query("SELECT cases.ID as CaseID,PatientProfile_ID FROM \
                             `callog` ,consultation,cases WHERE callog.Consultation_ID=consultation.ID and consultation.Case_ID = cases.ID and \
                             Doctor_ID="+str(Id)+" ORDER BY callog.CallDate DESC")
            rows=entries.list()
            return rows

Answer

This should handle things nicely.

def remove_dupes_keep_order(seq):
  seen = set()
  seen_add = seen.add
  return [ x for x in seq if not (x in seen or seen_add(x))]

Using seen_add speeds up the operation.

See also, this SO question.

Since the issue seems to be that your items are of type Storage, try this:

def remove_dupes_keep_order(seq):
  seen = set()
  seen_add = seen.add
  return [ x for x in seq if not (json.dumps(x, sort_keys=True) in seen or seen_add(json.dumps(x, sort_keys=True)))]