How to delete rows or remove duplicates with python

talebi

Forums

Hi Brian
I wanted to know if it is possible for me to apply python scripts to remove duplicates from a table. I would like to consider all columns (more than 100 columns) as duplicates.
I know how to write a function; the problem is I should add a column and apply the function in that column. I would like to apply Python scripts to whole table.
For instance, apply this Python script in a table; data.drop_duplicates(keep=False)
Or delete rows with python
 
Thanks,
 

Brian Element Mon, 12/12/2022 - 13:45

Hi talebi,

If you want to use Python then you would need to take the IDEA database and move it into Python, do your removal of columns and such and then send it back to IDEA.  You can use the IDEALib.py module found in IDEA to easily move the data to a Panda dataframe, then do your updating on the dataframe then use the IDEALib.py module to send the dataframe back to IDEA.  

Also this might be something you can do directly in IDEA with IDEAScript.

talebi Mon, 12/12/2022 - 16:38

Thank you BrianDo you have any link or example for that? That would be great if I see an example to make some scripts faster.
Thanks,