Working with Databases

VITOPIEPOLI

Forums

Good morning,
The scenario I am in is one in which IDEA,  I am using version 12, takes in a database, "send" it to Python to do something (mainly cruching data using Pandas) and then "receives" back a different one once the data crunching is done.  
I would be very grateful if you could direct me to some source to understand how Python access Idea DB and how then Idea can access the Python output.
I have watched videos and browsed around the IdeaLab but could noy find any clear guidance.
Thank you very much in advance for your help.
Vito

Brian Element Thu, 07/14/2022 - 07:27

Hi Vitopiepoli, I recently did a webinar for Sama where I hopefully explained this.  If you start watching around minute 48 this is where I start talking about the IDEALib.py module and how to use it and show how to do what you are trying to do.  Here is the link - https://www.youtube.com/watch?v=SxLShiRuRdY&t=1787s

Let me know if you have any questions on it.

VITOPIEPOLI Fri, 07/15/2022 - 09:42

Hi Brian,
that was exactly the type of info I was looking for.
I have two questions:
1)
When I open my IDE I get this error:

I think there is an issue with the environment related to version compatibility within which Python is working?
 
While if Import the IDEAlib pack i get:

 
Which packages version do you suggest to import?
Should I replicate the packages setting that come with IDEA in a dedicated environment?
I tried to run some code using the Clinet.RunPython function and it seem to be work but I would like to run the conde from an IDE.
 
2) While runnin the code on a dataset I acountered an issue with pandas importing the data:
ERROR - Issue: Out of bounds nanosecond timestamp: 1021-01-04 00:00:00 ( I standardised the date fields)
Do you suggest to add the option into IDEALib pd.detetime to coerce or ignore the errors  there is a better way to deal with it?
 
Many thaks for your help.
 
Regards,
Vito

klmi Fri, 07/15/2022 - 07:55

You will find the source code of IDEALib.py in your IDEA program folder. So it's basically possible to review or change the code.
IDEALib.py provides no direct interface between IDEA an Python. When data is transfered from IDEA to Python a CSV export from IDEA is done. Then the CSV file is imported into a panda dataframe. That's the secret! In opposite direction CSV exports/imports are done, too.

VITOPIEPOLI Fri, 07/15/2022 - 09:47

Hi klmi,
I see wha you mean, yes that makes sense to me.
I was wondering if the code can take input from users as parameters of the function.
I wish to run the IDEAlib within IDEA scripting environment. If I understand well I can use the Client.RunPythonEx to pass an array of parameters.
Am I correct?

klmi Mon, 07/18/2022 - 03:24

>I was wondering if the code can take input from users as parameters of the function.
???
 
>I wish to run the IDEAlib within IDEA scripting environment.
I wouldn't do that but it may depend on your special project.
 
>If I understand well I can use the Client.RunPythonEx to pass an array of parameters.
Yes

VITOPIEPOLI Mon, 07/18/2022 - 03:41

>???
Input from user passed to the Python code.
> I wouldn't do that but it may depend on your special project.
It is the only way I can use it at the moment as there are campatibility issues between IDEALLib and the Python environment I am using.
Why I should not use it inside Script env?

klmi Tue, 07/19/2022 - 09:17

>>???>Input from user passed to the Python code.
Sorry, maybe now I know what you mean: You can built an input window with Python's Tkinter module  and the Entry widget.