I need to import a very large number of files and append them into one master database.
I have written a simple script to import the files, and another simple scripte to append them into one database. So, I can just run the two scripts one after the other and be done.
However, due to the very large number of files (>500k), I am a bit worried about performance: IDEA needs to 'open-read-write-close' the files twice: first for the import-routine and then for the append-routine.
Is there a possibility in IDEA to import a file and write all the records directly to the master database, before going to the next file? (I'm looking for something similar to ACL's "EXTRACT RECORD TO filename APPEND"-command).
Thanks for any suggestions,
Last seen: 3 weeks 3 days ago
Joined: 02/25/2019 - 08:29