Way to get summary and pivot for more than 50K records

mohit7790

Forums

I have 30 million records in Idea and now i want to create pivot table and summarization based on certain fields but i am getting an error that it can not display more than 32K records. Please let me know the other way in idea (scripting etc) which might help me to analyse the information.

Steven Luciani Thu, 12/20/2018 - 10:56

Hello,

I assume you are using the summarization function in IDEA. The 32K records limit is only invoked in this feature when you are creating a result or you check off the box for use quick summarization.

Checking the quick summarization executes the summarization faster but limits you to sumarizing by just one field and that field cannot have more than 32K uniques values.

The default output for a summarization is a database and there is no limit to the amount of unique values in the field or fields by which you are summarizing. If you choose the create result box then the display in the result is limited to just 32K unique values.

In summary, don't check the quick summarization box and output your result to a database in order to see all your unique values.