Check a column if it is blank

10 posts / 0 new
Last post
CB's picture
CB
Offline
Joined: 10/19/2012 - 04:54
Check a column if it is blank

Dim iCheckDateColumnName As String
Dim iCheckStringColumnName As String

Sub Main

     iCheckDateColumnName = 0
     iCheckStringColumnName = 0

    RunSub_ColumnNullCheck

        sTableName = Dir(Client.Workingdirectory + "TableName.IMD")       
        Set db = Client.OpenDatabase(sTableName)
        Set task = db.Extraction
   
        task.AddFieldToInc "XXXXXXXXX"
        task.AddFieldToInc "XXXXXXX"
   
        'If Column is NULL then do not Extract
            If iCheckDateColumnName = 1 Then
                task.AddFieldToInc "DateColumnName"
            End If
           
            If iCheckStringColumnName = 1 Then
                task.AddFieldToInc "StringColumnName"
            End If
   
        task.AddFieldToInc "YYYYY"
        task.AddFieldToInc "YYYYY"
       
        task.AddKey "ABC", "A"
        dbName = Client.UniqueFileName ("PROTOTYP_CheckNull")
        task.AddExtraction dbName, "", ""
        task.PerformTask 1, db.Count
        Client.OpenDatabase (dbName)   

End Sub

Sub RunSub_ColumnNullCheck

    'declaration
    Dim percentComplete As Object
    Dim iCnt As Long
    Dim iLoopCounter As Long
    Dim sArrayFieldContent() As String
    Dim sTableName As String    
    iCnt = 0
    sTableName = ""
    

'Date Field
'---------------------------------------
    Set percentComplete = CreateObject ("CommonIdeaControls.StandaloneProgressCtl")
    iLoopCounter = 0    
    sTableName = Dir(Client.Workingdirectory + "TableName.IMD")
    Set db = Client.OpenDatabase(sTableName)
        iCnt = db.Count            
    Set table = db.TableDef
    Set field = Nothing
    Set field = table.GetField("DateColumnName")    
        ReDim sArrayFieldContent(iCnt)    
        Set RS = db.RecordSet    
        percentComplete.Start "Checking DateColumnName NULL"        
        RS.ToFirst            
        For iLoopCounter = 1 To iCnt    
            percentComplete.Progress Int(iLoopCounter * 100 /  iCnt)
            RS.Next        
            Set REC = RS.ActiveRecord        
            sArrayFieldContent(iLoopCounter-1) = REC.GetDateValue(field.name)    
            'If DateField has Content exit loop
            If iAbs(sArrayFieldContent(iLoopCounter-1)) <> "0" Then
                iCheckDateColumnName = 1
                Exit For
            End If        
        Next iLoopCounter        
    db.Close    
    
    Set Rec = Nothing    
    Set RS     = Nothing    
    Set table = Nothing
    Set field = Nothing    
    Set db = Nothing
    Set percentComplete = Nothing    
    
' String Field
'------------------------------------------------
    Set percentComplete = CreateObject ("CommonIdeaControls.StandaloneProgressCtl")
    iLoopCounter = 0    
    sTableName = Dir(Client.Workingdirectory + "TableName.IMD")
    Set db = Client.OpenDatabase(sTableName)
        iCnt = db.Count            
    Set table = db.TableDef
    Set field = Nothing
    Set field = table.GetField("StringColumnName")    
        ReDim sArrayFieldContent(iCnt)    
        Set RS = db.RecordSet    
        percentComplete.Start "Checking StringColumnName NULL"            
        RS.ToFirst            
        For iLoopCounter = 1 To iCnt    
        percentComplete.Progress Int(iLoopCounter * 100 /  iCnt)        
            RS.Next        
            Set REC = RS.ActiveRecord        
            sArrayFieldContent(iLoopCounter-1) = REC.GetCharValue(field.name)
            'If String Field has Content exit loop            
            If iIsBlank(sArrayFieldContent(iLoopCounter-1)) = 0 Then
                iCheckStringColumnName   = 1
                Exit For
            End If                        
        Next iLoopCounter    
    db.Close    

    Set Rec = Nothing    
    Set RS     = Nothing    
    Set table = Nothing
    Set field = Nothing    
    Set db = Nothing        
    Set percentComplete = Nothing    
            
End Sub

Brian Element's picture
Brian Element
Offline
Joined: 07/11/2012 - 19:57

Hi Chris,

Thanks for sharing the script.  One suggestion to speed up the process is for date fields is to use the field statistics instead of looking through the entire file.  If the #of Zero item stats is the same as the number of records you know that all the dates are set to 0.  You can also use this if you want to look at numeric or time fields.  The trick is you have to make sure that the field stats have been generated prior to verifying that there are 0 entires.

Thanks

Brian

CB's picture
CB
Offline
Joined: 10/19/2012 - 04:54

hi brianthanks for the tip... that is a thing i have never tested or thing about... but it should be very fast...i will try it soon... could be a really great performance step in the script,  especially if there are millions of datarows...cheers,chris

CB's picture
CB
Offline
Joined: 10/19/2012 - 04:54

hi brian,
FieldStats works really perfect for date-, time- and numeric fields... great..

cheers,
chris

Brian Element's picture
Brian Element
Offline
Joined: 07/11/2012 - 19:57

Chris, glad that the suggestion helped you out.  So just out of curiousity what type of scripting do you do?

CB's picture
CB
Offline
Joined: 10/19/2012 - 04:54

hi brian,
i am mainly a database programmer (oracle db 11g).
by the way i am developing for our users (tax auditors) standardized query in idea 8.5.
since the data basis for audits are from our databases i can work with a fixed data wreath.
in idea i develop graphical user interfaces and make all sorts of things with arrays, extractions, etc

just one information by using fieldstats ...
in idea 8.5 (german version), the table must contain a numeric field, which must be addressed before the date field,
otherwise idea raise an error..


so my redesigned example:


sTableName= Dir(Client.Workingdirectory + "TableName.IMD")
Set db = Client.OpenDatabase(sTableName)
iCntData= db.Count

'1st field must be a numField!
Set stats = db.FieldStats("NumFieldName")
Set stats = db.FieldStats("DateFieldName")

iCntNullData= stats.NumZeroItems()

If iCntNullData = iCntDataThen
   iCheckDateFieldName = 0
Else
   iCheckDateFieldName = 1
End If
db.Close        

sTableName= ""
iCntData= 0
iCntNullData= 0
Set stats = Nothing
Set db = Nothing

and iCntData and iCntNullData should be datatype LONG if you deal with big data..

what kind of scripting do you do with idea? i like your website, often looked for something similar like this and now i find it...

cheers,
chris

Brian Element's picture
Brian Element
Offline
Joined: 07/11/2012 - 19:57

Hi Chris,

Thanks for posting the updated script.  I am looking into finding a module so that will format the code better, I think I found one so I will try it out tonight.  That way the code will be easier to read!!!

I have been using IDEA for 23 years, I started using it when I worked with the Office of the Auditor General of Canada and have kept using in all my subsequent jobs.  I have also been a part-time trainer for the last 10 years for IDEA.  Last year I started getting into the scripting side and have been working with some friends that have great ideas for scripts.

I am glad you are enjoying the site, I was like you, always looking around for a site like this.  So several months ago I approached IDEA with the idea to create a site like this and they thought it was a great idea (sorry for all the ideas Smile )  So I started building the site and adding my own scripts and content and I hope to keep adding information for all the different users.

CB's picture
CB
Offline
Joined: 10/19/2012 - 04:54

..would be great..

hi brian,

i start using idea (i think it was the 2004 version?) in 2003 and it makes lot of fun to build scripts... sometimes it is hard to find workarounds when idea have bugs but it makes just fun...

the idea of an idea site is a really great idea ;-)
one thing i have noticed is that the website is a bit..how to say?!...slow... i take several seconds to navigate on the website...maybe you could speed it up??

if the code format is getting better i could share some longer code... :-) keep on going..

cheers,
chris

next snippet will be an actionfield example...

Brian Element's picture
Brian Element
Offline
Joined: 07/11/2012 - 19:57

Hi Chris,

Well it looks like you have been doing scripting longer than I have which is great, I only started last year and I am still learning.  So please keep posting I appreciate it and others will also appreciate your experience.

Yes, I have noticed it also, I am on a shared server so I will talk to my hosting company to see if there is something I can do to speed up the site as sometimes it works fine, like this morning, and other times it takes forever, like it was doing last night.

I am able to drop the code in properly as I edited your posts above but it doesn't look like you have the functionality, I will experiment to see what options you need to format it properly and for right now I can just reformat it for you until I figure it out.

Thanks

Brian

CB's picture
CB
Offline
Joined: 10/19/2012 - 04:54

... idea scipting is only additional for me.... i think you will get fast more experience if you work with it day after day....

so please share your experience..

cheers,
chris