0

Powershell Script to Delete Blank Columns from CSV

I have a spread sheet which I'm importing into a MySQL database, the import fails because of blank columns in the spread sheet.

Is there a powershell script I can run / create that will check any given CSV file and remove blank columns?

Col1,Col2,Col3,Col4,,,, Val1,Val2,Val3,Val4

2
  • You should give an example of what you've got and what you need. A short example of what you test so far would be nice too. Commented Feb 24, 2015 at 11:37
  • Example of what I have: Col1,Col2,Col3,Col4,,,, Val1,Val2,Val3,Val4 Example of what I want: Col1,Col2,Col3,Col4 Val1,Val2,Val3,Val4 Commented Feb 24, 2015 at 14:13

3 Answers 3

1

How about something like this:

$x = Import-Csv YourFile.csv
$f = $x[0] | Get-Member -MemberType NoteProperty | Select name
$f | Add-Member -Name count -Type NoteProperty -Value 0
$f | %{
  $n = $_.Name
  $_.Count = @($x | Select $n -ExpandProperty $n | ? {$_ -ne ''}).count
}
$f = @($f | ? {$_.count -gt 0} | Select Name -expandproperty Name)

$x | Select $f | Export-Csv NewFile.csv -NoTypeInformation

It uses Get-Member to get the column names, cycles though each one to check how many are not blank and then uses the results in a select.

Sign up to request clarification or add additional context in comments.

2 Comments

I get a few error messages: 1. Cannot process argument because the value of argument "name" is invalid. As far as I'm aware, this is because PS is trying to import a CSV with blank columns at the end. After using your answer: 2. Cannot index into a null array. 3. Add-Member : Cannot bind argument to parameter 'InputObject' because it is null.
If your files are not valid CSVs then you will need to do a little prepossessing first to clean them up. First step is to find out why Powershell can't load them, is it because each column does not have a unique header or are some rows missing columns, do your columns contain text with embedded commas and no quoting. Then you can load a file with Get-Content, clean it probably with a regex/replace and then convert to an object using ConvertFrom=Csv. If you can add example of your CSV to your question I could help you more.
0

When I run Dave Sexton's code, I get:

Select-Object : Cannot convert System.Management.Automation.PSObject to one of the following 
types {System.String, System.Management.Automation.ScriptBlock}.
At line:15 char:12
+ $x | Select <<<<  $f  | Export-Csv ColsRem.test.$time.csv -NoTypeInformation
+ CategoryInfo          : InvalidArgument: (:) [Select-Object], NotSupportedException
+ FullyQualifiedErrorId : 
                  DictionaryKeyUnknownType,Microsoft.PowerShell.Commands.SelectObjectCommand

I corrected this issue by adding one more line, to force each array element to be a string.

$x = Import-Csv YourFile.csv
$f = $x[0] | Get-Member -MemberType NoteProperty | Select name
$f | Add-Member -Name count -Type NoteProperty -Value 0
$f | %{
  $n = $_.Name
  $_.Count = @($x | Select $n -ExpandProperty $n | ? {$_ -ne ''}).count
}
$f = @($f | ? {$_.count -gt 0} | Select Name -expandproperty Name)

# I could get the select to work with strings separated by commas, but the array would 
# always produce the error until I added the following line, explicitly changing the 
#values to strings.
$f = $f | Foreach-Object { "$_" } 

$x | Select $f | Export-Csv NewFile.csv -NoTypeInformation

My import CSV contains a few hundred columns and about half likely won't be populated, so getting rid of the extra columns was necessary. Now I just need to figure out how to counteract the unintended re-ordering of the columns into alphabetical order by name, without changing the names.

Comments

0

I updated Dave Sexton's great code to not remove columns with $false and only remove empty ones

function Remove-EmptyColumns {

  Param(
    [Parameter(Mandatory=$True)]
    [PSCustomObject[]]$InputObject
  )

  $ColumnNames = $InputObject[0].PSObject.Properties.Name

  $ColumnsToRemove = foreach ($Column in $ColumnNames) {
    $ColumnValues = $InputObject.ForEach{ $_.$Column }
    # Explicitly check for $false or non-empty string
    if ($ColumnValues -contains $false -or 
        $ColumnValues.Where{ $_ -ne $null -and $_ -ne '' }) {
      # Dont remove 
    }
    else {
      $Column # Remove if all empty  
    }
  }

  return $InputObject | Select-Object * -ExcludeProperty $ColumnsToRemove
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.