Can't Extract Specific Value (variable) from an Array in PERL

Question

DATA FILE temp.csv

1,2015-08-20,00:00:00,89,1007.48,295.551,296.66,

2,2015-08-20,03:00:00,85,1006.49,295.947,296.99,

3,2015-08-20,06:00:00,86,1006.05,295.05,296.02,

4,2015-08-20,09:00:00,85,1005.87,296.026,296.93,

5,2015-08-20,12:00:00,77,1004.96,298.034,298.87,

I want to extract (column 6) value if (column 3 = 09:00:00)

push to $result variable. This needs to be the specific value/variable not an array. I need to do additional calculations with that value and move it to a database. Here is the code I used but I can't extract the specific value to a variable.

 my @temp_9 = map {
 chomp;
 my @t_fh_9 = split /,/;
 sprintf "%.0f", $t_fh_9[6] if $t_fh_9[2] eq '09:00:00';
 } <$ft1>;

When I attempt to subtract or do other math the numbers are nonsensical.

Your sample code compares the column with 9:00:00 -- is the missing leading zero an error in your paste or your code? — Jim Davis
– Jim Davis, Commented Aug 22, 2015 at 2:21
typo - above does work however the problem remains as its an array — BrianB
– BrianB, Commented Aug 22, 2015 at 3:50
@SinanÜnür At first glance that would seem to be the case, but it's not. grep is fine when you want to filter a list, but when you want to filter and transform it's not appropriate. Especially if the filter and transform have to do the same data processing (split the CSV line). — Schwern
– Schwern, Commented Aug 22, 2015 at 19:48
@SinanÜnür To avoid parsing the CSV line multiple times you need a map, grep, map. First map parses the line, then grep filters for the lines in question, and second map extracts just the desired column. At that point a while loop is simpler. — Schwern
– Schwern, Commented Aug 22, 2015 at 22:10

Sobrique · Accepted Answer · 2015-08-22 16:03:27Z

It seems one of the sources of confusion is extracting an element from an array. An array is zero or more scalar elements - you can't just assign one to the other, because .... well, what should happen if there isn't just one element (which is the usual case).

Given an array, we can:

pop @array will return the last element (and remove it from the array) so you could my $result = pop @array;
[0] is the first element of the array, so we can my $result = $array[0];
Or we can assign one array to another: my ( $result ) = @array; - because on the left hand side we have an array now, and it's a single element - the first element of @array goes into $result. (The rest isn't used in this scenario - but you could do my ( $result, @anything_else ) = @array;

So in your example - if what you're trying to do is retrieve a value matching a criteria - the normal tool for the job would be grep - which filters an array by applying a conditional test to each element.

So:

#!/usr/bin/env perl
use strict;
use warnings;
my @lines = grep { (split /,/)[2] eq "12:00:00" } <DATA>;
print "@lines";
print $lines[0];

__DATA__
1,2015-08-20,00:00:00,89,1007.48,295.551,296.66,
2,2015-08-20,03:00:00,85,1006.49,295.947,296.99,
3,2015-08-20,06:00:00,86,1006.05,295.05,296.02,
4,2015-08-20,09:00:00,85,1005.87,296.026,296.93,
5,2015-08-20,12:00:00,77,1004.96,298.034,298.87

Which we can reduce to:

my ( $firstresult ) = grep { (split /,/)[2] eq "12:00:00" } <DATA>;
print $firstresult;

But as we want to want to transform our array - map is the tool for the job.

my ( $result ) = map { (split /,/)[6] - 273.15 } grep { (split /,/)[2] eq "12:00:00" } <DATA>;
print $result;

First we:

use grep to extract the matching elements. (one in this case, but doesn't necessarily have to be!)
use map to transform the list, so that that we turn each element into just it's 6th field, and subtract 273.15
assign the whole lot to a list containing a single element - in effect just taking the first result, and throwing the rest away.

Or perhaps:

#!/usr/bin/env perl
use strict;
use warnings;

my ($result) = map {
          ( split /,/ )[2] eq "12:00:00"
        ? ( split /,/ )[6] - 273.15
        : ()
} <DATA>;
print $result;

But personally, I think that's getting a bit complicated and may be hard to understand. map is a powerful function, but can lead to code that's hard to read for future maintenance programmers.

So I would suggest instead:

my $result;
while (<DATA>) {
    my @fields = split /,/;
    if ( $fields[2] eq "12:00:00" ) {
        $result = $fields[6] - 273.15;
        last;
    }
}

print $result;

Iterate your data, split - and test - each line, and when you find one that matches the criteria - set $result and bail out of the loop.

Schwern · Accepted Answer · 2015-08-22 19:50:55Z

1

map always has a one to one mapping, if you put 10 things in, you will get a list of 10 things out. What you want is a while loop which selectively pushes onto an array.

use strict;
use warnings;
use v5.10;

my @nines;
while(<DATA>) {
    chomp;
    my @row = split /,/;
    next unless $row[2] eq '09:00:00';
    push @nines, sprintf("%.0f", $row[6]);
}

say join ", ", @nines;

__DATA__
1,2015-08-20,00:00:00,89,1007.48,295.551,296.66,
2,2015-08-20,03:00:00,85,1006.49,295.947,296.99,
3,2015-08-20,06:00:00,86,1006.05,295.05,296.02,
4,2015-08-20,09:00:00,85,1005.87,296.026,296.93,
5,2015-08-20,12:00:00,77,1004.96,298.034,298.87,

From your variable names like @temp_9 and @t_fh_9, I suspect you're hard coding the variable name with the particular time of day. This will lead to a lot of code duplication. Instead, write a little function that takes the time of day you're looking for.

sub extract_column_for_time_of_day {
    my($fh, $column_number, $time_of_day) = @_;

    my @extracts;
    while(<$fh>) {
        chomp;
        my @row = split /,/;
        next unless $row[2] eq $time_of_day;
        push @extracts, sprintf("%.0f", $row[$column_number]);
    }

    return @extracts;
}

say join ", ", extract_column_for_time_of_day(\*DATA, 6, '09:00:00');

Finally, I'm guessing you're going to get data for various times. This will lead to a pile of variables which will be awkward to pass around as a unit. Instead of having a variable for each time, put each list into a hash.

my $time = '09:00:00'; 
$extracts{$time} = [extract_column_for_time_of_day(\*DATA, 6, $time)];

edited Aug 22, 2015 at 19:50

answered Aug 22, 2015 at 4:30

Schwern

167k28 gold badges225 silver badges370 bronze badges

1 Comment

Borodin Over a year ago

"map always has a one to one mapping, if you put 10 things in, you will get a list of 10 things out" is misleading as tose things may be multi-element lists or empty lists. map { $_ > 2 ? ($_, $_, $_) : () } 1, 2, 3, 4 generates the six-element list 3, 3, 3, 4, 4, 4

Borodin · Accepted Answer · 2015-08-22 11:08:38Z

0

The map function typically returns a list with the same number of elements as the source, unless you return an empty or multi-element list. In fact, one of the first examples in the map perl doc's shows how to do this, once again proving you should always check the docs first.

my @temp_9 = map {
    chomp;
    my @t_fh_9 = split /,/;
    $t_fh_9[2] eq '09:00:00' ? ( $t_fh_9[6] ) : ();
} <$ft1>;

edited Aug 22, 2015 at 11:08

Borodin

127k9 gold badges72 silver badges146 bronze badges

answered Aug 22, 2015 at 4:40

elcaro

2,31716 silver badges15 bronze badges

1 Comment

BrianB Over a year ago

The "my ( $result ) = map" by Sobrique was the syntax that saved the day. The array was killing me, though it was producing the desired value at first blush, I was not able to use that variable in further calculations or functions. The above syntax has given me that much needed flexibility. Thanks everyone for your help and thanks Sobrique.

Collectives™ on Stack Overflow

Can't Extract Specific Value (variable) from an Array in PERL

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related