15 
Array
(
    [0] => Array
        (
            [file] => /var/websites/example.com/assets/images/200px/1419050406e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
            [md5] => 42479bee7a304d2318250de2ef1962a9
            [url] => http://example.com/assets/images/200px/1419050406e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
        )

    [1] => Array
        (
            [file] => /var/websites/example.com/assets/images/200px/21277792606e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
            [md5] => 42479bee7a304d2318250de2ef1962a9
            [url] => http://example.com/assets/images/200px/21277792606e6648e1c766551a0ffc91380fd6ff3406002011-10-233750.jpg
        )
)

How can I remove md5 key duplicates from above array?

Improve this question
3
  • 2
    are you trying to remove only the md5 key, or the whole array containing duplicate key? Commented Oct 23, 2011 at 16:15
  • Generic array or specific to some language? Commented Oct 23, 2011 at 16:15
  • @Tim, looking at the tags, I think it's about PHP Commented Oct 23, 2011 at 16:23

8 Answers 8

Reset to default
17 
<?php
$data = array(
  array(
    'md5' => 'alpha',
    'some' => 'value',
  ),
  array(
    'md5' => 'alpha',
    'some' => 'other value',
  ),
  array(
    'md5' => 'bravo',
    'some' => 'third value',
  ),
);
// walk input array
$_data = array();
foreach ($data as $v) {
  if (isset($_data[$v['md5']])) {
    // found duplicate
    continue;
  }
  // remember unique item
  $_data[$v['md5']] = $v;
}
// if you need a zero-based array, otheriwse work with $_data
$data = array_values($_data);
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

you're welcome. mind marking your question being answered? (Don't care which answer… Gumbo's is a good one, too…)
7

Here is my final function after your help guys.... Hope it helps somebody in the future...

    $data = array(
          array(
            'md5' => 'alpha',
            'some' => 'value',
          ),
          array(
            'md5' => 'alpha',
            'some' => 'other value',
          ),
          array(
            'md5' => 'bravo',
            'some' => 'third value',
          ),
        );
        // walk input array


        function remove_duplicateKeys($key,$data){

        $_data = array();

        foreach ($data as $v) {
          if (isset($_data[$v[$key]])) {
            // found duplicate
            continue;
          }
          // remember unique item
          $_data[$v[$key]] = $v;
        }
        // if you need a zero-based array
        // otherwise work with $_data
        $data = array_values($_data);
        return $data;
        }

$my_array = remove_duplicateKeys("md5",$data);
Improve this answer

1 Comment

this is great, but how do i add nested key in it? lets say inside "some" there's "name" now, how do i avoid duplication in ["some"]["name"]?
6

PHP does already have a function to remove duplicate elements from an array. But unfortunately, array_unique does only support string based comparisons:

Note: Two elements are considered equal if and only if (string) $elem1 === (string) $elem2. In words: when the string representation is the same. The first element will be used.

The problem is that any array turned into a string is equal to any other array:

Arrays are always converted to the string "Array"; […]

But you can use the uniqueness of array keys to solve this:

$index = array();
foreach ($arr as $key => $item) {
    if (isset($index[$item['md5']])) {
        unset($item[$key]);
    }
    $index[$item['md5']] = TRUE;
}
Improve this answer

1 Comment

I think you meant $item['md5'] instead of $arr['md5'].
3

As array_unique operates on flat arrays, you can not use it directly. But you can first map all 'md5' values to a flat array, unique it and then get the elements with array_intersect_key:

$allMd5s = array_map(function($v) {return $v['md5'];}, $array);

$uniqueMd5s = array_unique($md5);

$result = array_intersect_key($arr, $uniqueMd5s);
Improve this answer

Comments

3

Tldr.: The obligatory one-liner:

$out = array_values ( array_intersect_key( $in, array_unique( array_column( $in, 'md5' ) ) ) );

And step by step (with my own example):

$in = [
    [ 'k' => 'a' ],
    [ 'k' => 'a' ],
    [ 'k' => 'b' ]
];

print_r( array_column( $in, 'k' ) );

array_column returns a flat array, only with the values from all 'k'-keys:

Array
(
    [0] => a
    [1] => a
    [2] => b
)

We can now filter that with array_unique:

print_r( array_unique( array_column( $in, 'k' ) ) ) );

To get:

Array
(
    [0] => a
    [2] => b
)

See that we kept the keys of the original array (0 and 2)? We can use that with array_intersect_key. It computes the intersection of arrays using keys for comparison.

print_r( array_intersect_key( $in, array_unique( array_column( $in, 'k' ) ) ) );

We get the first [0] and third [2] entry of our original array.

Array
(
    [0] => Array
        (
            [k] => a
        )

    [2] => Array
        (
            [k] => b
        )

)

Now the keys are messed up (which might lead to problems if for loops), so we wrap the whole thing in array_values:

print_r(
  array_values( array_intersect_key( $in, array_unique( array_column( $in, 'k' ) ) ) ) 
);

Array
(
    [0] => Array
        (
            [k] => a
        )

    [1] => Array
        (
            [k] => b
        )

)
Improve this answer

Comments

1

Use array_filter(). Quick code test (doesn't necessarily mirror your situation, but you should get the idea):

<?php
header('Content-Type: Text/Plain');
$array = array(
    0 => array('name' => 'samson'),
    1 => array('name' => 'delilah'),
    2 => array('name' => 'samson'),
    3 => array('name' => 'goliath'),
);

$md5Processed = array();

$newArray = array_filter($array, "uniqueMD5");

print_r($newArray);

exit;

function uniqueMD5( $data ){
    global $md5Processed;

    if( !in_array( $data['name'], $md5Processed ) )
    {
        $md5Processed[] = $data['name'];
        return true;
    }
}
Improve this answer

Comments

1 
// your array
$array = array(...);
// will be used to store md5 hashes
$md5 = array();
// walk through array
foreach ($array as $key => $arr) {
  // have we already seen this md5 hash?
  if (in_array($arr['md5'], $md5)){
    // remove element
    unset($array[$key]);
  }else {
    // keep element, but add it's md5
    $md5[] = $arr['md5'];
  }
}
Improve this answer

Comments

-2

Take a look at the function array_unique.

Improve this answer

5 Comments

It's not going to remove the the md5s because to each array cell they are a unique value
array_unique won't work with multidimensional arrays. The function uses the string representation of each element to compare them.
Unfortunately array_unique does not work on multi-dimensional arrays. Also array_unique uses the === operator to compare two items - so you have to be careful how your data is stored.
@FelixKling: What is the string representation of an array?
@Eric: It's "Array" (afaik, or something similar). I.e. every element (array) has the same string representation and thus makes this function not useful.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.