I did a little thinking in my previous answer and it turns out it was not correct.
Firstly, do not use track by $index. It makes no sense in your case. track by is an optimisation of ng-repeat to correlate (potentially new) objects in the array that are "business-wise" equal with old objects in the array, so that it re-uses scopes and DOM elements in an effort to minimize DOM manipulation. That is: if you give ng-repeat a hint which new object in the new array is "equal" to an old object in the old array, it will reuse its scope and hopping that the new object is not dramatically different compared to the old one, less $watch callbacks will fire and less DOM updates will occur.
Your actual problem is that you are "statically" or "once-off" binding data with statements like:
$scope.gridIndex = $parse($attrs.gridIndex)($scope);
$scope.gridValues=$parse($attrs.appliedgrid)($scope);
$scope.gridParent=$parse($attrs.appliedgrids)($scope);
The first grid item is indeed removed from the array but ng-repeat does not remove its scope and DOM element because track by $index is used. But still, the new 0-index object (2nd, previously) is used to update the scope (the one created for the 1st object).
You do not see this reflecting to the UI because $scope.gridValues was evaluated in the beginning and is not evaluated again.
So, even though $scope.appliedgrid now points to [{span:12,data:[object]}], $scope.gridValues still points to [{span:6,data:[object]},{span:6,data:[grid2]}].
Removing track by $index solves the problem because ng-repeat tracks objects by reference so each object is associated with the same scope until it is removed from the array.
You can verify it with AngScope, a small Firebug-based scope inspector. You have to open it in a separate tab with "Launch the preview in a separate window" in order for it to work in plunker.
I tried to find a quick fix for it but there was no luck. I guess, you have to re-write it using isolated scopes and real 2-way binding.
&). That way,griddoes not need a reference to the whole array (and indexes etc).