AngularJS Dynamic Table Performance

I'm currently building a table for a fairly complex report. It is working but performance is very poor, building the table takes 4-5s.

The requirements for the table are as follows:

1. The table's columns are dynamically determined. There is a fixed set of columns to begin with, then a dynamic section, and then another fixed set.
2. The contents of each cell are also dynamically determined. By this I mean not just that the data for the cell changes, but the way it is presented also changes. Some cells simply output a value, some provide inputs etc.

The current solution works roughly as follows:

1. The column definitions are built and loaded into the table's scope.

2. ng-repeat is called over every row, and then over every column

3. Within each cell I make a call to a special directive to display that cell, which compiles another directive.

This special directive looks like this, it compiles the (programmatically determined) directive for the cell and replaces the 'cellDirective' node's contents:

directives.directive('cellDirective', ['$compile', function ($compile) {

    return {
        link: function (scope, elem, attrs) {
            scope.$watch(
                function (scope) {
                    return scope.$eval(attrs.cellDirective);
                },
                function (value) {
                    // profiler suggests that this is the bottleneck
                    elem.html(value);
                    $compile(elem.contents())(scope);
                }
            )
        }
    }

}]);

And the template calling it:

<tr ng-repeat="rows in rows">
    <td ng-repeat="column in columns">
        <div cell-directive="column.getDirective(row)"></div>
    </td>
</tr>

The problem I'm facing is poor performance. Generating the table takes 4-5 seconds, which is a problem because the user has options to filter the data displayed (including changing the columns) and has to go through the lag every time they do anything. I'm also worried about what will happen with larger data sets.

I've done some digging and found that:

1. For ~100 rows the application is binding 4500 watches. It seems that the suggested limit is 2000, so obviously I'm well over.
2. Looking at this in Chrome's profiler, I can see that the bulk of the execution time takes place within the directive compilation step

I'm looking for any suggestions on improving the performance of this table.

I have considered constructing a template for the whole row and compiling it in one step, but haven't been able to successfully implement such a system.

Thanks for any help.