3

I'm working with a Matlab API that loads data from a proprietary format into a series of structures. Here's an example of what a dataset looks like after loading a file:

>>fieldnames(data(1))

ans =

'Grid_Point_ID'
'Grid_Point_Latitude'
'Grid_Point_Longitude'
'Grid_Point_Altitude'
'Grid_Point_Mask'
'BT_Data'

>> data(1).BT_Data

ans =

BT_Data: [1x66 struct]

>> fieldnames(data(1).BT_Data(1))

ans =

'Flags'
'BT_Value_Real'
'BT_Value_Imag'
'Pixel_Radiometric_Accuracy'
'Incidence_Angle'
'Azimuth_Angle'
'Faraday_Rotation_Angle'
'Geometric_Rotation_Angle'
'Snapshot_ID_of_Pixel'
'Footprint_Axis1'
'Footprint_Axis2'

I want to loop over all data(i).BT_Data(j). I've already got the length of data fine, but I can't get the size/length of BT_Data (which varies for each data(i)):

>> length(data(1).BT_Data)

ans =

 1

>> size(data(1).BT_Data)

ans =

 1     1

My expected result here is ans = 66 (or equivalent array for size()).

I'm not terribly familiar with the structure data format, which may be part of my struggles. But length(data) worked fine, so I'm confused why it won't work on BT_Data (I've also tried BT_Data(:)).

The most relevant previous answer I can find is 1757250, but I couldn't get it to work (I think it doesn't apply here). Thanks for any insight you can provide.

------ EDIT ------

Here's a little more insight into how I have to use the API to even get to the point where I'm at:

>> system('ln -sf /opt/rwapi-matlab/lib/rwapi/smos/config/xml_rw_api.usr_conf.xml .');
setenv('XML_RW_API_HOME','/opt/rwapi-matlab/lib/rwapi');
path(path,'/opt/rwapi-matlab');

>> prod = RWAPI.product('SM_OPEB_MIR_SCLF1C_20110202T013659_20110202T014642_346_060_1')

Array SMOS Matlab Interface version 1.4
(c) 2010 Array Systems Computing Inc. of Canada (http://www.array.ca)
Distribution or modification of this software requires written permission from Array

prod = 

RWAPI.product handle
Package: RWAPI

Properties:
     filename: 'SM_OPEB_MIR_SCLF1C_20110202T013659_20110202T014642_346_060_1'
       header: [1x1 struct]
xml_datablock: []

Methods, Events, Superclasses

>> data = prod.dataset(2)

data =

RWAPI.dataset handle with no properties. Package: RWAPI

Methods, Events, Superclasses

>> data(1)

ans = 

       Grid_Point_ID: 251721
 Grid_Point_Latitude: 25.5000
Grid_Point_Longitude: -102.2590
 Grid_Point_Altitude: 1.4714e+03
     Grid_Point_Mask: 2
             BT_Data: [1x66 struct]

>> data(1).BT_Data

ans = 

BT_Data: [1x66 struct]

>> data(1).BT_Data(1)

ans = 

                     Flags: 6229
             BT_Value_Real: 262.5275
             BT_Value_Imag: 0
Pixel_Radiometric_Accuracy: 6160
           Incidence_Angle: 31966
             Azimuth_Angle: 10299
    Faraday_Rotation_Angle: 65277
  Geometric_Rotation_Angle: 58605
      Snapshot_ID_of_Pixel: 65752530
           Footprint_Axis1: 19649
           Footprint_Axis2: 14625

>> whos
Name Size Bytes Class Attributes

ans 1x1 1 logical
data 1x19091 112 RWAPI.dataset
prod 1x2 112 RWAPI.product

Improve this question

3 Answers 3

Reset to default
2

Okay, I really suspect it's an oddity in an overrridden subsref method in those RWAPI classes. I was able to reproduce all your observed behavior by defining a class with a slightly pathological subsref.

classdef stupidref
    %STUPIDREF Reproduce odd indexing behavior that jpatton saw. Buggy.
    properties
        BT_Data = repmat(struct('foo',42, 'bar',42), [1 66]);
    end
    methods
        function B = subsref(A,S)
            s = S(1);
            subs = s.subs;
            chain = S(2:end);

            switch s.type
                case '()'
                    B = builtin( 'subsref', A, s );
                    if ~isempty(chain)
                        B = subsref(B, chain);
                    end

                case '.'
                    % Non-standard behavior!
                    if ~isempty(chain) && isequal(chain(1).type, '()')
                        B = subsref(A.(s.subs), chain);
                    else
                        B = struct(s.subs, A.(s.subs));
                    end
            end
        end
    end
end

This is consistent with the weird difference between data(1).BT_Data and fieldnames(data(1).BT_Data(1)), and the tab-completion that repeatedly adds ".BT_Data".

>> data = stupidref;
>> data(1).BT_Data
ans = 
    BT_Data: [1x66 struct]
>> fieldnames(data(1).BT_Data)
ans = 
    'BT_Data'
>> fieldnames(data(1).BT_Data(1))
ans = 
    'foo'
    'bar'
>> length(data(1).BT_Data)
ans =
     1
>> data(1).BT_Data.BT_Data.BT_Data.BT_Data.BT_Data.BT_Data % produced by tab-completion
ans = 
    BT_Data: [1x66 struct]
>>

Your workaround is good - once you call a = data(1).BT_Data, you've got a normal struct, and the nonstandard subsref is out of the way. You can get the same effect in a one-liner with getfield.

>> btdata = getfield(data(1).BT_Data, 'BT_Data')
btdata = 
1x66 struct array with fields:
    foo
    bar

I would report this as a possible bug to the RWAPI library authors.

Feel free to just edit this code in to your own workaround answer; it's not really an answer so much as supporting diagnostics.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

+1: Nice! Looks like you found the smoking gun. I'd love to be a fly on the wall if ever the RWAPI library authors come to this thread and see a class named "stupidref" that reproduces what their class does! =D
Wow, awesome work! It's really above my head right now (I've never used Matlab for stuff beyond arrays/matrices/etc. before), but this will be a great reference as I'm trying to understand structs/objs more. +1 indeed (I can do that now!).
Thanks guys! @Jpatton, with any luck, you'll have to deal with very little of this sort of thing in your Matlab career.
2

I've found a workaround, though it's not terribly satisfying:

>> a = data(1).BT_Data

a = 

BT_Data: [1x66 struct]

>> length(a.BT_Data)

ans =

66

I'll mark this as the answer for now because I doubt there will be any other "proper" way to do it.

Andrew's answer really nailed down the issue (and why this workaround works).

Improve this answer

1 Comment

This makes sense if it's an oddity in the RWAPI's subsref methods. When you do a chain of references like data(1).BT_Data(3), all the steps - "(1)", ".BT_Data", and "(3)" - get passed to data's subsref() method, and it is free to handle or mishandle them how it wants. By capturing the results of a shorter reference in an intermediate variable, you get a struct, and can call normal indexing and length() functions.
1

Some of your results seem contradictory. For starters, if the field BT_Data contained a 1-by-66 structure array, I would expect to see output like this:

>> data(1).BT_Data

ans =

1x66 struct array with fields:
     Flags
     ...    %# etc.

The fact that you see this:

>> data(1).BT_Data

ans =

BT_Data: [1x66 struct]

suggests to me that BT_Data is actually a 1-by-1 structure with one field called BT_Data, and that field contains a 1-by-66 structure array. This would explain what you see when you get the length and size of the first BT_Data (a 1-by-1 structure). If this is the case, you should get the following results:

>> size(data(1).BT_Data.BT_Data)

ans =

     1     66

However, this still doesn't explain the output you see when you do:

fieldnames(data(1).BT_Data(1))

That's throwing me off. You may want to check that BT_Data is actually a structure and not some other type of object that may have different indexing and display behavior by doing this:

isstruct(data(1).BT_Data)

And this should return a 1 if BT_Data is a structure.

Improve this answer

9 Comments

That almost sounds like a corrupt struct. Does this proprietary API use MEX files to work with Matlab's internal data structures?
What's the output of whos after you've loaded the data? And how about builtin('isstruct', data(1).BT_Data)? (Just in case; user-defined classes can override isstruct() to masquerade as structs, builtin() bypasses that.)
@jpatton: Don't worry about the notification stuff I was talking about, you didn't do anything wrong. When someone comments on your post, you get notifications through SO. The comments Andrew left on my answer were only notifying me since there was no "@jpatton" in them, and I was just letting Andrew know because I think they contain helpful information for you.
@jpatton: Yeah. This looks like an issue in the RWAPI.product and RWAPI.dataset classes. These aren't structs (corrupt or otherwise), they're objects, and they probably override subsref and are doing non-standard indexing like gnovice suggests, and maybe mishandling "chaining". That's consistent with your workaround below, and I think the tab-completion behavior too. IMHO probably a bug in RWAPI.
@jpatton, @gnovice: Yep, got a pretty full repro using objects with a bogus subsref. Posted the code under a separate answer because it's too long for a comment.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.