Missing Decision Coverage (DC) in output json

Hello everyone,
I am planning to convert the coverage from the JSON output file to another format.
However, I am missing the Decision Coverage (DC) or the results of the test vectors. These are available in the HTML output.

In the source code, there is this section in the HTML: SourceCoverageViewHTML.cpp, but I cannot find anything similar in the JSON exporter e.g.: CoverageExporterJson.cpp. You can see how often a branch was executed and how often it was T/F, but I do not see any way to reconstruct the result. Am I missing something? Or is this information simply missing?

Hi @uthmanna,

The MCDC information is exported in the JSON output as “mcdc_records” in each file with summary marked as “mcdc” for the file and the executable. You can see this on line 259 of CoverageExporterJson.cpp, in the renderFile() function:

    File["mcdc_records"] = renderFileMCDC(FileCoverage, FileReport);

You can also see these markers documented at the top of CoverageExporterJson.cpp

//     -- Files: array => List of objects describing coverage for files
//       -- File: dict => Coverage for a single file
//         -- Branches: array => List of Branches in the file
//           -- Branch: dict => Describes a branch of the file with counters
//         -- MCDC Records: array => List of MCDC records in the file
//           -- MCDC Values: array => List of T/F covered condition values
//         -- Segments: array => List of Segments contained in the file
//           -- Segment: dict => Describes a segment of the file with a counter
...
//         -- Summary: dict => Object summarizing the coverage for this file
...
//           -- MCDCCoverage: dict => Object summarizing MC/DC coverage

Each “mcdc_record” represents a decision region with line information and an array of T/F corresponding to each condition and indicating whether or not the condition has been covered. To help you parse the information, here is an example. Suppose you have a decision region that looks like this in TEXT:

   20|      5|  if ((a && b) || c)                                                                                                                                                                                                
  ------------------                                                                                                                                                                             
  |---> MC/DC Decision Region (20:7) to (20:20)                                                                                          
  |                                                                                                                                                                                              
  |  Number of Conditions: 3                                                                                                             
  |     Condition C1 --> (20:8)                                                                                                                                                                  
  |     Condition C2 --> (20:13)                                                                                                         
  |     Condition C3 --> (20:19)                                                                                                                                                                 
  |                                                                                                                                      
  |  Executed MC/DC Test Vectors:                                                                                                                                                                
  |                                                                                                                                      
  |     C1, C2, C3    Result                                                                                                                                                                     
  |  1 { F,  -,  F  = F      }                                                                                                           
  |  2 { F,  -,  T  = T      }                                                                                                                                                                   
  |  3 { T,  F,  T  = T      }                                                                                                           
  |  4 { T,  T,  -  = T      }                                                                                                                                                                   
  |                                                                                                                                      
  |  C1-Pair: covered: (1,4)                                                                                                                                                                     
  |  C2-Pair: not covered                                                                                                                
  |  C3-Pair: covered: (1,2)                                                                                                                                                                     
  |  MC/DC Coverage for Decision: 66.67%                                                                                                 
  |                                                                                                                                                                                              
  ------------------

The corresponding json “mcdc_record” may look like this:

                        [
                            20,   <-- Line Start
                            7,    <-- Col Start
                            20,   <-- Line End
                            20,   <-- Col End
                            0,    <-- File ID
                            5,    <-- MCDCDecisionRegion Type (always 5)
                            [
                                true,   <-- C1-Pair Covered?
                                false,  <-- C2-Pair Covered?
                                true    <-- C3-Pair Covered?
                            ]
                        ],

Thanks!
-Alan

Hello @evodius96, thank you for your quick reply. I think I need to explain myself more clearly. I need the C/DC coverage “individually.” If you have 100% MC/DC coverage, you also have 100% C/DC by definition, but not vice versa. There are cases where C/DC coverage is sufficient for me as a quality criterion. In other cases, if 100% MC/DC has not yet been achieved, I still want to know how much C/DC I have. In HTML, this is available via the result vector, but not in the json output.

Thank you for the clarification. Your observation is correct in that the test vector results, which includes the evaluation of each individual condition as well as the ultimate decision result, are not emitted as part of JSON. I do not think it would be difficult to export this information into the JSON. I think we could file this as an open issue, and if someone (including you) would like to implement this support, that would be great.

-Alan

@evodius96 Thank you for confirming that. I don’t think it will be difficult to implement either. I’m happy to do it. (I’ve already done it for myself.) The question is, what should the change look like? I actually need to know whether the decision was T/F or both. My suggestion would be to include it in the mcdc_records. I’ve recorded the number of true and false decisions here as follows. (Similar to the branches.)

[
                            20,   <-- Line Start
                            7,    <-- Col Start
                            20,   <-- Line End
                            20,   <-- Col End
                            0,    <-- File ID
                            5,    <-- MCDCDecisionRegion Type (always 5)
                            3,    <-- Num decicion true
                            1,    <-- Num decicion false
                            [
                                true,   <-- C1-Pair Covered?
                                false,  <-- C2-Pair Covered?
                                true    <-- C3-Pair Covered?
                            ]
                        ],

Before I list 1000 other possibilities here, I would like to hear your opinion.

Unfortunately, I have never contributed anything before. I have read the “instructions,” but I’m not sure if I understood everything. Is an “open issue” necessary? I think I can create a PR with a commit. What about the review? Can you do that?

I think that’s reasonable and straightforward since the counts apply to the decision region itself, and like you said, it’s what we do for each branch condition in branch coverage. I would suggest placing the True and False Counts in between “Col End” and “File ID” – that better matches the format used for branch conditions.

[
                            20,   <-- Line Start
                            7,    <-- Col Start
                            20,   <-- Line End
                            20,   <-- Col End
                            3,    <-- Num decision true
                            1,    <-- Num decision false
                            0,    <-- File ID
                            5,    <-- MCDCDecisionRegion Type (always 5)

Since you plan to put up a PR soon, I don’t think opening an issue is necessary. I’d be happy to review your code. Thanks!