0

A continuation of my question here: Excel, query: How to add specific info from query 2 into query 1 with lookup

I have 2 queries, "Overview" and "Data". And I want to add the information from "Data" in the "Overview" query on the correct place. The information of columns ITEM, ITEM 2 and Y/N needs to be added to the correct row, determined by the value in NAME.1 and NAME.2.

These are the two queries. (Slightly different from my last question)

Overview

NAME.1 NAME.2 NUMBER RANDOM
Name11 243324 qwsa
Name22 6747 dsfsdf
Name13 455 yyu
Name14 908098 hfhn
Name25 34 ertew
132 uil
Name17 Name27 64 tgvc
Name18 Name28 876 iorts

Data

ITEM ITEM 2 Y/N NAME.1 NAME.2
123123 AA Y Name25
234324 BB Y Name11
345345 CC N Name17
456456 AA Y Name12 Name22
567567 AA N
678678 DD N Name13 Name23
789789 DD Y Name16
890890 AA N Name14

I want to add information ITEM, ITEM 2 and Y/N from query "Data" to query "Overview" on the correct row. So, the first row of "Overview" will be:

NAME.1 NAME.2 NUMBER RANDOM ITEM ITEM 2 Y/N
Name11 243324 qwsa 234324 BB Y

The code I got from Ron in my previous question is:

let
    Source = Excel.CurrentWorkbook(){[Name="Overview3"]}[Content],
    #"Overview" = Table.TransformColumnTypes(Source,{
        {"NAME.1", type text}, {"NAME.2", type text}, {"NUMBER", Int64.Type}, {"RANDOM", type text}}),
  
//Add index column to be able to go back to original table order
    #"Added Index" = Table.AddIndexColumn(Overview, "Index", 0, 1, Int64.Type),

//Create a Name column that we will expand to all names in a single column
    #"Added Custom" = Table.AddColumn(#"Added Index", "Name", each List.RemoveNulls({[NAME.1]} & {[NAME.2]})),
    #"Expanded Name" = Table.ExpandListColumn(#"Added Custom", "Name"),

//Since Data table could also have Names in both NAME columns, we can unpivot (and delete the unneeded columns)
    #"Unpivot Data" = Table.UnpivotOtherColumns(Table.RemoveColumns(Data3,"Y/N"),{"ITEM"},"Attribute","Name"),
    #"Removed Columns" = Table.RemoveColumns(#"Unpivot Data",{"Attribute"}),

//Now we can merge the tables and return the ITEM
    #"Add Item" = Table.TransformColumns(
        Table.NestedJoin(#"Expanded Name","Name",#"Removed Columns","Name","ITEM",JoinKind.LeftOuter),
        {"ITEM", each try [ITEM]{0} otherwise null, type text}),
    #"Removed Columns1" = Table.RemoveColumns(#"Add Item",{"Name"}),

//Group by Index to return initial table in order
    #"Grouped Rows" = Table.Group(#"Removed Columns1", {"Index"}, {

    //If two names in the same row of Overview have different ITEMs, we will returned a concatenated string
        {"All", each  Table.Distinct(Table.TransformColumns(_, {"ITEM", (c)=>Text.Combine(_[ITEM], ", ")}))  , 
        type table [NAME.1=nullable text, NAME.2=nullable text, NUMBER=nullable number, RANDOM=nullable text, Index=number, ITEM=text]}}),


    #"Removed Columns2" = Table.RemoveColumns(#"Grouped Rows",{"Index"}),
    #"Expanded All" = Table.ExpandTableColumn(#"Removed Columns2", "All", {"NAME.1", "NAME.2", "NUMBER", "RANDOM", "Index", "ITEM"}),
    #"Removed Columns3" = Table.RemoveColumns(#"Expanded All",{"Index"})
in
    #"Removed Columns3"

That worked for that question. However, I can't get it working when I try to add more columns in an efficient/clean way. (add all the columns at once instead of repeating the code 3 times. Repeating the code 3 times doesn't work so far, but I think I made a minor mistake somewhere. However, I think it should be possible to do it directly for 3 columns at once. In my actual file where I want to use it, I need to add multiple columns. So a more efficient method would be much better. Both for usage and for other people when someone wants to use/edit my file)


Additional information for reply on comment Ron Rosenfield:

When I use this code in my actual file, it works till #"Expanded Fields". However, it gives an error at #"Grouped Rows". It says: Expression.SyntaxError: Invalid identifier. And it points to [Sub-supplier]

    #"Expanded Fields" = Table.ExpandRecordColumn(#"Removed Columns1", "Fields", {"Report Number", "Sub-supplier", "text text text text.", "text text text", "text text text text text", "text text"}),
    #"Grouped Rows" = Table.Group(#"Expanded Fields", {"Index"}, {{"all", 
        each Table.TransformColumns(_, {
            {"Report Number", (c)=>Text.Combine(_[Report Number], ", ")},
            {"Sub-supplier",(c)=>Text.Combine(_[Sub-supplier], ", ")}, 
            {"text text text text.",(c)=>Text.Combine(_[text text text text.], ", ")}, 
            {"text text text",(c)=>Text.Combine(_[text text text], ", ")}, 
            {"text text text text text",(c)=>Text.Combine(_[text text text text text], ", ")}, 
            {"text text", (c)=>Text.Combine(_[#"text text"], ", ")}}
        ){0}}}),
4
  • In the #"Expanded All" step you can see a list of all the column names to be expanded {"NAME.1", "NAME.2", "NUMBER", "RANDOM", "Index", "ITEM"}. Just add your additional columns to the list. Or, like the other answer in your previous question, use the Power Query UI to expand the columns instead of copying pasting code. Commented Jan 4, 2024 at 22:48
  • If Overview has entries in both NAME.1 and NAME.2, but Data only has an entry for one of them, you won't know which name it is matching. Is that an issue? Commented Jan 4, 2024 at 23:59
  • @kevin if I do that, it just shows an empty column (null). It is also not available in the UI because the columns (ITEM 2 and Y/N) are not generated in the steps before #"Expanded All" Commented Jan 5, 2024 at 9:56
  • @RonRosenfeld that's not a problem. NAME.1 and NAME.2 are connected with each other. However, it is not standardized, causing sometimes only NAME.1 is given, sometimes only NAME.2. Most of the time both are given. And in rare cases non are given. And not for every given name in "Overview" there is info available in "Data" Commented Jan 5, 2024 at 10:00

1 Answer 1

1

Given your data, the following produces the result you show:

let
    Source = Excel.CurrentWorkbook(){[Name="Overview"]}[Content],
    #"Overview" = Table.TransformColumnTypes(Source,{
        {"NAME.1", type text}, {"NAME.2", type text}, {"NUMBER", Int64.Type}, {"RANDOM", type text}}),
  
//Add index column to be able to go back to original table order
    #"Added Index" = Table.AddIndexColumn(Overview, "Index", 0, 1, Int64.Type),

//Create a Name column that we will expand to all names in a single column
    #"Added Custom" = Table.AddColumn(#"Added Index", "Name", each List.RemoveNulls({[NAME.1]} & {[NAME.2]}), type {text}),
    #"Expanded Name" = Table.ExpandListColumn(#"Added Custom", "Name"),

//Use same technique on Data table
//Be certain the ITEM column is of "type text" if not set that way initially
    #"Added Custom2" =Table.AddColumn(Data,"dataName",each {[NAME.1],[NAME.2]}, type {text}),
    #"Expanded dataName" = Table.ExpandListColumn(#"Added Custom2", "dataName"),

//Extract these fields from the Data table
    #"Data Fields" = List.FirstN(Table.ColumnNames(#"Expanded dataName"),3),

//Merge the tables and return all the columns from the DATA table
    #"Merge Overview/Data" = Table.NestedJoin(#"Expanded Name","Name",#"Expanded dataName","dataName","Join",JoinKind.LeftOuter),

//Extract the desired fields
    #"Add Custom3" = Table.AddColumn(#"Merge Overview/Data", "Fields", each if [Name]=null then null
            else if Table.RowCount([Join]) = 0 then null
            else Record.SelectFields([Join]{0}, #"Data Fields"),type record),
    #"Removed Columns" = Table.RemoveColumns(#"Add Custom3",{"Name", "Join"}),
    #"Expanded Fields" = Table.ExpandRecordColumn(#"Removed Columns", "Fields", {"ITEM", "ITEM 2", "Y/N"}),
    #"Grouped Rows" = Table.Group(#"Expanded Fields", {"Index"}, {{"all", 
        each Table.TransformColumns(_, {
            {"ITEM", (c)=>Text.Combine(_[ITEM], ", ")},
            {"ITEM 2",(c)=>Text.Combine(_[ITEM 2], ", ")}, 
            {"Y/N", (c)=>Text.Combine(_[#"Y/N"], ", ")}}
        ){0}}}),
       
    #"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"Index"}),
    #"Expanded all" = Table.ExpandRecordColumn(#"Removed Columns1", "all", {"NAME.1", "NAME.2", "NUMBER", "RANDOM", "Index", "ITEM", "ITEM 2", "Y/N"}),
    #"Removed Columns2" = Table.RemoveColumns(#"Expanded all",{"Index"}),
    #"Changed Type" = Table.TransformColumnTypes(#"Removed Columns2",{
        {"NAME.1", type text}, {"NAME.2", type text}, {"NUMBER", Int64.Type}, 
        {"RANDOM", type text}, {"ITEM", type text}, {"ITEM 2", type text}, 
        {"Y/N", type text}})
in
    #"Changed Type"

Overview
enter image description here

Data
enter image description here

Results
enter image description here

Sign up to request clarification or add additional context in comments.

10 Comments

Can I also specify which fields needs to be extracted here: #"Data Fields" = List.FirstN(Table.ColumnNames(#"Expanded dataName"),3), Preferably by column name, and/or by column number. Or is by name just by using something like #"Data Fields" = {"ITEM", "ITEM 2", "Y/N"},? [[Yes this seems to work, is this the proper way to do so?]] And if I want to specify it by column number, can I then do something like #"Data Fields" = List.FirstN(Table.ColumnNames(#"Expanded dataName"),{0, 1, 2}), [[this doesn't seem to work]]
@MMM #"Data Fields" = {"ITEM", "ITEM 2", "Y/N"} is fine, or any formula that results in the same list. Given x = Table.ColumnNames(#"Expanded dataName"), if the numbers are contiguous but don't start at the beginning, you can use List.Range function. If the numbers are non-contiguous, you can use List.Accumulate function to create the list. eg: List.Accumulate({0,1,2}, {}, (s,c)=> s & {x{c}})
When I use this code in my actual file, it works till #"Expanded Fields". However, it gives an error at #"Grouped Rows". It says: Expression.SyntaxError: Invalid identifier. And it points to [Sub-supplier] . See my startpost here on top. I edited my question with a part of the code where the error points to
@MMM [Sub-supplier] should have been marked when you tried to type it in. For certain characters, you need to quote the column name. eg: [#"Sub-supplier"] would be the correct syntax because of the hyphen.
@MMM Since it seems to be what you require, kindly accept the answer.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.