Fetch multiple SAP Tables to Azure Data Factory?

Question

I am trying to connect multiple SAP tables in ADF. I have a SAP table connector with dataset but I am not able to connect multiple tables with this. It is working for a single table without any error. I have read the official document on Microsoft and there customRfcReadTableFunctionModule is mentioned, but there is no description as how to set parameters and all.

https://learn.microsoft.com/de-de/azure/data-factory/connector-sap-table?tabs=data-factory#copy-activity-properties

I have tried setting up dataset parameters but it is not working. Can anyone help me with this?

I have these parameters in my dataset.

I tried to setup parameters but my copy activity get failed. I get this error:

"Source". ErrorCode=SapRfcClientOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=SapRfcClient operation 'ReadTable' failed. Please contact SAP support if you need further help. Error message: 'Element QUERY_TABLE of container metadata ZZ_READ_VM unknown',Source=Microsoft.DataTransfer.Runtime.SapRfcHelper,''Type=SAP.Middleware.Connector.RfcInvalidParameterException,Message=Element QUERY_TABLE of container metadata ZZ_READ_VM unknown,Source=sapnco

what is your requirement? Do you want to copy multiple SAP tables using ADF? and Do you have list of tables that you want to copy? and you mentioned that dataset parameters are not working, what is the issue with those? you got any error? — Rakesh Govindula
– Rakesh Govindula, Commented Apr 18, 2024 at 10:30
@RakeshGovindula I want to copy multiple tables from SAP. I read the document from Microsoft and tried to setup parameters but my copy activity get failed. I get this error: — Ajax
– Ajax, Commented Apr 18, 2024 at 10:44
@RakeshGovindula "Source". ErrorCode=SapRfcClientOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=SapRfcClient operation 'ReadTable' failed. Please contact SAP support if you need further help. Error message: 'Element QUERY_TABLE of container metadata ZZ_READ_VM unknown',Source=Microsoft.DataTransfer.Runtime.SapRfcHelper,''Type=SAP.Middleware.Connector.RfcInvalidParameterException,Message=Element QUERY_TABLE of container metadata ZZ_READ_VM unknown,Source=sapnco I don't know if i am doing it right. — Ajax
– Ajax, Commented Apr 18, 2024 at 10:45
if customRfcReadTableFunctionModule works like I assume, and uses RFC_READ_TABLE under the hood, you are out of the luck. It is 1-by-run only, i.e. it can pull only 1 table at a time — Suncatcher
– Suncatcher, Commented Jul 16, 2024 at 1:16

Rakesh Govindula · Accepted Answer · 2024-04-18 11:40:45Z

I have a SAP table connector with dataset but i am not able to connect multiple tables with this

You can follow the below approach to achieve your requirement.

For this, first you need a list table names. Store this data in an array parameter in the pipeline.

["table1","table2"]

Here, I have used SQL dataset for sample. You can follow the same process for your case as well.

First create a string parameter in the dataset.

enter image description here

In your dataset, use this parameter @dataset().table_name in the table name.

enter image description here

In the pipeline, use a For-Each and give the array parameter as for-each input. Inside for-loop, take copy activity with your dataset as source. In the source, you need to provide the @item() from for-each to the dataset parameter.

enter image description here

In each iteration, the table will be copied to the given destination.

If you want to select the data from every table in the copy activity, then instead of using dataset parameters, you can directly use a query like below in the copy activity SAP source.

select * from @{item()};

enter image description here

In each iteration, the table name will be changed dynamically and all tables will be copied as expected.

Collectives™ on Stack Overflow

Fetch multiple SAP Tables to Azure Data Factory?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related