Reference data join on stream analytics input not giving output

Question

I'm trying to set a rule in Azure Stream Analytics job with the use of reference data and input stream which is coming from an event hub.

This is my reference data JSON packet in BLOB storage:

{
    "ruleId": 1234, 
    "Tag" : "TAG1", 
    "metricName": "velocity", 
    "alertName": "velocity over 500",
    "operator" : "AVGGREATEROREQUAL",
    "value": 500
}

And here is the transformation query in the stream analytics job:

WITH
transformedInput AS
(
    SELECT
        metric = GetArrayElement(DeviceInputStream.data,0),
        masterTag = rules.Tag, 
        ruleId = rules.ruleId, 
        alertName = rules.alertName,
        ruleOperator = rules.operator, 
        ruleValue = rules.value
    FROM 
        DeviceInputStream
        timestamp by EventProcessedUtcTime
    JOIN 
        rules
        ON DeviceInputStream.masterTag = rules.Tag
)
--rule output--
SELECT
    System.Timestamp as time,
    transformedInput.Tag as Tag,
    transformedInput.ruleId as ruleId,
    transformedInput.alertName as alert,
    AVG(metric.velocity) as avg
    INTO
    alertruleblob
FROM
    transformedInput
GROUP BY
    transformedInput.masterTag,
    transformedInput.ruleId,
    transformedInput.alertName,
    ruleOperator, 
    ruleValue, 
    TumblingWindow(second, 6)
HAVING
    ruleOperator = 'AVGGREATEROREQUAL' AND avg(metric.velocity) >= ruleValue

This is not yielding any results. However, when I do a test with sample input and reference data I get the expected results. But this doens't seem to be working with the streaming data. My use case is if the average velocity is greater than 500 for a 6 second window, store that result in another blob storage. The value of velocity has been greater than 500 for sometime, but I'm not getting any results.

What am I doing wrong?

Based on your description,it's difficult to reproduce your issue on my side.I suggest you trying to trace the log of ASA,please refer to this doc:azure.microsoft.com/en-us/blog/… — Jay Gong
– Jay Gong, Commented Sep 6, 2018 at 9:51

MAK · Accepted Answer · 2018-09-06 14:36:40Z

1

This was working all along. I just had to specify the input path of the reference blob in the reference input path of stream analytics including the file name. I was basically referencing only the blob container without the actual file. So when I changed the path pattern to "filename.json", I got the results. It was a stupid mistake.

answered Sep 6, 2018 at 14:36

MAK

1,3082 gold badges26 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Reference data join on stream analytics input not giving output

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related