SQL query with conditional join based on a table variable

The easiest way to do this is going to be to create two queries and UNION ALL them together. Each "half" of the UNION ALL will handle the possible values of @inputVar. One query for when the variable is NULL, and one for when it isn't.

DECLARE @inputIds VARCHAR(50) = NULL; -- This can have a comma-separated list of ids as well.

DECLARE @inputIdsVar TABLE (ID INT);
INSERT INTO @inputIdsVar SELECT DISTINCT CAST(value AS INT) FROM fnSplit(@inputIds, ','); -- Some custom function to split the Ids list.

SELECT 
    tbl1.JoinIndex1
    ,JoinIndex2 = NULL
    ,JoinIndex3 = NULL
FROM TABLE1 tbl1
WHERE @inputVar IS NULL
    AND tbl1.Key = 'blah'

UNION ALL

SELECT 
    tbl1.JoinIndex1
    ,tbl2.JoinIndex2
    ,tbl3.JoinIndex3
FROM TABLE1 tbl1
JOIN TABLE2 tbl2 ON tbl1.JoinIndex1 = tbl2.JoinIndex1 
JOIN TABLE2 tbl3 ON tbl2.JoinIndex2 = tbl3.JoinIndex2 
JOIN @inputIdsVar iiv ON tbl3.ID = iiv.ID 
WHERE @inputVar IS NOT NULL
    AND tbl1.Key = 'blah'

You should only ever get records from the first half or the second half of the UNION ALL. You'll also want to make sure your column list is the same in both halves. If you need any column names in the first half of the query that are only present in TABLE2, you can just stick a NULL with the correct alias. Notice the JoinIndex2 = NULL in the first half.

Digital aaron is on the right track, because OR kills performance. However, you are using select *, so you need to be careful about the columns.

I think the equivalent query with better performance is:

SELECT *
FROM TABLE1 tbl1 LEFT JOIN
     TABLE2 tbl2
     ON 1 = 0 LEFT JOIN  -- always false 
     TABLE2 tbl3
     ON 1 = 0 LEFT JOIN
     @inputIdsVar iiv
     ON 1 = 0 
WHERE tbl1.Key = 'blah'
UNION ALL
SELECT *
FROM TABLE1 tbl1 JOIN
     TABLE2 tbl2
     ON tbl1.JoinIndex1 = tbl2.JoinIndex1 AND
        @inputVar IS NOT NULL JOIN
     tbl3
     ON tbl2.JoinIndex2 = tbl3.JoinIndex2 AND
        @inputVar IS NOT NULL JOIn
     @inputIdsVar iiv
     ON tbl3.ID = iiv.ID AND @inputVar IS NOT NULL
WHERE tbl1.Key = 'blah';