Using CASE in the join statement in the AND clause. Where's the problem?

All,

I have my code, where based on the case statement, I would like to restrict the number of records in my JOIN condition. It's the CASE statement that I have used, that's giving me the problem. Not sure, what the problem is. The error says (in my case expression):

Select a.Col1, a.Col2, b.Col1, b.Col2, b.Col3
From Table1 a
Join Table2 b
On a.id = b.id
Join Table3 c
On c.FiltId = a.FiltIdAnd
Case
    When b.Col3 <> 5 --b.Col3 is an INT field
    Then (b.OutDate >= DATEADD(DAY, -30, c.InsertDate) And b.OutDate <= DATEADD(DAY, 30, c.InsertDate))

    When b.Col3 = 5
    Then (b.InDate >= DATEADD(DAY, 5, c.InsertDate))
End
Join Table 4 -- Other Join conditions and where clause conditions